[Fedora-livecd-list] Revised: [PATCH] turboLiveInst - improves livecd/usb installer speed by 15-20+%

[Douglas McClendon]

Mark McLoughlin wrote:
> Hi,
> 	This doesn't actually seem so bad to me ...

Thanks for taking a look.

> 
> 	The main suggestion I'd have is that if the osmin.size was dropped[1],

Actually in my first iteration of the patch, I was using dumpe2fs, but 
my parsing of it with awk and grep looked pretty ugly.  I'm not opposed 
to going back to that if you tell me that is what will get accepted. 
Another minor justification is to minimize the work done at install 
time.  (though I admit the bash+dumpe2fs+grep+sed is not going to add 
terribly significant time to the install).

historically-

"
echo "$(( $(/sbin/dumpe2fs -h /dev/mapper/live-osimg-min 2>/dev/null | 
grep "^Block size:" | sed -e 's/.*:\s*//') * $(/sbin/dumpe2fs -h 
/dev/mapper/live-osimg-min 2>/dev/null | grep "^Block count:" | sed -e 
's/.*:\s*//') / 512 ))" > /dev/shm/osmin.size
"


> and if we could reduce the size of the COW area of the snapshot to its
> minimal (i.e. look at how much is used and truncate it at that point),

This is pretty easy.  In the intervening time, I am now confidently 
aware that I can use dmsetup status on the snapshotted device to 
determine how much of the file can be truncated.

> then we can just include osmin.img directly and cut out a lot of the
> tricks.

It needs to be compressed, which is why the truncation didn't really 
matter.  The reason it needs to be compressed, is because it can't live 
in the squashfs, it has to live on the iso9660.  And note, that it 
compresses from 1.2M of actual data, to 7kb of actual data.

I'm all for cutting out the tricks, if it is possible.

Can you see an alternate way, with fewer tricks, to get osmin.img to be 
visible to anaconda?

You could try to force the mounting of /dev/live onto /mnt/live, but 
that breaks the copy-to-ram case, because in that case, the cdrom 
doesn't exist.  If breaking copy-to-ram is ok, and you prefer grabbing 
osmin.img from a mounted /dev/live, I can do that.

Honestly I thought that using the exact same mechanism to propogate 
osmin.img as was used to propogate os.img, was not a trick, but elegant.

> 
> 	Also, the snapshot device created by liveinst.sh should be readonly
> (dmsetup -r, I think).

Sounds good, and the loop devices for the components while we're at it.

> 
> 	I'd also probably move the snapshot creation code out of liveinst.sh
> into anaconda itself.

I'm not so sure about this.  liveinst.sh seemed appropriate, in that it 
was where the installable image was defined to anaconda.  This also 
depends on any suggestions you might have about alternate ways of 
exposing osmin.img, so I'll wait to hear what you think about that.

Thanks again for taking a look at this.  I'll be happy to do whatever 
you suggest to get it merge-worthy, but lets at least have one back and 
forth to make sure you understand why I made the design choices I did.

Regards,

-dmc


> 
> 	I'd tend towards not having --cleanup-deleted and --turbo-liveinst
> options and just make it the default.
> 
> 	Finally, perhaps explain this feature as:
> 
>   "Reduce installation time by not copying unused data to disk"
> 
> 	And if you want to explain how it actually works:
> 
>   "We avoid copying unused data by copying a filesystem image that has 
>    been reduced to the minimal possible size. This minimal image is not 
>    sufficient for a running LiveCD as applications need room to write 
>    more data, but the minimal image is efficiently created just before 
>    copying by applying a pre-calculated set of deltas to the original 
>    large filesystem image."
> 
> Cheers,
> Mark.
> 
> [1] - Just change getLiveSizeMb() to always "dumpe2fs -h" to get the
> filesystem size
> 
> --
> Fedora-livecd-list mailing list
> Fedora-livecd-list at redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-livecd-list