[Fedora-livecd-list] RFC- improving livecd installer efficiency

[Douglas McClendon]

Mark McLoughlin wrote:
> Hi,
> 	I've lost track of the all the details here but ...
> 
> On Tue, 2007-07-10 at 06:07 -0500, Douglas McClendon wrote:
>> Question:  Does the current livecd installer inefficiently write lots of 0's to 
>> the destination drive that it doesn't need to?
>>
>> I think it might.  The os.img on the F7 livecd is a 4G sparse file with about 
>> 2.3G of data.  Anaconda's livecdcopy backend uses python's os.read/write.  I 
>> would guess that that means that 4G of data is getting written, when 
>> theoretically only 2.3G needs to.
> 
> 	... how about something like this:
> 
>   http://www.gnome.org/~markmc/code/e2cp.c
> 
> 	i.e. read the ext3 metadata and copy everything but the  unallocated
> blocks.

I'm not as into low level filesystem internals as you are.  Tell me if this 
paraphrasing is accurate- e2cp is sort of like a dd for filesystems, that 
understands sparseness and/or unused/unallocated blocks, and handles them 
efficiently (ignores them).

The main downside I think it has as a solution to the issue, is that it doesn't 
fix the case of a destination volume of say 3.0G.  I.e. there is no reason why 
the livecd installer shouldn't be able to install it's 2.3G payload onto a 3.0G 
destination.  The solution I outlined covers that* case, while I don't think 
using e2cp does.

* _IF_ there aren't any flaws with anything I theorized

The most recent solution I was proposing, involved taking a second snapshot of 
the 4.0G (or perhaps now 1T) sparse ext3 os.img file, and resize2fs-ing it down 
to minimal, and then copying it.  The big question which I haven't tested yet, 
is whether or not such a resize2fs will happen quickly, with very few actual 
changes written to the image (i.e. the new overlay in ram).  If it does take 
less than a minute and use less than a 1MB of ram, it seems like a good solution 
to me.

Perhaps with your low level knowledge of ext2(/resize2fs?) you can answer that. 
  I.e. if you take an image, resize it to minimal, then resize it to 1TB, then 
how long and how many changes will it take to resize it back down to minimal? 
Of course I can just test it myself, and probably will soon enough.

-dmc



-dmc




> 
> 	(One thing from the above code you don't want to do is the
> check_all_zeros() bit - if an allocated block is all zeros, you *do*
> want to copy the zeros to the disk)
> 
> Cheers,
> Mark.
> 
> --
> Fedora-livecd-list mailing list
> Fedora-livecd-list at redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-livecd-list