[Fedora-livecd-list] [PATCH] cleanupDeleted and container_size - saves 5.5% output size on f7livecd-i686

[Jeremy Katz]

On Fri, 2007-08-17 at 16:02 -0500, Douglas McClendon wrote:
> Phillip Lougher wrote:
> > Douglas McClendon-2 wrote:
> >> Douglas McClendon wrote:
> > 
> > Some stats based on the Fedora-8-Test-1-Live-i686.iso 
> > 
> > original squashfs.img 712495104 (681M)
> > sparse squashfs.img 709820416 (678M)
> > sparse squashfs.img with 128Kbyte blocks 700100608 (669M)
> > 
> > So, the compressed 0s used to take up about 2.5 Mbytes.  More significant is
> > the data and seek time saving in not reading all these useless compressed
> > 0s.
> > 
> > time to do "dd if=os.img of=/dev/null" from CDROM
> > 
> > original squashfs.img, 221 seconds
> > sparse squashfs.img, 168 seconds
> > sparse squashfs.img with 128byte blocks, 171 seconds
> > 
> > on average about 20% speedup.
> 
> 20% speedup of reading data into /dev/null isn't all that useful.
> 
> You didn't really mention what you were planning on using it for. 
> Presumably you were also aware of my turboLiveInst patch, which 
> accomplishes the 20% speedup in an actual copy situation.
> 
> The sparse support could be used, perhaps with more code being written 
> to support sparse copying to block devices with python, to gain the 
> performance enhancements of turboLiveInst in an alternate manner.

I suspect that it can...

> For reasons that I can only speculate to, Jeremy, and everyone else 
> seems to have no interest in my turboLiveInst optimization approach. 
> Perhaps this method will be more palatable for them for some reason.

It's a bit more palatable because it's not playing tricks with
device-mapper and instead makes things know about sparse files and deal
with them as sparse files.  Which then makes the solution more general
and less dependent on knowing very very intimate details about how
things are constructed.

> This actually reopens the usefulness of the container-size option which 
> I had included with the first version of the cleanupDeleted patch.  I.e. 
> now it is no penalty whatsoever to go with a much larger container size. 
>   This will remove the need to perform ugly device-mapper-zero tricks to 
> expand the rootfs size at runtime, if you have enough memory in your 
> overlay device to support that.  And assuming of course that either you 
> are using the turboLiveInst patch, or the alternate method described 
> above which requires a little more code to be written.

The big question is what is trying to be enabled by it and what the user
experience of taking advantage of it is.  If it's something that you end
up passing lots of kernel command line options for, then I don't know if
the added complexity is worth it.  If it's something that can be a
benefit for all cases without requiring a lot of knowledge from the
user, then maybe it is more useful.

Jeremy