6:57 p.m.
Jeremy Katz wrote:
On Mon, 2007-07-23 at 03:06 -0500, Douglas McClendon wrote:
> They implement what I have described rather verbosely on fedora-livecd-list and
> in bug 248082.
>
> The short story is the fedora7 livecd installer works by copying a 4G ext3 image
> to the destination rootfs, and then resizing to maximal size.
>
> The attached patches improve the speed of this step by 10-30% (for cdrom vs fast
> livecd-iso-to-disk'd usbflash install media respectively).
>
> This is accomplished because the 4G image actually only holds 2G of data. The
> patch to livecd-creator, when invoked with --turbo-liveinst, will create a small
> (25kb) binary delta file on the livecd. The patch to anaconda, will detect the
> presence of the file, and if it is there, use it with devicemapper to create a
> 2G ext3 image, which can naturally be copied to the destination volume more
> quickly. (I.e. 2G of zeros don't get written to disk).
So I'm still not convinced that the wins here are really substantial
enough given the additional contortions that we have to go through to
get things going.
I understand your attitude. I will however do another round of polishing and
performance testing in an attempt to convince you. Ultimately, I hope to at
least convince you to do your own side-by-side taste test and confirm my
results. :)
Or that even if the win is worthwhile, that it's
really the best path to take. Continuing to layer more levels of
device-mapper indirection just leads to hitting ... interesting cases
within the kernel as well as making it a lot harder to tell what's going
on.
For the sake of argument (i.e. I'm not fanatically attached to this patch), I
would say that the interesting kernel cases, are probably useful ones to get the
bugs out of sooner rather than later. As for making it harder to tell whats
going on, I think after another round of polishing, it should actually not be
terribly complicated. Actually, after you see some of the patches I'll be
submitting in the coming months with other device mapper tricks, you might look
fondly on the days when you thought this was too much complexity :) :) :)
For avoiding the writes and speeding things up, could we instead not
be
smarter in the copying we use to actually take the sparseness into
account?
Definitely a valid and interesting option. In the same vein, as I alluded to
elsewhere, I think it might be interesting to see if squashfs could be made to
support sparse files. I _think_ it currently knows nothing of sparseness. And
the way I think it breaks up things into chunks, I suspect that the amount of
space the 2G of sparseness in os.img, takes up in the squashed image, is
significant. (I may test that one of these days and post my findings).
Alas, my specialty is in throwing together commandline tools in devious ways. I
could probably become knowledgable about ext3 and squashfs internals, but it's
not really what I want to do in the near future. The fact that I can work
around the absence of cool fs features with device mapper tricks... I just find
very amusing, in a brain twisting sort of way.
As for being able to get to a smaller rootfs, I think that if you really
want to get to requiring a small root, then we have to entirely change
the copying to be more of a "copy the bits from the filesystem" as
opposed to a block-level copy. That would also avoid the duplicated
copies if you have, eg, a separate /usr (and also avoid the overhead of
needing that space on the rootfs). This would also end up avoiding the
copying overhead as well as the filesystem resizing, although you'd then
have to format the entire fs. I think I'd be slightly more interested
in doing that, although care has to be taken to ensure that we really
are preserving everything then.
I agree with all of this. Still, I go back to the existing patch, which
hopefully should be a bit more elegant and polished soon, and suggest that it is
at least worth seriously considering for F8.
> This patch is less than polished and elegant. But I'm hoping it might make it
> into f8t1. Please review, and give suggestions for improvements. I'll plan on
> trying to clean it up in whatever ways seem best over the next couple of days.
I think trying to do this path for test1 is a little optimistic,
although now we've got a good start to be able to be doing some
experimentation immediately post-test1.
agreed. definitely not enough burn in testing for test1. But as with
persistence, if the test1 devfreeze contributed to my motivation to get
something out the door, all the better.
> (note, the cleanupDeleted patch I just posted to livecd-tools is included here,
> as it lays the foundation for this... as well as reclaiming lots (5-15+%) of
> wasted space on the livecd)
For future purposes, note that it's a lot easier if you don't do this.
I mean, interdiff works but it just makes it harder to do a direct
review. Saying that it depends on an earlier patch is simple to deal
with.
Yeah, it was just laziness on my part. Not entirely related, but I still need
to learn more about git, and where the heck 'diff --git' comes from.
-dmc