[Fedora-livecd-list] Re: Review? LiveUSB persistence

[Douglas McClendon]

Jeremy Katz wrote:
> Moving to the more appropriate list, original is at
> https://www.redhat.com/archives/fedora-devel-list/2008-January/msg03188.html for those not on fedora-devel
> 
> On Thu, 2008-01-31 at 16:11 -0600, Douglas McClendon wrote:
>> I keep hearing these crazy rumors that RedHat actually wants to subject 
>> actual users to my LiveUSB persistence feature next/this month.
> 
> Lots of people want lots of different things.  I've definitely left this
> sitting hanging too long and I apologize.  I just didn't have time to
> poke over the holidays and my January was a bitty nutty.  But I should
> be back to just my normal mail lag now ;-)
> 
>> I for one think the feature is really cool, useful, and know of no 
>> showstopping or even known bugs.  But I also would not subject actual 
>> users to it, without having had more people review it and sign off on 
>> it.  Anyway, to absolve myself of any responsibility, I once again 
>> request code review and feedback-
> 
> Overall, I think that it looks good.  I think the big thing to work out
> is how best to integrate the initscripts changes.  Especially as we're
> in parallel switching things over to upstart for F9.  Just to make sure
> that I'm clear about what's there so that when we try to bribe Bill, we
> can do so effectively...
> * The unmount loops in /etc/init.d/functions are modified so that we
> don't unmount the underlying fs of the overlay

Correct.  And the method currently in use is adding any subcomponents of 
the rootfs to /dev/.fstab.live.special.  I chose that location just 
because it is available from both the initramfs and then later during 
shutdown.

> * We don't want to sync the hwclock on shutdown of a live image
> * Do an overlay teardown on halt to replace the rw snapshot with a ro
> version

Yup, basically the old phase of 'remount rootfs readonly' just gets 
quite a bit more complicated, but still essentially just that.

> The latter feels like we're going to want a generic hook to be able to
> run, perhaps from back in the initramfs.  The middle feels largely
> unrelated to persistence in general, but just a "needs fixing" which
> will fit in nicely with other hwclock changes that are underway in
> rawhide.  And the first I'm less clear on I guess.
> 
> Outside of those changes, everything looks pretty straight-forward and
> reasonable.  We'll want a little more error checking in
> livecd-iso-to-disk and it might also be nice to have another tool that
> lets you create your usb stick to use with the actual cd form (I did
> this by hand and it was pretty nice that I could easily get it
> working!).  As for the findoverlay bits, for right now, leaving them as
> a separate script is fine.  In the longer term, we're hopefully going to
> kill mayflower and be able to use more "normal" initrds[1], at which
> point pulling it in more directly would be nice.

I have no preference where the code lives.  The main thing I would 
highlight is how originally I wrote findoverlay, with the mindset that 
the overlay file/layer could be located anywhere.  But then, to focus on 
getting something really usable, I honed in on the assumption of the 
overlay file living together with the LiveOS on a LiveUSB.  As a result 
the findoverlay script definitely does not look like entirely grade A 
code right now.  But I have no objection to it getting merged in any 
form, and perhaps later having cleanup/refactoring passes, perhaps with 
even later passes that add back the functionality I originally had in mind.

> As far as reliability... shutting down cleanly, I can't cause a problem.
> I pulled the usb stick from a box and got some garbage in a file I had
> written.  I want to do some more playing here, but I also wanted to get
> this mail out today.  Worst case, if we say "you need to shut down
> cleanly", then so be it.  It might be nice if we flag clean vs not
> shutdown so that we can force a fsck in unclean cases.  But no clue if
> that's even feasible; I'll take a look, though.

My vague understanding right now would that it would be treated just 
like a normal rootfs.  I.e. the same things happening if it is flagged 
as dirty, as if a normal system.  But...  I guess then there is the 
issue of fscking the host filesystem, e.g. vfat(ext3?) of the LiveUSB.  Ugh.

One immediate enhancement I thought of, is that perhaps the interface 
for initialization can be taken out of livecd-iso-to-disk, and put into 
the initramfs.  I.e. a new kernel cmdline parameter of

"initoverlay=256MB"

would just invoke the dd if=/dev/zero... during that boot.  Maybe even a 
default value that just uses half the free space on the LiveUSB.

Thus perhaps livecd-iso-to-disk just by default adds another couple menu 
entries to syslinux, one for 'boot with persistence' and one for 
'initialize persistence'.  Or perhaps the former automatically 
initializes if none is found, and the latter, or some post-boot system 
command could 'reinitialize persistence'...

-dmc