[Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer

[Douglas McClendon]

Tim Wood wrote:
> Before shooting from the hip anymore, I'll dig into my RHCT/CLP stuff and
> see what I dig out.  Homework... :-(

Just because I like to talk/type a lot...  I'll go ahead and instead of focusing 
on correcting you, try to actually answer your question.  Though first I'll say 
that my response about hibernated systems was a shot from the hip which is 
technically wrong and more likely to confuse than clarify.

Here is an attempt to outline what is going on with the rebootless installer 
idea, versus the fedora-7 official livecd, versus a normal installed fedora system.

Normal Fedora 7 Installed On Hard Disk System
(assume just 1 non lvm partition /dev/sda1)
----------------------------------------------
-The bios loads the grub boot loader from the MBR of /dev/sda
-grub knows how to read it's config from /dev/sda1:/boot/grub/grub.conf
-grub is configured to boot a specific kernel+ramdisk+appendstring, namely
  /dev/sda1:/boot/vmlinuz-someversion, /dev/sda1:/boot/initrd-someversion, 
someargs...
-control is thusly passed to the kernel, and the kernel then gunzips and 
extracts the cpio of the specified initrd (which I think grub copied to a well 
known place in ram.  Only reason I might know this is because qemu's crafty 
-initrd feature screwed it up for larger initrds recently, though it has been fixed)
-the kernel having then extracted the contents into a ram based filesystem, 
passes control to /init (or maybe /sbin/init, or maybe whatever init= was 
specified on the cmdline).
-now the fun starts.  This init is a nash or a bash script, whose job it is to 
mount the real root filesystem (e.g. the ext3fs on /dev/sda1) and then 
pivot_root to it.
-finally, control is passed to /sbin/init on the real disk-based root 
filesystem, at which point the contents of the well known /etc/inittab start to 
matter.

Now then, what the Fedora-7 livecd does is along these lines
------------------------------------------------------------
-instead of the bios booting grub loaded on the mbr of a disk, it boots grub(or 
perhaps isolinux) from the bootsector of the cdrom.
-this bootloader behaves much like above, but pulling a 
kernel+initrd+append_args from some place on the cdrom.
-now the fun begins after the initrd is extracted and mounted in a ram based fs 
as normal.  An entirely different /init script within the initrd will go about 
the business of mounting the 'real root filesystem'.  In this case, first the 
cdrom's iso9660 fs is mounted.  Then a squashfs image file is loopback mounted 
from within the iso9660 fs.  Then a sparse ext3fs image file is loopback mounted 
from within the squashfs.
- now the REAL fun begins.  A ram based filesystem is created.  A sparse file 
overlay is created within it.  Now a device mapper snapshot is created using the 
read-only ext3fs image, and the read-write overlay file.  (I'm skipping some 
loopback device associations, and in general probably misnaming a few things, as 
this is unashamedly from the hip, and not suitable to be published).  Now, this 
magic devicemapper snapshotted device appears as /dev/mapper/live-rw, and 
appears to be a read-write ext3 filesystem, except the writes really get tucked 
away in ram (which is going to eat away at your ram).
- That /dev/mapper/live-rw gets mounted as the 'real root filesystem', and 
pivot_root is called, and then things progress as normal.

Now then- what the rebootless installer patch does differently
--------------------------------------------------------------
-Just after the /dev/mapper/live-rw gets set up in the initrd, instead of 
mounting it as the real root filesystem, it gets used to create a raid1 
'mirror'.  quotes because in this case the 'mirror' only has 1 device, rather 
than the usual 2.  The mirror is visible as /dev/md7, and THAT gets mounted as 
the real root filesystem, before pivot_root is called, and everything progresses 
as normal.

But because of /dev/md7 being the 'real root filesystem', long after boot, you 
can hot-add another device to the mirror, in this case, the target volume that 
you want to install the system on (e.g. /dev/sda1).  After you hot-add, the 
raid/md driver starts synchronizing the data from /dev/mapper/live-rw to 
/dev/sda1.  When this finishes, you can hot-remove /dev/mapper/live-rw, at which 
point the system is running from /dev/sda1, just as if you had installed there 
and rebooted (with the caviat that there is this /dev/md7 layer sitting there 
until the next reboot).  And once /dev/mapper/live-rw is removed from the 
/dev/md7 array, the resources that constructed it (i.e. the files on the cdrom, 
and that overlay file in a ram-based fs) can be released/deleted/unmounted. 
Thus you stop suffering the penalty of that overlay eating up your ram, and you 
are free to eject the cdrom.

Now, for the sake of simplicity, we will assume that "mdadm /dev/md7 --grow 
--size=max" will actually work, and the 3.5G ext3fs that got migrated to your 
100G /dev/sda1 partition, can be grown with resize2fs's ability to online expand 
a live ext3 filesystem.  (technically this does not yet work, so refer to all 
those nasty workarounds).

Ideally, in the coming days/weeks/months/years I'll be able to publish far 
better patches that far more easily demonstrate the advantage of what is going 
on here (though the basic concept is simple- no reboot after install).

Maybe if people actually start using and liking this feature, I'll turn all of 
this into a nice whitepaper.

-dmc