[PATCH 0/2] Add directory install support to Anaconda

[Brian C. Lane]

From: "Brian C. Lane"

This is the result of an idea I had at fudcon. It turns off storage and
installs to /mnt/sysimage/ or whatever you have previously mounted on it.

I am using this with a modified version of livemedia-creator to make live isos
and various disk images. It speeds up the creation time for an iso by 'a bunch'
because it removes the need for copying the whole filesystem from one place to
another during the install sequence. It should also allow anaconda and
livemedia-creator to be run inside a mock environment because it removes the
need for device-mapper which has caused problems in the past inside a mock.

My plan for these patches is to apply them to both master+blivet and to
f18-branch and to do a new build of f18 (18.37.12) so that people using
livemedia-creator with f18 can take advantage of this without waiting for f19.

Brian C. Lane (2):
  Add mountOnly to turnOnFilesystems
  Add --dirinstall command

anaconda                            | 31 ++++++++++------
 pyanaconda/flags.py                 |  1 +
 pyanaconda/install.py               | 10 +++---
 pyanaconda/storage/__init__.py      | 70 +++++++++++++++++++++----------------
 pyanaconda/ui/gui/spokes/storage.py |  4 +++
 pyanaconda/ui/tui/spokes/storage.py |  4 +++
 6 files changed, 75 insertions(+), 45 deletions(-)

[f18-branch/blivet 1/2] Add mountOnly to turnOnFilesystems

[Brian C. Lane]

From: "Brian C. Lane"

This allows us to mount filesystems like dev, proc, ... to
/mnt/sysimage without activating the rest of storage.

NOTE: yes, update is unneeded, but I wanted to keep the changes to a minimum.
---
 pyanaconda/storage/__init__.py | 70 +++++++++++++++++++++++-------------------
 1 file changed, 39 insertions(+), 31 deletions(-)

diff --git a/pyanaconda/storage/__init__.py b/pyanaconda/storage/__init__.py
index e537500..40644db 100644
--- a/pyanaconda/storage/__init__.py
+++ b/pyanaconda/storage/__init__.py
@@ -153,45 +153,53 @@ def storageInitialize(storage, ksdata, protected):
                                          if d.name not in ksdata.ignoredisk.ignoredisk]
             log.debug("onlyuse is now: %s" % (",".join(ksdata.ignoredisk.onlyuse)))

-def turnOnFilesystems(storage):
-    upgrade = "preupgrade" in flags.cmdline
-
-    if not upgrade:
-        if (flags.livecdInstall and not flags.imageInstall and not storage.fsset.active):
-            # turn off any swaps that we didn't turn on
-            # needed for live installs
-            iutil.execWithRedirect("swapoff", ["-a"],
-                                   stdout = "/dev/tty5", stderr="/dev/tty5")
-        storage.devicetree.teardownAll()
-
-    upgrade_migrate = False
-    if upgrade:
-        for d in storage.migratableDevices:
-            if d.format.migrate:
-                upgrade_migrate = True
+def turnOnFilesystems(storage, mountOnly=False):
+    """ storage = storage object
+        mountOnly = only mount the filesystems, don't execute storage actions
+    """
+    if not mountOnly:
+        upgrade = "preupgrade" in flags.cmdline
+
+        if not upgrade:
+            if (flags.livecdInstall and not flags.imageInstall and not storage.fsset.active):
+                # turn off any swaps that we didn't turn on
+                # needed for live installs
+                iutil.execWithRedirect("swapoff", ["-a"],
+                                       stdout = "/dev/tty5", stderr="/dev/tty5")
+            storage.devicetree.teardownAll()
+
+        upgrade_migrate = False
+        if upgrade:
+            for d in storage.migratableDevices:
+                if d.format.migrate:
+                    upgrade_migrate = True

-    try:
-        storage.doIt()
-    except FSResizeError as e:
-        if os.path.exists("/tmp/resize.out"):
-            details = open("/tmp/resize.out", "r").read()
-        else:
-            details = e.args[1]
+        try:
+            storage.doIt()
+        except FSResizeError as e:
+            if os.path.exists("/tmp/resize.out"):
+                details = open("/tmp/resize.out", "r").read()
+            else:
+                details = e.args[1]

-        if errorHandler.cb(e, e.args[0], details=details) == ERROR_RAISE:
-            raise
-    except FSMigrateError as e:
-        if errorHandler.cb(e, e.args[0], e.args[1]) == ERROR_RAISE:
+            if errorHandler.cb(e, e.args[0], details=details) == ERROR_RAISE:
+                raise
+        except FSMigrateError as e:
+            if errorHandler.cb(e, e.args[0], e.args[1]) == ERROR_RAISE:
+                raise
+        except Exception as e:
             raise
-    except Exception as e:
-        raise

-    storage.turnOnSwap()
+        # TODO: Should we turn on swap?
+        storage.turnOnSwap()
+
     # FIXME:  For livecd, skipRoot needs to be True.
     storage.mountFilesystems(raiseErrors=False,
                              readOnly=False,
                              skipRoot=False)
-    writeEscrowPackets(storage)
+
+    if not mountOnly:
+        writeEscrowPackets(storage)

def writeEscrowPackets(storage):
     escrowDevices = filter(lambda d: d.format.type == "luks" and \

[PATCH 2/2] Add --dirinstall command

[Brian C. Lane]

From: "Brian C. Lane"

This disables the storage spoke and installs to /mnt/sysimage without
mounting any filesystems. If something is already mounted on
/mnt/sysimage it leaves it untouched.

This can be useful for utilities like livemedia-creator which will mount
a filesystem image on the directory and then run anaconda to install to
it.
---
 anaconda                            | 31 +++++++++++++++++++++----------
 pyanaconda/flags.py                 |  1 +
 pyanaconda/install.py               | 10 ++++++----
 pyanaconda/ui/gui/spokes/storage.py |  4 ++++
 pyanaconda/ui/tui/spokes/storage.py |  4 ++++
 5 files changed, 36 insertions(+), 14 deletions(-)

diff --git a/anaconda b/anaconda
index 0789094..8e32b87 100755
--- a/anaconda
+++ b/anaconda
@@ -50,7 +50,7 @@ def exitHandler(rebootData, storage, exitCode=None):
         while True:
             time.sleep(10000)

-    if image_count:
+    if image_count or flags.dirInstall:
         anaconda.storage.umountFilesystems(ignoreErrors=True, swapoff=False)
         devicetree = anaconda.storage.devicetree
         devicetree.teardownAll()
@@ -60,7 +60,8 @@ def exitHandler(rebootData, storage, exitCode=None):
                 loop.controllable = True
             device.deactivate(recursive=True)

-    if not flags.imageInstall and not flags.livecdInstall:
+    if not flags.imageInstall and not flags.livecdInstall \
+       and not flags.dirInstall:
         from pykickstart.constants import KS_SHUTDOWN, KS_WAIT, KS_REBOOT
         from pyanaconda.iutil import dracut_eject

@@ -281,6 +282,7 @@ def parseOptions(argv=None, cmdline=None):
     op.add_option("--updates", dest="updateSrc", action="store", type="string")
     op.add_option("--dlabel", action="store_true", default=False)
     op.add_option("--image", action="append", dest="images", default=[])
+    op.add_option("--dirinstall", action="store_true", default=False)
     op.add_option("--memcheck", action="store_true", default=True)
     op.add_option("--nomemcheck", action="store_false", dest="memcheck")
     op.add_option("--leavebootorder", action="store_true", default=False)
@@ -645,8 +647,6 @@ if __name__ == "__main__":
     # do this early so we can set flags before initializing logging
     from pyanaconda.flags import flags
     (opts, args, depr) = parseOptions(cmdline=flags.cmdline)
-    if opts.images:
-        flags.imageInstall = True

# Set up logging as early as possible.
     import logging
@@ -661,6 +661,14 @@ if __name__ == "__main__":
         stdoutLog.error("anaconda must be run as root.")
         sys.exit(0)

+    if opts.images and opts.dirinstall:
+        stdoutLog.error("--images and --dirinstall cannot be used at the same time")
+        sys.exit(0)
+    elif opts.images:
+        flags.imageInstall = True
+    elif opts.dirinstall:
+        flags.dirInstall = True
+
     # see if we're on s390x and if we've got an ssh connection
     uname = os.uname()
     if uname[4] == 's390x':
@@ -783,7 +791,8 @@ if __name__ == "__main__":
     flags.mpath = opts.mpath
     flags.selinux = opts.selinux

-    if not flags.livecdInstall and not flags.imageInstall:
+    if not flags.livecdInstall and not flags.imageInstall \
+      and not flags.dirInstall:
         startAuditDaemon()

# setup links required for all install types
@@ -972,18 +981,20 @@ if __name__ == "__main__":
         from pyanaconda.rescue import doRescue
         doRescue(anaconda.rescue_mount, ksdata, anaconda.platform)

-    threadMgr.add(AnacondaThread(name="AnaStorageThread", target=storageInitialize, args=(anaconda.storage, ksdata, anaconda.protected)))
+    if not flags.dirInstall:
+        threadMgr.add(AnacondaThread(name="AnaStorageThread", target=storageInitialize, args=(anaconda.storage, ksdata, anaconda.protected)))
     threadMgr.add(AnacondaThread(name="AnaNetworkThread", target=networkInitialize, args=(ksdata,)))
     threadMgr.add(AnacondaThread(name="AnaPayloadThread", target=payloadInitialize, args=(anaconda.storage, ksdata, anaconda.payload)))

atexit.register(exitHandler, ksdata.reboot, anaconda.storage)

# setup ntp servers and start NTP daemon if not requested otherwise
-    if (not flags.imageInstall) and anaconda.ksdata.timezone.ntpservers:
-        ntp.save_servers_to_config(anaconda.ksdata.timezone.ntpservers)
+    if not flags.imageInstall and not flags.dirInstall:
+        if anaconda.ksdata.timezone.ntpservers:
+            ntp.save_servers_to_config(anaconda.ksdata.timezone.ntpservers)

-    if not anaconda.ksdata.timezone.nontp:
-        iutil.start_service("chronyd")
+        if not anaconda.ksdata.timezone.nontp:
+            iutil.start_service("chronyd")

# FIXME:  This will need to be made cleaner once this file starts to take
     # shape with the new UI code.
diff --git a/pyanaconda/flags.py b/pyanaconda/flags.py
index fdbd504..8d24d13 100644
--- a/pyanaconda/flags.py
+++ b/pyanaconda/flags.py
@@ -67,6 +67,7 @@ class Flags(object):
         self.noverifyssl = False
         self.imageInstall = False
         self.automatedInstall = False
+        self.dirInstall = False
         # for non-physical consoles like some ppc and sgi altix,
         # we need to preserve the console device and not try to
         # do things like bogl on them.  this preserves what that
diff --git a/pyanaconda/install.py b/pyanaconda/install.py
index 4ed7cef..e8fb6a8 100644
--- a/pyanaconda/install.py
+++ b/pyanaconda/install.py
@@ -111,8 +111,9 @@ def doInstall(storage, payload, ksdata, instClass):

# Do partitioning.
     payload.preStorage()
-    turnOnFilesystems(storage)
-    if not flags.flags.livecdInstall:
+
+    turnOnFilesystems(storage, mountOnly=flags.flags.dirInstall)
+    if not flags.flags.livecdInstall and not flags.flags.dirInstall:
         storage.write()

# Do packaging.
@@ -131,7 +132,8 @@ def doInstall(storage, payload, ksdata, instClass):
         payload.postInstall()

# Do bootloader.
-    with progress_report(_("Installing bootloader")):
-        writeBootLoader(storage, payload, instClass, ksdata)
+    if not flags.flags.dirInstall:
+        with progress_report(_("Installing bootloader")):
+            writeBootLoader(storage, payload, instClass, ksdata)

progress.send_complete()
diff --git a/pyanaconda/ui/gui/spokes/storage.py b/pyanaconda/ui/gui/spokes/storage.py
index 1ac10f7..ea23432 100644
--- a/pyanaconda/ui/gui/spokes/storage.py
+++ b/pyanaconda/ui/gui/spokes/storage.py
@@ -434,6 +434,10 @@ class StorageSpoke(NormalSpoke, StorageChecker):
         return self._ready

@property
+    def showable(self):
+        return not flags.dirInstall
+
+    @property
     def status(self):
         """ A short string describing the current status of storage setup. """
         msg = _("No disks selected")
diff --git a/pyanaconda/ui/tui/spokes/storage.py b/pyanaconda/ui/tui/spokes/storage.py
index 0032f2a..e2d1974 100644
--- a/pyanaconda/ui/tui/spokes/storage.py
+++ b/pyanaconda/ui/tui/spokes/storage.py
@@ -115,6 +115,10 @@ class StorageSpoke(NormalTUISpoke):
         return self._ready and not threadMgr.get("AnaStorageWatcher")

@property
+    def showable(self):
+        return not flags.dirInstall
+
+    @property
     def status(self):
         """ A short string describing the current status of storage setup. """
         msg = _("No disks selected")

[PATCH 2/2] Add --dirinstall command

[Vratislav Podzimek]

On Wed, 2013-02-06 at 16:26 -0800, Brian C. Lane wrote:
> From: "Brian C. Lane"
>
> This disables the storage spoke and installs to /mnt/sysimage without
> mounting any filesystems. If something is already mounted on
> /mnt/sysimage it leaves it untouched.
>
> This can be useful for utilities like livemedia-creator which will mount
> a filesystem image on the directory and then run anaconda to install to
> it.
>
> anaconda                            | 31 +++++++++++++++++++++----------
>  pyanaconda/flags.py                 |  1 +
>  pyanaconda/install.py               | 10 ++++++----
>  pyanaconda/ui/gui/spokes/storage.py |  4 ++++
>  pyanaconda/ui/tui/spokes/storage.py |  4 ++++
>  5 files changed, 36 insertions(+), 14 deletions(-)
>
> diff --git a/anaconda b/anaconda
> index 0789094..8e32b87 100755
> --- a/anaconda
> +++ b/anaconda
> @@ -50,7 +50,7 @@ def exitHandler(rebootData, storage, exitCode=None):
>          while True:
>              time.sleep(10000)
> if image_count:
> if image_count or flags.dirInstall:
>     anaconda.storage.umountFilesystems(ignoreErrors=True, swapoff=False)
>     devicetree = anaconda.storage.devicetree
>     devicetree.teardownAll()
>
> @@ -60,7 +60,8 @@ def exitHandler(rebootData, storage, exitCode=None):
>                  loop.controllable = True
>              device.deactivate(recursive=True)
> if not flags.imageInstall and not flags.livecdInstall:
> if not flags.imageInstall and not flags.livecdInstall \
>   and not flags.dirInstall:
>    from pykickstart.constants import KS_SHUTDOWN, KS_WAIT, KS_REBOOT
>    from pyanaconda.iutil import dracut_eject
>
> @@ -281,6 +282,7 @@ def parseOptions(argv=None, cmdline=None):
>      op.add_option("--updates", dest="updateSrc", action="store", type="string")
>      op.add_option("--dlabel", action="store_true", default=False)
>      op.add_option("--image", action="append", dest="images", default=[])
> op.add_option("--dirinstall", action="store_true", default=False)
> op.add_option("--memcheck", action="store_true", default=True)
> op.add_option("--nomemcheck", action="store_false", dest="memcheck")
> op.add_option("--leavebootorder", action="store_true", default=False)
>
> @@ -645,8 +647,6 @@ if __name__ == "__main__":
>      # do this early so we can set flags before initializing logging
>      from pyanaconda.flags import flags
>      (opts, args, depr) = parseOptions(cmdline=flags.cmdline)
>
> if opts.images:
>    flags.imageInstall = True
>
> # Set up logging as early as possible.
>    import logging
>
> @@ -661,6 +661,14 @@ if __name__ == "__main__":
>          stdoutLog.error("anaconda must be run as root.")
>          sys.exit(0)
> if opts.images and opts.dirinstall:
>    stdoutLog.error("--images and --dirinstall cannot be used at the same time")
>    sys.exit(0)
> elif opts.images:
>    flags.imageInstall = True
> elif opts.dirinstall:
>    flags.dirInstall = True
> # see if we're on s390x and if we've got an ssh connection
>    uname = os.uname()
>    if uname[4] == 's390x':
>
> @@ -783,7 +791,8 @@ if __name__ == "__main__":
>      flags.mpath = opts.mpath
>      flags.selinux = opts.selinux
> if not flags.livecdInstall and not flags.imageInstall:
>
> if not flags.livecdInstall and not flags.imageInstall \
>  and not flags.dirInstall:
>    startAuditDaemon()
>
> # setup links required for all install types
>
> @@ -972,18 +981,20 @@ if __name__ == "__main__":
>          from pyanaconda.rescue import doRescue
>          doRescue(anaconda.rescue_mount, ksdata, anaconda.platform)
> threadMgr.add(AnacondaThread(name="AnaStorageThread", target=storageInitialize, args=(anaconda.storage, ksdata, anaconda.protected)))
>
> if not flags.dirInstall:
>    threadMgr.add(AnacondaThread(name="AnaStorageThread", target=storageInitialize, args=(anaconda.storage, ksdata, anaconda.protected)))
>
> threadMgr.add(AnacondaThread(name="AnaNetworkThread", target=networkInitialize, args=(ksdata,)))
>    threadMgr.add(AnacondaThread(name="AnaPayloadThread", target=payloadInitialize, args=(anaconda.storage, ksdata, anaconda.payload)))
>
> atexit.register(exitHandler, ksdata.reboot, anaconda.storage)
>
> # setup ntp servers and start NTP daemon if not requested otherwise
> if (not flags.imageInstall) and anaconda.ksdata.timezone.ntpservers:
>    ntp.save_servers_to_config(anaconda.ksdata.timezone.ntpservers)
> if not flags.imageInstall and not flags.dirInstall:
>    if anaconda.ksdata.timezone.ntpservers:
>        ntp.save_servers_to_config(anaconda.ksdata.timezone.ntpservers)

I think using 'and ' is better than using two ifs as it saves
indentation (which is good especially in this case of a long line
indented).

Also I believe pyanaconda.flags.can_touch_runtime_system should be
patched.

[PATCH 2/2] Add --dirinstall command

[Brian C. Lane]

On Thu, Feb 07, 2013 at 11:42:01AM +0100, Vratislav Podzimek wrote:
> On Wed, 2013-02-06 at 16:26 -0800, Brian C. Lane wrote:
> > From: "Brian C. Lane" 
> >      # setup ntp servers and start NTP daemon if not requested otherwise
> > if (not flags.imageInstall) and anaconda.ksdata.timezone.ntpservers:
> >    ntp.save_servers_to_config(anaconda.ksdata.timezone.ntpservers)
> > if not flags.imageInstall and not flags.dirInstall:
> >    if anaconda.ksdata.timezone.ntpservers:
> >        ntp.save_servers_to_config(anaconda.ksdata.timezone.ntpservers)
>
> I think using 'and ' is better than using two ifs as it saves
> indentation (which is good especially in this case of a long line
> indented).

Actually, that was on purpose, it makes the following block easier to
read. note that the complete section now looks like:

if not flags.imageInstall and not flags.dirInstall:
         if anaconda.ksdata.timezone.ntpservers:
             ntp.save_servers_to_config(anaconda.ksdata.timezone.ntpservers)

if not anaconda.ksdata.timezone.nontp:
             iutil.start_service("chronyd")

It would be redundant to repeat the test for the chronyd check which is now included in the block.
> Also I believe pyanaconda.flags.can_touch_runtime_system should be
> patched.

I think you're right. For some reason I was thinking that would cover too much but I think it would be ok.
So I'd replace the above *Install checks with can_touch_runtime. I'll
send an intermediate patch that will get squashed into the final one.

[Vratislav Podzimek]

On Wed, 2013-02-06 at 16:26 -0800, Brian C. Lane wrote:
> From: "Brian C. Lane"
>
> This is the result of an idea I had at fudcon. It turns off storage and
> installs to /mnt/sysimage/ or whatever you have previously mounted on it.
>
> I am using this with a modified version of livemedia-creator to make live isos
> and various disk images. It speeds up the creation time for an iso by 'a bunch'
> because it removes the need for copying the whole filesystem from one place to
> another during the install sequence. It should also allow anaconda and
> livemedia-creator to be run inside a mock environment because it removes the
> need for device-mapper which has caused problems in the past inside a mock.
>
> My plan for these patches is to apply them to both master+blivet and to
> f18-branch and to do a new build of f18 (18.37.12) so that people using
> livemedia-creator with f18 can take advantage of this without waiting for f19.
>
> Brian C. Lane (2):
>   Add mountOnly to turnOnFilesystems
>   Add --dirinstall command
>
> anaconda                            | 31 ++++++++++------
>  pyanaconda/flags.py                 |  1 +
>  pyanaconda/install.py               | 10 +++---
>  pyanaconda/storage/__init__.py      | 70 +++++++++++++++++++++----------------
>  pyanaconda/ui/gui/spokes/storage.py |  4 +++
>  pyanaconda/ui/tui/spokes/storage.py |  4 +++
>  6 files changed, 75 insertions(+), 45 deletions(-)

Apart from the two comments for 2/2 these both look good to me and
surprisingly straightforward. Nice idea!