[Fedora-livecd-list] PATCH: Disk (appliance) creator tool
Daniel P. Berrange
berrange at redhat.com
Sun Feb 17 02:14:43 UTC 2008
This patch adds support for another installation variant. The livecd-creator
produces ISO images which boot using syslinux. The image-creator creates a
single file containing a filesystem in which the OS is installated. This patch
adds a new disk-creator, which creates a single file containing a partitioned
disk with potentially many filesystems in which the OS is installed. Furthermore
it installs grub in the boot sector, so this image is immediately bootable in
a virtual machine.
The idea is that this serves as a tool for creating pre-installed appliances
suitable for distribution to end users, with no further installation steps
required, beyond potentially an appliance specific firstboot script.
I've used this tool to create an instance of the oVirt web management appliance
from its kickstart recipe. Currently developers building the appliance have to
boot a VM using the F8 boot.iso and run the kickstart script in the VM and so
on. While this works, involves many steps with potential for failure. This new
tool reduces the problem to simply
disk-creator --name ovirt-wui-demo.dsk ovirt-wui-appliance.ks
A short while later you end up with 'ovirt-wui-demo.dks' which you can boot
straight up in KVM / Xen / etc.
With the background information out of the way, here's some info about the
changes in this patch...
Most of the change here involves re-factoring the imgcreate/fs.py module.
The SparseExtLoopbackMount, SparseLoopbackMount, LoopbackMount classes all
have the built-in limitation that the image being produced corresponds to
a single filesystem / loop mount. With the disk creator, the image being
produced can be partitioned into multiple chunks, with many filesystems.
Furthermore, not all of them require loopback mounts, as the partitions
themselves are already visible via /dev/mapper/
So this patch separates the roles. There are now classes which deal with
accessing / creating disks:
Disk - generic base for disks
RawDisk - a disk backed by a block device
LoopbackDisk - a disk backed by a file
SparseLoopbackDisk - a disk backed by a sparse file
The 'create' method must make the disk visible as a block device - eg
by calling losetup. For RawDisk, this is obviously a no-op. The 'cleanup'
method must undo the 'create' operation.
There are then classes which deal with mounting things:
Mount - generic base for mounts
DiskMount - able to mount a Disk object
ExtDiskMount - able to format/resize ext3 filesystems when mounting
The livecd-creator/image-creator tools are updated to take account of these
API changes.
Next, up we have a new class 'PartitionedMount'. This takes a Disk object
and adds one or more partitions to it. It then creates further Disk and
Mount object instances for each partition. So, mounting a 'PartitionedMount'
object will in fact mount all its partitions (formatting the filesystems
or swap space as needed).
Currently only ext3/swap is supported for partition types. We use parted
to create partitions. This works well enough, but parted will spew lots
of scary looking warnings about being unable to tell the kernel to reload
its partition table. This is because loop devices don't support partition
tables. In this scenario though its not a problem because we then use the
kpartx tool to add entries to /dev/mapper for each partition.
When mounting partitions, PartitionedMount, figures out the correct order
based on the mount points, so it ensures the / is mounted before /boot.
The imgcreator/disk.py class contains the DiskImageCreator class which is
a subclass of ImageCreator. This uses the PartitionedMount object to
do an installation of the kickstart. It is fairly simple at the moment,
only being able to cope with 'part' entries in the kickstart which are
either ext3 or swap, and which have an explicit size given - ie --grow
is not supported. For testing I've been using
part /boot --fstype ext3 --size=100 --ondisk=sda
part swap --fstype swap --size=500 --ondisk=sda
part /var/lib/pgsql --fstype ext3 --size=300 --ondisk=sda
part / --fstype ext3 --size=3000 --ondisk=sda
part /home --fstype ext3 --size=500 --ondisk=sda
part /var --fstype ext3 --size=1000 --ondisk=sda
Which ensures it correctly deals with adding extended and logical partitions.
It doesn't currently care about --ondisk only outputting a single disk
image, but I intend to extend the CLI to allow creation of installs which
spawn multiple output disks.
Aside from partitioning, DiskImageCreator will setup an fstab file containing
entries for all the filesystems and swap space. It will add the grub device
map file, and create a grub.cfg based off the installed kernel. It is able
to cope with a layout where grub config is on /, or a separate /boot partition.
We can't use grub-install since it doesn't understand loopdevices, so to do
the MBR install we invoke grub manually.
If the disk image happens to be the same size or smaller than one of your
USB flash drives, you can also dd the entire image straight to the driver
with no modifications required. It should 'just boot' in any machine able
to boot off USB
Future enhancements for this:
- Autosize partitions when --grow is used
- Support LVM volumes
- Allow splitting across multiple disks (sda, sdb, etc)
- Deal with disk filesytem labels
- Deal with non-ext3 filesystems
- Install to pre-existing block device / LVM vol instead of file
- When staging final image, use qemu-img to generate qcow2, vmdk
or raw files at user's choice. qcow2 would give huge size savings
- Output XML file for use with 'virt-image' tool, so that the entire
build and deploy process consists of nothing more than:
# disk-creator ovirt-wui-appliance.ks
# virt-image ovirt-wui-appliance.xml
The last two options are probably my most immediate TODO items because this
will result in an excellant virtual appliance creation tool for Fedora
derived distros.
Perhaps it should be called appliance-creator instead of disk-creator ?
Final note - the patch contains a tonne of 'print' statements for debugging.
Debugging this stuff can be 'fun' when things go wrong, so I think it may
be useful if we make use of the python logging module throughout the livecd
tools. If folks agree, I'll switch the 'print' to logging.debug(), otherwise
I'll just remove them entirely before reposting this patch
The attached patch changes:
API | 4
Makefile | 3
imgcreate/__init__.py | 3
imgcreate/creator.py | 17 +
imgcreate/disk.py | 20 +-
imgcreate/fs.py | 425 ++++++++++++++++++++++++++++++++++++++-----------
imgcreate/kickstart.py | 3
imgcreate/live.py | 6
livecd-tools.spec | 1
9 files changed, 376 insertions(+), 106 deletions(-)
I'm also attaching the kickstart file I'm using as a demonstration...
Regards,
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
-------------- next part --------------
diff --git a/API b/API
index 9783c74..ef9c1aa 100644
--- a/API
+++ b/API
@@ -76,3 +76,7 @@ switching from the LiveImageCreator to another ImageCreator object.
build live images which use dm-snapshot, etc. This is what is used
by livecd-creator.
+* DiskImageCreator: This generates disk images containing multiple
+ partitions in a loopback file. It installs grub in the MBR and
+ can be directly booted in any virtual machine
+
diff --git a/Makefile b/Makefile
index 050fe88..d3d4ab7 100644
--- a/Makefile
+++ b/Makefile
@@ -18,6 +18,7 @@ all:
install:
$(INSTALL_PROGRAM) -D tools/livecd-creator $(DESTDIR)/usr/bin/livecd-creator
$(INSTALL_PROGRAM) -D tools/image-creator $(DESTDIR)/usr/bin/image-creator
+ $(INSTALL_PROGRAM) -D tools/disk-creator $(DESTDIR)/usr/bin/disk-creator
$(INSTALL_PROGRAM) -D tools/livecd-iso-to-disk.sh $(DESTDIR)/usr/bin/livecd-iso-to-disk
$(INSTALL_PROGRAM) -D tools/livecd-iso-to-pxeboot.sh $(DESTDIR)/usr/bin/livecd-iso-to-pxeboot
$(INSTALL_PROGRAM) -D tools/mayflower $(DESTDIR)/usr/lib/livecd-creator/mayflower
@@ -34,6 +35,8 @@ install:
uninstall:
rm -f $(DESTDIR)/usr/bin/livecd-creator
rm -rf $(DESTDIR)/usr/lib/livecd-creator
+ rm -rf $(DESTDIR)/usr/bin/disk-creator
+ rm -rf $(DESTDIR)/usr/bin/image-creator
rm -rf $(DESTDIR)/usr/share/doc/livecd-tools-$(VERSION)
rm -rf $(DESTDIR)/usr/share/livecd-tools
diff --git a/imgcreate/__init__.py b/imgcreate/__init__.py
index e535014..44f3ec5 100644
--- a/imgcreate/__init__.py
+++ b/imgcreate/__init__.py
@@ -21,6 +21,7 @@ from imgcreate.creator import *
from imgcreate.yuminst import *
from imgcreate.kickstart import *
from imgcreate.fs import *
+from imgcreate.disk import *
"""A set of classes for building Fedora system images.
@@ -28,6 +29,7 @@ The following image creators are available:
- ImageCreator - installs to a directory
- LoopImageCreator - installs to an ext3 image
- LiveImageCreator - installs to a bootable ISO
+ - DiskImageCreator - installs to a partitioned disk image
Also exported are:
- CreatorError - all exceptions throw are of this type
@@ -60,6 +62,7 @@ __all__ = (
'ImageCreator',
'LiveImageCreator',
'LoopImageCreator',
+ 'DiskImageCreator',
'FSLABEL_MAXLEN',
'read_kickstart',
'construct_name'
diff --git a/imgcreate/creator.py b/imgcreate/creator.py
index c2ed770..fb3b309 100644
--- a/imgcreate/creator.py
+++ b/imgcreate/creator.py
@@ -207,7 +207,11 @@ class ImageCreator(object):
"""
s = "/dev/root / %s defaults,noatime 0 0\n" %(self._fstype)
- s += "devpts /dev/pts devpts gid=5,mode=620 0 0\n"
+ s += self._get_fstab_special()
+ return s
+
+ def _get_fstab_special(self):
+ s = "devpts /dev/pts devpts gid=5,mode=620 0 0\n"
s += "tmpfs /dev/shm tmpfs defaults 0 0\n"
s += "proc /proc proc defaults 0 0\n"
s += "sysfs /sys sysfs defaults 0 0\n"
@@ -816,12 +820,11 @@ class LoopImageCreator(ImageCreator):
if not base_on is None:
shutil.copyfile(base_on, self._image)
- self.__instloop = SparseExtLoopbackMount(self._image,
- self._instroot,
- self.__image_size,
- self.__fstype,
- self.__blocksize,
- self.fslabel)
+ self.__instloop = ExtDiskMount(SparseLoopbackDisk(self._image, self.__image_size),
+ self._instroot,
+ self.__fstype,
+ self.__blocksize,
+ self.fslabel)
try:
self.__instloop.mount()
diff --git a/imgcreate/disk.py b/imgcreate/disk.py
index a0dd6e9..991db01 100644
--- a/imgcreate/disk.py
+++ b/imgcreate/disk.py
@@ -144,11 +144,12 @@ class DiskImageCreator(ImageCreator):
def _create_grub_config(self):
(bootdevnum, rootdevnum, rootdev, prefix) = self._get_grub_boot_config()
+ # XXX don't hardcode default - see livecd code
grub = ""
grub += "default=0\n"
grub += "timeout=5\n"
- grub += "splashimage=(hd0,%-d)%s/grub/splash.xpm.gz\n" % (rootdevnum, prefix)
- grub += "hiddenemnu\n"
+ grub += "splashimage=(hd0,%d)%s/grub/splash.xpm.gz\n" % (bootdevnum, prefix)
+ grub += "hiddenmenu\n"
versions = []
kernels = self._get_kernel_versions()
@@ -158,11 +159,11 @@ class DiskImageCreator(ImageCreator):
for v in versions:
grub += "title Fedora (%s)\n" % v
- grub += " root (hd0,%-d)\n" % bootdevnum
+ grub += " root (hd0,%d)\n" % bootdevnum
grub += " kernel %s/vmlinuz-%s ro root=%s\n" % (prefix, v, rootdev)
grub += " initrd %s/initrd-%s.img\n" % (prefix, v)
- cfg = open(self._instroot + "/boot/grub/grub.cfg", "w")
+ cfg = open(self._instroot + "/boot/grub/grub.conf", "w")
cfg.write(grub)
cfg.close()
@@ -185,11 +186,16 @@ class DiskImageCreator(ImageCreator):
def _install_grub(self):
(bootdevnum, rootdevnum, rootdev, prefix) = self._get_grub_boot_config()
+ # Ensure all data is flushed to disk before doing grub install
+ subprocess.call(["sync"])
+
+ stage2 = self._instroot + "/boot/grub/stage2"
+
setup = ""
setup += "device (hd0) %s\n" % (self.__instloop.disk.device)
- setup += "root (hd0,%d)\n" % rootdevnum
- setup += "configfile (hd0,%d)%s/grub/grub.cfg" % (bootdevnum, prefix)
- setup += "setup --prefix=%s/grub (hd0) (hd0,%d)\n" % (prefix, bootdevnum)
+ setup += "root (hd0,%d)\n" % bootdevnum
+ setup += "setup --stage2=%s --prefix=%s/grub (hd0)\n" % (stage2, prefix)
+ setup += "quit\n"
grub = subprocess.Popen(["grub", "--batch", "--no-floppy"],
stdin=subprocess.PIPE)
diff --git a/imgcreate/fs.py b/imgcreate/fs.py
index 9ca3a3e..b76f38b 100644
--- a/imgcreate/fs.py
+++ b/imgcreate/fs.py
@@ -86,42 +86,51 @@ class BindChrootMount:
subprocess.call(["/bin/umount", self.dest])
self.mounted = False
-class LoopbackMount:
- def __init__(self, lofile, mountdir, fstype = None):
- self.lofile = lofile
- self.mountdir = mountdir
- self.fstype = fstype
+class Disk:
+ def __init__(self, size, device = None):
+ self._device = device
+ self._size = size
- self.mounted = False
- self.losetup = False
- self.rmdir = False
- self.loopdev = None
+ def create(self):
+ pass
def cleanup(self):
- self.unmount()
- self.lounsetup()
+ pass
- def unmount(self):
- if self.mounted:
- rc = subprocess.call(["/bin/umount", self.mountdir])
- if rc == 0:
- self.mounted = False
+ def get_device(self):
+ return self._device
+ def set_device(self, path):
+ self._device = path
+ device = property(get_device, set_device)
+
+ def get_size(self):
+ return self._size
+ size = property(get_size)
- if self.rmdir and not self.mounted:
- try:
- os.rmdir(self.mountdir)
- except OSError, e:
- pass
- self.rmdir = False
- def lounsetup(self):
- if self.losetup:
- rc = subprocess.call(["/sbin/losetup", "-d", self.loopdev])
- self.losetup = False
- self.loopdev = None
+class RawDisk(Disk):
+ def __init__(self, size, device):
+ Disk.__init__(self, size, device)
+
+ def fixed(self):
+ return True
+
+ def exists(self):
+ return True
+
+class LoopbackDisk(Disk):
+ def __init__(self, lofile, size):
+ Disk.__init__(self, size)
+ self.lofile = lofile
- def loopsetup(self):
- if self.losetup:
+ def fixed(self):
+ return False
+
+ def exists(self):
+ return os.path.exists(self.lofile)
+
+ def create(self):
+ if self.device is not None:
return
losetupProc = subprocess.Popen(["/sbin/losetup", "-f"],
@@ -132,40 +141,27 @@ class LoopbackMount:
raise MountError("Failed to allocate loop device for '%s'" %
self.lofile)
- self.loopdev = losetupOutput.split()[0]
+ device = losetupOutput.split()[0]
- rc = subprocess.call(["/sbin/losetup", self.loopdev, self.lofile])
+ print "Losetup add %s %s" % (device, self.lofile)
+ rc = subprocess.call(["/sbin/losetup", device, self.lofile])
if rc != 0:
raise MountError("Failed to allocate loop device for '%s'" %
self.lofile)
+ self.device = device
- self.losetup = True
-
- def mount(self):
- if self.mounted:
+ def cleanup(self):
+ if self.device is None:
return
+ print "Losetup remove %s" % self.device
+ rc = subprocess.call(["/sbin/losetup", "-d", self.device])
+ self.device = None
- self.loopsetup()
- if not os.path.isdir(self.mountdir):
- os.makedirs(self.mountdir)
- self.rmdir = True
-
- args = [ "/bin/mount", self.loopdev, self.mountdir ]
- if self.fstype:
- args.extend(["-t", self.fstype])
- rc = subprocess.call(args)
- if rc != 0:
- raise MountError("Failed to mount '%s' to '%s'" %
- (self.loopdev, self.mountdir))
-
- self.mounted = True
-
-class SparseLoopbackMount(LoopbackMount):
- def __init__(self, lofile, mountdir, size, fstype = None):
- LoopbackMount.__init__(self, lofile, mountdir, fstype)
- self.size = size
+class SparseLoopbackDisk(LoopbackDisk):
+ def __init__(self, lofile, size):
+ LoopbackDisk.__init__(self, lofile, size)
def expand(self, create = False, size = None):
flags = os.O_WRONLY
@@ -191,10 +187,81 @@ class SparseLoopbackMount(LoopbackMount):
def create(self):
self.expand(create = True)
+ LoopbackDisk.create(self)
+
+class Mount:
+ def __init__(self, mountdir):
+ self.mountdir = mountdir
+
+ def cleanup(self):
+ self.unmount()
+
+ def mount(self):
+ pass
+
+ def unmount(self):
+ pass
+
+class DiskMount(Mount):
+ def __init__(self, disk, mountdir, fstype = None, rmmountdir = True):
+ Mount.__init__(self, mountdir)
+
+ self.disk = disk
+ self.fstype = fstype
+ self.rmmountdir = rmmountdir
+
+ self.mounted = False
+ self.rmdir = False
+
+ def cleanup(self):
+ Mount.cleanup(self)
+ self.disk.cleanup()
+
+ def unmount(self):
+ if self.mounted:
+ rc = subprocess.call(["/bin/umount", self.mountdir])
+ if rc == 0:
+ self.mounted = False
+
+ if self.rmdir and not self.mounted:
+ try:
+ os.rmdir(self.mountdir)
+ except OSError, e:
+ pass
+ self.rmdir = False
+
+
+ def __create(self):
+ print "Disk create %s" % str(self)
+ self.disk.create()
+
+
+ def mount(self):
+ if self.mounted:
+ return
+
+ print "Disk mount"
+ if not os.path.isdir(self.mountdir):
+ os.makedirs(self.mountdir)
+ self.rmdir = self.rmmountdir
+
+ self.__create()
+
+ print "Do mount %s " % self.disk.device
+ args = [ "/bin/mount", self.disk.device, self.mountdir ]
+ if self.fstype:
+ args.extend(["-t", self.fstype])
+
+ rc = subprocess.call(args)
+ if rc != 0:
+ raise MountError("Failed to mount '%s' to '%s'" %
+ (self.disk.device, self.mountdir))
+
+ self.mounted = True
-class SparseExtLoopbackMount(SparseLoopbackMount):
- def __init__(self, lofile, mountdir, size, fstype, blocksize, fslabel):
- SparseLoopbackMount.__init__(self, lofile, mountdir, size, fstype)
+class ExtDiskMount(DiskMount):
+ def __init__(self, disk, mountdir, fstype, blocksize, fslabel, rmmountdir=True):
+ DiskMount.__init__(self, disk, mountdir, fstype, rmmountdir)
self.blocksize = blocksize
self.fslabel = fslabel
@@ -202,19 +269,15 @@ class SparseExtLoopbackMount(SparseLoopbackMount):
rc = subprocess.call(["/sbin/mkfs." + self.fstype,
"-F", "-L", self.fslabel,
"-m", "1", "-b", str(self.blocksize),
- self.lofile,
- str(self.size / self.blocksize)])
+ self.disk.device])
+ # str(self.disk.size / self.blocksize)])
if rc != 0:
raise MountError("Error creating %s filesystem" % (self.fstype,))
subprocess.call(["/sbin/tune2fs", "-c0", "-i0", "-Odir_index",
- "-ouser_xattr,acl", self.lofile])
-
- def create(self):
- SparseLoopbackMount.create(self)
- self.__format_filesystem()
+ "-ouser_xattr,acl", self.disk.device])
- def resize(self, size = None):
- current_size = os.stat(self.lofile)[stat.ST_SIZE]
+ def __resize_filesystem(self, size = None):
+ current_size = os.stat(self.disk.lofile)[stat.ST_SIZE]
if size is None:
size = self.size
@@ -227,21 +290,31 @@ class SparseExtLoopbackMount(SparseLoopbackMount):
self.__fsck()
- resize2fs(self.lofile, size)
-
- if size < current_size:
- self.truncate(size)
+ resize2fs(self.disk.lofile, size)
return size
- def mount(self):
- if not os.path.isfile(self.lofile):
- self.create()
+ def __create(self):
+ print "Wibble"
+ resize = False
+ if not self.disk.fixed() and self.disk.exists():
+ resize = True
+
+ #DiskMount.__create(self)
+ self.disk.create()
+
+ if resize:
+ print "Fs resie"
+ self.__resize_filesystem()
else:
- self.resize()
- return SparseLoopbackMount.mount(self)
+ print "FS format"
+ self.__format_filesystem()
+
+ def mount(self):
+ self.__create()
+ DiskMount.mount(self)
def __fsck(self):
- subprocess.call(["/sbin/e2fsck", "-f", "-y", self.lofile])
+ subprocess.call(["/sbin/e2fsck", "-f", "-y", self.disk.lofile])
def __get_size_from_filesystem(self):
def parse_field(output, field):
@@ -253,7 +326,7 @@ class SparseExtLoopbackMount(SparseLoopbackMount):
dev_null = os.open("/dev/null", os.O_WRONLY)
try:
- out = subprocess.Popen(['/sbin/dumpe2fs', '-h', self.lofile],
+ out = subprocess.Popen(['/sbin/dumpe2fs', '-h', self.disk.lofile],
stdout = subprocess.PIPE,
stderr = dev_null).communicate()[0]
finally:
@@ -273,7 +346,7 @@ class SparseExtLoopbackMount(SparseLoopbackMount):
while top != (bot + 1):
t = bot + ((top - bot) / 2)
- if not resize2fs(self.lofile, t):
+ if not resize2fs(self.disk.lofile, t):
top = t
else:
bot = t
@@ -281,12 +354,187 @@ class SparseExtLoopbackMount(SparseLoopbackMount):
def resparse(self, size = None):
self.cleanup()
-
minsize = self.__resize_to_minimal()
+ self.disk.truncate(minsize)
+ return minsize
+
+class PartitionedMount(Mount):
+ def __init__(self, disk, mountdir):
+ Mount.__init__(self, mountdir)
+ self.disk = disk
+ self.partitions = []
+ self.mapped = False
+ self.mountOrder = []
+ self.unmountOrder = []
+
+ def add_partition(self, size, mountpoint, fstype = None):
+ self.partitions.append({'size': size, 'mountpoint': mountpoint, 'fstype': fstype, 'device': None, 'mount': None, 'num': None})
+
+ def __format_disk(self):
+ print "Formatting"
+ rc = subprocess.call(["/sbin/parted", "-s", self.disk.device, "mklabel", "msdos"])
+ if rc != 0:
+ raise MountError("Error writing partition table on %s" % self.disk.device)
+
+ print "Adding parts"
+ extSize = 0
+ num = 1
+ for p in self.partitions:
+ if num > 3:
+ extSize += p['size']
+ num += 1
+
+ # XXX we should probably work in cylinder units to keep fdisk happier..
+ start = 0
+ num = 1
+ for p in self.partitions:
+ if num == 4:
+ print "Added extened part at %d of size %d" %(start, extSize)
+ rc = subprocess.call(["/sbin/parted", "-s", self.disk.device, "mkpart", "extended", "%dM" % start, "%dM" % (start+extSize)])
+ num += 1
+
+ type = "primary"
+ if num > 3:
+ type = "logical"
+ print "Add %s part at %d of size %d" % (type, start, p['size'])
+ rc = subprocess.call(["/sbin/parted", "-s", self.disk.device, "mkpart", type, "%dM" % start, "%dM" % (start+p['size'])])
+ if rc != 0 and 1 == 0: # XXX disabled because parted always fails to reload part table with loop devices
+ raise MountError("Error creating partition on %s" % self.disk.device)
+ start = start + p['size']
+ p['num'] = num
+ num += 1
+
+ def __map_partitions(self):
+ if self.mapped:
+ return
+
+ kpartx = subprocess.Popen(["/sbin/kpartx", "-l", self.disk.device],
+ stdout=subprocess.PIPE)
+
+ kpartxOutput = kpartx.communicate()[0].split("\n")
+ # Strip trailing blank
+ kpartxOutput = kpartxOutput[0:len(kpartxOutput)-1]
+ print "Run %s" % str(kpartxOutput)
+ if kpartx.returncode:
+ raise MountError("Failed to query partition mapping for '%s'" %
+ self.disk.device)
+
+ if len(kpartxOutput) != len(self.partitions):
+ raise MountError("Unexpected number of partitions from kpartx: %d != %d",
+ len(kpartxOutput), len(self.partitions))
+
+ for i in range(len(kpartxOutput)):
+ print " Got %s" % (kpartxOutput[i])
+ line = kpartxOutput[i]
+ dev = line.split()[0]
+ mapperdev = "/dev/mapper/" + dev
+ loopdev = self.disk.device + dev[-1]
+ print "Dev %s: %s -> %s" % (dev, loopdev, mapperdev)
+ self.partitions[i]['device'] = loopdev
+
+ # grub's install wants partitions to be named
+ # to match their parent device + partition num
+ # kpartx doesn't work like this, so we add compat
+ # symlinks to point to /dev/mapper
+ os.symlink(mapperdev, loopdev)
+
+ print "Mapping"
+ rc = subprocess.call(["/sbin/kpartx", "-a", self.disk.device])
+ if rc != 0:
+ raise MountError("Failed to map partitions for '%s'" %
+ self.disk.device)
+ self.mapped = True
+
+
+ def __unmap_partitions(self):
+ if not self.mapped:
+ return
+
+ print "Removing compat symlinks"
+ for p in self.partitions:
+ if p['device'] != None:
+ os.unlink(p['device'])
+ p['device'] = None
+
+ print "Unmapping"
+ rc = subprocess.call(["/sbin/kpartx", "-d", self.disk.device])
+ if rc != 0:
+ raise MountError("Failed to unmap partitions for '%s'" %
+ self.disk.device)
+
+ self.mapped = False
+
+
+ def __calculate_mountorder(self):
+ for p in self.partitions:
+ self.mountOrder.append(p['mountpoint'])
+ self.unmountOrder.append(p['mountpoint'])
+
+ self.mountOrder.sort()
+ self.unmountOrder.sort()
+ self.unmountOrder.reverse()
+ print str(self.mountOrder)
- self.truncate(minsize)
+ def cleanup(self):
+ import time
+ time.sleep(20)
+ Mount.cleanup(self)
+ self.__unmap_partitions()
+ self.disk.cleanup()
+
+ def unmount(self):
+ for mp in self.unmountOrder:
+ if mp == 'swap':
+ continue
+ p = None
+ for p1 in self.partitions:
+ if p1['mountpoint'] == mp:
+ p = p1
+ break
+
+ if p['mount'] != None:
+ print "Clenaup %s " % p['mountpoint']
+ try:
+ p['mount'].cleanup()
+ except:
+ pass
+ p['mount'] = None
+
+ def mount(self):
+ self.disk.create()
+
+ self.__format_disk()
+ self.__map_partitions()
+ self.__calculate_mountorder()
+
+ for mp in self.mountOrder:
+ p = None
+ for p1 in self.partitions:
+ if p1['mountpoint'] == mp:
+ p = p1
+ break
+
+ if mp == 'swap':
+ subprocess.call(["/sbin/mkswap", p['device']])
+ continue
+
+ print "Submount %s " % p['mountpoint']
+ rmmountdir = False
+ if p['mountpoint'] == "/":
+ rmmountdir = True
+ pdisk = ExtDiskMount(RawDisk(p['size'] * 1024 * 1024, p['device']),
+ self.mountdir + p['mountpoint'],
+ p['fstype'],
+ 4096,
+ p['mountpoint'],
+ rmmountdir)
+ pdisk.mount()
+ p['mount'] = pdisk
+
+ def resparse(self, size = None):
+ # Can't re-sparse a disk image - too hard
+ pass
- return self.resize(size)
class DeviceMapperSnapshot(object):
def __init__(self, imgloop, cowloop):
@@ -306,8 +554,8 @@ class DeviceMapperSnapshot(object):
if self.__created:
return
- self.imgloop.loopsetup()
- self.cowloop.loopsetup()
+ self.imgloop.create()
+ self.cowloop.create()
self.__name = "imgcreate-%d-%d" % (os.getpid(),
random.randint(0, 2**16))
@@ -315,8 +563,8 @@ class DeviceMapperSnapshot(object):
size = os.stat(self.imgloop.lofile)[stat.ST_SIZE]
table = "0 %d snapshot %s %s p 8" % (size / 512,
- self.imgloop.loopdev,
- self.cowloop.loopdev)
+ self.imgloop.device,
+ self.cowloop.device)
args = ["/sbin/dmsetup", "create", self.__name, "--table", table]
if subprocess.call(args) != 0:
@@ -382,15 +630,14 @@ class DeviceMapperSnapshot(object):
# 8) Create a squashfs of the COW
#
def create_image_minimizer(path, image, minimal_size):
- imgloop = LoopbackMount(image, "None")
+ imgloop = LoopbackDisk(image, None) # Passing bogus size - doesn't matter
- cowloop = SparseLoopbackMount(os.path.join(os.path.dirname(path), "osmin"),
- None, 64L * 1024L * 1024L)
+ cowloop = SparseLoopbackDisk(os.path.join(os.path.dirname(path), "osmin"),
+ 64L * 1024L * 1024L)
snapshot = DeviceMapperSnapshot(imgloop, cowloop)
try:
- cowloop.create()
snapshot.create()
resize2fs(snapshot.path, minimal_size)
diff --git a/imgcreate/kickstart.py b/imgcreate/kickstart.py
index a7e0723..a92f877 100644
--- a/imgcreate/kickstart.py
+++ b/imgcreate/kickstart.py
@@ -468,6 +468,9 @@ def get_groups(ks, required = []):
def get_excluded(ks, required = []):
return ks.handler.packages.excludedList + required
+def get_partitions(ks, required = []):
+ return ks.handler.partition.partitions
+
def ignore_missing(ks):
return ks.handler.packages.handleMissing == ksconstants.KS_MISSING_IGNORE
diff --git a/imgcreate/live.py b/imgcreate/live.py
index bbb17ef..dc2aea9 100644
--- a/imgcreate/live.py
+++ b/imgcreate/live.py
@@ -130,7 +130,7 @@ class LiveImageCreatorBase(LoopImageCreator):
#
def __base_on_iso(self, base_on):
"""helper function to extract ext3 file system from a live CD ISO"""
- isoloop = LoopbackMount(base_on, self._mkdtemp())
+ isoloop = Mount(LoopbackDisk(base_on), self._mkdtemp())
try:
isoloop.mount()
@@ -144,10 +144,10 @@ class LiveImageCreatorBase(LoopImageCreator):
else:
squashimg = isoloop.mountdir + "/LiveOS/squashfs.img"
- squashloop = LoopbackMount(squashimg, self._mkdtemp(), "squashfs")
+ squashloop = Mount(LoopbackDisk(squashimg), self._mkdtemp(), "squashfs")
try:
- if not os.path.exists(squashloop.lofile):
+ if not squashloop.disk.exists():
raise CreatorError("'%s' is not a valid live CD ISO : "
"squashfs.img doesn't exist" % base_on)
diff --git a/livecd-tools.spec b/livecd-tools.spec
index a789460..d9118ba 100644
--- a/livecd-tools.spec
+++ b/livecd-tools.spec
@@ -58,6 +58,7 @@ rm -rf $RPM_BUILD_ROOT
%dir %{_datadir}/livecd-tools
%{_datadir}/livecd-tools/*
%{_bindir}/image-creator
+%{_bindir}/disk-creator
%dir %{python_sitelib}/imgcreate
%{python_sitelib}/imgcreate/*.py
%{python_sitelib}/imgcreate/*.pyo
-------------- next part --------------
# Kickstart file automatically generated by anaconda.
install
url --url http://download.fedora.redhat.com/pub/fedora/linux/releases/8/Fedora/x86_64/os/
lang en_US.UTF-8
keyboard us
network --device eth0 --bootproto dhcp
rootpw --iscrypted $1$HNOucon/$m69RprODwQn4XjzVUi9TU0
firewall --disabled
authconfig --enableshadow --enablemd5
selinux --disabled
services --disabled=iptables,yum-updatesd,libvirtd,bluetooth,cups,gpm,pcscd --enabled=ntpd,dhcpd,xinetd,httpd,postgresql,ovirt-wui
timezone --utc America/New_York
text
bootloader --location=mbr --driveorder=sda
# The following is the partition information you requested
# Note that any partitions you deleted are not expressed
# here so unless you clear all partitions first, this is
# not guaranteed to work
zerombr
clearpart --all --drives=sda
part /boot --fstype ext3 --size=100 --ondisk=sda
part swap --fstype swap --size=500 --grow --ondisk=sda
part /var/lib/pgsql --fstype ext3 --size=300 --ondisk=sda
part / --fstype ext3 --size=3000 --grow --ondisk=sda
part /home --fstype ext3 --size=500 --grow --ondisk=sda
part /var --fstype ext3 --size=1000 --ondisk=sda
#part pv.2 --size=3000 --grow --ondisk=sda
#volgroup VolGroup00 --pesize=32768 pv.2
#logvol swap --fstype swap --name=LogVol01 --vgname=VolGroup00 --size=512
#logvol / --fstype ext3 --name=LogVol00 --vgname=VolGroup00 --size=1024 --grow
repo --name=f8 --mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=fedora-8&arch=$basearch
repo --name=f8-updates --mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=updates-released-f8&arch=$basearch
# Not using rawhide currently
#repo --name=rawhide --mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=rawhide&arch=$basearch
repo --name=freeipa --baseurl=http://freeipa.com/downloads/devel/rpms/F7/$basearch/
repo --name=ovirt --baseurl=http://ovirt.et.redhat.com/repos/ovirt/$basearch/
%packages
@admin-tools
@editors
@system-tools
@text-internet
@core
@base
@hardware-support
@web-server
@sql-server
@development-libs
@legacy-fonts
@development-tools
radeontool
fuse
pax
imake
dhcp
tftp-server
tftp
dhclient
ipa-server
ipa-admintools
xinetd
libvirt
cyrus-sasl-gssapi
iscsi-initiator-utils
collectd
ruby-libvirt
ruby-postgres
ovirt-wui
firefox
xorg-x11-xauth
virt-viewer
-libgcj
-glib-java
-valgrind
-boost-devel
-frysk
-bittorrent
-fetchmail
-slrn
-cadaver
-mutt
%post
cat > /root/create_default_principals.py << \EOF
#!/usr/bin/python
import krbV
import os, string, re
import socket
import shutil
def kadmin_local(command):
ret = os.system("/usr/kerberos/sbin/kadmin.local -q '" + command + "'")
if ret != 0:
raise
default_realm = krbV.Context().default_realm
# here, generate the libvirt/ principle for this machine, necessary
# for taskomatic and host-browser
this_libvirt_princ = 'libvirt/' + socket.gethostname() + '@' + default_realm
kadmin_local('addprinc -randkey +requires_preauth ' + this_libvirt_princ)
kadmin_local('ktadd -k /usr/share/ovirt-wui/ovirt.keytab ' + this_libvirt_princ)
# We need to replace the KrbAuthRealms in the ovirt-wui http configuration
# file to be the correct Realm (i.e. default_realm)
ovirtconfname = '/etc/httpd/conf.d/ovirt-wui.conf'
ipaconfname = '/etc/httpd/conf.d/ipa.conf'
# make sure we skip this on subsequent runs of this script
if string.find(file(ipaconfname, 'rb').read(), '<VirtualHost *:8089>') < 0:
ipaconf = open(ipaconfname, 'r')
ipatext = ipaconf.readlines()
ipaconf.close()
ipaconf2 = open(ipaconfname, 'w')
print >>ipaconf2, "Listen 8089"
print >>ipaconf2, "NameVirtualHost *:8089"
print >>ipaconf2, "<VirtualHost *:8089>"
for line in ipatext:
newline = re.sub(r'(.*RewriteCond %{HTTP_HOST}.*)', r'#\1', line)
newline = re.sub(r'(.*RewriteRule \^/\(.*\).*)', r'#\1', newline)
newline = re.sub(r'(.*RewriteCond %{SERVER_PORT}.*)', r'#\1', newline)
newline = re.sub(r'(.*RewriteCond %{REQUEST_URI}.*)', r'#\1', newline)
ipaconf2.write(newline)
print >>ipaconf2, "</VirtualHost>"
ipaconf2.close()
if string.find(file(ovirtconfname, 'rb').read(), '<VirtualHost *:80>') < 0:
ovirtconf = open(ovirtconfname, 'r')
ovirttext = ovirtconf.readlines()
ovirtconf.close()
ovirtconf2 = open(ovirtconfname, 'w')
print >>ovirtconf2, "NameVirtualHost *:80"
print >>ovirtconf2, "<VirtualHost *:80>"
for line in ovirttext:
newline = re.sub(r'(.*)KrbAuthRealms.*', r'\1KrbAuthRealms ' + default_realm, line)
newline = re.sub(r'(.*)Krb5KeyTab.*', r'\1Krb5KeyTab /etc/httpd/conf/ipa.keytab', newline)
ovirtconf2.write(newline)
print >>ovirtconf2, "</VirtualHost>"
ovirtconf2.close()
EOF
chmod +x /root/create_default_principals.py
cat > /root/add_host_principal.py << \EOF
#!/usr/bin/python
import krbV
import os
import socket
import shutil
import sys
def kadmin_local(command):
ret = os.system("/usr/kerberos/sbin/kadmin.local -q '" + command + "'")
if ret != 0:
raise
def get_ip(hostname):
return socket.gethostbyname(hostname)
if len(sys.argv) != 2:
print "Usage: add_host_principal.py <hostname>"
sys.exit(1)
default_realm = krbV.Context().default_realm
ipaddr = get_ip(sys.argv[1])
libvirt_princ = 'libvirt/' + sys.argv[1] + '@' + default_realm
outname = '/usr/share/ipa/html/' + ipaddr + '-libvirt.tab'
# here, generate the libvirt/ principle for this machine, necessary
# for taskomatic and host-browser
kadmin_local('addprinc -randkey +requires_preauth ' + libvirt_princ)
kadmin_local('ktadd -k ' + outname + ' ' + libvirt_princ)
# make sure it is readable by apache
os.chmod(outname, 0644)
EOF
chmod +x /root/add_host_principal.py
cat > /usr/share/ovirt-wui/psql.cmds << \EOF
CREATE USER ovirt WITH PASSWORD 'v23zj59an';
CREATE DATABASE ovirt;
GRANT ALL PRIVILEGES ON DATABASE ovirt to ovirt;
CREATE DATABASE ovirt_test;
GRANT ALL PRIVILEGES ON DATABASE ovirt_test to ovirt;
EOF
chmod a+r /usr/share/ovirt-wui/psql.cmds
cat > /etc/init.d/ovirt-app-first-run << \EOF
#!/bin/bash
#
# ovirt-app-first-run First run configuration for Ovirt WUI appliance
#
# chkconfig: 3 99 01
# description: ovirt appliance first run configuration
#
# Source functions library
. /etc/init.d/functions
start() {
service postgresql initdb
echo "local all all trust" > /var/lib/pgsql/data/pg_hba.conf
echo "host all all 127.0.0.1 255.255.255.0 trust" >> /var/lib/pgsql/data/pg_hba.conf
service postgresql start
su - postgres -c "/usr/bin/psql -f /usr/share/ovirt-wui/psql.cmds"
cd /usr/share/ovirt-wui ; rake db:migrate
/usr/bin/ovirt_grant_admin_privileges.sh admin
}
case "$1" in
start)
start
;;
*)
echo "Usage: ovirt {start}"
exit 2
esac
chkconfig ovirt-app-first-run off
EOF
chmod +x /etc/init.d/ovirt-app-first-run
chkconfig ovirt-app-first-run on
sed -i -e 's/\(.*\)disable\(.*\)= yes/\1disable\2= no/' /etc/xinetd.d/tftp
# set up the yum repos
cat > /etc/yum.repos.d/freeipa.repo << \EOF
[freeipa]
name=FreeIPA Development
baseurl=http://freeipa.com/downloads/devel/rpms/F7/x86_64/
enabled=1
gpgcheck=0
EOF
cat > /etc/yum.repos.d/ovirt.repo << \EOF
[ovirt]
name=Ovirt
baseurl=http://ovirt.et.redhat.com/repos/ovirt/x86_64
enabled=1
gpgcheck=0
EOF
# pretty login screen..
echo -e "" > /etc/issue
echo -e " 888 888 \\033[0;32md8b\\033[0;39m 888 " >> /etc/issue
echo -e " 888 888 \\033[0;32mY8P\\033[0;39m 888 " >> /etc/issue
echo -e " 888 888 888 " >> /etc/issue
echo -e " .d88b. Y88b d88P 888 888d888 888888 " >> /etc/issue
echo -e " d88''88b Y88b d88P 888 888P' 888 " >> /etc/issue
echo -e " 888 888 Y88o88P 888 888 888 " >> /etc/issue
echo -e " Y88..88P Y888P 888 888 Y88b. " >> /etc/issue
echo -e " 'Y88P' Y8P 888 888 'Y888 " >> /etc/issue
echo -e "" >> /etc/issue
echo -e " Admin node \\\\n " >> /etc/issue
echo -e "" >> /etc/issue
echo -e " Virtualization just got the \\033[0;32mGreen Light\\033[0;39m" >> /etc/issue
echo -e "" >> /etc/issue
cp /etc/issue /etc/issue.net
echo "0.fedora.pool.ntp.org" >> /etc/ntp/step-tickers
# remove the mod_auth_kerb, and make sure we get the one from freeipa
rpm -e --nodeps mod_auth_kerb
yum -y --disablerepo=* --enablerepo=freeipa install mod_auth_kerb
# with the new libvirt (0.4.0), make sure we we setup gssapi in the mech_list
sed -i -e 's/mech_list: digest-md5/#mech_list: digest-md5/' /etc/sasl2/libvirt.conf
sed -i -e 's/#mech_list: gssapi/mech_list: gssapi/' /etc/sasl2/libvirt.conf
# for firefox, we need to add the following to ~/.mozilla/firefox/zzzzz/prefs.js
#echo 'user_pref("network.negotiate-auth.delegation-uris", ".redhat.com");' >> ~/.mozilla/firefox/zzzz/prefs.js
#echo 'user_pref("network.negotiate-auth.trusted-uris", ".redhat.com");' >> ~/.mozilla/firefox/zzzz/prefs.js
%end
More information about the livecd
mailing list