1:45 p.m.
On Mon, Aug 10, 2009 at 10:40:16AM -0700, Jesse Keating wrote:
I'm looking at the spec file for libguestfs, and all I can say is
WTF.
There is a lot of crazyness going on in this spec, chroots within
chroots, making a repo of yum cache packages and using it again, calling
qemu, and none of it is really documented in the spec as for what and
why things are done this way.
Hey Jesse, I'll explain below what the general idea is, then provide
at the end some commentary inline on specific points within the
specfile. It certainly is a complex specfile.
In case you aren't familiar with it, libguestfs (http://libguestfs.org)
is a tool for modifying VM disk images. For example:
$ guestfish -a Fedora10.img
Welcome to guestfish, the libguestfs filesystem interactive shell for
editing virtual machine filesystems.
Type: 'help' for help with commands
'quit' to quit the shell
<fs> run
<fs> list-devices
/dev/vda
<fs> list-partitions
/dev/vda1
/dev/vda2
/dev/vda3
<fs> lvs
/dev/VolGroup00/LogVol00
/dev/VolGroup00/LogVol01
<fs> file /dev/VolGroup00/LogVol00
Linux rev 1.0 ext3
filesystem data (large files)
<fs> mount /dev/VolGroup00/LogVol00 /
<fs> cat /etc/redhat-release
Fedora release 10 (Cambridge)
<fs> vi /etc/passwd
<fs> ^D
What we do to make this happen is run a small Fedora distro (called
the "appliance") under qemu which is attached to the disk image in
question. The appliance runs a Linux kernel, LVM tools etc etc and
thus has full access to the disk. You can also run qemu as non-root,
which means you don't need to be root to edit disk images.
The complexity in the specfile is mainly around building this
appliance.
We use a tool called febootstrap which is basically a shell script
that runs 'yum' as non-root, using Debian's fakeroot and fakechroot
programs.
febootstrap can take any yum repository and turn it into an appliance,
without needing root permission:
$ febootstrap fedora-11 ./f11
$ febootstrap-to-initramfs ./f11 > initrd.img
Of course during the Koji build we don't have a yum repository, and we
don't have (or want) network access. So we make one using createrepo
+ the BuildRequired rpms from the mock cache.
There's a further complexity here -- a normal appliance would actually
contain the kernel + glibc + libraries + programs. However that is
undesirable from a Fedora point of view, since it's very similar to
static linking. If we really shipped that as an appliance then there
would be obvious security / patching headaches if (for example) glibc
had a security bug. Therefore we don't actually bundle any binaries.
We remove those from the final appliance, and get them from the host
system at runtime. The technique is called building a "supermin
appliance" and it's described here: http://libguestfs.org/README.txt
under "supermin appliance" It is important to reiterate that the
appliance we ship in the RPM *does not* contain any libraries or
binaries. We just remember where those files were, and pull them out
of the host's filesystem at runtime.
From the point of view of improving Koji (or mock) it would be nice to
have a more official method to access the build RPMs, although the
technique we are using works fine.
On to the specfile itself:
# Enable to build w/o network.
%global buildnonet 1
Summary: Access and modify virtual machine disk images
Name: libguestfs
[...]
# Basic build requirements:
BuildRequires: /usr/bin/pod2man
BuildRequires: /usr/bin/pod2text
BuildRequires: febootstrap >= 2.3
BuildRequires: augeas-devel >= 0.5.0
BuildRequires: readline-devel
BuildRequires: squashfs-tools
BuildRequires: qemu-kvm >= 0.10-7
BuildRequires: createrepo
BuildRequires: glibc-static
# This is only needed for RHEL 5 because readline-devel doesn't
# properly depend on it, but doesn't do any harm on other platforms:
BuildRequires: ncurses-devel
These are the basic build requires for making the actual package.
# Build requirements for the appliance (see 'make.sh.in' in the source):
BuildRequires: kernel, bash, coreutils, lvm2, ntfs-3g, util-linux-ng
BuildRequires: MAKEDEV, net-tools, augeas-libs, file
BuildRequires: module-init-tools, procps, strace, iputils
BuildRequires: dosfstools, zerofree, lsof, scrub
%ifarch %{ix86} x86_64
BuildRequires: grub, ntfsprogs
%endif
These are the BRs reflecting what is inside the appliance ...
# Must match the above set of BuildRequires exactly!
Requires: kernel, bash, coreutils, lvm2, ntfs-3g, util-linux-ng
Requires: MAKEDEV, net-tools, augeas-libs, file
Requires: module-init-tools, procps, strace, iputils
Requires: dosfstools, zerofree, lsof, scrub
%ifarch %{ix86} x86_64
Requires: grub, ntfsprogs
%endif
... and at runtime we need the same set of Requires, because of the
supermin appliance described above. Those files that we evicted from
the appliance at build time must exist in the host system at runtime.
[...]
%build
%if %{buildnonet}
mkdir repo
find /var/cache/yum -type f -name '*.rpm' -print0 | xargs -0 cp -t repo
createrepo repo
%define extra --with-mirror=file://$(pwd)/repo --with-repo=fedora-12
--with-updates=none
%else
%define extra %nil
%endif
This is the part where we create the local repository from the mock
cache. The %{buildnonet} switch allows people to build from the
Fedora repos instead if they do have network access.
# --with-net-if=ne2k_pci is a workaround for RHBZ#516022.
./configure \
--prefix=%{_prefix} --libdir=%{_libdir} \
--mandir=%{_mandir} \
--with-qemu="qemu-kvm qemu-system-%{_build_arch} qemu" \
--enable-debug-command \
--enable-supermin \
--with-net-if=ne2k_pci \
%{extra}
# This ensures that /usr/sbin/chroot is on the path. Not needed
# except for RHEL 5, it shouldn't do any harm on other platforms.
export PATH=/usr/sbin:$PATH
# 'INSTALLDIRS' ensures that perl libs are installed in the vendor dir
# not the site dir.
make INSTALLDIRS=vendor %{?_smp_mflags}
This bit is building the library + appliance + 6 language bindings.
%check
# Enable debugging - very useful if a test does fail, although
# it produces masses of output in the build.log.
export LIBGUESTFS_DEBUG=1
# Uncomment one of these, depending on whether you want to
# do a very long and thorough test ('make check') or just
# a quick test to see if things generally work.
# Tracking test issues:
# BZ archs branch reason
# 494075 ppc, ppc64 openbios bug causes "invalid/unsupported
opcode"
# 504273 ppc, ppc64 "no opcode defined"
# 505109 ppc, ppc64 "Boot failure! No secondary bootloader
specified"
# 502058 i386, x86-64 F-11 need to boot with noapic (WORKAROUND ENABLED)
# 502074 i386 F-11 commands segfault randomly
# 503236 i386 F-12 cryptomgr_test at doublefault_fn
# 507066 all F-12 sequence of chroot calls (FIXED)
# 513249 all F-12 guestfwd broken in qemu (FIXED)
# 516022 all F-12 virtio-net gives "Network is unreachable"
errors
# (WORKAROUND ENABLED)
# 516096 ? F-11 race condition in swapoff/blockdev --rereadpt
# 516543 ? F-12 qemu-kvm segfaults when run inside a VM
#%ifarch x86_64
#make check
#%endif
Normally we run the extensive test suite. However because of a bug in
qemu (#516543) we are unable to run that at the moment.
And the rest is just concerned with the language bindings.
Rich.
--
Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top