ANNOUNCE: libguestfs 1.20 - tools for accessing and modifying virtual machine disk images

[Richard W.M. Jones]

I'm very pleased to announce the release of libguestfs 1.20.

Libguestfs is a library and a comprehensive set of tools for accessing
and modifying virtual machine (VM) disk images.  For more information
see http://libguestfs.org

Libguestfs 1.20 represents 7 months of upstream work, dozens of major new
features and bug fixes.  For full details read the release notes below.

You can download the libguestfs source from
http://libguestfs.org/download/1.20-stable
and packages will soon be available for Fedora 18 and Debian Wheezy.

For additional notes on Fedora 18, see:
https://www.redhat.com/archives/libguestfs/2012-December/msg00047.html

For additional notes on Debian Wheezy, see:
https://www.redhat.com/archives/libguestfs/2012-December/msg00046.html

----------------------------------------

You can also read the release notes online at:
http://libguestfs.org/guestfs-release-notes.1.html

RELEASE NOTES FOR LIBGUESTFS 1.20

New features

libvirt and sVirt (SELinux confinement)

You can optionally use libvirt to launch and manage the backend
    appliance by:

* setting LIBGUESTFS_ATTACH_METHOD=libvirt[:URI]

* using ./configure --with-default-attach-method=libvirt[:URI]

* or calling the API method guestfs_set_attach_method

SVirt (SELinux confinement) is used to secure the libguestfs appliance,
    if the libvirt backend is selected.

virtio-scsi, larger number of drives, hotplugging

Virtio-scsi is now used by default, if it is available. This greatly
    increases the number of drives that can be added to a single libguestfs
    handle (up to 255).

You can now hotplug drives (add and remove drives after launch).

Libguestfs can now handle > 25 disks, in all APIs, tools and tests.

You can label drives when adding them, then refer to them by label
    (/dev/disk/guestfs/LABEL) instead of having to use device names
    (/dev/sda).

new library features

POSIX Access Control Lists (ACLs) and Linux filesystem capabilities can
    now be read and written through the API.

Windows Dynamic Disks are now supported. The metadata can only be read
    (volumes cannot be modified or created). However the data/filesystems
    in the volumes themselves can be mounted, read and written (Matthew
    Booth).

Hivex (Windows Registry editing) is now directly supported through the
    API, making reading or editing the Windows Registry much more
    efficient.

Several libguestfs APIs were reimplemented so they no longer have any
    limits on output. The reimplemented APIs are: guestfs_cat,
    guestfs_find, guestfs_read_file, guestfs_read_lines, guestfs_write,
    guestfs_write_append, guestfs_lstatlist, guestfs_lxattrlist,
    guestfs_readlinklist, guestfs_ls.

virt tools

guestfish touch 'win:c:\new_file' now works as expected.

guestfish has a new --network option, which enables the user network in
    libguestfs.

You can set GUESTFISH_PS1 to use fancy prompts in guestfish.

rsync, ssh, less and lrzip commands are now available in virt-rescue.
    In addition, rsync can be used through the API for doing incremental
    copies.

virt-sparsify uses qcow2 v3 for increased efficiency.

virt-sysprep enhancements:

* generate new UUIDs for PVs and VGs

* remove the local machine ID (Wanlong Gao)

* remove ABRT data (Wanlong Gao)

* remove puppet keys and log files (Wanlong Gao)

* remove process accounting files (Wanlong Gao)

* new --firstboot parameter allowing firstboot scripts for
      customizing guests

* remove HOSTNAME field from ifcfg-eth* files (Wanlong Gao)

* miscellaneous improvements for SuSE (Olaf Hering)

* remove man pages cache (Wanlong Gao)

* remove crash data generated by kexec-tools (Wanlong Gao)

virt-make-fs can now create virtual floppy disks (VFDs).

guestmount has a --pid-file option, which can be used to avoid a race
    condition between unmounting the filesystem and using the underlying
    disk image.

language bindings

Full-featured Lua bindings have been added in this release (thanks to
    Jerome Vuarand for many suggestions).

In Ruby, optional arguments hash is optional.

In Python, optional arguments now don't use special "sentinel" values,
    so any integer can be passed as an optional argument.

Erlang bindings now come with examples and tests.

Erlang bindings now handle 64 bit integer parameters correctly.

The automated 'bindtests' now test for 64 bit cleanliness in parameter
    passing. Several bugs in the bindings were found and fixed as a result.

Better handling of blocking and non-blocking functions should make
    libguestfs calls more efficient.

inspection

Windows guests with unlimited-sized Registries are now supported. By
    adding the hivex API to the libguestfs API, it is much more efficient
    and easier to inspect and modify the Windows Registry in guests.

Better support for SuSE guests (Olaf Hering).

Return the architecture of installed applications (John Eckersberg).

Windows 8 is now supported.

Fedora 18 is now supported.

Inspection of ISOs/CDs/DVDs can now use the libosinfo database.

ports

Libguestfs without the appliance can be compiled on Mac OS X (Masami
    HIRATA).

RHEL 5 is now explicitly supported through the 'oldlinux' branch in
    upstream git.

PowerPC 64 bit is now supported. Also libguestfs has been examined for
    endianness bugs and these have been fixed.

daemon

A man page for the daemon (guestfsd(8)) is included.

Security

guestfish history file

The $HOME/.guestfish history file is now created with 0600 permissions
    (instead of 0644 before) so it is no longer world readable.

CVE-2012-2690

Old versions of both virt-edit and the guestfish edit command created a
    new file containing the changes but did not set the permissions, etc of
    the new file to match the old one. The result of this was that if you
    edited a security sensitive file such as /etc/shadow then it would be
    left world-readable after the edit.

This issue was assigned CVE-2012-2690, and is fixed in libguestfs ≥
    1.16.

For further information, see
    https://bugzilla.redhat.com/show_bug.cgi?id=788642

New APIs

acl-delete-def-file
     acl-get-file
     acl-set-file
     canonical-device-name
     cap-get-file
     cap-set-file
     create-flags
     device-index
     disk-has-backing-file
     disk-format
     disk-virtual-size
     filesystem-available
     fill-dir
     fstrim
     get-cachedir
     get-libvirt-requested-credentials
     get-libvirt-requested-credential-prompt
     get-libvirt-requested-credential-challenge
     get-libvirt-requested-credential-defresult
     get-tmpdir
     hivex-close
     hivex-commit
     hivex-node-add-child
     hivex-node-children
     hivex-node-delete-child
     hivex-node-get-child
     hivex-node-get-value
     hivex-node-name
     hivex-node-parent
     hivex-node-set-value
     hivex-node-values
     hivex-open
     hivex-root
     hivex-value-key
     hivex-value-type
     hivex-value-utf8
     hivex-value-value
     inspect-list-applications2 (John Eckersberg)
     list-ldm-volumes
     list-ldm-partitions
     ldmtool-create-all
     ldmtool-diskgroup-disks
     ldmtool-diskgroup-name
     ldmtool-diskgroup-volumes
     ldmtool-remove-all
     ldmtool-scan
     ldmtool-scan-devices
     ldmtool-volume-hint
     ldmtool-volume-partitions
     ldmtool-volume-type
     ls0
     max-disks
     mke2fs (Wanlong Gao)
     mklost-and-found
     mkswap [added label and uuid optional arguments]
     mktemp (Wanlong Gao)
     nr-devices
     parse-environment
     parse-environment-list
     rm-f
     rsync
     rsync-in
     rsync-out
     set-cachedir
     set-libvirt-supported-credentials
     set-libvirt-requested-credential
     set-tmpdir
     shutdown [backported to 1.16 and 1.18]
     tar-in [added compress flag]
     tar-out [added compress, numericowner, excludes flags]
     umount [added force and lazy optional arguments]
     utsname
     xfs-admin (Wanlong Gao)
     xfs-growfs (Wanlong Gao)
     xfs-info (Wanlong Gao)
     xfs-repair (Wanlong Gao)

In the C API only:

guestfs_push_error_handler
     guestfs_pop_error_handler

Internals

qemu ≥ 1.1 is required (≥ 1.2 preferred).

febootstrap ≥ 3.20 is required.

libvirt is optional, but if you want to use the new libvirt backend for
    launching the appliance then libvirt ≥ 0.10.2 would be required.

Coverity has been run over the complete codebase, and many issues fixed
    (thanks Ondrej Vasik, Kamil Dudka).

Functions which previously had no optargs can now be converted to ones
    with optargs, making extending the API much more flexible. Source and
    binary backwards compatibility is preserved for users of the API.

The way that libguestfs APIs and structures are represented in the
    generator has changed to use an OCaml struct instead of a tuple. This
    makes generator descriptions more flexible and easier to understand.
    For details see commits 39d1a7db and eb185eef.

Separation of the library code into more files:

* Launch backends are now located in separate files eg.
      src/launch-appliance.c, src/launch-libvirt.c.

* Generated action code is now split over several src/action*.c
      files, for faster compilation.

* The huge src/guestfs.c file is now split into smaller logical
      units.

POD (documentation) is now generated using a rewritten Perl program
    instead of pod2* + shell scripts.

Man pages now contain stable dates (Hilko Bengen).

Skipped tests now exit with code 77, so they appear as SKIP: in make
    check output.

The parallel mount-local test has been rewritten in C (RHBZ#838081).

Ruby 1.8.5 is now supported (though Ruby ≥ 1.9 is preferred).

Perl bindings can be disabled via ./configure --disable-perl (Wulf C.
    Krueger). Note that Perl is still required in order to build
    libguestfs.

Java bindings are now enabled/disabled using ./configure --with-java or
    ./configure --without-java (Wulf C. Krueger).

New configure options ./configure --enable-code-profiling and
    ./configure --enable-code-coverage.

Multiple fixes to use of 64 bit integers in language bindings.

The appliance backend now uses sgabios instead of vgabios (Dan
    Berrange).

The ./run script now sets enough environment variables that you can run
    OCaml, Python, Ruby, Java, GJS, Erlang, Lua programs.

./run --test flag for running tests with minimal output. It also prints
    the time taken to run each test.

The ./run script now builds up paths cumulatively, meaning that you can
    use ./run twice, or use the libguestfs and libvirt ./run scripts
    together.

You can extract a list of external commands required by the daemon,
    making building the appliance on certain distros easier (Olaf Hering).

The virt-rescue command is now tested during make check.

The generator now removes unused generated files. This helps when going
    back and forth with git rebase, git bisect, etc.

Tests now run in a separate toplevel tmp/ directory in the source. This
    allows the directory to be labelled for SELinux (sVirt), and also makes
    it easier to clean up.

make syntax-check now works to a greater extent, and many problems in
    the main code that were found by syntax-check have been fixed (thanks
    Jim Meyering).

Emacs mode (-*- foo -*-) has been added to generated files.

Progress bar output is now sent to /dev/tty so it doesn't end up in the
    regular output of the program. virt-resize and virt-sparsify now
    suppress progress bars if stdout is not a tty.

There is now a ./configure --without-libvirt option. This is useful for
    testing that the code still compiles without libvirt.

There is now an internal mini-library for running commands. This allows
    us to redirect errors from external commands into events.

Code for handling temporary directories and the appliance cache was
    completely overhauled.

Code for temporarily ignoring/disabling errors now looks like this:

guestfs_push_error_handler (g, NULL, NULL);
     guestfs_mkdir (g, "/foo"); /* We don't care if this fails. */
     guestfs_pop_error_handler (g);

The tests/extra directory has gone. The "extra tests" are now split
    into separately runnable targets, such as make check-valgrind. Use make
    help to get a summary of the targets.

The lpj option is passed to the appliance kernel when using TCG. This
    should improve clock stability (thanks Marcelo Tosatti, Olaf Hering).

Bugs fixed

https://bugzilla.redhat.com/882417

libguestfs tools fail with odd error messages if TMPDIR is not an
      absolute path

https://bugzilla.redhat.com/882299

Windows 8 guest disks can't be mounted: "The NTFS partition is in an
      unsafe state. Please resume and shutdown Windows fully (no
      hibernation or fast restarting), or mount the volume read-only with
      the 'ro' mount option."

https://bugzilla.redhat.com/881953

libguestfs: migration to /etc/hostname, /etc/vconsole.conf,
      /etc/locale.conf

https://bugzilla.redhat.com/880801

virt-df with two -a options displays incorrect disk image name

https://bugzilla.redhat.com/879416

libguestfs-test-tool pauses when you use --help option

https://bugzilla.redhat.com/876579

mke2fs API does not apply block device naming translation to
      journaldevice optarg

https://bugzilla.redhat.com/860235

SELinux policy ought to allow qemu to write to
      unconfined_u:object_r:user_tmp_t:s0

https://bugzilla.redhat.com/859949

RFE: inspect-list-applications does not return the architecture of
      RPM packages

https://bugzilla.redhat.com/859885

inspect-list-applications does not list all installed RPM packages
      with same name and different versions

https://bugzilla.redhat.com/859876

guestfish printed paths are not canonicalized

https://bugzilla.redhat.com/859875

Progress bar output should go to tty(?) stderr(?)

https://bugzilla.redhat.com/858696

virt-sysprep reports Guestfs.Error("read_lines: fopen:
      /etc/sysconfig/network: No such file or directory") on some Fedora
      guests

https://bugzilla.redhat.com/858128

libguestfs fail to list devices added by add-drive-ro-with-if twice

https://bugzilla.redhat.com/858126

virt-inspector fail to work with some windows guests

https://bugzilla.redhat.com/853762

virt-sparsify should use a more robust method to detect the input
      format

https://bugzilla.redhat.com/853393

libvirt doesn't label console, serial sockets

https://bugzilla.redhat.com/853159

virt-rescue in Fedora 18 is completely broken

https://bugzilla.redhat.com/852394

libguestfs inspection limits registries to 100 MiB

https://bugzilla.redhat.com/852194

virt-sparsify --compress fails if output is raw format

https://bugzilla.redhat.com/847881

RFE: allow extra arguments (like --exclude) to tar-out

https://bugzilla.redhat.com/847880

tar-out should allow (or force) --numeric-owner

https://bugzilla.redhat.com/845522

guestfish "copy-out / localdir" command fails with "No such file or
      directory"

https://bugzilla.redhat.com/845488

Long filenames on NTFS cause tar-out, copy-out etc to fail with error
      "Cannot open: File name too long"

https://bugzilla.redhat.com/842307

RFE: Need help designing and implementing selinux policy for
      libguestfs/sVirt

https://bugzilla.redhat.com/840572

virt-make-fs / tar-in should support vfat properly

https://bugzilla.redhat.com/840115

guestfish touch problem - case_sensitive_path API expects the file to
      exist

https://bugzilla.redhat.com/836710

Data loss when writing to qcow2-format disk files

https://bugzilla.redhat.com/834712

lvresize, lvresize-free fail unnecessarily if you don't change the
      size of the LV: "New size (nn extents) matches existing size (nn
      extents)"

https://bugzilla.redhat.com/824716

compress-device-out didn't support bzip2

https://bugzilla.redhat.com/824043

guestfish unrecognized mount option gives confusing error message

https://bugzilla.redhat.com/823887

Filenames containing Chinese characters can't be created on vfat
      filesystems

https://bugzilla.redhat.com/823885

virt-make-fs cannot create vfat filesystem containing filesystems
      with chinese characters

https://bugzilla.redhat.com/823883

virt-make-fs -t fat fails with an obscure error message

https://bugzilla.redhat.com/823821

Inspection fails when /etc/HOSTNAME is empty

https://bugzilla.redhat.com/801117

libguestfs cannot get icon for Windows 8

https://bugzilla.redhat.com/798979

Ubuntu install CDs from oneiric onwards are not recognized:
      "multi-boot operating systems are not supported"

https://bugzilla.redhat.com/782167

libguestfs doesn't recognize Windows Dynamic disks in some
      configurations, eg. spanned

https://bugzilla.redhat.com/713678

Not all febootstrap messages are redirected to log callbacks

https://bugzilla.redhat.com/627675

libguestfs inspector code cannot handle /dev/disk/by-id/* paths

https://bugzilla.redhat.com/602997

part-get-bootable gives wrong result with an unordered part layout

----------------------------------------

Rich.