qcow2 compression [was Re: F20 System Wide Change: Visible Cloud]

[Richard W.M. Jones]

On Mon, Jul 15, 2013 at 01:36:57PM -0400, Matthew Miller wrote:
> On Mon, Jul 15, 2013 at 07:35:49PM +0200, Juerg Haefliger wrote:
> > > Or, bigger picture, maybe a future qcow2 format could support this natively?
> > Confused. Support what? Your comment above indicates that compressed
> > qcow2 images are seekable already.
> 
> Seekable xz/lzma compression instead of zlib/deflate.

In qcow2, each [by default] 64K cluster is compressed independently.

So seeking is great, but the compression cannot be very effective.
In my rather unscientific test I found that there is a 25% overhead
for using lzma with a 64K block size versus the default block size.

Also:

- qcow2 wastes up to 511 bytes per cluster when compression is used.
  Ironically.

- The qcow2 format doesn't store any indication that zlib is used vs
  some other compression format.  (To be fair this one could be fixed
  pretty easily).

- It's read-only -- if you write to a compressed qcow2 file, it
  gradually uncompresses.  I suspect this one could also be fixed, at
  the cost of slow writes.

My main point however was that there's no need to invent a new file
format variant.  xz-compressed raw files Just Work, provided they are
prepared using the xz `--block-size' parameter.  If you slap a qcow2
overlay on top, you can even make them writable.

Adding an xz driver to qemu based on the one I wrote for nbdkit would
be the way to go IMHO.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/