zlib-ng as a compat replacement for zlib

[Richard W.M. Jones]

The background to this is I've spent far too long trying to optimize
the conversion of qcow2 files to raw files.  Most existing qcow2 files
that you can find online are zlib compressed, including the qcow2
images provided by Fedora.  Each cluster in the file is separately
compressed as a zlib stream, and qemu uses zlib library functions to
decompress them.  When downloading and decompressing these files, I
measured 40%+ of the total CPU time is doing zlib decompression.

[You don't need to tell me how great Zstd is, qcow2 supports this for
compression also, but it is not widely used by existing content.]

I was looking around for alternative zlib implementations and there
are several.  This AWS blog is a decent summary:

 
https://aws.amazon.com/blogs/opensource/improving-zlib-cloudflare-and-com...

We already package zlib-ng in Fedora:

  https://src.fedoraproject.org/rpms/zlib-ng/blob/rawhide/f/zlib-ng.spec
  https://github.com/zlib-ng/zlib-ng

In my test zlib-ng is about 40% faster.

Sadly zlib-ng is not compiled with the ZLIB_COMPAT option.  What this
means in practice is that the zlib functions have different names
(eg. zng_inflateInit instead of inflateInit).  It is not a drop-in
replacement for zlib and software would need to be adjusted to use it.

However there is this bug / RFE to package the compat library.

  https://bugzilla.redhat.com/show_bug.cgi?id=2145239

It literally replaces /usr/lib64/libz.so.1, so pretty high stakes.  It
has the appropriate Provides/Conflicts.

Anyway, 40% (or whatever, but significant) performance improvement
sounds good for a very widely used operation.

So I'd like to ask Fedora ...

What do we think about the opt in approach of adding a patch similar
to the one proposed in bug 2145239, where I think you could "simply"
install zlib-ng to get better performance with existing software?
("simply" because it seems high risk of going wrong)

What about replacing zlib with zlib-ng?

Other ideas ...?  Does anyone know what other distros are doing?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v