F21 System Wide Change: lbzip2 as default bzip2 implementation

[Mikolaj Izdebski]

> On Wed, Apr 2, 2014 at 11:39 PM, Chris Adams <linux at cmadams.net> wrote:
> > Once upon a time, Mikolaj Izdebski <mizdebsk at redhat.com> said:
> >> lbzip2 does not (at least not yet) provide interfaces of libbz2 library,
> >> only command line tools.  This Change does *not* affect users of libbz2.
> >
> > Is there enough of a gain to the system to only partially replace a core
> > program like this (especially with alternatives)?  This seems like a
> > case where either we get a "new and improved and replaces the old"
> > version (where the new one just obsoletes the old one, such as the
> > jpeg-turbo change), or just leave it alone.

Eventually lbzip2 may replace bzip2, but I don't want to make any drastic
changes.  Using alternatives allows us to have a nice contingency plan
in case something goes unexpected.  Once lbzip2 is used by default,
further chnages will be just a metter of agreement between maintainers.

> > Please understand, I don't mean to "attack" you or your code.  I just
> > think adding a second implementation of a core utility like bzip2 is a
> > bad idea unless there is a significant gain.  If there's a point where
> > lbzip2 can fully replace bzip2 (so all CLI and API uses), and there are
> > good benefits, then Fedora should just replace the old implementation
> > with a new one.

I believe there is a signifficant gain, see below.

> Well the change says " multi-threaded operation for both compression
> and decompression, with almost linear scalability" linear scalability
> means speed ups on the range of 2-8x on current desktop / laptop
> systems.
> Which I'd call a "significant gain" ;)

I don't have any recent benchmark comparing performance of lbzip2 with bzip2
because doing them doesn't make much sense for me -- lbzip2 is so much faster.
I compare only different versions to make sure there are no performance
regressions.

However I have one *old* benchmark made just after lbzip2 2.0 release
2 and half years ago (current version is 2.5).  But since then lbzip2
was improved.

  https://github.com/kjn/lbzip2/wiki/Benchmark:-Opteron-24

I will prepare more benchmarks of recent lbzip2 version for desktop
(1, 2, 4, 8 cores) and bigger multicore server systems.

A test of scalability (pretty old too) is available at:

  http://archive.lbzip2.org/scaling/scaling.html

I hope that's convincing enough for now.

--
Mikolaj Izdebski