F21 System Wide Change: lbzip2 as default bzip2 implementation

[Mikolaj Izdebski]

On 04/04/2014 05:16 PM, Michal Schmidt wrote:
> On 04/04/2014 04:15 PM, Mikolaj Izdebski wrote:
>> Compression of payload.tar
>> --------------------------
>>
>> command    |   real |   user |  sys | memory | compr. size
>> -----------+--------+--------+------+--------+------------
>> lbzip2     |   3.36 | 170.07 | 6.38 | 380448 | 424676188
>> lbzip2 -u  |   6.45 | 123.14 | 3.80 | 255524 | 424518771
>> pbzip2     |   6.78 | 288.33 | 8.90 | 491644 | 425213134
>> bzip2      | 176.68 | 175.76 | 0.67 |   8000 | 425108407
>>
>>
>> Conclusions
>> ===========
>> [...]
>> "lbzip2 -u" always produced smallest files (even smaller than bzip2)
>> while consuming the least amount of resources (CPU power and memory).
> 
> The table above says it needs about 30 times *more* memory than bzip2.

No, it shows that it *used* that much memory.

The system had 32 GB of RAM, lbzip2 using all 56 CPUs used less than 1.2
% of available memory.  That is *very* conservative.

Memory usage can be limited by lowering number of threads used (-n) or
by specifying explicit memory limit (-m, undocumented for now, it will
be fully supported in future version of lbzip2 after it gets enough
testing).

-- 
Mikolaj Izdebski
Software Engineer, Red Hat
IRC: mizdebsk