save 50MB? And why is the effect on QA less important?
From my perspective, the storage on the installation medium should be
efficiently used. Even though the optimization is just 50MiB, it is an
optimization.
And this optimization is transparent in terms of content.
The QA team is only one group of users. We should rather orient at end
users, and the way those users install Fedora.
On 15/09/2020 12:03, Zbigniew Jędrzejewski-Szmek wrote:
On Tue, Sep 15, 2020 at 11:18:04AM +0200, Bohdan Khomutskyi wrote:
> Hello everyone,
>
> Thanks for sharing your ideas and comments about this change.
>
> Thanks to Mohan Boddu, I got RawHide DVD and netinstall images of
> RawHide with the optimization features enabled. Those test composes
> are available at the following locations:
>
https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-2020082...
> (Plain SquashFS)
>
>
https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-2020090...
> (Plain SquashFS and different xz compression options)
>
> To select images from the test composes, I applied a patch from
>
https://github.com/rhinstaller/anaconda/pull/2829, so you can
> download and run those images yourself to test the change:
>
>
https://drive.google.com/drive/folders/1tGsoqFMg2A6dQZHfuQNb9uDqYu9XEiPI
>
> The result of benchmark will supplement already available data I
> previously posted for Fedora Live images at
>
https://fedoraproject.org/wiki/Changes/OptimizeSquashFS
>
> As a reminder, there are 2 levels for this optimization:
>
> 1. Making the SquashFS filesystem on the DVD plain (i.e. without
> embedded EXT4 inside it) -- has the suffix plain-xz-128k.iso
This sounds excellent. We should get both better time and space efficiency.
> 2. In addition to #1, using a different compression parameters for
> the SquashFS filesystem on the installation medium -- has the suffix
> plain-xz-1M.iso
And this ones trades times for space. Considering that the space
difference is only ~50 MB, I don't think it's worth it, for all the
reasons outlined before.
You haven't really answered the "why" part: why is it so important to
save 50MB? And why is the effect on QA less important?
Zbyszek
> I propose to apply both of optimizations, using the highest possible
> compression ratio supported by SquashFS. The highest compression
> ratio in SquashFS corresponds to xz level 1 (out of 9 available
> presets).
>
> Making the first change will reduce both x86-64 Boot and the x86-64
> DVD ISO by approximately 24 MiB. With both changes combined, the
> reduction of size will be 70 MiB.
>
> Because the SquashFS filesystem on the Live installation medium is
> of bigger size, storage savings on the Live images are even higher
> (I documented it in
>
https://fedoraproject.org/wiki/Changes/OptimizeSquashFS)
>
> On 05/09/2020 12:43, Zbigniew Jędrzejewski-Szmek wrote:
>> On Thu, Aug 27, 2020 at 11:13:26AM -0400, Ben Cotton wrote:
>>>
https://fedoraproject.org/wiki/Changes/OptimizeSquashFS
>> ...
>>> Based on the results above, I'd suggest selecting the following
>>> ''optimal configuration'': XZ algorithm, with block size of
1MiB and
>>> without BCJ filter (plain xz -b 1M, without -Xbcj x86).
>>> On the right, you can see the impact of the compression algorithms on
>>> installation time.
>>>
>>> As can be seen from the picture on the right hand side, selecting
>>> 'plain xz -b 1M configuration' has minimal impact on the
installation
>>> time and CPU usage during the installation. The compression will
>>> result in +6.51% or, in real terms, +24.94s additional installation
>>> time, compared to the savings of 142 MiB on the installation media,
>>> == Benefit to Fedora ==
>>> * Reduction of the installation media size and the cost of storing and
>>> distributing Fedora.
>>> * Reduction of the CPU usage at build time. Depending on which
>>> compression parameters chosen.
>> Hi Bohdan,
>>
>> I think there's a misalignment of priorities.
>>
>> My evaluation is the following: users won't care. The image size difference
>> is not big enough for people to notice. OTOH, slower installation will
>> impact QA and VM installations. We're doing more and more automated
>> installations and tests, and this change impacts those tests negatively.
>>
>>> This increase in installation time will be compensated by the change
>>> in the
installer:https://github.com/rhinstaller/anaconda/pull/2292
>> This PR is very interesting. But it seems that the more we optimize
>> things, the more slow decompression will be noticable. I.e. in some
>> way, right now, the slow decompression is obscured by the slow IO
>> speed, multiple levels of block device, or slow processing of the
>> uncompressed data. Any time the input or output speed is improved,
>> slow compression will be more of a bottleneck. So the approach of
>> increasing XZ compression levels has bad perspectives.