Dear fellow users,
I want to pose a quick questions. I know Mr. A Robatino posts deltaisos regularly which amounts to a certain amount of savings when one downloads a deltaiso. Although I have never downloaded one, I ask if one can save more bandwidth if we use lzma compression. I ask this question because TeXLive2009 comes in a iso.xz format which is lzma compression and saves some bandwidth. Would Fedora welcome such savings and distributing isos, whether test or official releases as lzma compressed?
Has this been thought of before?
Thank you in advance for commenting/answering this question.
Regards,
Antonio
On Tue, Mar 09, 2010 at 20:06:00 -0800, Antonio Olivares olivares14031@yahoo.com wrote:
Dear fellow users,
I want to pose a quick questions. I know Mr. A Robatino posts deltaisos regularly which amounts to a certain amount of savings when one downloads a deltaiso. Although I have never downloaded one, I ask if one can save more bandwidth if we use lzma compression. I ask this question because TeXLive2009 comes in a iso.xz format which is lzma compression and saves some bandwidth. Would Fedora welcome such savings and distributing isos, whether test or official releases as lzma compressed?
Has this been thought of before?
For the Live images you can keep an eye on the proposed feature page: https://fedoraproject.org/wiki/Features/LZMA_for_Live_Images
Current status is that support in the 2.6.34 kernel is iffy. Linus bounced Lougher's initial pull request and only some clean up pacthes were accepted before the merge window closed. Potentially he might pull in a patch providing support for lmza squashfs later since there was an initial pull request in time and it sat in linux-next for essentially all of 2.6.33 development, but I wouldn't bet the farm on that.
I did test an lzma compressed live image (non-functional due to no kernel support) for the games spin and found it was 10% (400 MiB) smaller than the zlib compressed version.
Bruno Wolff III wrote:
I did test an lzma compressed live image (non-functional due to no kernel support) for the games spin and found it was 10% (400 MiB) smaller than the zlib compressed version.
Isn't the basic strength of lzma a very big dictionary window? I would have said this is not easily paired to random access (squashfs). 10% (of the compressed size) looks like a very good gain.
On Wed, Mar 10, 2010 at 16:06:35 +0100, Roberto Ragusa mail@robertoragusa.it wrote:
Bruno Wolff III wrote:
I did test an lzma compressed live image (non-functional due to no kernel support) for the games spin and found it was 10% (400 MiB) smaller than the zlib compressed version.
Isn't the basic strength of lzma a very big dictionary window? I would have said this is not easily paired to random access (squashfs). 10% (of the compressed size) looks like a very good gain.
I haven't done performance testing on live images. It is claimed that lzma decompresses at good speeds compared to other options. My expectation is reduced accesses to slow media (especially CDs and DVDs) will outweigh other effects. I have been holding off on testing things that need kernel support. If Kyle lets me get squashfs 4.1 in rawhide or F13 and Lougher posts a revised set of patches that will work with his clean up patches that have already been pulled, then I'll look at doing some testing of live images and mounted file systems. I wasn't planning on detailed benchmarks for that, just seeing if things obviously sucked compared to zlib.
Bruno Wolff III wrote:
On Wed, Mar 10, 2010 at 16:06:35 +0100, Roberto Ragusa mail@robertoragusa.it wrote:
Bruno Wolff III wrote:
I did test an lzma compressed live image (non-functional due to no kernel support) for the games spin and found it was 10% (400 MiB) smaller than the zlib compressed version.
Isn't the basic strength of lzma a very big dictionary window? I would have said this is not easily paired to random access (squashfs). 10% (of the compressed size) looks like a very good gain.
I haven't done performance testing on live images. It is claimed that lzma decompresses at good speeds compared to other options. My expectation is reduced accesses to slow media (especially CDs and DVDs) will outweigh other effects. I have been holding off on testing things that need kernel support. If Kyle lets me get squashfs 4.1 in rawhide or F13 and Lougher posts a revised set of patches that will work with his clean up patches that have already been pulled, then I'll look at doing some testing of live images and mounted file systems. I wasn't planning on detailed benchmarks for that, just seeing if things obviously sucked compared to zlib.
Wow, this could get complex fast! Let me start by saying I am not disagreeing with you.
That said, I have noticed that install off USB thumb drive is faster than DVD by a good bit, even though the transfer rate off a DVD is almost certainly faster. This suggests that seek is the limiting factor delay rather than transfer rate. Therefore you comment about seek and squashfs is interesting, and open the question of whether it would be better to use an uncompressed ISO filesystem and better compression on the RPMs. I wouldn't guess without testing.
I share the impression that a huge window is the cause of both better better and slower compression.
On Mon, Mar 15, 2010 at 17:32:14 -0400, Bill Davidsen davidsen@tmr.com wrote:
That said, I have noticed that install off USB thumb drive is faster than DVD by a good bit, even though the transfer rate off a DVD is almost certainly faster. This suggests that seek is the limiting factor delay rather than transfer rate.
That's what I would expect. live DVD images are pretty unusable for doing real stuff on them. They are very slow to boot and unless you stay in a few applications (so that what you need is in ram), it's very slow to do things.
Live USB images are a lot more usable.
Therefore you comment about seek and squashfs is interesting, and open the question of whether it would be better to use an uncompressed ISO filesystem and better compression on the RPMs. I wouldn't guess without testing.
For live images better compression for rpms isn't really relevant as they aren't on the final image. The rpms are installed into an image which is shrunk and then written to a squashfs file system that is then written to the iso image.
-
Antonio Olivares -
Bill Davidsen -
Bruno Wolff III -
Roberto Ragusa