Fwd: [Libjpeg-turbo-users] jpeg-9, API/ABI compatibility, and the future role of this project

[Adam Tkac]

On Mon, Jan 14, 2013 at 04:42:59PM +0100, Xose Vazquez Perez wrote:
> hi,
> 
> As Fedora was pioneering in the libjpeg-turbo inclusion
> maybe you are interested in this discussion.
> 
> http://sourceforge.net/mailarchive/forum.php?thread_name=50E4AF86.50203%40users.sourceforge.net&forum_name=libjpeg-turbo-users

Another interesting thread is
http://sourceforge.net/mailarchive/message.php?msg_id=30352453

We are currently discussing drop of the
https://fedoraproject.org/wiki/Features/libjpeg-turbo-jpeg8-ABI feature because
SmartScale encoding (only reason of the jpeg7 and jpeg 8 ABI bumps) was rejected
by ISO and actually doesn't bring any benefit over widely used png/webp codecs.

So Fedora will probably stick with old jpeg6 API/ABI unless IJG maintainer
reveals some new facts about SmartScale. All facts are currently against
SmartScale (image quality/size, compression/decompression speed).

Regards, Adam

> -------- Original Message --------
> Subject: [Libjpeg-turbo-users] jpeg-9, API/ABI compatibility, and the future role of this project
> Date: 2013-01-01 08:32
> From: DRC dcommander
> To: libjpeg-turbo-users
> 
> I am not casting blame, because I realize that the libjpeg API, by 
> virtue of exposing all of its data structures, necessitates breaking 
> backward API/ABI compatibility whenever additional features are added. 
> Unfortunately, however, the reality is that jpeg-9 (due to be released 
> in a few weeks) has broken API/ABI compatibility yet again, introducing 
> a new field called "color_transform" to both jpeg_compress_struct and 
> jpeg_decompress_struct.  This field was implemented to further support 
> generating lossless RGB JPEG files, which require the SmartScale format 
> that can only (currently) be decoded using the IJG's software.
> 
> As most of you know, libjpeg-turbo has never really tried to be anything 
> other than a very fast implementation of baseline JPEG.  We implemented 
> jpeg-7 and jpeg-8 API/ABI emulation at the behest of a commercial 
> software company, primarily so that their application could take 
> advantage of turbo baseline encoding/decoding without recompiling. 
> However, this also opened the door for libjpeg-turbo to be adopted by 
> other projects (including several O/S distros) that had already made the 
> switch to jpeg-7 or jpeg-8.  I never actively sought this out, but I am 
> happy that others have found uses for libjpeg-turbo that go beyond the 
> initial scope of the project.  Even though I'm the only maintainer, this 
> is a community project, and almost 100% of the modifications that have 
> been made to it have either been ideas that others paid me to implement 
> or code that others have contributed.
> 
> Thus, I'm curious to hear your thoughts regarding jpeg-9 and the future 
> role of this project.  My background is in computer architecture, 
> performance optimization, HPC, and visualization, so providing faster 
> implementations of existing algorithms is my strong suit, but improving 
> the algorithms themselves or supporting new formats isn't.  I've never 
> tried to make libjpeg-turbo into a reference implementation or a home 
> for such new formats, but it seems like most people don't care about 
> anything but baseline anyhow.  Thus, even though libjpeg-turbo doesn't 
> accelerate any other type of encoding/decoding, that has not seemingly 
> been a hindrance to its adoption.
> 
> Although it is possible to stub out the new "color_transform" field as a 
> GNDN (goes nowhere/does nothing) feature and thus provide API/ABI 
> compatibility with jpeg-9, I don't really see the point of this unless 
> there are projects that might upgrade to jpeg-9 and then later want to 
> cross-grade to libjpeg-turbo (in which case, it seems more prudent to 
> address that situation when or if it arises.)  It seems like most of the 
> justification for emulating the v7 and later libjpeg API/ABI may have 
> passed.  It was needed to enable cross-grading for those who had already 
> upgraded, but at this point, projects have probably made the decision to 
> either go with more speed or to go with the format extensions offered in 
> jpeg-8 and later.
> 
> My personal take on it is that tracking the upstream code may no longer 
> be a battle worth fighting.  Most of the recent IJG changes (post 
> jpeg-8b) have been related to lossless JPEG encoding or SmartScale. 
> Best case, SmartScale is a new format that has not been adopted as a 
> standard yet and is not widely used, and worst case, it may be a mostly 
> useless extension.  The IJG's method for generating lossless JPEG files 
> using SmartScale is interesting, but I struggle to think of a reason why 
> one would want to use SmartScale for any other purpose (apparently I'm 
> not the only one who struggles with this: 
> http://hardwarebug.org/2010/02/01/ijg-swings-again-and-misses/).  And it 
> hasn't been proven that the use of this extension for lossless encoding 
> is significantly better than, for instance, PNG.  In fact, from my 
> admittedly quick & dirty testing, it seems to be worse:
> 
> Using jpeg-8d:
> > time ./cjpeg -rgb -block 1 -arithmetic <~/test_images/nightshot_iso_100.ppm >nightshot_iso_100.jpg
> real	0m7.025s
> user	0m6.468s
> sys	0m0.087s
> Original size: 22127633, Compressed size: 7537012, Ratio: 2.936
> 
> Using the new color transform in jpeg-9:
> > time ./cjpeg -rgb1 -block 1 -arithmetic <~/test_images/nightshot_iso_100.ppm >nightshot_iso_100.jpg
> real	0m7.316s
> user	0m6.753s
> sys	0m0.081s
> Original size: 22127633, Compressed size: 7554020, Ratio: 2.929
> 
> Using ImageMagick 6.7.9 to compress as PNG:
> > time convert -quality 6 ~/test_images/nightshot_iso_100.ppm nightshot_iso_100.png
> real	0m2.647s
> user	0m1.689s
> sys	0m0.114s
> Original size: 22127633, Compressed size: 7157427, Ratio: 3.092
> 
> Maybe I'm missing something?  Anyhow, until/unless there is community 
> support behind SmartScale, it is unlikely that it will ever be adopted 
> in libjpeg-turbo (I don't have any need for it in my own work, so it 
> would pretty much have to be a funded development sort of deal.)  Thus, 
> the implementation of the libjpeg v7+ API and ABI would remain just 
> that:  an emulation and not a feature-complete implementation of jpeg-7 
> or jpeg-8.  Without provable metrics to indicate its usefulness, it's 
> unlikely that the SmartScale format will garner community support or 
> gain official adoption as a standard.  I don't see any of that happening 
> as long as the current IJG maintainers take the impolitic attitude that 
> anyone (including the ISO standards committee) who doesn't recognize the 
> brilliance of SmartScale is "corrupt" or "ignorant."
> 
> I guess what I'm saying is-- libjpeg-turbo may have reached a point at 
> which there isn't really a whole lot more we can add to it feature-wise 
> without either adopting the unproven SmartScale technology or diverging 
> from IJG to implement some other format, like JPEG XR.  Personally, I 
> feel that both would be out of scope for what is still, at the end of 
> the day, a turbo baseline JPEG library.  I've always believed that new 
> formats should be implemented by a new library.  The libjpeg API is 
> dated and really ill-equipped to handle new formats, which is why these 
> API/ABI incompatibilities keep popping up with the IJG's software.
> 
> However, I want this project to be whatever the community wants it to 
> be.  I don't think we're well-positioned to be a haven for new formats, 
> but if enough people are interested in one that they want to either pay 
> for the implementation or contribute code, I'm definitely open to that. 
>   "Keep things the way they are" is also a perfectly acceptable answer, 
> as is "continue focusing on baseline and coming up with new ways to make 
> it faster."
> 
> Comments encouraged.
> 
> DRC
> --end--
> 
> FYI: libjpeg 9 was released today
> http://www.h-online.com/open/news/item/Libjpeg-9-improves-lossless-JPEG-compression-1783311.html

-- 
Adam Tkac, Red Hat, Inc.