Fwd: [Libjpeg-turbo-users] jpeg-9, API/ABI compatibility, and the future role of this project

[Xose Vazquez Perez]

hi,

As Fedora was pioneering in the libjpeg-turbo inclusion
maybe you are interested in this discussion.

http://sourceforge.net/mailarchive/forum.php?thread_name=50E4AF86.50203%40users.sourceforge.net&forum_name=libjpeg-turbo-users

-------- Original Message --------
Subject: [Libjpeg-turbo-users] jpeg-9, API/ABI compatibility, and the future role of this project
Date: 2013-01-01 08:32
From: DRC dcommander
To: libjpeg-turbo-users

I am not casting blame, because I realize that the libjpeg API, by 
virtue of exposing all of its data structures, necessitates breaking 
backward API/ABI compatibility whenever additional features are added. 
Unfortunately, however, the reality is that jpeg-9 (due to be released 
in a few weeks) has broken API/ABI compatibility yet again, introducing 
a new field called "color_transform" to both jpeg_compress_struct and 
jpeg_decompress_struct.  This field was implemented to further support 
generating lossless RGB JPEG files, which require the SmartScale format 
that can only (currently) be decoded using the IJG's software.

As most of you know, libjpeg-turbo has never really tried to be anything 
other than a very fast implementation of baseline JPEG.  We implemented 
jpeg-7 and jpeg-8 API/ABI emulation at the behest of a commercial 
software company, primarily so that their application could take 
advantage of turbo baseline encoding/decoding without recompiling. 
However, this also opened the door for libjpeg-turbo to be adopted by 
other projects (including several O/S distros) that had already made the 
switch to jpeg-7 or jpeg-8.  I never actively sought this out, but I am 
happy that others have found uses for libjpeg-turbo that go beyond the 
initial scope of the project.  Even though I'm the only maintainer, this 
is a community project, and almost 100% of the modifications that have 
been made to it have either been ideas that others paid me to implement 
or code that others have contributed.

Thus, I'm curious to hear your thoughts regarding jpeg-9 and the future 
role of this project.  My background is in computer architecture, 
performance optimization, HPC, and visualization, so providing faster 
implementations of existing algorithms is my strong suit, but improving 
the algorithms themselves or supporting new formats isn't.  I've never 
tried to make libjpeg-turbo into a reference implementation or a home 
for such new formats, but it seems like most people don't care about 
anything but baseline anyhow.  Thus, even though libjpeg-turbo doesn't 
accelerate any other type of encoding/decoding, that has not seemingly 
been a hindrance to its adoption.

Although it is possible to stub out the new "color_transform" field as a 
GNDN (goes nowhere/does nothing) feature and thus provide API/ABI 
compatibility with jpeg-9, I don't really see the point of this unless 
there are projects that might upgrade to jpeg-9 and then later want to 
cross-grade to libjpeg-turbo (in which case, it seems more prudent to 
address that situation when or if it arises.)  It seems like most of the 
justification for emulating the v7 and later libjpeg API/ABI may have 
passed.  It was needed to enable cross-grading for those who had already 
upgraded, but at this point, projects have probably made the decision to 
either go with more speed or to go with the format extensions offered in 
jpeg-8 and later.

My personal take on it is that tracking the upstream code may no longer 
be a battle worth fighting.  Most of the recent IJG changes (post 
jpeg-8b) have been related to lossless JPEG encoding or SmartScale. 
Best case, SmartScale is a new format that has not been adopted as a 
standard yet and is not widely used, and worst case, it may be a mostly 
useless extension.  The IJG's method for generating lossless JPEG files 
using SmartScale is interesting, but I struggle to think of a reason why 
one would want to use SmartScale for any other purpose (apparently I'm 
not the only one who struggles with this: 
http://hardwarebug.org/2010/02/01/ijg-swings-again-and-misses/).  And it 
hasn't been proven that the use of this extension for lossless encoding 
is significantly better than, for instance, PNG.  In fact, from my 
admittedly quick & dirty testing, it seems to be worse:

Using jpeg-8d:
> time ./cjpeg -rgb -block 1 -arithmetic <~/test_images/nightshot_iso_100.ppm >nightshot_iso_100.jpg
real	0m7.025s
user	0m6.468s
sys	0m0.087s
Original size: 22127633, Compressed size: 7537012, Ratio: 2.936

Using the new color transform in jpeg-9:
> time ./cjpeg -rgb1 -block 1 -arithmetic <~/test_images/nightshot_iso_100.ppm >nightshot_iso_100.jpg
real	0m7.316s
user	0m6.753s
sys	0m0.081s
Original size: 22127633, Compressed size: 7554020, Ratio: 2.929

Using ImageMagick 6.7.9 to compress as PNG:
> time convert -quality 6 ~/test_images/nightshot_iso_100.ppm nightshot_iso_100.png
real	0m2.647s
user	0m1.689s
sys	0m0.114s
Original size: 22127633, Compressed size: 7157427, Ratio: 3.092

Maybe I'm missing something?  Anyhow, until/unless there is community 
support behind SmartScale, it is unlikely that it will ever be adopted 
in libjpeg-turbo (I don't have any need for it in my own work, so it 
would pretty much have to be a funded development sort of deal.)  Thus, 
the implementation of the libjpeg v7+ API and ABI would remain just 
that:  an emulation and not a feature-complete implementation of jpeg-7 
or jpeg-8.  Without provable metrics to indicate its usefulness, it's 
unlikely that the SmartScale format will garner community support or 
gain official adoption as a standard.  I don't see any of that happening 
as long as the current IJG maintainers take the impolitic attitude that 
anyone (including the ISO standards committee) who doesn't recognize the 
brilliance of SmartScale is "corrupt" or "ignorant."

I guess what I'm saying is-- libjpeg-turbo may have reached a point at 
which there isn't really a whole lot more we can add to it feature-wise 
without either adopting the unproven SmartScale technology or diverging 
from IJG to implement some other format, like JPEG XR.  Personally, I 
feel that both would be out of scope for what is still, at the end of 
the day, a turbo baseline JPEG library.  I've always believed that new 
formats should be implemented by a new library.  The libjpeg API is 
dated and really ill-equipped to handle new formats, which is why these 
API/ABI incompatibilities keep popping up with the IJG's software.

However, I want this project to be whatever the community wants it to 
be.  I don't think we're well-positioned to be a haven for new formats, 
but if enough people are interested in one that they want to either pay 
for the implementation or contribute code, I'm definitely open to that. 
  "Keep things the way they are" is also a perfectly acceptable answer, 
as is "continue focusing on baseline and coming up with new ways to make 
it faster."

Comments encouraged.

DRC
--end--

FYI: libjpeg 9 was released today
http://www.h-online.com/open/news/item/Libjpeg-9-improves-lossless-JPEG-compression-1783311.html