My apologies, I had accidentally dropped out of computer world for a while, and here a
world spins ahead! Thanx a lot for many valuable feedback!
see the clarifications of most concerns:
Might be a copy-paste error there, the last sentence is incomplete.
ty, fixed: "We already made a heavy testing of the behavior, and user should
not face negative experience. Still I'm not sure if this testing can ever be enough,
considering all the use-cases we do not know. "
Bundled versions are always outdated and may be even vulnerable.
Not necessarily. In small project, sure, bundled libraries will get rotten, but project
like OpenJDK, where 99% of its builds uses the in tree copies, can not allow itself to
have security holes in them. It already happened in past that we
have switched to the in-tree library because the CVE there was already fixed, and moved
back to system library once the fix landed to live Fedora.
Using bundled freetype, fontconfig and harfbuzz will result in ugly
fonts (without system configuration support, including subpixel rendering, hinting, etc).
This is probably main issue we are aware about. Especially the system configuration will
be a hard issue, if solvable at all.
However IIRC, those features do nor work properly in java event hose days, as java have to
support all what lies below, and it simply can not, so the intree libraries come to play.
Note that we had to patch jdk in past to work properly after changes in the system
libraries. And that is not easy task.
Please take my last three sentences with bit of reserve, a better upstream engineer then
me should comment.
Applications linking against libraries SHOULD link against shared
libraries not static versions.
right, should. So strictly from law of guidelines, there is no rule to stop this change.
The bundling was always bad, but is necessary evil. See eg rsync - the bundled zlib is
there because it is technically not possible to use dynamic one.
Java is in the border - yes, you can use the dynamic libraries (and it really took several
excellent engineers a decade to manage), but it have pros and cons. Unluckily, the
negatives are multiplying for years. Although we are not happy
with the change, it must be done if JDKs should remain maintained and healthy in Fedora.
And upstream only incorporates security fixes once per quarter
Right, but downstream works well too. If such vulnerability occurs, be sure we will patch
RPMs asap, and also upstream project - maybe without release - will react to.
Unluckily, I'm unable to comment on the demanglers, can you elaborate more?
In this case upstream might actually get there first because this CVE
is not yet fixed in Fedora
And it already happeed in past, that jdk swithced to bundled version for a while, and back
to dynamic once the library was fixed in live Fedora.
This sounds fascinating. Can anyone share details about this? On the
I'm aware of some codecs, which are built in Fedora, then the binary is sent to
.. cisco(?), and if passed, they are repacked into all live fedoras.
As super cornercase of this may be packing of some firmwares as binary blobs.
versions in an RPM sounds like it would violate all the guidelines
$foo to meet this threshold and claim it needs to avoid rebuilding for certification?
Right, you need fesco exception.
I'm very confused on why this reduces certification burden. In
our crypto libraries this is exactly the kind of behavior we would *NOT* want packages
to do because it increases our certification and support burden.
I believe this are
First - our burden. We ahve to certify each binary. This is quite long and lenghty
process. Onl once it is certified, we can release it (with small unwritten exception in
So with repacked portabel build as described in
will allow us to certify
once (and even stop hoping for that small exception in rawhide). Considering 4 jdks in 3
fedoras, that is incredible work. If it reduced to 4 we will manage again.
Second - I can understand that you certify your libray, but if others bundle it, then it
is theirs choice, and they are on thier own, arent they? Also you highlight crypto lib, we
are not going to embed any crypto library. Crypto is loaded
in runtime, and build time have no idea about future system in this case. Can you please
elaborate more if still needed?
I'm confused how this would not negatively impact the user
experience, because things like FreeType and HarfBuzz in Fedora have features and
configuration that are non-default that improve the font rendering capabilities of
that link to FreeType.
As explained above. This is knwon issue. I was naively considereding it as minor, as it
nearly non-fixable, and to my half blind eyes some subixels and shadows says nothing. (/me
on non patched fluxbox, which I guess explains all). This
is also how I wrote it to the
. As I
realy ahve only vague knowledge of
what you are speaking about, can you please ehnace those two paragraphs?
I would rather have our shared maintenance and evolution of font
stuff be reused in Java too...
This is holy grail we have been pursuing for last 10 years. But now we gave up. To keep
java alive in Fedora, we have to take this one step back.
Are people really installing OpenJDK RPM packages, taking the
"/usr/bin/java" binary, and putting it onto some other system?
No, no and
Please see the linked https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs
I cannot see how Fedora RPM packages for OpenJDK can redistributing
pre-built binaries would ever be considered acceptable.
no, no and no again, Please
see the linked https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs
In long run, we will build in koji once, and repack this to all live fedoras. There alre
already prcednets, and even worse things - like packing blobish binary driver, are here.
I guess you noticed, that it was clarified already in the discussion. Thanx a lot for
highlighting this though.
I agree, I don't think there's positives for the user
experience here. And I don't understand what actual problem this change is trying to
As described above, keeping dynamic jdk proeprly working requires huge maintainance. In
addition, jdks are expected to be static (only distribution rpms do differently, and thats
why developers often unpack 3rd party static jdk to develop
on), and have futures like generating standalone binary native images and so, which are
impossible with dynamic linking. Future of java looks pointing pretty stright forward to
such changes, so we have to move to.
When the change talks about being "portable", my
understanding is that means within the context of Fedora. eg they want to be able to build
the JDK version in a Fedora 35 build root once, and then ship that built binary in Fedora
Fedora 36 and Fedora rawhide (37) by just tagging the F35 build into the newer
Fedora koji tags too, instead of rebuilding for each Fedora version.
Exactly. With one addition - the jdk will be indeed portable, you should be able to bring
it to most of the system,and it should work. Still glibc limitations remains here. To
stand on the safe ground, "usual" non distribution providers of
openjdk simply builds on oldest possible supported system.
Also there si one more advantage - the native images you produce form suchjdk will be
trully portable. Unlike now, when they are simply unusable. I know distro-based people
dont like this. But that si current state of world. By hart I'm
distribution person living on Fedora since Core 4 with few steps to other distros, but as
java developer I;m rather distributing final self contianed blob to custommmer so he have
everything in place. So despite being java maintainer for
decade, I' have to unpack and use 3rd party jdk much more often then I would like.
This case appears different in that Fedora is still building the
binaries in one release stream,
Right. Thanx for describing it.
One downside with building in F35 and then re-tagging into newer
Fedora releases, is that we loose any insight into problems coming down the pipe.
Currently a Fedora rawhide mass rebuild will often highlight pre-existing bugs in
applications, and/or highlight bugs in newly rebased GCC toolchain....Detecting bugs
early in both applications and GCC toolchain, via builds in rawhide...
Right you are, and it is already described in
We are very well aware of this benefit, and we will do our best to keep it running - eg to
build portbales also in rawhide, although not to repack them. Just for this single - but
huge - case.
We deeply agree with your description of rawhide and gcc bumps. Thanx a lot for
This is so bad. The final death of Java on Fedora.
Actually just a opposite. This is future of java and without it java in fedora my diminish
and fade away. I personally really do not like this change, and I agree with all rebukes
taken here. But current OpenJDK maintainers (which are the
same for last decade) do not see other way.
If it will really go bad, we will withdraw and continue fighting.
I consider this a step entirely in the wrong direction. The JDK
should be linked to system libraries wherever possible just like our other packages.
Language interpreters/JITs are not exempt from that. In fact, I see very little value
in providing JDK packages at all if they are built that way.
Where you are right with the interpreters/jits not being exempt, I'm afraid yo do not
see current flow of java - to self contained native images, and to share OpenJDK
marketpalce. Even if wesomhow will be able to pathc jdk to live with
system libraries, which is already close to impossible, and drop all features which
allows generate portable images, the problem with the certification will persists. It
isnot in human powers to maintain 4 JDKS in 3 Fedoras.
And this plan is entirely unacceptable. It is just plain not allowed
in Fedora to ship prebuilt binary blobs (even if they are built by Fedora developers)
This is wrong. The JDK will be always build from sources in koji. The main reduce of load
will be that webuilt once in koji, in oldest Fedora, and repack to newer.
In additon there are many excludes in various binary drivers.
If passing the TCK is such an issue, then please just go back to
shipping the packages under the name IcedTea or some other name not trademarked by
It is not as simple. You are right that to call it java yoou must pass tck. But unluckily
even IcedTea had to pass TCK. Maybe it is possible to patch out all "java" from
it, but I doubt jdk will remain working and usefull,.
No way. This violates Fedora policy. All packages **must** be built
from sources. No exceptions.
Quite a few packages are dleivered as blobs... Still. Be sure we are NOT going to do that.
We will continue to build from sources, in koji, and with luck only repack build form
latest stable Fedora to all,
Fedora 38. So the decision on whether to approve this first feature
really needs to consider the acceptability of the overall plan, not just this first step
You are right, that the https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs
is the key, but by aproving this, the whole chain isnotyet there. Especially if we found
that the static jdks are really not viable, then this
) proposal will be
reverted, and the https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs
But to be able to move forward, we have to try. If we dont, it will not know we are no
wrong track, and thus be unabel to try to figure backup plan.
IIUC, it is implied that when a new JDK comes out (jdk19 next), it
would be added immediately to all Fedora streams (35, 36, 37rawhide).
Yes, jdk 19 will be added to all fedoras. But not as new package - it will replace jdk18
inside java-latest-openjdk. jdk19 is STS, simialrly as jdk18 was. Once next LTS is out
jdk-21 in 2024, I hope we wil finally drop jdk8, so we wills
stay in 4 jdks.
introduce new JDKs to all existing Fedora streams, only add it to
rawhide so certification is only needed once.
That will unluckily not help. there would remain previous JDK in older Fedoras. In
addition, EOLed one.
certification; form same email as abvoe two huks, from Daniel P.
+ Why do we need such certification? Fedora is a separate distribution, not related to
Oracle at all.
To call build of OpenJDK java, each binary ahve to pas TCK(as mentioned in thread). And
you have to do it on each binary you produce. So with dynamic lining in fedoras, I have to
certifie 12+ builds.
And update once they pass (with small exception in rawhide). With portable build, I can
build once, TCK it eg on super custom opensuse, and still do update (from this binary) to
TCK - are testing if all public apis forks as expected. Despite some tck are really weird,
and to pass we eg have to have balck background and shadows disabled and so, the TCK are
pretty good testsuite. Corenrcases are in other testssuites,
which we run too. As our build is based on OpenJDK with Hotspot, we do not need to prove
it, but must provide once asked.
Also the "one would have thought that any time a dependency got an update it would
invalidate certification too, even right down to any glibc update, or even kernel update
" is close to reality. But we really have to pass the binary on ANY
system. Including the one before release. So luckily not.
Is there an actual contractual requirement for Fedora to distribute
OpenJDK builds only after they have passed the TCK? That's just impossible with the
Fedora build system, and we would have to remove OpenJDK from Fedora to comply.
I'm not sure I follow. Why would we need to remove OpenJDK from fedora to comply? The
only issue I see is the Rawhide. But considering it is not officially released.. It should
Each build we pass to updates, we are bound to prove it passes TCK. As we control
environment, we are usually able to do so.
Runing them is mandatory. The results reporting is on demand. The simple rename is not so
easy. I "m afraid that striping all "java" from bnaries and thus from
sources is maybe possbel to be done, but human impossible to maintian, and
will leave JDK mostly not working, and for sure useless and incomaptible with other
One of the side steps of this proposal is to be more compatible with other javas.
If current maintainers can't continue maintaining the
well-packaged OpenJDK, I think it's time to retire it. It would be better
we can no longer maintian it. And I must contradict you - Seeing all the dependencies, we
really need them all. Loosing any one, wiould kill java ecosystemin Fedora.
You do not need to rebuild OpenJDK after the system library update
We rebuild so often that this one is irrelevant.
That would also mean that the JDKs would lag any other Fedora
system-wide changes, such as compiler/library updates, build options that affect security,
Well, yes. But except GCC which we will try to support as writen above, and in
But please not, that that is very common troubles for us. We do adapt to fedora changes -
eg crypto policies or so - but the bumps of libraries are unnecessary burden. As those
usully need bigger patching upstream. But please takeme with
reserve - a more upstream/native libs development included persn should elaborate.
in last decade, we proved one thing - each fedora change like crypto policies or so, when
finally tuned in JDK, was possible to backport to older fedoras. However changes due to
shared libs, not.
F35, F36, and rawhide need to be updated to a JDK built on F35...
which would mean that F36 gets released with an immediately obsolete JDK, to be replaced
with an untested (in the general sense, didn't go through OS beta and such) build.
Interesting point of view. Tyvm for it!
If the binary is identical, then what can go wrong? Already now tests portable comunity
builds on all live fedoras, and a distribution-specific issue is so rare, that I think
there was none even on long run.
Still I see yor point and possible issues you suggests out. Trhough tuccrent plan, your
assumption "F36, and rawhide need to be updated to a JDK built on F35." is
However the "t F36 gets released with an immediately obsolete JDK, to be replaced
with an untested (in the general sense, didn't go through OS beta and such)"
should not be correct.
There is ususaly no need to update immediately. It is actually even not desired. And here
is few weeks where original builder Fedora is still alive, and we can stay on it for a
while. But sooner or later the new "untested" from your point
of view Jdk will get there. The onl assurance it careful testing of Openjdk maintianers.
To make your sleep a bit easier,:
gJobsAllOtool | grep -e f35 -e f34 -e f36 | wc -l
We run 2736 testsuites jenkins jobs on Fedoras. From those,
gJobsAllOtool | grep -e f35 -e f34 -e f36 | grep tck | wc -l
197 are TCK (which each includes over 100k of tests) jenkins jobs, as there are several
architectures, and even several variants of JDK which we wish to esnure. I expect the
testing burden to drop to 1/2, without actually loosing necessary
coveragein matrix, after the whole
Or would rawhide get a build built on (oldest+1) rather than
(oldest)? So F36 would be tested and released with a build built on ...
Interesting idea. Tyvm for it!
No, I was considering simply latest live one. But no imediate repack is needed once it is
eoled. We can stay for a bit on eoled one, or not update from new oldest. In all cases the
heavy testing will preceded
In your description. It is indeed a bit of confusing
> JDKs from other vendors(Amazon, Azul, Oracle, etc.) are built in
exactly this way. W
Because they build it for all available GNU/Linux distributions. We shouldn't focus
on bad practices.
Well and there are many reasons to do so, as jdk is designed to be like this. for 10
years we tried to go against the main stream, and we managed a lot. But to keep peace and
compatibility and proper developer's support, we have to move
with mian stream now. The divergences we keep in rpms are right now blocking devloeprs to
use system jdsk as proper JDKs and are enforcing them to download Amazon, Azul, Oracle,
To keep Fedora competitive, this is currenlty necessary step.
I hope I addressed all,
Thanx a lot for contributions!
Jiri Vanek Mgr.
Principal QA Software Engineer
Red Hat Inc.
+420 775 39 01 09