F39 Change Proposal: Build JDKs once, repack everywhere (System-Wide) - Proposal Addendum
https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#including_...)
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.
== Summary ==
This is the last step in https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs effort. JDKs in fedora are already static, and we repack portable tarballs into RPMs. Currently, the portable tarball is built for each Fedora and EPEL version. Goal here is to build each JDK (8,11,17,21,latest (20)) only once, in oldest live Fedora repack in all live Fedoras. If jdk is buitl in epel, it will be built in oldest possible epel and repacked in newer live epels.
== Owner == * Name: [[User:jvanek| Jiri Vanek]]
* Email: jvanek@redhat.com
== Detailed Description ==
As described in https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs ; during last year, packaging of JDKs had changed dramatically. As described in the same wiki page and in individual sub changes and devel threads, the primary reason for this is to lower maintenance and still keep Fedora Java friendly.
* In the first system wide change, we have changed the JDKs to build properly as standalone, portable JDK - the way JDK is supposed to be built. I repeat, we spent ten years by patching JDK to become properly dynamic against system libs, and all patches went upstream, but this has become a fight which can not be won.
* As a second step we introduced portable RPMs, which do not have any system integration, only build JDK and pack the final tarball in RPM for Fedora use.
* In third step - without any noise, just verified with fesco - https://pagure.io/fesco/issue/2907 - we stopped building JDK in fully integrated RPMs. Instead of this, normal RPMs BUildRequire portable RPMs and just unpack it, and repack it.
Now last step is ahead - to build portable LTS JDKs 8,11,17 and 21 in oldest live Fedora, and repack everywhere. java-latest-openjdk, which contains latests STS JDK - currently 20, soon briefly 21 and a bit after 22... If we would built java-latest-openjdk in oldest live EPEL - epel8 now, we have verified, that such repacked JDKs works fine, however repack from epel seem to not be acceptable, thus ajva-latest-openjdk will be built twice - one in oldest live fedora, and once in oldest live epel. Build forme oldest possible epel will be repacked to that one or newer epels, and build from oldest live fedroa to all fedoras.
=== theoretical tagging solution ===
1. request side tags for all releases 2. build the actual Java in the side tag for the oldest thing 3. tag the result ot (2) to all side tags from (1) 4. waitrepo them 5. build the repacked java packages in all the side tags from (1) 6. untag the result of (2) from all the side tags from (1) 7. ship bodhi updates from side tags OR retag the builds to candidate tags (and delete the side tags)
The build from (2) will be eventually garbage collected. To prevent that, it might be re-tagged regularly. This is where releng might be able to help by creating a long lived tag to tag this into for preserving.
Yes, we could make a 'fN-openjdk' tag and mark it protected... that part would be easy enough.
==== including portable srpms in release (improving of step 6) ====
To include portable rpms in all live Fedoras is currently not possible. Best solution would be simply make and bodhi update of one portable rpm to all live fedoras. Bodhi is currenlty not capable to do so, issue was raised: https://github.com/fedora-infra/bodhi/issues/5387 investigating possibility to deliver single build as update to several releases.
"..It's not possible ATM, it would require a heavy rewrite of the code, starting from the database structure (every build is now related to a single release)..." Maybe on long run..."
On long run, if bodhi will allow this, that will be way to go. On short run, there are following options: a) ask releng to tag the portables directly - this needs manual approach of rare humans, thus no go unless strictly enforced by unpredicted conditions - this walks around whole testing repos. For portables tarballs, as nothing should depend on them, and are tested indirectly after repack, this should be technically ok, but is heavily discouraged in principle. b) build portable for all OSes, but do not ship them (don't do bodhi update) - this would probably work for all frontiers, only the real repacked JDK will be different - pros is, that we will be sure that portables builds on live fedoras - cons is, that the portable JDK will not be available by dnf install anyway c) build portable for all OSes, including bodhi update - pros is, that we will be sure that portables builds on live fedoras - another pros is that the portable JDK will be available by dnf install anyway - there may be clash during the build which will cause to repack wrong (newer, non certified) portables d) include SRPM_REBUILD.readme in srpm and generated PORTABLES_INSTALL.readme in RPMs, which will ideally at least contain: - instruction why you need portables - instruction how to find the portables - from SRPM_REBUILD.readme pointing to PORTABLES_INSTALL.readme - generated link to the koji, allowing to download the SRPM - generated link to the koji, allowing to download the binaries - generated instruction how to dnf install used portables
I would currently vote for d). If there will be complains about broken SRPM rebuild, or need to install portables without hacking, then fall-back a, b or c via Change Proposal. Once Bodhi allows single build to be tagged to several release, I will move to that.
== Feedback ==
== Benefit to Fedora ==
Java maintainers will finally have some free time... No kidding - maintenance and *certification* of so much supported JDKs on so much Fedora versions is brutal. By building once, and repack, we will regain cycles to continue support Fedora with all LTS and one STS JDK.
If we fail to build once and repack everywhere, Java maintainers will most likely need to lower the number of JDKs in fedora to system one only.
== Scope == * Proposal owners: Technically all JDKs (except 8, where some more tuning is needed, and EPEL for java-latest) are prepared, as they have a portable version, and RPMs just repack it. Except tuning up the JDK8 and EPEL for latest, scope owners are done.
* Other developers: There will be needed significant support from RCM and maybe senior Fedora leadership to help to finish the build in oldest and enable to repack everywhere
* '''Release engineering: [https://pagure.io/releng/issue/11438 #11438]''' There will be needed significant support from RCM, where I'm actually unsure what they will have to do to enable this. The mas rebuild will not be needed.
* Policies and guidelines: AFAIK none (not needed for this Change) * Trademark approval: N/A (not needed for this Change)
* Alignment with Community Initiatives: All supported JDKs will remain in Fedora in highest possible quality with full QA and certification, and its packagers will not lose their minds. Note that QA will still run on all live Fedoras, not only on the builder one.
== Upgrade/compatibility impact ==
The change should be completely transparent to any user.
== How To Test ==
`sudo dnf update/install "java*"` will install expected set of working packages.
SRPM rebuild of both portables (which were built once) and of any rpms (from this freshly rebuild portbales) have to remain possible
== User Experience == The change should be absolutely transparent to any user.
== Dependencies == To finish this we will need heavy support from RCM, and maybe others. Although there are precedents with such pacakge, they all bites. From SW point of view, the dependece chain is `normal RPMs build requires portable RPMs` and thats all.
== Contingency Plan == * Contingency mechanism: Even if It should be straight forward to revert back to building per OS, it '''may be impossible for current maintainers to save time''' for it. If this change is approved, we will be building '''4-5''' (jdk8,11,17,sts and 21) builds for all fedoras. If this change is not finished in time, we may '''need to orphan some of the JDKs'''. In better case, we will be able to keep living '''one LTS as system JDK, and java-latest-openjdk''' as future system JDK. That is 2*(3-5) builds (rawhide, (forked,), latest live, oldest live (oldest not yet dropped)). '''In worst case''', we may be able to maintain only java-latest-openjdk. On long run changing it to '''rolling system JDK,''' which are the expected 3-5 builds. * Contingency deadline: N/A * Blocks release? No. The change can be introduced even on the fly to live distributions.
== Documentation ==
N/A (not a System Wide Change)
== Release Notes ==
On 6/26/23 12:21, Aoife Moloney wrote:
https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#including_...)
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.
== Summary ==
This is the last step in https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs effort. JDKs in fedora are already static, and we repack portable tarballs into RPMs. Currently, the portable tarball is built for each Fedora and EPEL version. Goal here is to build each JDK (8,11,17,21,latest (20)) only once, in oldest live Fedora repack in all live Fedoras. If jdk is buitl in epel, it will be built in oldest possible epel and repacked in newer live epels.
== Owner ==
Name: [[User:jvanek| Jiri Vanek]]
Email: jvanek@redhat.com
== Detailed Description ==
As described in https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs ; during last year, packaging of JDKs had changed dramatically. As described in the same wiki page and in individual sub changes and devel threads, the primary reason for this is to lower maintenance and still keep Fedora Java friendly.
- In the first system wide change, we have changed the JDKs to build
properly as standalone, portable JDK - the way JDK is supposed to be built. I repeat, we spent ten years by patching JDK to become properly dynamic against system libs, and all patches went upstream, but this has become a fight which can not be won.
- As a second step we introduced portable RPMs, which do not have any
system integration, only build JDK and pack the final tarball in RPM for Fedora use.
- In third step - without any noise, just verified with fesco -
https://pagure.io/fesco/issue/2907 - we stopped building JDK in fully integrated RPMs. Instead of this, normal RPMs BUildRequire portable RPMs and just unpack it, and repack it.
Now last step is ahead - to build portable LTS JDKs 8,11,17 and 21 in oldest live Fedora, and repack everywhere. java-latest-openjdk, which contains latests STS JDK - currently 20, soon briefly 21 and a bit after 22... If we would built java-latest-openjdk in oldest live EPEL
- epel8 now, we have verified, that such repacked JDKs works fine,
however repack from epel seem to not be acceptable, thus ajva-latest-openjdk will be built twice - one in oldest live fedora, and once in oldest live epel. Build forme oldest possible epel will be repacked to that one or newer epels, and build from oldest live fedroa to all fedoras.
=== theoretical tagging solution ===
- request side tags for all releases
- build the actual Java in the side tag for the oldest thing
- tag the result ot (2) to all side tags from (1)
- waitrepo them
- build the repacked java packages in all the side tags from (1)
- untag the result of (2) from all the side tags from (1)
- ship bodhi updates from side tags OR retag the builds to candidate tags (and delete the side tags)
The build from (2) will be eventually garbage collected. To prevent that, it might be re-tagged regularly. This is where releng might be able to help by creating a long lived tag to tag this into for preserving.
Yes, we could make a 'fN-openjdk' tag and mark it protected... that part would be easy enough.
==== including portable srpms in release (improving of step 6) ====
To include portable rpms in all live Fedoras is currently not possible. Best solution would be simply make and bodhi update of one portable rpm to all live fedoras. Bodhi is currenlty not capable to do so, issue was raised: https://github.com/fedora-infra/bodhi/issues/5387 investigating possibility to deliver single build as update to several releases.
"..It's not possible ATM, it would require a heavy rewrite of the code, starting from the database structure (every build is now related to a single release)..." Maybe on long run..."
On long run, if bodhi will allow this, that will be way to go. On short run, there are following options: a) ask releng to tag the portables directly
- this needs manual approach of rare humans, thus no go unless
strictly enforced by unpredicted conditions
- this walks around whole testing repos. For portables tarballs, as
nothing should depend on them, and are tested indirectly after repack, this should be technically ok, but is heavily discouraged in principle. b) build portable for all OSes, but do not ship them (don't do bodhi update)
- this would probably work for all frontiers, only the real
repacked JDK will be different
- pros is, that we will be sure that portables builds on live fedoras
- cons is, that the portable JDK will not be available by dnf install anyway
c) build portable for all OSes, including bodhi update
- pros is, that we will be sure that portables builds on live fedoras
- another pros is that the portable JDK will be available by dnf
install anyway
- there may be clash during the build which will cause to repack
wrong (newer, non certified) portables d) include SRPM_REBUILD.readme in srpm and generated PORTABLES_INSTALL.readme in RPMs, which will ideally at least contain:
- instruction why you need portables
- instruction how to find the portables
- from SRPM_REBUILD.readme pointing to PORTABLES_INSTALL.readme
- generated link to the koji, allowing to download the SRPM
- generated link to the koji, allowing to download the binaries
- generated instruction how to dnf install used portables
I would currently vote for d). If there will be complains about broken SRPM rebuild, or need to install portables without hacking, then fall-back a, b or c via Change Proposal. Once Bodhi allows single build to be tagged to several release, I will move to that.
== Feedback ==
== Benefit to Fedora ==
Java maintainers will finally have some free time... No kidding - maintenance and *certification* of so much supported JDKs on so much Fedora versions is brutal. By building once, and repack, we will regain cycles to continue support Fedora with all LTS and one STS JDK.
What is the reason certification is so hard in terms of human (as opposed to machine) time? If the TCK fails intermittently then that needs to be fixed in the TCK or OpenJDK. It seems to me that an upstream OpenJDK release should not happen unless the TCK passes, and if it passes once it should pass everywhere. Otherwise, there is a bug somewhere, most likely upstream.
On 6/26/23 09:21, Aoife Moloney wrote:
https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#including_...)
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.
== Summary ==
This is the last step in https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs effort. JDKs in fedora are already static, and we repack portable tarballs into RPMs. Currently, the portable tarball is built for each Fedora and EPEL version. Goal here is to build each JDK (8,11,17,21,latest (20)) only once, in oldest live Fedora repack in all live Fedoras. If jdk is buitl in epel, it will be built in oldest possible epel and repacked in newer live epels.
== Owner ==
Name: [[User:jvanek| Jiri Vanek]]
Email: jvanek@redhat.com
== Detailed Description ==
As described in https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs ; during last year, packaging of JDKs had changed dramatically. As described in the same wiki page and in individual sub changes and devel threads, the primary reason for this is to lower maintenance and still keep Fedora Java friendly.
- In the first system wide change, we have changed the JDKs to build
properly as standalone, portable JDK - the way JDK is supposed to be built. I repeat, we spent ten years by patching JDK to become properly dynamic against system libs, and all patches went upstream, but this has become a fight which can not be won.
- As a second step we introduced portable RPMs, which do not have any
system integration, only build JDK and pack the final tarball in RPM for Fedora use.
- In third step - without any noise, just verified with fesco -
https://pagure.io/fesco/issue/2907 - we stopped building JDK in fully integrated RPMs. Instead of this, normal RPMs BUildRequire portable RPMs and just unpack it, and repack it.
Now last step is ahead - to build portable LTS JDKs 8,11,17 and 21 in oldest live Fedora, and repack everywhere. java-latest-openjdk, which contains latests STS JDK - currently 20, soon briefly 21 and a bit after 22... If we would built java-latest-openjdk in oldest live EPEL
- epel8 now, we have verified, that such repacked JDKs works fine,
however repack from epel seem to not be acceptable, thus ajva-latest-openjdk will be built twice - one in oldest live fedora, and once in oldest live epel. Build forme oldest possible epel will be repacked to that one or newer epels, and build from oldest live fedroa to all fedoras.
=== theoretical tagging solution ===
- request side tags for all releases
- build the actual Java in the side tag for the oldest thing
Could you use the real package name here. I think that will make it easier to understand. You can still put 'actual Java' in parens or something.
- tag the result ot (2) to all side tags from (1)
- waitrepo them
- build the repacked java packages in all the side tags from (1)
Same thing here, can you use the real package name.
- untag the result of (2) from all the side tags from (1)
- ship bodhi updates from side tags OR retag the builds to candidate tags (and delete the side tags)
The build from (2) will be eventually garbage collected. To prevent that, it might be re-tagged regularly. This is where releng might be able to help by creating a long lived tag to tag this into for preserving.
Yes, we could make a 'fN-openjdk' tag and mark it protected... that part would be easy enough.
I think this process could be improved if the side-tags in 1. are permanent side-tags, and step 6 is skipped. That would make it easier to rebuild the packages from step 5 if needed.
Also, can you put a config file in the dist-git repos to tell fedpkg which target to build against? I thought I remembered seeing that feature in the past. If so, then you could configure the dist-gits for the repacked javas to automatically build from those side-tags, which I think would be a lot easier for package maintainers and may help make automated rebuilds possible.
-Tom
==== including portable srpms in release (improving of step 6) ====
To include portable rpms in all live Fedoras is currently not possible. Best solution would be simply make and bodhi update of one portable rpm to all live fedoras. Bodhi is currenlty not capable to do so, issue was raised: https://github.com/fedora-infra/bodhi/issues/5387 investigating possibility to deliver single build as update to several releases.
"..It's not possible ATM, it would require a heavy rewrite of the code, starting from the database structure (every build is now related to a single release)..." Maybe on long run..."
On long run, if bodhi will allow this, that will be way to go. On short run, there are following options: a) ask releng to tag the portables directly - this needs manual approach of rare humans, thus no go unless strictly enforced by unpredicted conditions - this walks around whole testing repos. For portables tarballs, as nothing should depend on them, and are tested indirectly after repack, this should be technically ok, but is heavily discouraged in principle. b) build portable for all OSes, but do not ship them (don't do bodhi update) - this would probably work for all frontiers, only the real repacked JDK will be different - pros is, that we will be sure that portables builds on live fedoras - cons is, that the portable JDK will not be available by dnf install anyway c) build portable for all OSes, including bodhi update - pros is, that we will be sure that portables builds on live fedoras - another pros is that the portable JDK will be available by dnf install anyway - there may be clash during the build which will cause to repack wrong (newer, non certified) portables d) include SRPM_REBUILD.readme in srpm and generated PORTABLES_INSTALL.readme in RPMs, which will ideally at least contain: - instruction why you need portables - instruction how to find the portables - from SRPM_REBUILD.readme pointing to PORTABLES_INSTALL.readme - generated link to the koji, allowing to download the SRPM - generated link to the koji, allowing to download the binaries - generated instruction how to dnf install used portables
I would currently vote for d). If there will be complains about broken SRPM rebuild, or need to install portables without hacking, then fall-back a, b or c via Change Proposal. Once Bodhi allows single build to be tagged to several release, I will move to that.
== Feedback ==
== Benefit to Fedora ==
Java maintainers will finally have some free time... No kidding - maintenance and *certification* of so much supported JDKs on so much Fedora versions is brutal. By building once, and repack, we will regain cycles to continue support Fedora with all LTS and one STS JDK.
If we fail to build once and repack everywhere, Java maintainers will most likely need to lower the number of JDKs in fedora to system one only.
== Scope ==
- Proposal owners: Technically all JDKs (except 8, where some more
tuning is needed, and EPEL for java-latest) are prepared, as they have a portable version, and RPMs just repack it. Except tuning up the JDK8 and EPEL for latest, scope owners are done.
- Other developers: There will be needed significant support from RCM
and maybe senior Fedora leadership to help to finish the build in oldest and enable to repack everywhere
- '''Release engineering: [https://pagure.io/releng/issue/11438
#11438]''' There will be needed significant support from RCM, where I'm actually unsure what they will have to do to enable this. The mas rebuild will not be needed.
Policies and guidelines: AFAIK none (not needed for this Change)
Trademark approval: N/A (not needed for this Change)
Alignment with Community Initiatives: All supported JDKs will remain
in Fedora in highest possible quality with full QA and certification, and its packagers will not lose their minds. Note that QA will still run on all live Fedoras, not only on the builder one.
== Upgrade/compatibility impact ==
The change should be completely transparent to any user.
== How To Test ==
`sudo dnf update/install "java*"` will install expected set of working packages.
SRPM rebuild of both portables (which were built once) and of any rpms (from this freshly rebuild portbales) have to remain possible
== User Experience == The change should be absolutely transparent to any user.
== Dependencies == To finish this we will need heavy support from RCM, and maybe others. Although there are precedents with such pacakge, they all bites. From SW point of view, the dependece chain is `normal RPMs build requires portable RPMs` and thats all.
== Contingency Plan ==
- Contingency mechanism: Even if It should be straight forward to
revert back to building per OS, it '''may be impossible for current maintainers to save time''' for it. If this change is approved, we will be building '''4-5''' (jdk8,11,17,sts and 21) builds for all fedoras. If this change is not finished in time, we may '''need to orphan some of the JDKs'''. In better case, we will be able to keep living '''one LTS as system JDK, and java-latest-openjdk''' as future system JDK. That is 2*(3-5) builds (rawhide, (forked,), latest live, oldest live (oldest not yet dropped)). '''In worst case''', we may be able to maintain only java-latest-openjdk. On long run changing it to '''rolling system JDK,''' which are the expected 3-5 builds.
- Contingency deadline: N/A
- Blocks release? No. The change can be introduced even on the fly to
live distributions.
== Documentation ==
N/A (not a System Wide Change)
== Release Notes ==
Hi Tom!
Thanx a lot of for input. Although I did my bes with the tagging, it will be learning on the go. I had adapted https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretica... as you suggested. It is great improvement.
Especially the config, I was not aware about. That woudl indeed help a lot. The usage of pernament tag is someging I have to learn, but is moreover necessary for proper srpm rebuil.
TYVM! J.
On Wed, 28 Jun 2023 at 21:31, Tom Stellard tstellar@redhat.com wrote:
On 6/26/23 09:21, Aoife Moloney wrote:
https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#including_...)
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.
== Summary ==
This is the last step in https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs effort. JDKs in fedora are already static, and we repack portable tarballs into RPMs. Currently, the portable tarball is built for each Fedora and EPEL version. Goal here is to build each JDK (8,11,17,21,latest (20)) only once, in oldest live Fedora repack in all live Fedoras. If jdk is buitl in epel, it will be built in oldest possible epel and repacked in newer live epels.
== Owner ==
Name: [[User:jvanek| Jiri Vanek]]
Email: jvanek@redhat.com
== Detailed Description ==
As described in https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs ; during last year, packaging of JDKs had changed dramatically. As described in the same wiki page and in individual sub changes and devel threads, the primary reason for this is to lower maintenance and still keep Fedora Java friendly.
- In the first system wide change, we have changed the JDKs to build
properly as standalone, portable JDK - the way JDK is supposed to be built. I repeat, we spent ten years by patching JDK to become properly dynamic against system libs, and all patches went upstream, but this has become a fight which can not be won.
- As a second step we introduced portable RPMs, which do not have any
system integration, only build JDK and pack the final tarball in RPM for Fedora use.
- In third step - without any noise, just verified with fesco -
https://pagure.io/fesco/issue/2907 - we stopped building JDK in fully integrated RPMs. Instead of this, normal RPMs BUildRequire portable RPMs and just unpack it, and repack it.
Now last step is ahead - to build portable LTS JDKs 8,11,17 and 21 in oldest live Fedora, and repack everywhere. java-latest-openjdk, which contains latests STS JDK - currently 20, soon briefly 21 and a bit after 22... If we would built java-latest-openjdk in oldest live EPEL
- epel8 now, we have verified, that such repacked JDKs works fine,
however repack from epel seem to not be acceptable, thus ajva-latest-openjdk will be built twice - one in oldest live fedora, and once in oldest live epel. Build forme oldest possible epel will be repacked to that one or newer epels, and build from oldest live fedroa to all fedoras.
=== theoretical tagging solution ===
- request side tags for all releases
- build the actual Java in the side tag for the oldest thing
Could you use the real package name here. I think that will make it easier to understand. You can still put 'actual Java' in parens or something.
- tag the result ot (2) to all side tags from (1)
- waitrepo them
- build the repacked java packages in all the side tags from (1)
Same thing here, can you use the real package name.
- untag the result of (2) from all the side tags from (1)
- ship bodhi updates from side tags OR retag the builds to candidate tags (and delete the side tags)
The build from (2) will be eventually garbage collected. To prevent that, it might be re-tagged regularly. This is where releng might be able to help by creating a long lived tag to tag this into for preserving.
Yes, we could make a 'fN-openjdk' tag and mark it protected... that part would be easy enough.
I think this process could be improved if the side-tags in 1. are permanent side-tags, and step 6 is skipped. That would make it easier to rebuild the packages from step 5 if needed.
Also, can you put a config file in the dist-git repos to tell fedpkg which target to build against? I thought I remembered seeing that feature in the past. If so, then you could configure the dist-gits for the repacked javas to automatically build from those side-tags, which I think would be a lot easier for package maintainers and may help make automated rebuilds possible.
-Tom
==== including portable srpms in release (improving of step 6) ====
To include portable rpms in all live Fedoras is currently not possible. Best solution would be simply make and bodhi update of one portable rpm to all live fedoras. Bodhi is currenlty not capable to do so, issue was raised: https://github.com/fedora-infra/bodhi/issues/5387 investigating possibility to deliver single build as update to several releases.
"..It's not possible ATM, it would require a heavy rewrite of the code, starting from the database structure (every build is now related to a single release)..." Maybe on long run..."
On long run, if bodhi will allow this, that will be way to go. On short run, there are following options: a) ask releng to tag the portables directly - this needs manual approach of rare humans, thus no go unless strictly enforced by unpredicted conditions - this walks around whole testing repos. For portables tarballs, as nothing should depend on them, and are tested indirectly after repack, this should be technically ok, but is heavily discouraged in principle. b) build portable for all OSes, but do not ship them (don't do bodhi update) - this would probably work for all frontiers, only the real repacked JDK will be different - pros is, that we will be sure that portables builds on live fedoras - cons is, that the portable JDK will not be available by dnf install anyway c) build portable for all OSes, including bodhi update - pros is, that we will be sure that portables builds on live fedoras - another pros is that the portable JDK will be available by dnf install anyway - there may be clash during the build which will cause to repack wrong (newer, non certified) portables d) include SRPM_REBUILD.readme in srpm and generated PORTABLES_INSTALL.readme in RPMs, which will ideally at least contain: - instruction why you need portables - instruction how to find the portables - from SRPM_REBUILD.readme pointing to PORTABLES_INSTALL.readme - generated link to the koji, allowing to download the SRPM - generated link to the koji, allowing to download the binaries - generated instruction how to dnf install used portables
I would currently vote for d). If there will be complains about broken SRPM rebuild, or need to install portables without hacking, then fall-back a, b or c via Change Proposal. Once Bodhi allows single build to be tagged to several release, I will move to that.
== Feedback ==
== Benefit to Fedora ==
Java maintainers will finally have some free time... No kidding - maintenance and *certification* of so much supported JDKs on so much Fedora versions is brutal. By building once, and repack, we will regain cycles to continue support Fedora with all LTS and one STS JDK.
If we fail to build once and repack everywhere, Java maintainers will most likely need to lower the number of JDKs in fedora to system one only.
== Scope ==
- Proposal owners: Technically all JDKs (except 8, where some more
tuning is needed, and EPEL for java-latest) are prepared, as they have a portable version, and RPMs just repack it. Except tuning up the JDK8 and EPEL for latest, scope owners are done.
- Other developers: There will be needed significant support from RCM
and maybe senior Fedora leadership to help to finish the build in oldest and enable to repack everywhere
- '''Release engineering: [https://pagure.io/releng/issue/11438
#11438]''' There will be needed significant support from RCM, where I'm actually unsure what they will have to do to enable this. The mas rebuild will not be needed.
Policies and guidelines: AFAIK none (not needed for this Change)
Trademark approval: N/A (not needed for this Change)
Alignment with Community Initiatives: All supported JDKs will remain
in Fedora in highest possible quality with full QA and certification, and its packagers will not lose their minds. Note that QA will still run on all live Fedoras, not only on the builder one.
== Upgrade/compatibility impact ==
The change should be completely transparent to any user.
== How To Test ==
`sudo dnf update/install "java*"` will install expected set of working packages.
SRPM rebuild of both portables (which were built once) and of any rpms (from this freshly rebuild portbales) have to remain possible
== User Experience == The change should be absolutely transparent to any user.
== Dependencies == To finish this we will need heavy support from RCM, and maybe others. Although there are precedents with such pacakge, they all bites. From SW point of view, the dependece chain is `normal RPMs build requires portable RPMs` and thats all.
== Contingency Plan ==
- Contingency mechanism: Even if It should be straight forward to
revert back to building per OS, it '''may be impossible for current maintainers to save time''' for it. If this change is approved, we will be building '''4-5''' (jdk8,11,17,sts and 21) builds for all fedoras. If this change is not finished in time, we may '''need to orphan some of the JDKs'''. In better case, we will be able to keep living '''one LTS as system JDK, and java-latest-openjdk''' as future system JDK. That is 2*(3-5) builds (rawhide, (forked,), latest live, oldest live (oldest not yet dropped)). '''In worst case''', we may be able to maintain only java-latest-openjdk. On long run changing it to '''rolling system JDK,''' which are the expected 3-5 builds.
- Contingency deadline: N/A
- Blocks release? No. The change can be introduced even on the fly to
live distributions.
== Documentation ==
N/A (not a System Wide Change)
== Release Notes ==
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On 6/29/23 09:52, Jiri Vanek wrote:
Hi Tom!
Thanx a lot of for input. Although I did my bes with the tagging, it will be learning on the go. I had adapted https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretica... as you suggested. It is great improvement.
Thanks, this looks better.
For step 5. should the first mention of java-xy-openjdk-portable actually be java-xy-openjdk ? Same question for step 7.
-Tom
Especially the config, I was not aware about. That woudl indeed help a lot. The usage of pernament tag is someging I have to learn, but is moreover necessary for proper srpm rebuil.
TYVM! J.
On Wed, 28 Jun 2023 at 21:31, Tom Stellard tstellar@redhat.com wrote:
On 6/26/23 09:21, Aoife Moloney wrote:
https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#including_...)
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.
== Summary ==
This is the last step in https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs effort. JDKs in fedora are already static, and we repack portable tarballs into RPMs. Currently, the portable tarball is built for each Fedora and EPEL version. Goal here is to build each JDK (8,11,17,21,latest (20)) only once, in oldest live Fedora repack in all live Fedoras. If jdk is buitl in epel, it will be built in oldest possible epel and repacked in newer live epels.
== Owner ==
Name: [[User:jvanek| Jiri Vanek]]
Email: jvanek@redhat.com
== Detailed Description ==
As described in https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs ; during last year, packaging of JDKs had changed dramatically. As described in the same wiki page and in individual sub changes and devel threads, the primary reason for this is to lower maintenance and still keep Fedora Java friendly.
- In the first system wide change, we have changed the JDKs to build
properly as standalone, portable JDK - the way JDK is supposed to be built. I repeat, we spent ten years by patching JDK to become properly dynamic against system libs, and all patches went upstream, but this has become a fight which can not be won.
- As a second step we introduced portable RPMs, which do not have any
system integration, only build JDK and pack the final tarball in RPM for Fedora use.
- In third step - without any noise, just verified with fesco -
https://pagure.io/fesco/issue/2907 - we stopped building JDK in fully integrated RPMs. Instead of this, normal RPMs BUildRequire portable RPMs and just unpack it, and repack it.
Now last step is ahead - to build portable LTS JDKs 8,11,17 and 21 in oldest live Fedora, and repack everywhere. java-latest-openjdk, which contains latests STS JDK - currently 20, soon briefly 21 and a bit after 22... If we would built java-latest-openjdk in oldest live EPEL
- epel8 now, we have verified, that such repacked JDKs works fine,
however repack from epel seem to not be acceptable, thus ajva-latest-openjdk will be built twice - one in oldest live fedora, and once in oldest live epel. Build forme oldest possible epel will be repacked to that one or newer epels, and build from oldest live fedroa to all fedoras.
=== theoretical tagging solution ===
1. request side tags for all releases 2. build the actual Java in the side tag for the oldest thingCould you use the real package name here. I think that will make it easier to understand. You can still put 'actual Java' in parens or something.
3. tag the result ot (2) to all side tags from (1) 4. waitrepo them 5. build the repacked java packages in all the side tags from (1)Same thing here, can you use the real package name.
6. untag the result of (2) from all the side tags from (1) 7. ship bodhi updates from side tags OR retag the builds to candidate tags (and delete the side tags)The build from (2) will be eventually garbage collected. To prevent that, it might be re-tagged regularly. This is where releng might be able to help by creating a long lived tag to tag this into for preserving.
Yes, we could make a 'fN-openjdk' tag and mark it protected... that part would be easy enough.
I think this process could be improved if the side-tags in 1. are permanent side-tags, and step 6 is skipped. That would make it easier to rebuild the packages from step 5 if needed.
Also, can you put a config file in the dist-git repos to tell fedpkg which target to build against? I thought I remembered seeing that feature in the past. If so, then you could configure the dist-gits for the repacked javas to automatically build from those side-tags, which I think would be a lot easier for package maintainers and may help make automated rebuilds possible.
-Tom
==== including portable srpms in release (improving of step 6) ====
To include portable rpms in all live Fedoras is currently not possible. Best solution would be simply make and bodhi update of one portable rpm to all live fedoras. Bodhi is currenlty not capable to do so, issue was raised: https://github.com/fedora-infra/bodhi/issues/5387 investigating possibility to deliver single build as update to several releases.
"..It's not possible ATM, it would require a heavy rewrite of the code, starting from the database structure (every build is now related to a single release)..." Maybe on long run..."
On long run, if bodhi will allow this, that will be way to go. On short run, there are following options: a) ask releng to tag the portables directly - this needs manual approach of rare humans, thus no go unless strictly enforced by unpredicted conditions - this walks around whole testing repos. For portables tarballs, as nothing should depend on them, and are tested indirectly after repack, this should be technically ok, but is heavily discouraged in principle. b) build portable for all OSes, but do not ship them (don't do bodhi update) - this would probably work for all frontiers, only the real repacked JDK will be different - pros is, that we will be sure that portables builds on live fedoras - cons is, that the portable JDK will not be available by dnf install anyway c) build portable for all OSes, including bodhi update - pros is, that we will be sure that portables builds on live fedoras - another pros is that the portable JDK will be available by dnf install anyway - there may be clash during the build which will cause to repack wrong (newer, non certified) portables d) include SRPM_REBUILD.readme in srpm and generated PORTABLES_INSTALL.readme in RPMs, which will ideally at least contain: - instruction why you need portables - instruction how to find the portables - from SRPM_REBUILD.readme pointing to PORTABLES_INSTALL.readme - generated link to the koji, allowing to download the SRPM - generated link to the koji, allowing to download the binaries - generated instruction how to dnf install used portables
I would currently vote for d). If there will be complains about broken SRPM rebuild, or need to install portables without hacking, then fall-back a, b or c via Change Proposal. Once Bodhi allows single build to be tagged to several release, I will move to that.
== Feedback ==
== Benefit to Fedora ==
Java maintainers will finally have some free time... No kidding - maintenance and *certification* of so much supported JDKs on so much Fedora versions is brutal. By building once, and repack, we will regain cycles to continue support Fedora with all LTS and one STS JDK.
If we fail to build once and repack everywhere, Java maintainers will most likely need to lower the number of JDKs in fedora to system one only.
== Scope ==
- Proposal owners: Technically all JDKs (except 8, where some more
tuning is needed, and EPEL for java-latest) are prepared, as they have a portable version, and RPMs just repack it. Except tuning up the JDK8 and EPEL for latest, scope owners are done.
- Other developers: There will be needed significant support from RCM
and maybe senior Fedora leadership to help to finish the build in oldest and enable to repack everywhere
- '''Release engineering: [https://pagure.io/releng/issue/11438
#11438]''' There will be needed significant support from RCM, where I'm actually unsure what they will have to do to enable this. The mas rebuild will not be needed.
Policies and guidelines: AFAIK none (not needed for this Change)
Trademark approval: N/A (not needed for this Change)
Alignment with Community Initiatives: All supported JDKs will remain
in Fedora in highest possible quality with full QA and certification, and its packagers will not lose their minds. Note that QA will still run on all live Fedoras, not only on the builder one.
== Upgrade/compatibility impact ==
The change should be completely transparent to any user.
== How To Test ==
`sudo dnf update/install "java*"` will install expected set of working packages.
SRPM rebuild of both portables (which were built once) and of any rpms (from this freshly rebuild portbales) have to remain possible
== User Experience == The change should be absolutely transparent to any user.
== Dependencies == To finish this we will need heavy support from RCM, and maybe others. Although there are precedents with such pacakge, they all bites. From SW point of view, the dependece chain is `normal RPMs build requires portable RPMs` and thats all.
== Contingency Plan ==
- Contingency mechanism: Even if It should be straight forward to
revert back to building per OS, it '''may be impossible for current maintainers to save time''' for it. If this change is approved, we will be building '''4-5''' (jdk8,11,17,sts and 21) builds for all fedoras. If this change is not finished in time, we may '''need to orphan some of the JDKs'''. In better case, we will be able to keep living '''one LTS as system JDK, and java-latest-openjdk''' as future system JDK. That is 2*(3-5) builds (rawhide, (forked,), latest live, oldest live (oldest not yet dropped)). '''In worst case''', we may be able to maintain only java-latest-openjdk. On long run changing it to '''rolling system JDK,''' which are the expected 3-5 builds.
- Contingency deadline: N/A
- Blocks release? No. The change can be introduced even on the fly to
live distributions.
== Documentation ==
N/A (not a System Wide Change)
== Release Notes ==
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On Thu, Jun 29, 2023 at 10:14:31AM -0700, Tom Stellard wrote:
On 6/29/23 09:52, Jiri Vanek wrote:
Hi Tom!
Thanx a lot of for input. Although I did my bes with the tagging, it will be learning on the go. I had adapted https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretica... as you suggested. It is great improvement.
Thanks, this looks better.
For step 5. should the first mention of java-xy-openjdk-portable actually be java-xy-openjdk ? Same question for step 7.
I don't think the fixed sidetag idea will work. (Unless you are planning on doing something different with rawhide).
Bodhi won't let you make an update from a non sidetag tag, so you would have to tag them into fN-updates-candidate, but then they would go one by one into rawhide and not be tested/gated as a unit.
But of course we could do fixed tags for stable releases and have a sidetag flow for rawhide, but that might make things more confusing.
kevin
Hi Kevin! I'm unabel to answer that. First release will be a bit experiemntal for sure. But final target is sure - to persist the underlying portables and to enable srpm rebuild as comfortably as possible. As far rawhide itself, as it is unreleased distro, if no other paths will remain, rawhide may remian self building. Aka using protbales from rawhide to buidl rawhide's rpms.
Thax!
On Thu, 29 Jun 2023 at 19:43, Kevin Fenzi kevin@scrye.com wrote:
On Thu, Jun 29, 2023 at 10:14:31AM -0700, Tom Stellard wrote:
On 6/29/23 09:52, Jiri Vanek wrote:
Hi Tom!
Thanx a lot of for input. Although I did my bes with the tagging, it will be learning on the go. I had adapted https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretica... as you suggested. It is great improvement.
Thanks, this looks better.
For step 5. should the first mention of java-xy-openjdk-portable actually be java-xy-openjdk ? Same question for step 7.
I don't think the fixed sidetag idea will work. (Unless you are planning on doing something different with rawhide).
Bodhi won't let you make an update from a non sidetag tag, so you would have to tag them into fN-updates-candidate, but then they would go one by one into rawhide and not be tested/gated as a unit.
But of course we could do fixed tags for stable releases and have a sidetag flow for rawhide, but that might make things more confusing.
kevin _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On Thu, Jun 29, 2023 at 08:14:01PM +0200, Jiri Vanek wrote:
Hi Kevin! I'm unabel to answer that. First release will be a bit experiemntal for sure. But final target is sure - to persist the underlying portables and to enable srpm rebuild as comfortably as possible.
Sure.
As far rawhide itself, as it is unreleased distro, if no other paths will remain, rawhide may remian self building. Aka using protbales from rawhide to buidl rawhide's rpms.
But is that not distributing a non validated version? Or you mean you would run that through the tests also?
rawhide is released... (almost) every single day. ;)
kevin
On 6/29/23 10:42, Kevin Fenzi wrote:
On Thu, Jun 29, 2023 at 10:14:31AM -0700, Tom Stellard wrote:
On 6/29/23 09:52, Jiri Vanek wrote:
Hi Tom!
Thanx a lot of for input. Although I did my bes with the tagging, it will be learning on the go. I had adapted https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretica... as you suggested. It is great improvement.
Thanks, this looks better.
For step 5. should the first mention of java-xy-openjdk-portable actually be java-xy-openjdk ? Same question for step 7.
I don't think the fixed sidetag idea will work. (Unless you are planning on doing something different with rawhide).
Bodhi won't let you make an update from a non sidetag tag, so you would have to tag them into fN-updates-candidate, but then they would go one by one into rawhide and not be tested/gated as a unit.
What the difference between a fixed side-tag and a normal side-tag from bodhi's perspective?
-Tom
But of course we could do fixed tags for stable releases and have a sidetag flow for rawhide, but that might make things more confusing.
kevin
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On Thu, Jun 29, 2023 at 01:23:42PM -0700, Tom Stellard wrote:
On 6/29/23 10:42, Kevin Fenzi wrote:
On Thu, Jun 29, 2023 at 10:14:31AM -0700, Tom Stellard wrote:
On 6/29/23 09:52, Jiri Vanek wrote:
Hi Tom!
Thanx a lot of for input. Although I did my bes with the tagging, it will be learning on the go. I had adapted https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretica... as you suggested. It is great improvement.
Thanks, this looks better.
For step 5. should the first mention of java-xy-openjdk-portable actually be java-xy-openjdk ? Same question for step 7.
I don't think the fixed sidetag idea will work. (Unless you are planning on doing something different with rawhide).
Bodhi won't let you make an update from a non sidetag tag, so you would have to tag them into fN-updates-candidate, but then they would go one by one into rawhide and not be tested/gated as a unit.
What the difference between a fixed side-tag and a normal side-tag from bodhi's perspective?
Well, koji marks sidetags as being such and bodhi only operates on them. You won't see non sidetags available to make updates from or the like.
We have some non sidetags like 'f38-overrides' that are all the buildroot overrides, or 'f39-compose' which is the tag we use to add packages to a compose that aren't yet stable, etc. It would be pretty bad for bodhi to try and pull all the things in those into an update.
kevin
Nope, xy stands for 1.8.0, 11, 17 and latest. It is enumerated several time in the proposal. Still the https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretica... adjusted
Tahnx!
On Thu, 29 Jun 2023 at 19:14, Tom Stellard tstellar@redhat.com wrote:
On 6/29/23 09:52, Jiri Vanek wrote:
Hi Tom!
Thanx a lot of for input. Although I did my bes with the tagging, it will be learning on the go. I had adapted https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretica... as you suggested. It is great improvement.
Thanks, this looks better.
For step 5. should the first mention of java-xy-openjdk-portable actually be java-xy-openjdk ? Same question for step 7.
-Tom
Especially the config, I was not aware about. That woudl indeed help a lot. The usage of pernament tag is someging I have to learn, but is moreover necessary for proper srpm rebuil.
TYVM! J.
On Wed, 28 Jun 2023 at 21:31, Tom Stellard tstellar@redhat.com wrote:
On 6/26/23 09:21, Aoife Moloney wrote:
https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#including_...)
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.
== Summary ==
This is the last step in https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs effort. JDKs in fedora are already static, and we repack portable tarballs into RPMs. Currently, the portable tarball is built for each Fedora and EPEL version. Goal here is to build each JDK (8,11,17,21,latest (20)) only once, in oldest live Fedora repack in all live Fedoras. If jdk is buitl in epel, it will be built in oldest possible epel and repacked in newer live epels.
== Owner ==
Name: [[User:jvanek| Jiri Vanek]]
Email: jvanek@redhat.com
== Detailed Description ==
As described in https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs ; during last year, packaging of JDKs had changed dramatically. As described in the same wiki page and in individual sub changes and devel threads, the primary reason for this is to lower maintenance and still keep Fedora Java friendly.
- In the first system wide change, we have changed the JDKs to build
properly as standalone, portable JDK - the way JDK is supposed to be built. I repeat, we spent ten years by patching JDK to become properly dynamic against system libs, and all patches went upstream, but this has become a fight which can not be won.
- As a second step we introduced portable RPMs, which do not have any
system integration, only build JDK and pack the final tarball in RPM for Fedora use.
- In third step - without any noise, just verified with fesco -
https://pagure.io/fesco/issue/2907 - we stopped building JDK in fully integrated RPMs. Instead of this, normal RPMs BUildRequire portable RPMs and just unpack it, and repack it.
Now last step is ahead - to build portable LTS JDKs 8,11,17 and 21 in oldest live Fedora, and repack everywhere. java-latest-openjdk, which contains latests STS JDK - currently 20, soon briefly 21 and a bit after 22... If we would built java-latest-openjdk in oldest live EPEL
- epel8 now, we have verified, that such repacked JDKs works fine,
however repack from epel seem to not be acceptable, thus ajva-latest-openjdk will be built twice - one in oldest live fedora, and once in oldest live epel. Build forme oldest possible epel will be repacked to that one or newer epels, and build from oldest live fedroa to all fedoras.
=== theoretical tagging solution ===
1. request side tags for all releases 2. build the actual Java in the side tag for the oldest thingCould you use the real package name here. I think that will make it easier to understand. You can still put 'actual Java' in parens or something.
3. tag the result ot (2) to all side tags from (1) 4. waitrepo them 5. build the repacked java packages in all the side tags from (1)Same thing here, can you use the real package name.
6. untag the result of (2) from all the side tags from (1) 7. ship bodhi updates from side tags OR retag the builds to candidate tags (and delete the side tags)The build from (2) will be eventually garbage collected. To prevent that, it might be re-tagged regularly. This is where releng might be able to help by creating a long lived tag to tag this into for preserving.
Yes, we could make a 'fN-openjdk' tag and mark it protected... that part would be easy enough.
I think this process could be improved if the side-tags in 1. are permanent side-tags, and step 6 is skipped. That would make it easier to rebuild the packages from step 5 if needed.
Also, can you put a config file in the dist-git repos to tell fedpkg which target to build against? I thought I remembered seeing that feature in the past. If so, then you could configure the dist-gits for the repacked javas to automatically build from those side-tags, which I think would be a lot easier for package maintainers and may help make automated rebuilds possible.
-Tom
==== including portable srpms in release (improving of step 6) ====
To include portable rpms in all live Fedoras is currently not possible. Best solution would be simply make and bodhi update of one portable rpm to all live fedoras. Bodhi is currenlty not capable to do so, issue was raised: https://github.com/fedora-infra/bodhi/issues/5387 investigating possibility to deliver single build as update to several releases.
"..It's not possible ATM, it would require a heavy rewrite of the code, starting from the database structure (every build is now related to a single release)..." Maybe on long run..."
On long run, if bodhi will allow this, that will be way to go. On short run, there are following options: a) ask releng to tag the portables directly - this needs manual approach of rare humans, thus no go unless strictly enforced by unpredicted conditions - this walks around whole testing repos. For portables tarballs, as nothing should depend on them, and are tested indirectly after repack, this should be technically ok, but is heavily discouraged in principle. b) build portable for all OSes, but do not ship them (don't do bodhi update) - this would probably work for all frontiers, only the real repacked JDK will be different - pros is, that we will be sure that portables builds on live fedoras - cons is, that the portable JDK will not be available by dnf install anyway c) build portable for all OSes, including bodhi update - pros is, that we will be sure that portables builds on live fedoras - another pros is that the portable JDK will be available by dnf install anyway - there may be clash during the build which will cause to repack wrong (newer, non certified) portables d) include SRPM_REBUILD.readme in srpm and generated PORTABLES_INSTALL.readme in RPMs, which will ideally at least contain: - instruction why you need portables - instruction how to find the portables - from SRPM_REBUILD.readme pointing to PORTABLES_INSTALL.readme - generated link to the koji, allowing to download the SRPM - generated link to the koji, allowing to download the binaries - generated instruction how to dnf install used portables
I would currently vote for d). If there will be complains about broken SRPM rebuild, or need to install portables without hacking, then fall-back a, b or c via Change Proposal. Once Bodhi allows single build to be tagged to several release, I will move to that.
== Feedback ==
== Benefit to Fedora ==
Java maintainers will finally have some free time... No kidding - maintenance and *certification* of so much supported JDKs on so much Fedora versions is brutal. By building once, and repack, we will regain cycles to continue support Fedora with all LTS and one STS JDK.
If we fail to build once and repack everywhere, Java maintainers will most likely need to lower the number of JDKs in fedora to system one only.
== Scope ==
- Proposal owners: Technically all JDKs (except 8, where some more
tuning is needed, and EPEL for java-latest) are prepared, as they have a portable version, and RPMs just repack it. Except tuning up the JDK8 and EPEL for latest, scope owners are done.
- Other developers: There will be needed significant support from RCM
and maybe senior Fedora leadership to help to finish the build in oldest and enable to repack everywhere
- '''Release engineering: [https://pagure.io/releng/issue/11438
#11438]''' There will be needed significant support from RCM, where I'm actually unsure what they will have to do to enable this. The mas rebuild will not be needed.
Policies and guidelines: AFAIK none (not needed for this Change)
Trademark approval: N/A (not needed for this Change)
Alignment with Community Initiatives: All supported JDKs will remain
in Fedora in highest possible quality with full QA and certification, and its packagers will not lose their minds. Note that QA will still run on all live Fedoras, not only on the builder one.
== Upgrade/compatibility impact ==
The change should be completely transparent to any user.
== How To Test ==
`sudo dnf update/install "java*"` will install expected set of working packages.
SRPM rebuild of both portables (which were built once) and of any rpms (from this freshly rebuild portbales) have to remain possible
== User Experience == The change should be absolutely transparent to any user.
== Dependencies == To finish this we will need heavy support from RCM, and maybe others. Although there are precedents with such pacakge, they all bites. From SW point of view, the dependece chain is `normal RPMs build requires portable RPMs` and thats all.
== Contingency Plan ==
- Contingency mechanism: Even if It should be straight forward to
revert back to building per OS, it '''may be impossible for current maintainers to save time''' for it. If this change is approved, we will be building '''4-5''' (jdk8,11,17,sts and 21) builds for all fedoras. If this change is not finished in time, we may '''need to orphan some of the JDKs'''. In better case, we will be able to keep living '''one LTS as system JDK, and java-latest-openjdk''' as future system JDK. That is 2*(3-5) builds (rawhide, (forked,), latest live, oldest live (oldest not yet dropped)). '''In worst case''', we may be able to maintain only java-latest-openjdk. On long run changing it to '''rolling system JDK,''' which are the expected 3-5 builds.
- Contingency deadline: N/A
- Blocks release? No. The change can be introduced even on the fly to
live distributions.
== Documentation ==
N/A (not a System Wide Change)
== Release Notes ==
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On 6/29/23 11:06, Jiri Vanek wrote:
Nope, xy stands for 1.8.0, 11, 17 and latest. It is enumerated several time in the proposal. Still the https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretica... adjusted
OK, I see. I thought there were going to be two different packages. java-xy-openjdk-portable and java-xy-openjdk. Where java-xy-openjdk is the one that gets shipped in Fedora and java-xy-openjdk-portable lives only in the side-tags.
-Tom
Tahnx!
On Thu, 29 Jun 2023 at 19:14, Tom Stellard tstellar@redhat.com wrote:
On 6/29/23 09:52, Jiri Vanek wrote:
Hi Tom!
Thanx a lot of for input. Although I did my bes with the tagging, it will be learning on the go. I had adapted https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretica... as you suggested. It is great improvement.
Thanks, this looks better.
For step 5. should the first mention of java-xy-openjdk-portable actually be java-xy-openjdk ? Same question for step 7.
-Tom
Especially the config, I was not aware about. That woudl indeed help a lot. The usage of pernament tag is someging I have to learn, but is moreover necessary for proper srpm rebuil.
TYVM! J.
On Wed, 28 Jun 2023 at 21:31, Tom Stellard tstellar@redhat.com wrote:
On 6/26/23 09:21, Aoife Moloney wrote:
https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#including_...)
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.
== Summary ==
This is the last step in https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs effort. JDKs in fedora are already static, and we repack portable tarballs into RPMs. Currently, the portable tarball is built for each Fedora and EPEL version. Goal here is to build each JDK (8,11,17,21,latest (20)) only once, in oldest live Fedora repack in all live Fedoras. If jdk is buitl in epel, it will be built in oldest possible epel and repacked in newer live epels.
== Owner ==
Name: [[User:jvanek| Jiri Vanek]]
Email: jvanek@redhat.com
== Detailed Description ==
As described in https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs ; during last year, packaging of JDKs had changed dramatically. As described in the same wiki page and in individual sub changes and devel threads, the primary reason for this is to lower maintenance and still keep Fedora Java friendly.
- In the first system wide change, we have changed the JDKs to build
properly as standalone, portable JDK - the way JDK is supposed to be built. I repeat, we spent ten years by patching JDK to become properly dynamic against system libs, and all patches went upstream, but this has become a fight which can not be won.
- As a second step we introduced portable RPMs, which do not have any
system integration, only build JDK and pack the final tarball in RPM for Fedora use.
- In third step - without any noise, just verified with fesco -
https://pagure.io/fesco/issue/2907 - we stopped building JDK in fully integrated RPMs. Instead of this, normal RPMs BUildRequire portable RPMs and just unpack it, and repack it.
Now last step is ahead - to build portable LTS JDKs 8,11,17 and 21 in oldest live Fedora, and repack everywhere. java-latest-openjdk, which contains latests STS JDK - currently 20, soon briefly 21 and a bit after 22... If we would built java-latest-openjdk in oldest live EPEL
- epel8 now, we have verified, that such repacked JDKs works fine,
however repack from epel seem to not be acceptable, thus ajva-latest-openjdk will be built twice - one in oldest live fedora, and once in oldest live epel. Build forme oldest possible epel will be repacked to that one or newer epels, and build from oldest live fedroa to all fedoras.
=== theoretical tagging solution ===
1. request side tags for all releases 2. build the actual Java in the side tag for the oldest thingCould you use the real package name here. I think that will make it easier to understand. You can still put 'actual Java' in parens or something.
3. tag the result ot (2) to all side tags from (1) 4. waitrepo them 5. build the repacked java packages in all the side tags from (1)Same thing here, can you use the real package name.
6. untag the result of (2) from all the side tags from (1) 7. ship bodhi updates from side tags OR retag the builds to candidate tags (and delete the side tags)The build from (2) will be eventually garbage collected. To prevent that, it might be re-tagged regularly. This is where releng might be able to help by creating a long lived tag to tag this into for preserving.
Yes, we could make a 'fN-openjdk' tag and mark it protected... that part would be easy enough.
I think this process could be improved if the side-tags in 1. are permanent side-tags, and step 6 is skipped. That would make it easier to rebuild the packages from step 5 if needed.
Also, can you put a config file in the dist-git repos to tell fedpkg which target to build against? I thought I remembered seeing that feature in the past. If so, then you could configure the dist-gits for the repacked javas to automatically build from those side-tags, which I think would be a lot easier for package maintainers and may help make automated rebuilds possible.
-Tom
==== including portable srpms in release (improving of step 6) ====
To include portable rpms in all live Fedoras is currently not possible. Best solution would be simply make and bodhi update of one portable rpm to all live fedoras. Bodhi is currenlty not capable to do so, issue was raised: https://github.com/fedora-infra/bodhi/issues/5387 investigating possibility to deliver single build as update to several releases.
"..It's not possible ATM, it would require a heavy rewrite of the code, starting from the database structure (every build is now related to a single release)..." Maybe on long run..."
On long run, if bodhi will allow this, that will be way to go. On short run, there are following options: a) ask releng to tag the portables directly - this needs manual approach of rare humans, thus no go unless strictly enforced by unpredicted conditions - this walks around whole testing repos. For portables tarballs, as nothing should depend on them, and are tested indirectly after repack, this should be technically ok, but is heavily discouraged in principle. b) build portable for all OSes, but do not ship them (don't do bodhi update) - this would probably work for all frontiers, only the real repacked JDK will be different - pros is, that we will be sure that portables builds on live fedoras - cons is, that the portable JDK will not be available by dnf install anyway c) build portable for all OSes, including bodhi update - pros is, that we will be sure that portables builds on live fedoras - another pros is that the portable JDK will be available by dnf install anyway - there may be clash during the build which will cause to repack wrong (newer, non certified) portables d) include SRPM_REBUILD.readme in srpm and generated PORTABLES_INSTALL.readme in RPMs, which will ideally at least contain: - instruction why you need portables - instruction how to find the portables - from SRPM_REBUILD.readme pointing to PORTABLES_INSTALL.readme - generated link to the koji, allowing to download the SRPM - generated link to the koji, allowing to download the binaries - generated instruction how to dnf install used portables
I would currently vote for d). If there will be complains about broken SRPM rebuild, or need to install portables without hacking, then fall-back a, b or c via Change Proposal. Once Bodhi allows single build to be tagged to several release, I will move to that.
== Feedback ==
== Benefit to Fedora ==
Java maintainers will finally have some free time... No kidding - maintenance and *certification* of so much supported JDKs on so much Fedora versions is brutal. By building once, and repack, we will regain cycles to continue support Fedora with all LTS and one STS JDK.
If we fail to build once and repack everywhere, Java maintainers will most likely need to lower the number of JDKs in fedora to system one only.
== Scope ==
- Proposal owners: Technically all JDKs (except 8, where some more
tuning is needed, and EPEL for java-latest) are prepared, as they have a portable version, and RPMs just repack it. Except tuning up the JDK8 and EPEL for latest, scope owners are done.
- Other developers: There will be needed significant support from RCM
and maybe senior Fedora leadership to help to finish the build in oldest and enable to repack everywhere
- '''Release engineering: [https://pagure.io/releng/issue/11438
#11438]''' There will be needed significant support from RCM, where I'm actually unsure what they will have to do to enable this. The mas rebuild will not be needed.
Policies and guidelines: AFAIK none (not needed for this Change)
Trademark approval: N/A (not needed for this Change)
Alignment with Community Initiatives: All supported JDKs will remain
in Fedora in highest possible quality with full QA and certification, and its packagers will not lose their minds. Note that QA will still run on all live Fedoras, not only on the builder one.
== Upgrade/compatibility impact ==
The change should be completely transparent to any user.
== How To Test ==
`sudo dnf update/install "java*"` will install expected set of working packages.
SRPM rebuild of both portables (which were built once) and of any rpms (from this freshly rebuild portbales) have to remain possible
== User Experience == The change should be absolutely transparent to any user.
== Dependencies == To finish this we will need heavy support from RCM, and maybe others. Although there are precedents with such pacakge, they all bites. From SW point of view, the dependece chain is `normal RPMs build requires portable RPMs` and thats all.
== Contingency Plan ==
- Contingency mechanism: Even if It should be straight forward to
revert back to building per OS, it '''may be impossible for current maintainers to save time''' for it. If this change is approved, we will be building '''4-5''' (jdk8,11,17,sts and 21) builds for all fedoras. If this change is not finished in time, we may '''need to orphan some of the JDKs'''. In better case, we will be able to keep living '''one LTS as system JDK, and java-latest-openjdk''' as future system JDK. That is 2*(3-5) builds (rawhide, (forked,), latest live, oldest live (oldest not yet dropped)). '''In worst case''', we may be able to maintain only java-latest-openjdk. On long run changing it to '''rolling system JDK,''' which are the expected 3-5 builds.
- Contingency deadline: N/A
- Blocks release? No. The change can be introduced even on the fly to
live distributions.
== Documentation ==
N/A (not a System Wide Change)
== Release Notes ==
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
nn. You were right. There are going to be two separated packages. Portable, built once in oldest live, and "normal" which is going to repack them for all and shipp them. My apologise for typo in last second change: https://fedoraproject.org/w/index.php?title=Changes%2FBuildJdkOncePackEveryw...
narrowed.
On Thu, 29 Jun 2023 at 21:16, Tom Stellard tstellar@redhat.com wrote:
On 6/29/23 11:06, Jiri Vanek wrote:
Nope, xy stands for 1.8.0, 11, 17 and latest. It is enumerated several time in the proposal. Still the https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretica... adjusted
OK, I see. I thought there were going to be two different packages. java-xy-openjdk-portable and java-xy-openjdk. Where java-xy-openjdk is the one that gets shipped in Fedora and java-xy-openjdk-portable lives only in the side-tags.
-Tom
Tahnx!
On Thu, 29 Jun 2023 at 19:14, Tom Stellard tstellar@redhat.com wrote:
On 6/29/23 09:52, Jiri Vanek wrote:
Hi Tom!
Thanx a lot of for input. Although I did my bes with the tagging, it will be learning on the go. I had adapted https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretica... as you suggested. It is great improvement.
Thanks, this looks better.
For step 5. should the first mention of java-xy-openjdk-portable actually be java-xy-openjdk ? Same question for step 7.
-Tom
Especially the config, I was not aware about. That woudl indeed help a lot. The usage of pernament tag is someging I have to learn, but is moreover necessary for proper srpm rebuil.
TYVM! J.
On Wed, 28 Jun 2023 at 21:31, Tom Stellard tstellar@redhat.com wrote:
On 6/26/23 09:21, Aoife Moloney wrote:
https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#including_...)
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.
== Summary ==
This is the last step in https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs effort. JDKs in fedora are already static, and we repack portable tarballs into RPMs. Currently, the portable tarball is built for each Fedora and EPEL version. Goal here is to build each JDK (8,11,17,21,latest (20)) only once, in oldest live Fedora repack in all live Fedoras. If jdk is buitl in epel, it will be built in oldest possible epel and repacked in newer live epels.
== Owner ==
Name: [[User:jvanek| Jiri Vanek]]
Email: jvanek@redhat.com
== Detailed Description ==
As described in https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs ; during last year, packaging of JDKs had changed dramatically. As described in the same wiki page and in individual sub changes and devel threads, the primary reason for this is to lower maintenance and still keep Fedora Java friendly.
- In the first system wide change, we have changed the JDKs to build
properly as standalone, portable JDK - the way JDK is supposed to be built. I repeat, we spent ten years by patching JDK to become properly dynamic against system libs, and all patches went upstream, but this has become a fight which can not be won.
- As a second step we introduced portable RPMs, which do not have any
system integration, only build JDK and pack the final tarball in RPM for Fedora use.
- In third step - without any noise, just verified with fesco -
https://pagure.io/fesco/issue/2907 - we stopped building JDK in fully integrated RPMs. Instead of this, normal RPMs BUildRequire portable RPMs and just unpack it, and repack it.
Now last step is ahead - to build portable LTS JDKs 8,11,17 and 21 in oldest live Fedora, and repack everywhere. java-latest-openjdk, which contains latests STS JDK - currently 20, soon briefly 21 and a bit after 22... If we would built java-latest-openjdk in oldest live EPEL
- epel8 now, we have verified, that such repacked JDKs works fine,
however repack from epel seem to not be acceptable, thus ajva-latest-openjdk will be built twice - one in oldest live fedora, and once in oldest live epel. Build forme oldest possible epel will be repacked to that one or newer epels, and build from oldest live fedroa to all fedoras.
=== theoretical tagging solution ===
1. request side tags for all releases 2. build the actual Java in the side tag for the oldest thingCould you use the real package name here. I think that will make it easier to understand. You can still put 'actual Java' in parens or something.
3. tag the result ot (2) to all side tags from (1) 4. waitrepo them 5. build the repacked java packages in all the side tags from (1)Same thing here, can you use the real package name.
6. untag the result of (2) from all the side tags from (1) 7. ship bodhi updates from side tags OR retag the builds to candidate tags (and delete the side tags)The build from (2) will be eventually garbage collected. To prevent that, it might be re-tagged regularly. This is where releng might be able to help by creating a long lived tag to tag this into for preserving.
Yes, we could make a 'fN-openjdk' tag and mark it protected... that part would be easy enough.
I think this process could be improved if the side-tags in 1. are permanent side-tags, and step 6 is skipped. That would make it easier to rebuild the packages from step 5 if needed.
Also, can you put a config file in the dist-git repos to tell fedpkg which target to build against? I thought I remembered seeing that feature in the past. If so, then you could configure the dist-gits for the repacked javas to automatically build from those side-tags, which I think would be a lot easier for package maintainers and may help make automated rebuilds possible.
-Tom
==== including portable srpms in release (improving of step 6) ====
To include portable rpms in all live Fedoras is currently not possible. Best solution would be simply make and bodhi update of one portable rpm to all live fedoras. Bodhi is currenlty not capable to do so, issue was raised: https://github.com/fedora-infra/bodhi/issues/5387 investigating possibility to deliver single build as update to several releases.
"..It's not possible ATM, it would require a heavy rewrite of the code, starting from the database structure (every build is now related to a single release)..." Maybe on long run..."
On long run, if bodhi will allow this, that will be way to go. On short run, there are following options: a) ask releng to tag the portables directly - this needs manual approach of rare humans, thus no go unless strictly enforced by unpredicted conditions - this walks around whole testing repos. For portables tarballs, as nothing should depend on them, and are tested indirectly after repack, this should be technically ok, but is heavily discouraged in principle. b) build portable for all OSes, but do not ship them (don't do bodhi update) - this would probably work for all frontiers, only the real repacked JDK will be different - pros is, that we will be sure that portables builds on live fedoras - cons is, that the portable JDK will not be available by dnf install anyway c) build portable for all OSes, including bodhi update - pros is, that we will be sure that portables builds on live fedoras - another pros is that the portable JDK will be available by dnf install anyway - there may be clash during the build which will cause to repack wrong (newer, non certified) portables d) include SRPM_REBUILD.readme in srpm and generated PORTABLES_INSTALL.readme in RPMs, which will ideally at least contain: - instruction why you need portables - instruction how to find the portables - from SRPM_REBUILD.readme pointing to PORTABLES_INSTALL.readme - generated link to the koji, allowing to download the SRPM - generated link to the koji, allowing to download the binaries - generated instruction how to dnf install used portables
I would currently vote for d). If there will be complains about broken SRPM rebuild, or need to install portables without hacking, then fall-back a, b or c via Change Proposal. Once Bodhi allows single build to be tagged to several release, I will move to that.
== Feedback ==
== Benefit to Fedora ==
Java maintainers will finally have some free time... No kidding - maintenance and *certification* of so much supported JDKs on so much Fedora versions is brutal. By building once, and repack, we will regain cycles to continue support Fedora with all LTS and one STS JDK.
If we fail to build once and repack everywhere, Java maintainers will most likely need to lower the number of JDKs in fedora to system one only.
== Scope ==
- Proposal owners: Technically all JDKs (except 8, where some more
tuning is needed, and EPEL for java-latest) are prepared, as they have a portable version, and RPMs just repack it. Except tuning up the JDK8 and EPEL for latest, scope owners are done.
- Other developers: There will be needed significant support from RCM
and maybe senior Fedora leadership to help to finish the build in oldest and enable to repack everywhere
- '''Release engineering: [https://pagure.io/releng/issue/11438
#11438]''' There will be needed significant support from RCM, where I'm actually unsure what they will have to do to enable this. The mas rebuild will not be needed.
Policies and guidelines: AFAIK none (not needed for this Change)
Trademark approval: N/A (not needed for this Change)
Alignment with Community Initiatives: All supported JDKs will remain
in Fedora in highest possible quality with full QA and certification, and its packagers will not lose their minds. Note that QA will still run on all live Fedoras, not only on the builder one.
== Upgrade/compatibility impact ==
The change should be completely transparent to any user.
== How To Test ==
`sudo dnf update/install "java*"` will install expected set of working packages.
SRPM rebuild of both portables (which were built once) and of any rpms (from this freshly rebuild portbales) have to remain possible
== User Experience == The change should be absolutely transparent to any user.
== Dependencies == To finish this we will need heavy support from RCM, and maybe others. Although there are precedents with such pacakge, they all bites. From SW point of view, the dependece chain is `normal RPMs build requires portable RPMs` and thats all.
== Contingency Plan ==
- Contingency mechanism: Even if It should be straight forward to
revert back to building per OS, it '''may be impossible for current maintainers to save time''' for it. If this change is approved, we will be building '''4-5''' (jdk8,11,17,sts and 21) builds for all fedoras. If this change is not finished in time, we may '''need to orphan some of the JDKs'''. In better case, we will be able to keep living '''one LTS as system JDK, and java-latest-openjdk''' as future system JDK. That is 2*(3-5) builds (rawhide, (forked,), latest live, oldest live (oldest not yet dropped)). '''In worst case''', we may be able to maintain only java-latest-openjdk. On long run changing it to '''rolling system JDK,''' which are the expected 3-5 builds.
- Contingency deadline: N/A
- Blocks release? No. The change can be introduced even on the fly to
live distributions.
== Documentation ==
N/A (not a System Wide Change)
== Release Notes ==
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On 6/29/23 13:07, Jiri Vanek wrote:
nn. You were right. There are going to be two separated packages. Portable, built once in oldest live, and "normal" which is going to repack them for all and shipp them. My apologise for typo in last second change: https://fedoraproject.org/w/index.php?title=Changes%2FBuildJdkOncePackEveryw...
narrowed.
Ok, that makes sense now.
My next question is what is the difference between the java-xy-openjdk-portable package that is built on f37 and the repacked java-xy-openjdk package that is shipped on f38. Are the bits inside the package exactly the same?
-Tom
On Thu, 29 Jun 2023 at 21:16, Tom Stellard tstellar@redhat.com wrote:
On 6/29/23 11:06, Jiri Vanek wrote:
Nope, xy stands for 1.8.0, 11, 17 and latest. It is enumerated several time in the proposal. Still the https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretica... adjusted
OK, I see. I thought there were going to be two different packages. java-xy-openjdk-portable and java-xy-openjdk. Where java-xy-openjdk is the one that gets shipped in Fedora and java-xy-openjdk-portable lives only in the side-tags.
-Tom
Tahnx!
On Thu, 29 Jun 2023 at 19:14, Tom Stellard tstellar@redhat.com wrote:
On 6/29/23 09:52, Jiri Vanek wrote:
Hi Tom!
Thanx a lot of for input. Although I did my bes with the tagging, it will be learning on the go. I had adapted https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretica... as you suggested. It is great improvement.
Thanks, this looks better.
For step 5. should the first mention of java-xy-openjdk-portable actually be java-xy-openjdk ? Same question for step 7.
-Tom
Especially the config, I was not aware about. That woudl indeed help a lot. The usage of pernament tag is someging I have to learn, but is moreover necessary for proper srpm rebuil.
TYVM! J.
On Wed, 28 Jun 2023 at 21:31, Tom Stellard tstellar@redhat.com wrote:
On 6/26/23 09:21, Aoife Moloney wrote: > https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#including_...) > > This document represents a proposed Change. As part of the Changes > process, proposals are publicly announced in order to receive > community feedback. This proposal will only be implemented if approved > by the Fedora Engineering Steering Committee. > > > == Summary == > > This is the last step in > https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs > effort. JDKs in fedora are already static, and we repack portable > tarballs into RPMs. Currently, the portable tarball is built for each > Fedora and EPEL version. Goal here is to build each JDK > (8,11,17,21,latest (20)) only once, in oldest live Fedora repack in > all live Fedoras. If jdk is buitl in epel, it will be built in oldest > possible epel and repacked in newer live epels. > > > == Owner == > * Name: [[User:jvanek| Jiri Vanek]] > > * Email: jvanek@redhat.com > > > == Detailed Description == > > As described in > https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs ; > during last year, packaging of JDKs had changed dramatically. As > described in the same wiki page and in individual sub changes and > devel threads, the primary reason for this is to lower maintenance and > still keep Fedora Java friendly. > > * In the first system wide change, we have changed the JDKs to build > properly as standalone, portable JDK - the way JDK is supposed to be > built. I repeat, we spent ten years by patching JDK to become properly > dynamic against system libs, and all patches went upstream, but this > has become a fight which can not be won. > > * As a second step we introduced portable RPMs, which do not have any > system integration, only build JDK and pack the final tarball in RPM > for Fedora use. > > * In third step - without any noise, just verified with fesco - > https://pagure.io/fesco/issue/2907 - we stopped building JDK in fully > integrated RPMs. Instead of this, normal RPMs BUildRequire portable > RPMs and just unpack it, and repack it. > > Now last step is ahead - to build portable LTS JDKs 8,11,17 and 21 in > oldest live Fedora, and repack everywhere. java-latest-openjdk, which > contains latests STS JDK - currently 20, soon briefly 21 and a bit > after 22... If we would built java-latest-openjdk in oldest live EPEL > - epel8 now, we have verified, that such repacked JDKs works fine, > however repack from epel seem to not be acceptable, thus > ajva-latest-openjdk will be built twice - one in oldest live fedora, > and once in oldest live epel. Build forme oldest possible epel will be > repacked to that one or newer epels, and build from oldest live fedroa > to all fedoras. > > === theoretical tagging solution === > > 1. request side tags for all releases > 2. build the actual Java in the side tag for the oldest thing
Could you use the real package name here. I think that will make it easier to understand. You can still put 'actual Java' in parens or something.
> 3. tag the result ot (2) to all side tags from (1) > 4. waitrepo them > 5. build the repacked java packages in all the side tags from (1)
Same thing here, can you use the real package name.
> 6. untag the result of (2) from all the side tags from (1) > 7. ship bodhi updates from side tags OR retag the builds to candidate tags > (and delete the side tags) > > The build from (2) will be eventually garbage collected. To prevent that, it > might be re-tagged regularly. This is where releng might be able to help by > creating a long lived tag to tag this into for preserving. > > Yes, we could make a 'fN-openjdk' tag and mark it protected... that part > would be easy enough. >
I think this process could be improved if the side-tags in 1. are permanent side-tags, and step 6 is skipped. That would make it easier to rebuild the packages from step 5 if needed.
Also, can you put a config file in the dist-git repos to tell fedpkg which target to build against? I thought I remembered seeing that feature in the past. If so, then you could configure the dist-gits for the repacked javas to automatically build from those side-tags, which I think would be a lot easier for package maintainers and may help make automated rebuilds possible.
-Tom
> > ==== including portable srpms in release (improving of step 6) ==== > > To include portable rpms in all live Fedoras is currently not > possible. Best solution would be simply make and bodhi update of one > portable rpm to all live fedoras. Bodhi is currenlty not capable to do > so, issue was raised: > https://github.com/fedora-infra/bodhi/issues/5387 investigating > possibility to deliver single build as update to several releases. > > "..It's not possible ATM, it would require a heavy rewrite of the > code, starting from the database structure (every build is now related > to a single release)..." Maybe on long run..." > > On long run, if bodhi will allow this, that will be way to go. > On short run, there are following options: > a) ask releng to tag the portables directly > - this needs manual approach of rare humans, thus no go unless > strictly enforced by unpredicted conditions > - this walks around whole testing repos. For portables tarballs, as > nothing should depend on them, and are tested indirectly after repack, > this should be technically ok, but is heavily discouraged in > principle. > b) build portable for all OSes, but do not ship them (don't do bodhi update) > - this would probably work for all frontiers, only the real > repacked JDK will be different > - pros is, that we will be sure that portables builds on live fedoras > - cons is, that the portable JDK will not be available by dnf install anyway > c) build portable for all OSes, including bodhi update > - pros is, that we will be sure that portables builds on live fedoras > - another pros is that the portable JDK will be available by dnf > install anyway > - there may be clash during the build which will cause to repack > wrong (newer, non certified) portables > d) include SRPM_REBUILD.readme in srpm and generated > PORTABLES_INSTALL.readme in RPMs, which will ideally at least contain: > - instruction why you need portables > - instruction how to find the portables > - from SRPM_REBUILD.readme pointing to PORTABLES_INSTALL.readme > - generated link to the koji, allowing to download the SRPM > - generated link to the koji, allowing to download the binaries > - generated instruction how to dnf install used portables > > I would currently vote for d). If there will be complains about broken > SRPM rebuild, or need to install portables without hacking, then > fall-back a, b or c via Change Proposal. > Once Bodhi allows single build to be tagged to several release, I will > move to that. > > == Feedback == > > > == Benefit to Fedora == > > Java maintainers will finally have some free time... No kidding - > maintenance and *certification* of so much supported JDKs on so much > Fedora versions is brutal. By building once, and repack, we will > regain cycles to continue support Fedora with all LTS and one STS JDK. > > If we fail to build once and repack everywhere, Java maintainers will > most likely need to lower the number of JDKs in fedora to system one > only. > > == Scope == > * Proposal owners: Technically all JDKs (except 8, where some more > tuning is needed, and EPEL for java-latest) are prepared, as they have > a portable version, and RPMs just repack it. Except tuning up the JDK8 > and EPEL for latest, scope owners are done. > > * Other developers: There will be needed significant support from RCM > and maybe senior Fedora leadership to help to finish the build in > oldest and enable to repack everywhere > > * '''Release engineering: [https://pagure.io/releng/issue/11438 > #11438]''' There will be needed significant support from RCM, where > I'm actually unsure what they will have to do to enable this. The mas > rebuild will not be needed. > > * Policies and guidelines: AFAIK none (not needed for this Change) > * Trademark approval: N/A (not needed for this Change) > > * Alignment with Community Initiatives: All supported JDKs will remain > in Fedora in highest possible quality with full QA and certification, > and its packagers will not lose their minds. Note that QA will still > run on all live Fedoras, not only on the builder one. > > > == Upgrade/compatibility impact == > > The change should be completely transparent to any user. > > > == How To Test == > > `sudo dnf update/install "java*"` will install expected set of working packages. > > SRPM rebuild of both portables (which were built once) and of any rpms > (from this freshly rebuild portbales) have to remain possible > > > == User Experience == > The change should be absolutely transparent to any user. > > > == Dependencies == > To finish this we will need heavy support from RCM, and maybe others. > Although there are precedents with such pacakge, they all bites. From > SW point of view, the dependece chain is `normal RPMs build requires > portable RPMs` and thats all. > > > == Contingency Plan == > * Contingency mechanism: Even if It should be straight forward to > revert back to building per OS, it '''may be impossible for current > maintainers to save time''' for it. If this change is approved, we > will be building '''4-5''' (jdk8,11,17,sts and 21) builds for all > fedoras. If this change is not finished in time, we may '''need to > orphan some of the JDKs'''. In better case, we will be able to keep > living '''one LTS as system JDK, and java-latest-openjdk''' as future > system JDK. That is 2*(3-5) builds (rawhide, (forked,), latest live, > oldest live (oldest not yet dropped)). '''In worst case''', we may be > able to maintain only java-latest-openjdk. On long run changing it to > '''rolling system JDK,''' which are the expected 3-5 builds. > * Contingency deadline: N/A > * Blocks release? No. The change can be introduced even on the fly to > live distributions. > > == Documentation == > > N/A (not a System Wide Change) > > == Release Notes == > > > > _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Once the portables will be built in f37, the repacked binary in f37 and in f38 (39...)will be identical. Rpms have some additional value - split to subpakcages, few symlinks due to system integration, system tzdata, system crypto binding and so on.
On Thu, 29 Jun 2023 at 22:20, Tom Stellard tstellar@redhat.com wrote:
On 6/29/23 13:07, Jiri Vanek wrote:
nn. You were right. There are going to be two separated packages. Portable, built once in oldest live, and "normal" which is going to repack them for all and shipp them. My apologise for typo in last second change: https://fedoraproject.org/w/index.php?title=Changes%2FBuildJdkOncePackEveryw...
narrowed.
Ok, that makes sense now.
My next question is what is the difference between the java-xy-openjdk-portable package that is built on f37 and the repacked java-xy-openjdk package that is shipped on f38. Are the bits inside the package exactly the same?
-Tom
On Thu, 29 Jun 2023 at 21:16, Tom Stellard tstellar@redhat.com wrote:
On 6/29/23 11:06, Jiri Vanek wrote:
Nope, xy stands for 1.8.0, 11, 17 and latest. It is enumerated several time in the proposal. Still the https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretica... adjusted
OK, I see. I thought there were going to be two different packages. java-xy-openjdk-portable and java-xy-openjdk. Where java-xy-openjdk is the one that gets shipped in Fedora and java-xy-openjdk-portable lives only in the side-tags.
-Tom
Tahnx!
On Thu, 29 Jun 2023 at 19:14, Tom Stellard tstellar@redhat.com wrote:
On 6/29/23 09:52, Jiri Vanek wrote:
Hi Tom!
Thanx a lot of for input. Although I did my bes with the tagging, it will be learning on the go. I had adapted https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoretica... as you suggested. It is great improvement.
Thanks, this looks better.
For step 5. should the first mention of java-xy-openjdk-portable actually be java-xy-openjdk ? Same question for step 7.
-Tom
Especially the config, I was not aware about. That woudl indeed help a lot. The usage of pernament tag is someging I have to learn, but is moreover necessary for proper srpm rebuil.
TYVM! J.
On Wed, 28 Jun 2023 at 21:31, Tom Stellard tstellar@redhat.com wrote: > > On 6/26/23 09:21, Aoife Moloney wrote: >> https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#including_...) >> >> This document represents a proposed Change. As part of the Changes >> process, proposals are publicly announced in order to receive >> community feedback. This proposal will only be implemented if approved >> by the Fedora Engineering Steering Committee. >> >> >> == Summary == >> >> This is the last step in >> https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs >> effort. JDKs in fedora are already static, and we repack portable >> tarballs into RPMs. Currently, the portable tarball is built for each >> Fedora and EPEL version. Goal here is to build each JDK >> (8,11,17,21,latest (20)) only once, in oldest live Fedora repack in >> all live Fedoras. If jdk is buitl in epel, it will be built in oldest >> possible epel and repacked in newer live epels. >> >> >> == Owner == >> * Name: [[User:jvanek| Jiri Vanek]] >> >> * Email: jvanek@redhat.com >> >> >> == Detailed Description == >> >> As described in >> https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs ; >> during last year, packaging of JDKs had changed dramatically. As >> described in the same wiki page and in individual sub changes and >> devel threads, the primary reason for this is to lower maintenance and >> still keep Fedora Java friendly. >> >> * In the first system wide change, we have changed the JDKs to build >> properly as standalone, portable JDK - the way JDK is supposed to be >> built. I repeat, we spent ten years by patching JDK to become properly >> dynamic against system libs, and all patches went upstream, but this >> has become a fight which can not be won. >> >> * As a second step we introduced portable RPMs, which do not have any >> system integration, only build JDK and pack the final tarball in RPM >> for Fedora use. >> >> * In third step - without any noise, just verified with fesco - >> https://pagure.io/fesco/issue/2907 - we stopped building JDK in fully >> integrated RPMs. Instead of this, normal RPMs BUildRequire portable >> RPMs and just unpack it, and repack it. >> >> Now last step is ahead - to build portable LTS JDKs 8,11,17 and 21 in >> oldest live Fedora, and repack everywhere. java-latest-openjdk, which >> contains latests STS JDK - currently 20, soon briefly 21 and a bit >> after 22... If we would built java-latest-openjdk in oldest live EPEL >> - epel8 now, we have verified, that such repacked JDKs works fine, >> however repack from epel seem to not be acceptable, thus >> ajva-latest-openjdk will be built twice - one in oldest live fedora, >> and once in oldest live epel. Build forme oldest possible epel will be >> repacked to that one or newer epels, and build from oldest live fedroa >> to all fedoras. >> >> === theoretical tagging solution === >> >> 1. request side tags for all releases >> 2. build the actual Java in the side tag for the oldest thing > > Could you use the real package name here. I think that will make it easier > to understand. You can still put 'actual Java' in parens or something. > >> 3. tag the result ot (2) to all side tags from (1) >> 4. waitrepo them >> 5. build the repacked java packages in all the side tags from (1) > > Same thing here, can you use the real package name. > >> 6. untag the result of (2) from all the side tags from (1) >> 7. ship bodhi updates from side tags OR retag the builds to candidate tags >> (and delete the side tags) >> >> The build from (2) will be eventually garbage collected. To prevent that, it >> might be re-tagged regularly. This is where releng might be able to help by >> creating a long lived tag to tag this into for preserving. >> >> Yes, we could make a 'fN-openjdk' tag and mark it protected... that part >> would be easy enough. >> > > I think this process could be improved if the side-tags in 1. are permanent side-tags, > and step 6 is skipped. That would make it easier to rebuild the packages from > step 5 if needed. > > Also, can you put a config file in the dist-git repos to tell fedpkg which target > to build against? I thought I remembered seeing that feature in the past. If so, > then you could configure the dist-gits for the repacked javas to automatically build > from those side-tags, which I think would be a lot easier for package maintainers and > may help make automated rebuilds possible. > > -Tom > > >> >> ==== including portable srpms in release (improving of step 6) ==== >> >> To include portable rpms in all live Fedoras is currently not >> possible. Best solution would be simply make and bodhi update of one >> portable rpm to all live fedoras. Bodhi is currenlty not capable to do >> so, issue was raised: >> https://github.com/fedora-infra/bodhi/issues/5387 investigating >> possibility to deliver single build as update to several releases. >> >> "..It's not possible ATM, it would require a heavy rewrite of the >> code, starting from the database structure (every build is now related >> to a single release)..." Maybe on long run..." >> >> On long run, if bodhi will allow this, that will be way to go. >> On short run, there are following options: >> a) ask releng to tag the portables directly >> - this needs manual approach of rare humans, thus no go unless >> strictly enforced by unpredicted conditions >> - this walks around whole testing repos. For portables tarballs, as >> nothing should depend on them, and are tested indirectly after repack, >> this should be technically ok, but is heavily discouraged in >> principle. >> b) build portable for all OSes, but do not ship them (don't do bodhi update) >> - this would probably work for all frontiers, only the real >> repacked JDK will be different >> - pros is, that we will be sure that portables builds on live fedoras >> - cons is, that the portable JDK will not be available by dnf install anyway >> c) build portable for all OSes, including bodhi update >> - pros is, that we will be sure that portables builds on live fedoras >> - another pros is that the portable JDK will be available by dnf >> install anyway >> - there may be clash during the build which will cause to repack >> wrong (newer, non certified) portables >> d) include SRPM_REBUILD.readme in srpm and generated >> PORTABLES_INSTALL.readme in RPMs, which will ideally at least contain: >> - instruction why you need portables >> - instruction how to find the portables >> - from SRPM_REBUILD.readme pointing to PORTABLES_INSTALL.readme >> - generated link to the koji, allowing to download the SRPM >> - generated link to the koji, allowing to download the binaries >> - generated instruction how to dnf install used portables >> >> I would currently vote for d). If there will be complains about broken >> SRPM rebuild, or need to install portables without hacking, then >> fall-back a, b or c via Change Proposal. >> Once Bodhi allows single build to be tagged to several release, I will >> move to that. >> >> == Feedback == >> >> >> == Benefit to Fedora == >> >> Java maintainers will finally have some free time... No kidding - >> maintenance and *certification* of so much supported JDKs on so much >> Fedora versions is brutal. By building once, and repack, we will >> regain cycles to continue support Fedora with all LTS and one STS JDK. >> >> If we fail to build once and repack everywhere, Java maintainers will >> most likely need to lower the number of JDKs in fedora to system one >> only. >> >> == Scope == >> * Proposal owners: Technically all JDKs (except 8, where some more >> tuning is needed, and EPEL for java-latest) are prepared, as they have >> a portable version, and RPMs just repack it. Except tuning up the JDK8 >> and EPEL for latest, scope owners are done. >> >> * Other developers: There will be needed significant support from RCM >> and maybe senior Fedora leadership to help to finish the build in >> oldest and enable to repack everywhere >> >> * '''Release engineering: [https://pagure.io/releng/issue/11438 >> #11438]''' There will be needed significant support from RCM, where >> I'm actually unsure what they will have to do to enable this. The mas >> rebuild will not be needed. >> >> * Policies and guidelines: AFAIK none (not needed for this Change) >> * Trademark approval: N/A (not needed for this Change) >> >> * Alignment with Community Initiatives: All supported JDKs will remain >> in Fedora in highest possible quality with full QA and certification, >> and its packagers will not lose their minds. Note that QA will still >> run on all live Fedoras, not only on the builder one. >> >> >> == Upgrade/compatibility impact == >> >> The change should be completely transparent to any user. >> >> >> == How To Test == >> >> `sudo dnf update/install "java*"` will install expected set of working packages. >> >> SRPM rebuild of both portables (which were built once) and of any rpms >> (from this freshly rebuild portbales) have to remain possible >> >> >> == User Experience == >> The change should be absolutely transparent to any user. >> >> >> == Dependencies == >> To finish this we will need heavy support from RCM, and maybe others. >> Although there are precedents with such pacakge, they all bites. From >> SW point of view, the dependece chain is `normal RPMs build requires >> portable RPMs` and thats all. >> >> >> == Contingency Plan == >> * Contingency mechanism: Even if It should be straight forward to >> revert back to building per OS, it '''may be impossible for current >> maintainers to save time''' for it. If this change is approved, we >> will be building '''4-5''' (jdk8,11,17,sts and 21) builds for all >> fedoras. If this change is not finished in time, we may '''need to >> orphan some of the JDKs'''. In better case, we will be able to keep >> living '''one LTS as system JDK, and java-latest-openjdk''' as future >> system JDK. That is 2*(3-5) builds (rawhide, (forked,), latest live, >> oldest live (oldest not yet dropped)). '''In worst case''', we may be >> able to maintain only java-latest-openjdk. On long run changing it to >> '''rolling system JDK,''' which are the expected 3-5 builds. >> * Contingency deadline: N/A >> * Blocks release? No. The change can be introduced even on the fly to >> live distributions. >> >> == Documentation == >> >> N/A (not a System Wide Change) >> >> == Release Notes == >> >> >> >> > _______________________________________________ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-leave@lists.fedoraproject.org > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
-
Aoife Moloney -
Demi Marie Obenour -
Jiri Vanek -
Kevin Fenzi -
Tom Stellard