Wiki - https://fedoraproject.org/wiki/Changes/Deprecate_Zezere
Discussion thread -
This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
Deprecate use of the {{package|Zezere}} provisioning server, currently
used to configure Fedora IoT devices.
== Owner ==
* Name: [[User: pwhalen| Paul Whalen]]
* Email: pwhalen(a)fedoraproject.org
* Name: Fedora IoT SIG
== Detailed Description ==
Currently, Fedora IoT users can add an SSH key to the root user
account using the Zezere provisioning tool. While convenient for most
use cases, users have given feedback that this does not work for all.
In Fedora 42 we plan to deprecate the Zezere provisioning server in
favour of offering a local means for user configuaration -
`systemd-firstboot` - as well as the existing options of `FDO` or
`ignition`.
== Feedback ==
== Benefit to Fedora ==
The Zezere provisioning tool has not worked well for all Fedora IoT
users. Deprecation will allow us to replace this configuration method
with something that is more robust, well tested and already installed
by default with `systemd`.
== Scope ==
* Proposal owners:
** Remove Zezere from the installed packages, enable
`systemd-firstboot` and ensure its compatible with IoT systems
** Document the change
* Other developers: N/A
* Release engineering: [https://pagure.io/releng/issues #Releng issue number]
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with the Fedora Strategy:
== Upgrade/compatibility impact ==
None.
== Early Testing (Optional) ==
== How To Test ==
To test, users will need to provision a new Fedora IoT system after
the change is made to enable `systemd-firstboot`.
== User Experience ==
Users who have been unable to use Zezere will have an easier and more
straight forward way to configure their system resulting in less
frustration during the critical first boot experience.
== Dependencies ==
== Contingency Plan ==
* Contingency mechanism: Continue to include Zezere as we do today.
== Documentation ==
* Fedora IoT Getting started guide will be updated to reflect the
change and new configuration option.
== Release Notes ==
--
Aoife Moloney
Fedora Operations Architect
Fedora Project
Matrix: @amoloney:fedora.im
IRC: amoloney
Wiki - https://fedoraproject.org/wiki/Changes/CoreOSOstree2OCIUpdates
Discussion thread -
https://discussion.fedoraproject.org/t/f42-change-proposal-move-fedora-core…
== Summary ==
Change Fedora CoreOS to receive updates from quay.io/fedora/fedora-coreos
instead of the Fedora OSTree repository.
== Owner ==
* Name/Email: [[jbtrystram | Jean-Baptiste Trystram]], jbtrystram(a)redhat.com
* Name/Email: [[User:Siosm| Timothée Ravier]], siosm(a)fedoraproject.org
* Name/Email: [[User:jlebon|Jonathan Lebon]], jonathan(a)jlebon.com
* Name/Email: [[User:Dustymabe|Dusty Mabe]], dusty(a)dustymabe.com
== Detailed Description ==
Currently, Fedora CoreOS hosts pull updates from the OSTree repository.
With this change, the hosts will pull updates from the Quay.io container
registry instead. At first, this should be a transparent change. We will
notably keep using rpm-ostree for updates (and not yet bootc).
This is preliminary work to switching to bootc to manage the system and
will enable us to deliver the following changes in the future:
* Moving from rpm-ostree to bootc, which only supports OCI.
* Better support for mirroring updates in disconnected setups.
* Moving away from maintaining a Cincinnati server towards having the graph
live in an OCI registry alongside the update payload. This also allows
users to maintain their own update graphs.
* Users will be able to create their own customized versions of Fedora
CoreOS by building a derived container image.
== Feedback ==
None yet.
== Benefit to Fedora ==
Alignment with the work happening in the [https://
https://docs.fedoraproject.org/en-US/bootc/getting-started/#_what_is_a_boot…
Bootable Containers] initiative.
== Scope ==
* Proposal owners:
** Publish an upgrade graph containing the digest pullspec for each FCOS
release. This mirrors the current update graph containing the same
information, but pointing at OSTree commit checksums.
** Change new nodes on `next` to use OCI from the start.
** After a number of FCOS releases, ship a migration script to switch
existing `next` nodes to use OCI.
** Repeat the last two steps for `testing`, and then `stable`.
** Down the line, stop publishing new OSTree commits to the OSTree repo.
This will not happen until at least f43.
* Other developers: N/A (not needed for this Change)
* Release engineering: N/A (not needed for this Change)
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with the Fedora Strategy:
** Part of the work to align with Bootable Containers
== Upgrade/compatibility impact ==
We will issue a barrier release to migrate users to switch to OCI images.
== How To Test ==
Once the changes are ready, it will be possible to test it on the
'''next''' stream before it gets rolled out there. This can be done by
switching a '''next''' node from the OSTree remote to the OCI remote:
rpm-ostree rebase ostree-remote-image:fedora:registry:
quay.io/fedora/fedora-coreos:$NEXT_VERSION
where `$NEXT_VERSION` is a tag for a '''next''' release that's _not_ the
latest. Then, watch Zincati fetch the latest '''next''' release using OCI.
== User Experience ==
This change won't be visible to users running auto-updates, except cosmetic
changes in `rpm-ostree status` output.
== Contingency Plan ==
Revert the change to switch back to the OSTree repo. Both will be active
until the Fedora 43 release.
== Documentation ==
We will update the Fedora CoreOS documentation alongside the transition.
This is currently tracked in:
https://github.com/coreos/fedora-coreos-tracker/issues/1823.
== Release Notes ==
--
Aoife Moloney
Fedora Operations Architect
Fedora Project
Matrix: @amoloney:fedora.im
IRC: amoloney
Greetings.
Last year, as part of https://fedoraproject.org/wiki/Changes/ChangeComposeSettings
we were going to change rawhide (and then f41) repo metadata to use the
new upstream createrepo_c default (zstd).
We unfortunately had to partially back this change out because at the
time there were still consumers using older OSes (in particular rhel7)
that were unable to handle the repodata.
We are going to switch rawhide back on the next compose.
RHEL8/9 dnf can handle this repodata fine, however, if you use
mergerepo_c or the like you may need to use
https://copr.fedorainfracloud.org/coprs/amatej/createrepo_c/
until createrepo_c is updated.
kevin
Hi all,
Per the Fedora Linux 42 schedule [1], we will start a mass rebuild for
Fedora 42 on 2025-01-15.
We will be conducting this mass rebuild for the changes listed in:
https://pagure.io/releng/issues?status=Open&tags=mass+rebuild; We have
a tracker at releng for the same, in case of any queries and requests
feel free to comment on the issue
https://pagure.io/releng/issue/12527.
This mass rebuild will be done in a side tag (f42-rebuild) and merged
when completed. FTBFS (Fails To Build From Source) bugs will be filed
shortly after the mass rebuild.
Just so you know, if you need to exclude any packages from the
rebuild, you can use the PKG_SKIP_LIST or add a noautobuild file to
the root of your distgit repository. This will instruct the mass
rebuild script to skip these packages. Previously, some OCaml packages
were rebuilt, but they can now be opted out using this method if
required.
If you encounter any bugs or issues in the reporting, please let Releng know.
You can contact us in the #fedora-releng channel on Libera.Chat, the
#releng.org room on Matrix, email our list [2], or file an issue in
pagure [3].
Regards,
Samyak Jain
Fedora Release Engineering
[1] https://fedorapeople.org/groups/schedule/f-42/f-42key-tasks.html
[2] https://lists.fedoraproject.org/admin/lists/rel-eng.lists.fedoraproject.org/
[3] https://pagure.io/releng/
Wiki - https://fedoraproject.org/wiki/Changes/BootLoaderUpdatesPhase1
Discussion thread -
https://discussion.fedoraproject.org/t/f42-change-proposal-unification-of-b…
This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
Begin the process of using single tool (likely bootupd) for most boot
loader updates, especially grub and shim. This decouples package
installation, e.g., via rpm transaction, from actual update in `/boot`
and `/boot/efi`. This change does not affect zipl and systemd-boot.
{{admon/important||The implementation details for this change are not
fully fleshed out yet but we believe that there is a general agreement
that this is the right way to go, so we are submitting it to start the
discussion around how we should implement it. We will update the
change based on feedback from those discussions.}}
== Owner ==
* Name/Emails:
** [[User:Siosm| Timothée Ravier]] siosm(a)fedoraproject.org (Atomic
Desktops, CoreOS, bootc)
** [[User:Walters| Colin Walters]] walters(a)verbum.org (bootupd, bootc)
** [[User:mlewando| Marta Lewandowska]] bazylia(a)pm.me (grub2, shim)
== Detailed Description ==
Our goal is to make boot loader updates in all environments: rpm mode,
image mode, etc., safer and more reliable, and to add a fallback
option so that a system is always bootable. We want to use a single
interface, bootupd, to perform the actual boot loader update on a
system once all new packages have been installed on that system.
On package mode systems (i.e. Workstation, Server, etc.), most
components are updated monolithically along with boot loader packages,
which automatically install into the `/boot` and/or `/boot/efi`
directories, overwriting binaries that were there. Notably, kernel
installation is performed via systemd script kernel-install, which
runs at the end of the rpm transaction. Each distribution has a single
directory structure constituting its bootpath, and if anything goes
wrong during the subsequent boot, repair can be difficult or even
impossible.
On image mode systems (i.e. bootc, CoreOS, IoT, Atomic Desktops), the
boot loader update is decoupled from the system update, but, bootupd,
once triggered, will also overwrite entries in `/boot` & `/boot/efi`,
and any problem with the boot loaders has the potential to create an
unbootable and unrecoverable system.
Rather than using different, yet equally precarious, methods for
updating these critical components, we want to adapt bootupd to work
on all systems in order to make boot loader updates easier and safer.
It would additionally be aware of dependencies between packages, such
as boot loaders and kernels. bootupd would (likely) be the interface
that is used on all systems to perform the actual update once all new
packages are available, and it would in the future retain a fallback
option in case any problem arises during the subsequent boot process.
=== This change will be split into two phases ===
''Phase 1 (this change, proposed for Fedora 42)''
* Install the following packages into `/usr/lib/<pkg>/<arch>` rather
than `/boot` & `/boot/efi`
** grub2
** shim
* Separate rpm actions that happen at build time vs. runtime:
** Add a mechanism that installs the bootloader for package mode
installations as part of a RPM post trans. It is not yet defined how
that will happen. Options that were suggested:
*** Add a dumb Bash script that copies the bootloader to the ESP
*** Add a new action verb to kernel-install, which will run script(s)
for given component
*** Adapt bootupd to cover this case
* Teach new boot loader location (`/usr/…`) to bootupd (needed for
rpm-ostree & Bootable Containers variants)
* Extend systemd package-notes to support PE files
''Phase 2 (will be proposed for Fedora 43)''
* Add A/B support, BootNext to bootupd
https://github.com/coreos/bootupd/issues/440
* Unify on bootupd to perform all boot loader updates at both system
install (e.g. Anaconda) time and for updates
* Install bootupd on dnf and image based systems by default
* Consider switching to a static GRUB configuration file (to be detailed)
=== Phase 1 (this change, proposed for Fedora 42) ===
In phase 1 of this change, we will focus on moving the content
included in the grub and shim packages from `/boot` and `/boot/efi` to
`/usr` and adding the logic needed to install them to `/boot` as part
of a posttrans script.
Moving the content from `/boot` and `/boot/efi` to `/usr` (in the RPM
packages) is a pre-requisite for all the potential future improvements
as it decouples the bootloader installation phase via the RPM (that
happens offline for all image mode systems and image builds in
general) from the actual update in the ESP and any runtime EFI
variable updates.
As we move the bootloader and shim to `/usr`, we now need a mechanism
to sync the content from `/usr` to `/boot` and the ESP (and the MBR
for BIOS systems, but we already have to do that). In all cases, the
current suggestion is to do this synchronization step as part of a
posttrans trigger at the end of the RPM transaction.
The first suggested option is to add a “dumb” Bash script that copies
the new bootloader to the ESP partition.
The second suggested option is to add scripts to the kernel/install.d
directory. Currently, when a new kernel gets installed, kernel-install
(systemd) runs and invokes a number of bash and python scripts in
`/usr/lib/kernel/install.d/` which make the initramfs, create BLS
entries, etc. We could extend kernel-install to add a new action that
will be triggered when the relevant components (shim, grub2, sd-boot)
are updated.
The third option is to adapt bootupd to install the updated bootloader
to cover this case and run as part of the posttrans trigger.
=== Phase 2 (will be proposed for Fedora 43) ===
In phase 2, we will look at unifying in a single tool how we do
bootloader and shim updates, likely using bootupd, and extend the safe
of boot loader updates, mainly on EFI based systems.
We will teach bootupd how to create a full new boot chain, e.g., shim
-> grub2 -> kernel, based on newly installed components while
retaining the working boot chain as a fallback option. Upon reboot,
the new boot chain will be attempted, and if successful, set as the
new default/fallback.
We will also need to choose how to trigger the script(s) so that in
the end all new content is in a single directory in /boot, and a new
efi entry is created for that directory and set to BootNext.
It has not yet been decided what trigger will be used to initiate the
update. There is an idea to create a new systemd target, but obviously
this has to happen before the system is rebooted.
One possibility is to use the reboot/poweroff target as the trigger:
in case the system is powered off accidentally, no update would occur.
If the user chooses to reboot, the new boot chain will be created just
before shutdown so that it can be set as BootNext.
It has been proposed to use systemd package-notes for version numbers
to track dependencies between kernels and boot loaders. This would
remove bootupd’s dependency on rpm to scrape package numbers.
=== Architecture comments ===
Phase 1 is architecture agnostic.
Our current targets for phase 2 are UEFI based systems: x86_64 and
aarch64. Additionally, we believe ppc64le will want this support and
will thus make itself more compliant in the meantime. s390 systems are
out of scope at this time.
== Feedback ==
Q: What about UKIs?
A: This should not impact setups with UKIs. We will not touch the
bootloader configuration itself or how GRUB works as part of this
change.
== Benefit to Fedora ==
Once phase 1 and 2 are complete:
* Boot loader updates are safer and more reliable.
* The same mechanism is used for boot loader updates across Fedora products.
* Fallback to known working boot loader is always available.
== Scope ==
* Proposal owners:
** Packaging changes to move the content from `/boot` to `/usr/lib/…`
for the shim and grub2 packages.
** Update the scripts needed to copy those files to `/boot` as part of
a post-trans scriptlet
** Teach bootupd about these new paths
* Other developers:
** Other packages that install files in `/boot` and `/boot/efi` and
other bootloader packages should adapt for this change as well. It
should still continue to work if they don’t, but they won’t benefit
from the changes.
* Release engineering: [https://pagure.io/releng/issues #Releng issue number]
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with the Fedora Strategy: Nothing specific
== Upgrade/compatibility impact ==
Changes will be self-consistent, so upgrading from the previous
release should not be affected.
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
== Early Testing (Optional) ==
<!-- This is an optional step for system-wide changes to avail of. If
you would like to build an initial proof of concept of your change and
have a member of Fedora QA help you write and/or run some initial
basic tests on your code, please email tests(a)fedoraproject.org and
include the link to your change proposal. This step is *optional*. -->
Do you require 'QA Blueprint' support? Unclear yet
We will reach out to QA as soon as we have a proof of concept ready for testing.
== How To Test ==
<!-- This does not need to be a full-fledged document. Describe the
dimensions of tests that this change implementation is expected to
pass when it is done. This can be based off of the above section if
early testing has been completed. If it needs to be tested with
different hardware or software configurations, indicate them. The
more specific you can be, the better the community testing can be.
Remember that you are writing this how to for interested testers to
use to check out your change implementation - documenting what you do
for testing is OK, but it's much better to document what *I* can do to
test your change.
A good "how to test" should answer these four questions:
0. What special hardware / data / etc. is needed (if any)?
1. How do I prepare my system to test this change? What packages
need to be installed, config files edited, etc.?
2. What specific actions do I perform to check that the change is
working like it's supposed to?
3. What are the expected results of those actions?
-->
We will make a COPR available with updated packages. Updating an
existing system with the packages from this COPR should work and
result in a working system without any specific action.
== User Experience ==
If this is done properly, this should be transparent for users.
fwupd does this already, as does systemd-boot (installs itself into
/usr/lib/systemd/boot/efi). Other distributions, like Arch and Debian,
install boot loaders into /usr/lib/grub/<platform> rather than into
/boot directly.
== Dependencies ==
systemd:
* Extend package-notes to support PE files
* Add a new action to kernel-install script if this is the option that
is decided
bootupd:
* Update to handle the new filesystem paths for the bootloader and
shim components
* Add support to run as posttrans is this is the option that is decided
<!-- What other packages (RPMs) depend on this package? Are there
changes outside the developers' control on which completion of this
change depends? In other words, completion of another change owned by
someone else and might cause you to not be able to finish on time or
that you would need to coordinate? Other upstream projects like the
kernel (if this is not a kernel change)? -->
== Contingency Plan ==
If each component is updated consistently, then dependency on other
components is minimal. Components that are dependent, will wait until
dependencies have been fulfilled before making changes.
Example:
i) shim changes its installation location to `/usr/lib/shim/x86_64/`
and then runs a simple `cp` to move its components to /boot in
posttrans;
ii) shim script is added to `/usr/lib/kernel/install.d/` and shim
posttrans triggers the script, etc.
* Contingency deadline: Beta release
* Blocks release? Yes if not completed and not reverted
== Documentation ==
Documentation will have to be written.
== Release Notes ==
Fedora shim and grub2 packages are now installing content in
`/usr/lib/…` and not in /boot and `/boot/efi` directly.
--
Aoife Moloney
Fedora Operations Architect
Fedora Project
Matrix: @amoloney:fedora.im
IRC: amoloney
Wiki - https://fedoraproject.org/w/index.php?title=Changes/Trafficserver_10.0
Discussion thread -
https://discussion.fedoraproject.org/t/f42-change-proposal-trafficserver-10…
This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
Upgrade Apache Traffic Server in Fedora to version 10.0.
== Owner ==
* Name: [[User:jered| Jered Floyd]]
* Email: jered(a)redhat.com
== Detailed Description ==
Apache Traffic Server in Fedora will be upgraded to v10.0.
More details on Traffic Server 10.x are available from
[https://docs.trafficserver.apache.org/en/latest/release-notes/whats-new.en.…
the upstream What's New page]. Notably, there is a breaking change in
configuration file format, as ATS 10.x changes records.conf to
records.yaml and does not support the previous configuration format.
Many additional parameter changes and removals are also documented in
[https://docs.trafficserver.apache.org/en/latest/release-notes/upgrading.en.…
the upstream Upgrading page].
== Feedback ==
== Benefit to Fedora ==
Fedora includes the latest version of ATS, which will also be proposed
for EPEL 10.
== Scope ==
* Proposal owners:
** Upgrade trafficserver package to 10.x
** DIscuss proprietary of running convert2yaml.py on records.config as
part of post-install script
== Upgrade/compatibility impact ==
Users will be upgraded to Traffic Server 10.x. Daemon will not start
until records.config is updated to records.yaml.
== How To Test ==
Install package, or upgrade from older release. Configure, or update
configuration, then verify operation.
== User Experience ==
The user must update records.config to records.yaml before
trafficserver will restart after upgrade.
Alternatively, this could be triggered as a post-install scriptlet.
In most cases the daemon would run normally after upgrade, as long as
no removed features are in use (see upstream documentation).
== Dependencies ==
None outside of this Change.
== Contingency Plan ==
* Contingency mechanism: Roll back the trafficserver packages.
* Contingency deadline: Beta freeze
* Blocks release? No.
== Documentation ==
N/A (not a System Wide Change)
== Release Notes ==
Apache Traffic Server (trafficserver) in Fedora has been upgraded to
version 10.x. This introduces a breaking configuration change where
records.config must be updated to records.yaml. Additional upgrade
steps may be required if removed features or APIs are in use; please
review the upstream documentation at
https://docs.trafficserver.apache.org/en/latest/release-notes/upgrading.en.…
--
Aoife Moloney
Fedora Operations Architect
Fedora Project
Matrix: @amoloney:fedora.im
IRC: amoloney
Wiki - https://fedoraproject.org/wiki/Changes/Optimized_Binaries_for_the_AMD64_Arc…
Discussion thread -
https://discussion.fedoraproject.org/t/f42-change-proposal-optimized-binari…
This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
Individual packages can provide already optimized libraries via the
glibc-hwcaps mechanism. This approach will be extended to executables.
The package provides an optimized variant of a binary in a different
directory. A symlink to small program which replaces the binary in
`/usr/bin`. At runtime, this program will find the most appropriate
variant and execute it.
''Which'' packages provide the optimized code and at which level will
be made by individual package maintainers based on benchmark results.
A few programs/packages will be updated by the Change Owners to show
how the mechanism works.
== Owner ==
* Name: [[User:Zbyszek| Zbigniew Jędrzejewski-Szmek]]
* Name: [[User:Salimma| Michel Lind]]
* Name: José Relvas
* Emails: zbyszek(a)in.waw.pl, salimma(a)fedoraproject.org
== Detailed Description ==
This is an updated version of
[[Changes/Optimized_Binaries_for_the_AMD64_Architecture]].
Fedora binaries for the AMD64 / x86_64 architecture are compiled with
code-generation flags that support almost all CPU variants. But newer
generations of processors gained additional instructions that may be
used to generate faster code. A vendor-independent x86-64 psABI
supplement defines four "microachitecture levels": `x86-64-v1` (the
baseline, our code targets this), `x86-64-v2` (+`SSE3`, CentoOS
targets this), `x86-64-v3` (+`AVX`), `x86-64-v4` (+`AVX512`) [1]. When
code is compiled for a higher microarchitecture level it will crash
(with `SIGILL`, "illegal instruction") on CPUs which do not support
it. Benchmark results show small differences in performance: usually
in the range from -5% to 10%, with no discernible difference for most
code, but '''some''' applications benefit, with gains of 120% in some
benchmarks [e.g. 2, 4].
Over the years, various people have expressed interest in raising the
required microarchitecture levels. But we have been very conservative
in making changes, because support is missing in many older CPUs that
are still in use, and in fact, even in some CPUs produced and sold
today. By raising the required level we would make Fedora completely
unusable on many machines. It also seems that recompiling ''all''
packages with the changed options would largely be a waste of
resources, because for most code it makes no difference. But for some
of the numerical or cryptographic code there are noticeable gains and
it seems to be worth the effort to provide optimized code. This also
makes Fedora more attractive to people interested in optimization.
The dynamic linker already has the `glibc-hwcaps` mechanism to load
optimized implementations of ''shared objects'' [3]. This means that
packages can provide optimized libraries and they linker will be
automatically load them from separate directories if appropriate.
(For AMD64, this is `/usr/lib64/glibc-hwcaps/x86-64-v{2,3,4}/`.)
'''This Change is about extending the glibc-hwcaps mechanism to
''executables''. A small helper binary is provided. A program in
`/usr/bin` (or another path) is symlinked to this helper. When
executed, the helper checks the capabilities of the CPU and searches
for the most appropriate variant of the target program in a separate
directory hierarchy. If then launches one of the optimized binaries or
the "generic" one compiled for the baseline.'''
This means that individual packages "opt in", by moving their binary
to the alternative directory hierarchy and replacing it by a symlink,
and also providing one or more optimized variants.
Note: the ELF format provides the IFUNC mechanism to dynamically
select a variant of a function (symbol) when an executable is loaded
[5]. This is in particular used to load code using specific CPU
instructions when those are supported. This mechanism is both more
general (because it allows arbitrary selection criteria), more
fine-grained (because there can be other variants than just a few
fixed microarchitecture levels), and more efficient (because only the
parts of the code that benefit from this need to be provided in
multiple variants). In particular, glibc already makes extensive use
of this to provide optimized code, which is then widely used by other
libraries and programs. This means that even though we compile code in
a way where the lowest baseline is supported, modern CPU instructions
are already widely used. This is one of the reasons why compiling for
a higher baseline often doesn't make any difference in benchmarks. The
IFUNC mechanism or an equivalent mechanism should generally be
preferred. Nevertheless, that needs to be implemented in the program
or library itself, which is not trivial. The mechanism in this
Proposal is intended for the code which do not use IFUNCs or some
other similar mechanism.
[1] https://hackweek.opensuse.org/all/projects/support-glibc-hwcaps-and-micro-a…<BR>
[2] https://gitlab.archlinux.org/archlinux/rfcs/-/blob/master/rfcs/0002-march.r…<BR>
[3] https://sourceware.org/pipermail/libc-alpha/2021-February/122207.html<BR>
[4] https://blog.centos.org/2023/08/centos-isa-sig-performance-investigation/<BR>
[5] https://jasoncc.github.io/gnu_gcc_glibc/gnu-ifunc.html<BR>
Glibc-hwcaps together with the new helper provide a generic mechanism.
It will be up to individual packages to actually provide code which
makes use of it. Individual package maintainers are encouraged to
benchmark their packages after recompilation, and provide the
optimized variants if useful. (I.e. the code in question is measurably
faster '''and''' the program is run often enough for this to make a
difference.)
The Change Owners will implement the packaging changes for a few
packages while developing the general mechanism and will submit those
as pull requests. Other maintainers are asked to do the same for their
packages if desired.
Optimized variants of programs and libraries MAY be packaged in a
separate subpackage. The general packaging rules should be applied,
i.e. a separate package or packages SHOULD be created if it is files
are large enough.
Available benchmark results [2,4] are narrow and not very convincing.
We should plan an evaluation of results after one release. If it
turns out that the real gains are too small, we can scrap the effort.
On the other hand, we should also consider other architectures. For
example, microarchitecture levels `z{14,15}` for `s390x` or
`power{9,10}` for `ppc64le`. Other architectures are not included in
this Change Proposal to reduce its scope.
== Feedback ==
== Benefit to Fedora ==
The developers who are interested in this kind of optimization work
can perform it within Fedora, without having to build separate
repositories. The users who have the appropriate hardware will gain
performance benefits. Faster code is also more energy-efficient. The
change will be automatic and transparent to users.
Note that other distributions use higher microarchitecture levels. For
example RHEL 9 uses x86-64-v2 as the baseline, RHEL 10 uses x86-64-v3,
and other distros provide optimized variants
([https://en.opensuse.org/openSUSE:X86-64-Architecture-Levels
OpenSUSE], Arch Linux,
[https://ubuntu.com/blog/optimising-ubuntu-performance-on-amd64-architecture
Ubuntu]). We implement the same change in Fedora in a way that is
scoped more narrowly, and thus vastly cheaper in the sense of
development effort, code compilation time, storage and distribution
overhead, but should provide the same performance and energy benefits.
== Scope ==
* Proposal owners:
** Package [https://github.com/jrelvas-ipc/hwcaps-loader hwcaps-loader].
** Find some example packages to convert (the code must do "number
crunching" or string processing, and must not already use IFUNCs or
glibc-hwcaps or some other mechanism).
** Convert a few packages and submit the changes as pull requests.
** Submit a draft change to Packaging Guidelines
** Do benchmarks.
* Other developers:
** Consider converting some additional packages.
** Review and merge the Packaging Guidelines change
* Release engineering: [https://pagure.io/releng/issues #Releng issue number]
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with the Fedora Strategy:
== Upgrade/compatibility impact ==
== Early Testing (Optional) ==
Do you require 'QA Blueprint' support? N
== How To Test ==
== User Experience ==
The change should be invisible to users, except that some programs may
execute more quickly.
== Dependencies ==
== Contingency Plan ==
* Contingency mechanism: Revert changes in individual packages. This
can be either by the maintainers of those packages or by the Change
Owners using provenpackager privileges.
* Contingency deadline: any time really. The changes are independent
between packages, so we can trivially convert and uncovert individual
programs even after release.
* Blocks release? No
== Documentation ==
N/A (not a System Wide Change)
== Release Notes ==
--
Aoife Moloney
Fedora Operations Architect
Fedora Project
Matrix: @amoloney:fedora.im
IRC: amoloney
Wiki - https://fedoraproject.org/wiki/Changes/KojiLocalImageBuilder
Discussion thread -
https://discussion.fedoraproject.org/t/f42-change-proposal-koji-uses-red-ha…
This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
Switch the [https://osbuild.org/ Red Hat Image Builder]-built images
in Koji to not build through a service, but locally.
== Owner ==
* Name: [[User:supakeen| Simon de Vlieger]]
* Email: cmdr(a)supakeen.com
== Detailed Description ==
Several Fedora images are being built by the Red Hat Image Builder
service (Fedora IoT, Fedora Minimal). Image Builder acts as a
[https://docs.pagure.org/koji/content_generators/ content generator]
in Koji. Images are built external to Fedora infrastructure as
requested by Koji and then delivered back.
This has led to a lot of issues over time. Amongst them instability
and the inability of Fedora release engineering and infrastructure
teams to intervene and the inability to freeze the external
infrastructure during a release window.
I plan to replace the current approach in `koji-osbuild` with a
local-only build. Image Builder machinery and definitions will still
be used for the mentioned images but the build will run locally on
Fedora infrastructure and builders. In the same way that `kiwi` or
`livemediacreator`-images are built.
My plan is to start with adding these tasks to `koji-osbuild`, keeping
them there for a cycle and if they are stable to upstream them into
`koji` directly; deprecating `koji-osbuild` at that point in the
future.
`image-builder-cli` built images can be easily rebuilt locally from
their manifests uploaded in `koji`. The images can be easily built
locally and support
[https://osbuild.org/docs/user-guide/blueprint-reference all end-user
customizations available]. Work is ongoing in the Image Builder stack
to also move the definitions of the distribution to a declarative
format.
== Feedback ==
This change proposal does not yet provide a way for the distribution
definitions to be owned by, and live inside, Fedora infrastructure. I
plan to address this in a later change proposal.
== Benefit to Fedora ==
Control over Image Builder on its own infrastructure, lessen remove
the need to reach out to external parties when releases are blocked.
== Scope ==
=== Proposal Owners ===
* Submit the `image-builder-cli` package for inclusion in Fedora 42.
* Request a `image-builder-build` comp group for the build
requirements in koji roots.
* Change the `koji-osbuild` plugin to provide a new task type that
uses `image-builder-cli`.
* Changes to pungi to be able to schedule these tasks.
* Changes to pungi config for Fedora IoT.
* Changes to pungi config for Fedora Minimal.
== Contingency Plan ==
Independent of where on the above tasks I get stuck images can
continue to be built with the current setup.
== Release Notes ==
--
Aoife Moloney
Fedora Operations Architect
Fedora Project
Matrix: @amoloney:fedora.im
IRC: amoloney