Wiki - https://fedoraproject.org/wiki/Changes/Python_built_with_gcc_O3
Discussion.fpo -
https://discussion.fedoraproject.org/t/f41-change-proposal-python-built-wit…
This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
Instead of [https://docs.fedoraproject.org/en-US/packaging-guidelines/#_compiler_flags
Fedora's default `-O2` compiler flag], we will use `-O3` to build
CPython.
This only impacts the interpreter and Python standard library, not any
3rd party extension modules built as RPM or on developer machines.
This aligns with the way Python is built upstream.
According to our performance measurements, it makes Python
significantly faster (pyperformance geometric mean: 1.04x faster).
== Owner ==
* Name: [[User:churchyard|Miro Hrončok]]
* Email: mhroncok(a)redhat.com
== Detailed Description ==
We will replace the `-O2` compiler flag with `-O3` when building the
python3.13 package. This change may be backported to older Pythons if
desired. [[Changes/Python3.13|Python 3.13 should be the main Python
version in Fedora 41+]].
The [https://docs.fedoraproject.org/en-US/packaging-guidelines/#_compiler_flags
Fedora packaging guidelines] about compiler flags explicitly say:
''> Overriding these flags for performance optimizations (for
instance, `-O3` instead of `-O2`) is generally discouraged. If you can
present benchmarks that show a significant speedup for this particular
code, this could be revisited on a case-by-case basis.''
This change proposal presents such benchmarks and a case for Python to
use `-O3`.
This change is limited to CPython interpreter and extension modules
from the Python standard library only thanks to
[[Changes/Python_Extension_Flags_Reduction]] (since Fedora 39). Other
Python extension modules will remain bulidng as before, e.g. in RPM
packages, they will still be built with `-O2`, unless Fedora changes
that globally. The extension modules built with `-O2` still work with
Python built with `-O3`.
== Feedback ==
== Benefit to Fedora ==
Upstream already builds Python with `-O3` by default. Fedora's Python
built with `-O3` is faster (1.04x):
{| class="wikitable sortable"
|+ Benchmark with python3.12-3.12.2-3.fc41
|-
! Benchmark !! -O2 !! -O3 !!
Change !! Significance
|-
| 2to3 || 465 ms || 446 ms ||
1.04x faster || Significant (t=21.72)
|-
| async_generators || 853 ms || 784 ms ||
1.09x faster || Significant (t=36.61)
|-
| async_tree_cpu_io_mixed || 1.19 sec || 1.11 sec ||
1.08x faster || Significant (t=13.38)
|-
| async_tree_cpu_io_mixed_tg || 1.17 sec || 1.09 sec ||
1.08x faster || Significant (t=18.69)
|-
| async_tree_eager || 202 ms || 189 ms ||
1.07x faster || Significant (t=7.99)
|-
| async_tree_eager_cpu_io_mixed || 727 ms || 664 ms ||
1.09x faster || Significant (t=18.56)
|-
| async_tree_eager_cpu_io_mixed_tg || 633 ms || 558 ms ||
1.13x faster || Significant (t=24.53)
|-
| async_tree_eager_io || 1.72 sec || 1.68 sec ||
1.03x faster || Significant (t=6.13)
|-
| async_tree_eager_io_tg || 1.65 sec || 1.62 sec ||
1.02x faster || Significant (t=4.65)
|-
| async_tree_eager_memoization || 437 ms || 422 ms ||
1.04x faster || Significant (t=5.09)
|-
| async_tree_eager_memoization_tg || 330 ms || 322 ms ||
1.03x faster || Significant (t=2.60)
|-
| async_tree_eager_tg || 137 ms || 125 ms ||
1.09x faster || Significant (t=16.94)
|-
| async_tree_io || 1.64 sec || 1.60 sec ||
1.02x faster || Significant (t=9.49)
|-
| async_tree_io_tg || 1.65 sec || 1.61 sec ||
1.02x faster || Not significant
|-
| async_tree_memoization || 895 ms || 871 ms ||
1.03x faster || Significant (t=3.73)
|-
| async_tree_memoization_tg || 848 ms || 836 ms ||
1.01x faster || Not significant
|-
| async_tree_none || 718 ms || 700 ms ||
1.03x faster || Significant (t=6.90)
|-
| async_tree_none_tg || 686 ms || 659 ms ||
1.04x faster || Significant (t=13.11)
|-
| asyncio_tcp || 757 ms || 748 ms ||
1.01x faster || Not significant
|-
| asyncio_tcp_ssl || 2.58 sec || 2.56 sec ||
1.01x faster || Not significant
|-
| asyncio_websockets || 419 ms || 418 ms ||
1.00x faster || Not significant
|-
| bench_mp_pool || 10.7 ms || 10.7 ms ||
1.00x faster || Not significant
|-
| bench_thread_pool || 1.62 ms || 1.61 ms ||
1.01x faster || Not significant
|-
| chameleon || 12.2 ms || 12.0 ms ||
1.02x faster || Not significant
|-
| chaos || 113 ms || 105 ms ||
1.07x faster || Significant (t=46.23)
|-
| comprehensions || 37.4 us || 35.1 us ||
1.07x faster || Significant (t=49.72)
|-
| coroutines || 42.4 ms || 41.4 ms ||
1.02x faster || Significant (t=18.68)
|-
| coverage || 109 ms || 104 ms ||
1.05x faster || Significant (t=33.91)
|-
| create_gc_cycles || 1.84 ms || 1.79 ms ||
1.02x faster || Significant (t=5.50)
|-
| crypto_pyaes || 141 ms || 127 ms ||
1.11x faster || Significant (t=86.61)
|-
| dask || 766 ms || 769 ms ||
1.00x slower || Not significant
|-
| deepcopy || 619 us || 614 us ||
1.01x faster || Not significant
|-
| deepcopy_memo || 71.3 us || 68.3 us ||
1.04x faster || Significant (t=26.58)
|-
| deepcopy_reduce || 5.62 us || 5.56 us ||
1.01x faster || Not significant
|-
| deltablue || 5.76 ms || 5.49 ms ||
1.05x faster || Significant (t=7.97)
|-
| django_template || 62.8 ms || 59.7 ms ||
1.05x faster || Significant (t=27.05)
|-
| docutils || 4.38 sec || 4.29 sec ||
1.02x faster || Significant (t=11.25)
|-
| fannkuch || 706 ms || 667 ms ||
1.06x faster || Significant (t=75.80)
|-
| float || 144 ms || 137 ms ||
1.05x faster || Significant (t=24.66)
|-
| gc_traversal || 5.73 ms || 5.81 ms ||
1.01x slower || Not significant
|-
| generators || 56.0 ms || 58.2 ms ||
1.04x slower || Significant (t=-16.25)
|-
| genshi_text || 40.8 ms || 39.5 ms ||
1.03x faster || Significant (t=17.64)
|-
| genshi_xml || 88.2 ms || 86.3 ms ||
1.02x faster || Significant (t=6.96)
|-
| go || 223 ms || 217 ms ||
1.03x faster || Significant (t=19.92)
|-
| hexiom || 10.3 ms || 9.76 ms ||
1.05x faster || Significant (t=42.15)
|-
| html5lib || 109 ms || 108 ms ||
1.01x faster || Not significant
|-
| json_dumps || 17.4 ms || 16.3 ms ||
1.06x faster || Significant (t=45.38)
|-
| json_loads || 44.2 us || 42.3 us ||
1.04x faster || Significant (t=27.71)
|-
| logging_format || 12.9 us || 12.4 us ||
1.04x faster || Significant (t=9.81)
|-
| logging_silent || 176 ns || 174 ns ||
1.01x faster || Not significant
|-
| logging_simple || 11.4 us || 11.0 us ||
1.03x faster || Significant (t=9.94)
|-
| mako || 19.2 ms || 18.1 ms ||
1.06x faster || Significant (t=54.89)
|-
| mdp || 4.46 sec || 4.33 sec ||
1.03x faster || Significant (t=30.14)
|-
| meteor_contest || 189 ms || 167 ms ||
1.13x faster || Significant (t=60.31)
|-
| nbody || 157 ms || 153 ms ||
1.03x faster || Significant (t=4.34)
|-
| nqueens || 153 ms || 140 ms ||
1.09x faster || Significant (t=63.60)
|-
| pathlib || 32.9 ms || 32.6 ms ||
1.01x faster || Not significant
|-
| pickle || 18.6 us || 16.0 us ||
1.16x faster || Significant (t=23.88)
|-
| pickle_dict || 45.8 us || 44.6 us ||
1.03x faster || Significant (t=16.51)
|-
| pickle_list || 6.86 us || 6.59 us ||
1.04x faster || Significant (t=19.65)
|-
| pickle_pure_python || 515 us || 505 us ||
1.02x faster || Not significant
|-
| pidigits || 285 ms || 284 ms ||
1.00x faster || Not significant
|-
| pprint_pformat || 2.72 sec || 2.54 sec ||
1.07x faster || Significant (t=40.28)
|-
| pprint_safe_repr || 1.34 sec || 1.25 sec ||
1.08x faster || Significant (t=58.43)
|-
| pyflate || 738 ms || 724 ms ||
1.02x faster || Not significant
|-
| python_startup || 15.5 ms || 15.3 ms ||
1.01x faster || Not significant
|-
| python_startup_no_site || 11.2 ms || 11.0 ms ||
1.01x faster || Not significant
|-
| raytrace || 549 ms || 514 ms ||
1.07x faster || Significant (t=45.37)
|-
| regex_compile || 245 ms || 233 ms ||
1.05x faster || Significant (t=13.30)
|-
| regex_dna || 269 ms || 268 ms ||
1.00x faster || Not significant
|-
| regex_effbot || 4.83 ms || 4.95 ms ||
1.03x slower || Significant (t=-12.52)
|-
| regex_v8 || 33.7 ms || 33.1 ms ||
1.02x faster || Not significant
|-
| richards || 75.7 ms || 71.9 ms ||
1.05x faster || Significant (t=18.30)
|-
| richards_super || 85.2 ms || 81.4 ms ||
1.05x faster || Significant (t=31.25)
|-
| scimark_fft || 662 ms || 587 ms ||
1.13x faster || Significant (t=71.10)
|-
| scimark_lu || 199 ms || 190 ms ||
1.04x faster || Significant (t=26.77)
|-
| scimark_monte_carlo || 123 ms || 117 ms ||
1.05x faster || Significant (t=37.45)
|-
| scimark_sor || 217 ms || 210 ms ||
1.04x faster || Significant (t=10.68)
|-
| scimark_sparse_mat_mult || 8.51 ms || 7.42 ms ||
1.15x faster || Significant (t=62.99)
|-
| spectral_norm || 196 ms || 183 ms ||
1.07x faster || Significant (t=95.78)
|-
| sqlalchemy_declarative || 239 ms || 234 ms ||
1.02x faster || Significant (t=4.81)
|-
| sqlalchemy_imperative || 33.1 ms || 33.4 ms ||
1.01x slower || Not significant
|-
| sqlglot_normalize || 197 ms || 187 ms ||
1.05x faster || Significant (t=39.81)
|-
| sqlglot_optimize || 97.1 ms || 91.3 ms ||
1.06x faster || Significant (t=47.14)
|-
| sqlglot_parse || 2.29 ms || 2.18 ms ||
1.05x faster || Significant (t=14.70)
|-
| sqlglot_transpile || 2.79 ms || 2.67 ms ||
1.04x faster || Significant (t=11.76)
|-
| sqlite_synth || 3.97 us || 3.90 us ||
1.02x faster || Not significant
|-
| sympy_expand || 833 ms || 802 ms ||
1.04x faster || Significant (t=19.41)
|-
| sympy_integrate || 34.7 ms || 33.8 ms ||
1.03x faster || Significant (t=9.99)
|-
| sympy_str || 511 ms || 489 ms ||
1.04x faster || Significant (t=18.17)
|-
| sympy_sum || 286 ms || 278 ms ||
1.03x faster || Significant (t=14.46)
|-
| telco || 12.6 ms || 11.7 ms ||
1.08x faster || Significant (t=9.31)
|-
| tomli_loads || 3.91 sec || 3.56 sec ||
1.10x faster || Significant (t=46.29)
|-
| tornado_http || 213 ms || 212 ms ||
1.01x faster || Not significant
|-
| typing_runtime_protocols || 214 us || 196 us ||
1.09x faster || Significant (t=24.74)
|-
| unpack_sequence || 70.5 ns || 66.0 ns ||
1.07x faster || Significant (t=8.58)
|-
| unpickle || 24.3 us || 22.0 us ||
1.10x faster || Significant (t=10.67)
|-
| unpickle_list || 7.44 us || 8.61 us ||
1.16x slower || Significant (t=-45.10)
|-
| unpickle_pure_python || 390 us || 360 us ||
1.08x faster || Significant (t=37.48)
|-
| xml_etree_generate || 160 ms || 145 ms ||
1.10x faster || Significant (t=44.33)
|-
| xml_etree_iterparse || 189 ms || 180 ms ||
1.05x faster || Significant (t=20.16)
|-
| xml_etree_parse || 275 ms || 257 ms ||
1.07x faster || Significant (t=20.58)
|-
| xml_etree_process || 106 ms || 98.6 ms ||
1.08x faster || Significant (t=46.73)
|-
| Geometric mean || || ||
1.04x faster ||
|}
Generated by `pyperformance run -o Ox.json` and `pyperformance compare
-O table O2.json O3.json` on Fedora 40 x86_64 with rawhide-built
Python, python3.12-3.12.2-3.fc41, on Lenovo X1 Carbon 3rd gen.
The benchmark was performed on Python 3.12 because it uses 3rd party
Python packages lacking support for Python 3.13. Once it is possible
to run such a benchmark for Python 3.13, we will do so.
The benchmark was performed on x86_64. Until somebody presents a
contradicting benchmark (or gives explicit reason for us to measure
it), we believe the change makes sense on all architectures.
== Scope ==
* Proposal owners:
** Change python3.13 to build with `-O3` instead of `-O2`
** Backport the change to older Pythons if desired
* Other developers: no action expected, report bugs when found
* Release engineering: no action expected
* Policies and guidelines: this change is following the spirit of the guidelines
* Trademark approval: not needed for this Change
* Alignment with Community Initiatives: faster Python, happier users,
more contributors?
== Upgrade/compatibility impact ==
None expected.
== How To Test ==
To verify this change has landed, inspect the build.log of python3.13.
It should be built with `gcc ... -O3`.
To test this change, test Fedora as you would normally do and assert
there are no regressions.
Run benchmarks, and report slowdowns if found.
== User Experience ==
Faster Python, faster Fedora.
== Dependencies ==
* [[Changes/Python_Extension_Flags_Reduction]] landed in Fedora 39
* [[Changes/Python3.13]] is expected to land in Fedora 41. If not, we
will apply this on Python 3.12.
== Contingency Plan ==
* Contingency mechanism: revert the change, rebuild Python
* Contingency deadline: Final Freeze
* Blocks release? No
== Documentation ==
N/A (not a System Wide Change)
== Release Notes ==
--
Aoife Moloney
Fedora Operations Architect
Fedora Project
Matrix: @amoloney:fedora.im
IRC: amoloney
There will be an outage starting at 2024-06-27 21:00 UTC,
which will last approximately 1 hour.
To convert UTC to your local time, take a look at
http://fedoraproject.org/wiki/Infrastructure/UTCHowto
or run:
date -d '2024-06-27 21:00 UTC'
Reason for outage:
We will be moving the fedorapeople.org vm to a RHEL9 install. During the outage repos and access to the server will be unavailable as we sync data from the old vm.
Additionally, we will be pointing fedoraplanet.org to a new application that pulls rss feeds from the fedora account system instead of using .planet files.
Please make sure your rss feed(s) are set in your account to continue sindicating them on fedoraplanet.org
Affected Services:
fedorapeople.org and all data stored there.
Ticket Link:
https://pagure.io/fedora-infrastructure/issue/12008
Please join #fedora-admin or #fedora-noc on irc.libera.chat
or #admin:fedoraproject.org / #noc:fedoraproject.org on matrix.
Please add comments to the ticket for this outage above.
Updated status for this outage may be available at
https://www.fedorastatus.org/
Planned Outage - mailman - 2024-06-27 10:00 UTC
There will be an outage starting at 2024-06-27 10:00 UTC
which will last approximately 1 hour.
To convert UTC to your local time, take a look at
http://fedoraproject.org/wiki/Infrastructure/UTCHowto
or run:
date -d '2024-06-27 10:00UTC'
Reason for outage:
We will be upgrading the mailman to a newer version. During this outage
the mailing lists will be down.
Affected Services:
https://lists.fedoraproject.org/
Ticket Link:
https://pagure.io/fedora-infrastructure/issue/12004
Please join #fedora-admin or #fedora-noc on irc.libera.chat
or #admin:fedoraproject.org / #noc:fedoraproject.org on matrix.
Please add comments to the ticket for this outage above.
Updated status for this outage may be available at
https://www.fedorastatus.org/
Wiki - https://fedoraproject.org/wiki/Changes/ComposefsAtomicCoreOSIoT
Discussion Thread -
https://discussion.fedoraproject.org/t/f41-change-proposal-enabling-compose…
This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
We want to enable composefs by default for Fedora Atomic Desktops,
Fedora CoreOS and Fedora IoT. This makes the root mount of the system
(`/`) a truly read only filesystem, increasing the system integrity
and robustness. This is the first step toward a full ''at runtime''
verification of filesystem integrity.
This change will be enabled only for the Bootable Container images of
Fedora Atomic Desktops and not the classic ostree ones.
== Owner ==
* [[User:jbtrystram| Jean-Baptiste Trystram]], jbtrystram(a)redhat.com
* [[User:Siosm| Timothée Ravier]], siosm(a)fedoraproject.org
* [[User: pwhalen| Paul Whalen]], pwhalen(a)fedoraproject.org
== Detailed Description ==
Ostree based systems currently have `/usr` mounted as read-only and
managed by ostree/rpm-ostree. The integrity of the content of `/usr`
is only validated by ostree/rpm-ostree during updates and deployment
operations, but not at "runtime". If a file is corrupted on disk
(maliciously or not), it will only be detected if a full check is
performed using `ostree fsck`.
On those systems, the runtime root (`/`) of the system is currently
mounted as read-write but with the `immutable` bit set (`chattr +i /`)
to prevent accidental modifications.
[https://github.com/containers/composefs composefs] is a new project
that combines several existing filesystems (overlayfs, EROFS) to
provide a very flexible mechanism to support read-only mountable
filesystem trees, stacking on top of an underlying "lower" Linux
filesystem.
Using composefs, it will no longer be possible to mutate the
underlaying file content that is part of the system (`/usr`) nor the
layout of the root directory. It will result in I/O errors at the
kernel level.
The content is `/etc` and `/var` will remain writtable as it is today.
This change is part of the
[https://fedoraproject.org/wiki/Initiatives/Fedora_bootc Fedora
Bootable Containers Initiative]. The `bootc` container images already
enable composefs thus this change is to align existing variants to the
new Bootable Containers defaults.
It is tracked in:
* Fedora Atomic Desktops: https://gitlab.com/fedora/ostree/sig/-/issues/35
* Fedora CoreOS: https://github.com/coreos/fedora-coreos-tracker/issues/1718
* Fedora IoT: https://github.com/fedora-iot/iot-distro/issues/52
This is the first step toward a full boot chain integrity, that will
requiring signing the composefs metadata during composes and using
Unified Kernel Images (UKI). See:
https://gitlab.com/fedora/bootc/tracker/-/issues/14
As podman also use composefs to store containers layers, this enable
deduplication of files between containers and host. This will result
in less disk usage but also faster container startup and less memory
use. See https://github.com/containers/composefs/issues/125
== Feedback ==
Nothing specific so far.
We have the following "known issues":
* Conflicts with `ostree-grub2`, which impacts Dual Boot support:
** https://github.com/ostreedev/ostree/issues/3198
** We will remove ostree-grub2 from Fedora Atomic Desktops bootable
container images
** Related to: https://fedoraproject.org/wiki/Changes/FedoraSilverblueBootupd
** We can do this ''now'' for the container images as they are not
officially released for Fedora, and generally ''newer'', so it's more
likely that the bootloader on those systems are already BLS capable.
** We can not do this for "classic ostree" Atomic Desktops yet as we
need a transition period with bootupd enabled by default before
removing ostree-grub2.
** However with the recent Secure Boot issue
(https://github.com/fedora-silverblue/issue-tracker/issues/543)
forcing everybody to manually update their bootloader, we might be
able to shorten this transition period.
** See for Dual Boot:
https://github.com/fedora-silverblue/issue-tracker/issues/530
* No longer possible to create root level direcotries (`chattr -i` workaround):
** Requires derivation, thus the container flow
** https://github.com/coreos/rpm-ostree/issues/337
** Alternative: https://github.com/ostreedev/ostree/pull/3114
** Might impact Podman Desktop for Fedora CoreOS. They will likely
disable it until a solution is found.
* Issues with kdump:
** https://bugzilla.redhat.com/show_bug.cgi?id=2284097
** https://gitlab.com/fedora/bootc/tracker/-/issues/19
== Benefit to Fedora ==
This will increase the robustness of image based Fedora systems and
prepare them for future increased security guarantees.
This will align the existing image based variants of Fedora (Atomic
Desktops, CoreOS, IoT) to the work that is done as part of the
Bootable Containers Initiative.
== Scope ==
* Proposal owners:
** Enable composefs in Atomic Desktops (bootable containers only)
** Enable composefs in CoreOS
** Enable composefs in IoT
* Other developers:
** Applications doing disk-full checks on `/` will have to be updated
to look at other places as `/` will be small (a few MB) and full (100%
used).
* Release engineering: N/A (not needed for this Change)
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with the Fedora Strategy 2028:
** Aligns with the goal: "Immutable variants are the majority of
Fedora Linux in use"
== Upgrade/compatibility impact ==
To be fleshed out
== Early Testing (Optional) ==
Do you require 'QA Blueprint' support? N
== How To Test ==
* Make sure that you do not rely on Dual Boot support
* Make sure that you bootloader is recent enough to support BLS configs
** If you don't know, update it using the instructions from
https://github.com/fedora-silverblue/issue-tracker/issues/543#issuecomment-…
first
* Remove `ostree-grub2` from the upcoming deployment: `rpm-ostree
override remove ostree-grub2`
* Enable composefs: `sudo ostree config set ex-integrity.composefs yes`
* Update your system to a new version: `rpm-ostree update`
** Or do a manual (re)deploy of the current version: `sudo ostree
admin deploy fedora/39/x86_64/silverblue`
* Reboot into the new deployment
== User Experience ==
The main visible change will be that the root filesystem (`/`) is now
small and full (a few MB, 100% used). The real root is mounted in
`/sysroot` and most of the data is stored in `/var`.
== Dependencies ==
For the Atomic Desktops, this change depends on:
* Bootupd support:
** https://gitlab.com/fedora/ostree/sig/-/issues/1
** https://fedoraproject.org/wiki/Changes/FedoraSilverblueBootupd
CoreOS and IoT already do not depends on `ostree-grub2`.
== Contingency Plan ==
* Contingency mechanism: (What to do? Who will do it?) Undo the
change. It's a single line change in a configuration file.
* Contingency deadline: Beta Freeze / Release Freeze
* Blocks release? No
== Documentation ==
To be written.
== Release Notes ==
To be written once the change is accepted.
--
Aoife Moloney
Fedora Operations Architect
Fedora Project
Matrix: @amoloney:fedora.im
IRC: amoloney
Wiki - https://fedoraproject.org/wiki/Changes/Acpica-tools_Deprecate-Big_Endian_Su…
Discussion Thread -
https://discussion.fedoraproject.org/t/f41-change-proposal-acpica-tools-dep…
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
The acpica-tools package has supported big-endian architectures for
several years, but it has few uses. For Fedora 41, remove all of the
patches for big-endian support and remove s390x from the list of
supported architectures.
== Owner ==
* Name: [[User:ahs3| Al Stone]]
* Email: ahs3(a)fedoraproject.org
== Detailed Description ==
For many years, the acpica-tools package has supported the big-endian
s390x architecture, despite ACPI being a little-endian ''only''
standard. This made sense originally, as some s390x packages had to
have mechanisms for dealing with ACPI tables, and there were several
packages with build dependencies since they created ACPI tables.
Neither of these is really true any longer.
What's more, big-endian support has always had a fairly high cost. In
the first place, upstream wants nothing to do with big-endian support;
it is completely irrelevant, despite several attempts to get them to
accept it. Secondly, every release of the upstream source requires
significant backporting of the big-endian patches, creating new
patches to support new features of, or changes in, the ACPI standard,
and then testing and debugging the results. In the last ten years,
the least amount of time to do this was two full days; the longest it
has taken has been two full weeks. One has to plan for at least a
week. And finally, the need for big-endian has diminished
considerably. At one point, some virtualization functions required
these tools so that there was no virtualization on s390x without them;
again, this is no longer true.
The actual changes to the acpica-tools package will actually simplify
maintenance significantly. The change itself is fairly
straightforward: remove the big-endian patches, and add s390x to
ExcludeArch.
== Feedback ==
This proposal was actually prompted by a recent PR on acpica-tools
requesting a way to ignore the big-endian patches:
[https://src.fedoraproject.org/rpms/acpica-tools/pull-request/5]
Upstream has ''never'' accepted big-endian support. Should this
proposal be accepted, it would be one way of meeting the request
behind the PR, however. Some discussion has occurred with s390x
supporters (both recently, and in the past) that would indicate this
would be something that is not ideal (no one likes to lose packages),
but doesn't really create problems. Part of the reason for making
this proposal is to be sure I'm not missing something.
== Benefit to Fedora ==
The primary benefit to this proposal is that it simplifies maintenance
significantly. Of the 69 patches for the package, 49 can simply be
removed. The longest and most difficult builds will not be needed;
the s390x testing that always takes most of the package update time
will no longer be needed either. This will make it easier to ensure
that acpica-tools more closely matches upstream than has been possible
recently, producing a higher quality package quicker.
== Scope ==
* Proposal owners: The actual changes to the acpica-tools package will
be to remove the big-endian patches, and add s390x to ExcludeArch.
This is quite straightforward.
* Other developers: Based on an initial investigation, it appears that
most changes will be to simply exclude s390x. So far, the only
packages that seem to be affected are hw-probe, vim-syntastic-asl, and
edk2. There appears to be some use outside of Fedora
(xorg-x11-drv-nvidia, their proprietary drivers, for some reason).
* Release engineering: This change only affects the specific packages
mentioned so far.
* Policies and guidelines: N/A
* Trademark approval: N/A
== Upgrade/compatibility impact ==
Only s390x systems would notice a change in that this package would no
longer be available.
== How To Test ==
There is a test suite included in the acpica-tools package that is
always run as part of the %check step in the spec file.
No new testing should be required.
== User Experience ==
Only s390x systems would notice a change in that this package would no
longer be available.
== Dependencies ==
The packages hw-probe, vim-syntastic-asl, and edk2 depend on
acpica-tools. However, to make the changes to acpica-tools, there are
no other dependencies.
== Contingency Plan ==
* Contingency mechanism: do nothing; the existing f40 package can
continue to be used.
* Contingency deadline: beta freeze
* Blocks release? No
== Documentation ==
These changes actually put the Fedora version of the package back into
a state where it matches upstream more closely.
== Release Notes ==
--
Aoife Moloney
Fedora Operations Architect
Fedora Project
Matrix: @amoloney:fedora.im
IRC: amoloney
Planned Outage - wiki - 2024-06-18 21:00 UTC
There will be an outage starting at 2024-06-18 21:00 UTC
which will last approximately 2 hours.
To convert UTC to your local time, take a look at
http://fedoraproject.org/wiki/Infrastructure/UTCHowto
or run:
date -d '2024-06-18 21:00UTC'
Reason for outage:
We will be upgrading the wiki to a newer version. During this outage the wiki will be down.
Affected Services:
https://fedoraproject.org/wiki
Ticket Link:
https://pagure.io/fedora-infrastructure/issue/11994
Please join #fedora-admin or #fedora-noc on irc.libera.chat
or #admin:fedoraproject.org / #noc:fedoraproject.org on matrix.
Please add comments to the ticket for this outage above.
Updated status for this outage may be available at
https://www.fedorastatus.org/
Wiki - https://fedoraproject.org/wiki/Changes/NvidiaInstallationWithSecureboot
Discussion Thread -
https://discussion.fedoraproject.org/t/f41-change-proposal-nvidia-driver-in…
This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
Nvidia Drivers have been removed from GNOME Software because it didn't
support Secure Boot which is increasingly often enabled. This change
brings the option back with Secure Boot supported.
== Owner ==
* Name: [[User:eischmann|Jiří Eischmann]]
* Name: Milan Crha
* Email: eischmann(a)redhat.com
* Email: mcrha(a)redhat.com
== Detailed Description ==
The goal is this change is to provide an easy way to install Nvidia
drivers in Fedora Workstation. It was removed from GNOME Software
because the original mechanism didn't support Secure Boot. When users
installed the drivers with Secure Boot enabled, they could not boot
the OS.
What we're doing this time is using mokutil to create a key for the
user to self-sign the drivers. When installing the drivers, the user
is asked to provide a password for the key. On the next reboot the
user is presented with the mokutil interface to enroll the key.
See the [https://gitlab.gnome.org/GNOME/gnome-software/-/merge_requests/2034
upstream merge request] for more details and screenshots.
== Feedback ==
== Benefit to Fedora ==
The Nvidia drivers are necessary not only for gaming, but especially
for CUDA and AI/LLM workloads. The Nvidia drivers can't be part of
Fedora because of their license, but Fedora should offer an easy
installation of them to stay relevant in the respective fields.
== Scope ==
* Proposal Owners: The feature will be implemented in GNOME Software
47 and will be shipped in the gnome-software package in Fedora Linux
41.
* Other Developers: No work required from other Fedora developers. The
only requirement outside of the scope of the proposal owners is to
reintroduce AppStream metadata into the Nvidia driver repo on
RPMFusion.org.
* Release Engineering:
* Policies and Guidelines:
* Trademark approval:
* Alignment with Community Initiatives:
== Upgrade/compatibility impact ==
No impact is expected.
== How To Test ==
1. Open GNOME Software.<br>
2. Search for "nvidia".<br>
3. Choose the Nvidia driver, click Install and follow the prompts.<br>
4. Reboot and enroll the self-signing key in the mokutil tool
following <<the documentation will be added>><br>
5. The OS should boot up with the Nvidia driver enabled.<br>
== User Experience ==
This change aims to improve user experience of installing the
proprietary Nvidia driver.
== Contingency Plan ==
If the feature is not implemented on time for Fedora Linux 41, we can
simply remove AppStream metadata from the Nvidia driver repo and the
driver will not show up in GNOME Software like in Fedora Linux 40.
== Documentation ==
The GNOME Software part is intuitive and doesn't require
documentation. The mokutil part is less intuitive and will be
documented in the Fedora Workstation section on
docs.fedoraproject.org. The docs will be published when the feature
lands in Fedora Linux 41.
== Release Notes ==
--
Aoife Moloney
Fedora Operations Architect
Fedora Project
Matrix: @amoloney:fedora.im
IRC: amoloney
Wiki - https://fedoraproject.org/wiki/Changes/LibvirtVirtualNetworkNFTables
Discussion Thread -
https://discussion.fedoraproject.org/t/f41-change-proposl-libvirt-virtual-n…
This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
The default firewall backend for the default libvirt virtual network
(the virbr0 bridge device), will change from 'iptables' to 'nftables'.
== Owner ==
* Name: [[User:berrange| Daniel Berrange]]
* Email: berrange(a)redhat.com
== Detailed Description ==
A default installation of libvirt will create an isolated bridge
device (virbr0), to which guest virtual machines can have their NIC
connected. Firewall rules are used to provide NAT based connectivity
on this bridge device, as well as allow access to the DNS/DHCP
services provided by the dnsmasq it launches on the host.
Historically libvirt has always used the iptables tools, which first
talked to the iptables kernel modules, but in recent Fedora uses the
nftables kernel modules. When Fedora switched Firewalld to use
nftables for its own firewall rules, libvirt kept using its iptables
userspace backend, which then indirectly created kernel nftables
rules. To work correctly, libvirt added the ability to associate its
bridge devices with a 'libvirt' zone in firewall to ensure guest->host
DHCP/DNS/SSH traffic was not blocked by firewall.
With this change, libvirt will now prefer to directly use the nftables
userspace tools to create its firewall rules, where they are
installed:
* nftables only installed => libvirt uses its nftables backend
* iptables only intsalled => libvirt uses its iptables backend
* nftables and iptables installed => libvirt uses its nftables backend
This default behaviour can be overridden to force a specific backend
by editting settings in /etc/libvirt/network.conf.
Note that this change only applies to libvirt's virtual network
functionality. Libvirt's nwfilter (network filter) functionality is
still exclusively using iptables/ebtables. This will be switched to
nftables in a future Fedora release, timeframe to be determined.
== Feedback ==
TBD
== Benefit to Fedora ==
* Libvirt will be using the same recommended modern nftables userspace
as firewalld instead of the legacy iptables compatibility tools which
secretly talk to nftables behind the scenes.
* All the libvirt nftables rules will be self-contained in a dedicated
'libvirt_network' nftables table, reducing the scope for other
applications accidentally changing/removing them.
* Improved performance scalability since libvirt's nftables rules are
written to use an interface ID match which is a simple lookup, while
traditional string based interface name matches in iptables require
multiple string comparisons.
== Scope ==
* Proposal owners: Change the libvirt RPM spec to set the 'nftables'
backend as the preferred default
* Other developers: N/A
* Release engineering: N/A
* Policies and guidelines: N/A
* Trademark approval: N/A
* Alignment with the Fedora Strategy:
== Upgrade/compatibility impact ==
Existing hosts with libvirt deployed will be using iptables for their
firewall rules. When the Fedora upgrade brings in the new libvirt, it
will detect the switch to nftables and attempt to remove the
historically created iptables rules, and create new nftables rules
providing equivalent functionality.
A number of iptables chains will remain present after an upgrade
* Chain LIBVIRT_FWI
* Chain LIBVIRT_FWO
* Chain LIBVIRT_FWX
* Chain LIBVIRT_INP
* Chain LIBVIRT_OUT
These chains should not contain rules and thus be harmless. They will
go away permanently the next time the host is rebooted, or can be
deleted manually if desired.
=== Reverting to iptables for compatibility ===
If an incompatibility is discovered between libvirt's nftables backend
and some other use case, it is possible to tell libvirt to revert to
its iptables backend.
* Edit /etc/libvirt/network.conf and set 'firewall_backend = "iptables"'
* systemctl restart virtnetworkd
Libvirt should then automatically delete its nftables rules and create
equivalent iptables rules as present on previous Fedora releases.
Alternatively, if the 'nftables' userspace tools are not installed at
all, libvirt will attempt to use the 'iptables' userspace tools
instead, if present.
=== Known issue: docker ===
The Docker iptables integration force changes the iptables FORWARD
chain policy from ACCEPT to DENY. This makes '''Docker incompatible
with any application that is using nftables''' to try to allow
forwarding:
https://docs.docker.com/network/packet-filtering-firewalls/#docker-on-a-rou…
When libvirt uses its iptables backend, its FORWARD rules will end up
in the same table as docker's, and so override the DENY policy docker
had created. When libvirt uses its nftables backend, its FORWARD rules
end up in a separate table from dockers'. Since nftables requires a
packet to be allowed by *all* top level tables, docker's DENY rules
will block the libvirt traffic.
The possible workarounds are:
* Reconfigure libvirt to use iptables when compatibility with docker is required
* Use podman instead of docker, since podman does not create
problematic iptables rules
* TBD: whether libvirt can do something automagic to workaround
docker's DENY policy
=== Known issue: non-firewalld firewall mgmt tools ===
Libvirt requires the ability to mark traffic from the guest to the
host, for DHCP and DNS as permitted.
When using iptables kernel modules, libvirt could achieve this by
inserting some rules in the INPUT chain to allow DHCP/DNS.
When using nftables kernel modules, there are an arbitrary number of
application defined top level tables with unknown names. All top level
tables must allow a packet in order for it to pass. libvirt must not
touch tables created by other applications, thus it is no longer
practical to seemlessly allow DHCP & DNS when nftables userspace is
used by any component on the system.
To address this, when Fedora 32 switched firewall to its nftables
backend, libvirt gained ability to install a firewalld zone files to
allow guest traffic. Libvirt does not have equivalent integration for
any other nftables based firewall mgmt tool, but other tools using
iptables remained compatible as long as libvirt also kept using
iptables.
With libvirt changing to prefer its nftables backend, libvirt is will
be incompatible with any other non-firewalld firewall management
tools.
The compatibility matrix is as follows
# Kernel module: iptables, firewall mgmt userspace: iptables, libvirt
backend: iptables => WORKS for all firewall mgmt tools
# Kernel module: nftables, firewall mgmt userspace: iptables, libvirt
backend: iptables => WORKS for all firewall mgmt tools
# Kernel module: nftables, firewall mgmt userspace: iptables, libvirt
backend: nftables => WORKS for firewalld, other tools require
workaround
# Kernel module: nftables, firewall mgmt userspace: nftables, libvirt
backend: iptables => WORKS for firewalld, other tools require
workaround
# Kernel module: nftables, firewall mgmt userspace: nftables, libvirt
backend: nftables => WORKS for firewalld, other tools require
workaround
The two main workaround options:
* Change /etc/libvirt/network.conf 'firewall_backend' setting to 'iptables'
* Manually add rules to the firewall mgmt tool to allow DHCP, DNS &
SSH from guests.
The long term answer is to enhance libvirt upstream to natively learn
about integration for more firewall mgmt tools than just firewalld. eg
UFW: https://gitlab.com/libvirt/libvirt/-/issues/644
== Early Testing (Optional) ==
Do you require 'QA Blueprint' support? N
== How To Test ==
* Perform a default libvirt KVM install 'dnf groupinstall @Virtualization'
* Create a KVM guest using virt-install, virt-manager, or cockpit, and
configure it to use the 'default' virtual network
* Boot the KVM guest
* Login to the KVM guest graphical console, and attempt to connect to
the internet. eg curl https://google.com
* Configure the KVM guest to enable SSH (if not already allowed & started)
* Attempt to SSH to the guest from the host
== User Experience ==
* Libvirt firewall rules will no longer be present when running 'iptables -L'
* A new 'libvirt_network' table will be present when running 'nft list ruleset'
== Dependencies ==
None
== Contingency Plan ==
* Contingency mechanism: Libvirt maintainers to change the
libvirt.spec to set 'prefer_nftables' variable to '0' for Fedora 41
* Contingency deadline: Final freeze
* Blocks release? No
== Documentation ==
General libvirt network documentation at https://libvirt.org/formatnetwork.html
== Release Notes ==
The libvirt virtual network has been changed to prefer use of the
nftables firewall backend instead of iptables.
--
Aoife Moloney
Fedora Operations Architect
Fedora Project
Matrix: @amoloney:fedora.im
IRC: amoloney
Wiki - https://fedoraproject.org/wiki/Changes/ConfidentialVirtHostAMDSEVSNP
Discussion Thread -
https://discussion.fedoraproject.org/t/f41-change-proposal-confidential-vir…
This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
This enables Fedora virtualization hosts to launch confidential
virtual machines using AMD's SEV-SNP technology. Confidential
virtualization prevents admins with root shell access, or a
compromised host software stack, from accessing memory of any running
guest. SEV-SNP is an evolution of previously provided SEV and SEV-ES
technologies providing stronger protection and unlocking new features
such as a secure virtual TPM.
== Owner ==
* Name: [[User:berrange| Daniel P. Berrangé]]
* Email: berrange(a)redhat.com
== Detailed Description ==
Fedora has provided support for launching confidential virtual
machines using KVM on x86_64 hosts for several years, using the SEV
and SEV-ES technologies available from AMD CPUs. These technologies
have a number of design limitations, however, that make them less
secure than is desired, and prevent exposure of desirable features
such as secure TPMs. The SEV-SNP technology is a significant design
enhancement and architectural change to addresses the key gaps,
increasing security and unlocking more powerful use cases for
confidential virtual machines.
With SEV/SEV-ES, attestation of the launched VM has to be initiated
from the host, which means for a guest owner to attest their VM, they
need to rely on API services exposed by the virtualization management
stack which will vary across applications. With SEV-SNP, guest owners
can initiate attestation from within guest context, providing a
standard mechanism across any virtualization environment running
SEV-SNP, avoiding a reliance on any host platform tools/APIs.
The SEV-SNP technology also unlocks the ability to have a paravisor
run in guest context, which is a miniature OS that runs at a higher
privilege level (VMPL 0) than the guest firmware / OS (VMPL 3). The
paravisor, specifically the [https://github.com/coconut-svsm/svsm
Coconut SVSM] implementation, is able to expose trusted services to
the guest firmware and OS, which also remain inaccessible to the host.
This makes it possible to provide a secure virtual TPM for
confidential guests, thus allowing guest OS software to be better
secured than is the case with normal virtual TPMs running in host OS
context.
== Feedback ==
* TBD
== Benefit to Fedora ==
Confidential guests running under a Fedora SEV-SNP enabled KVM host
will be able to:
* Self initiate an VM attestation to prove integrity of their running
guest machine. This guarantees their guest is running on AMD hardware
with SEV-SNP setup in a given configuration, running a particular
build for EDK2 firmware, providing data confidentiality even if the
host is compromised or malicious.
* Measure all aspects of the guest machine boot process into PCRs in a
securely hosted virtual TPM
* Protect against various known weaknesses of the traditional SEV and
SEV-ES technologies
== Scope ==
=== Proposal owners ===
* Ensure kernel packages built in Fedora have SEV-SNP feature
integrated and enabled. Code is in kvm/next tree, targetted to merge
to linus's tree when 6.11 merge window opens. Will require backport to
Fedora's 6.10 tree.
* Ensure QEMU packages built in Fedora have SEV-SNP feature integrated
and enabled. Queued for merge into the forthcoming
[https://wiki.qemu.org/Planning/9.1 9.1.0 upstream release] - rc0 Jul
30th, GA Sep 3rd. F41 beta will ship a QEMU rc, and F41 GA will have
QEMU GA release.
* Ensure libvirt packages built in Fedora have SEV-SNP feature
integrated. Targeted to release immediately following merge into QEMU
git HEAD. Releases on 1st of each month, anticipated Sept 1st release
at latest, version 10.7.0, but ideally Aug 1st release.
* Ensure EDK2 packages built in Fedora have a EFI binary built
suitable for use with SEV-SNP guests with SVSM paravisor. EDK2
currently in rawhide has support for SNP + SVSM. Only lacking vTPM
support.
* Add new Coconut SVSM package, to provide paravisor for SEV-SNP
guests. Work in progress adding required rust deps.
* Add new IGVM library package, to enable bundling of Coconut SVSM and
EDK2 firmware into one launch binary. Work in progress adding required
rust deps.
* Ensure that an IGVM binary is built containing paired EDK2 and SVSM binaries.
* Ensure that virt-install is able to launch an SEV-SNP guest with SVSM and EDK2
=== Other developers ===
* Kernel maintainers: responsible for building new kernel that
includes SEV-SNP feature. Currently code is in kvm/next tree,
targetted for linus' tree when the 6.11 merge window opens. '''ONLY'''
if it merges to linus' tree, then proposal owners are to provide
Fedora kernel maintainers with a set of backported patches against
6.10 prior to Fedora Beta. Fedora 41 will get 6.11 kernels after GA.
=== Release engineering ===
N/A
=== Policies and guidelines ===
N/A
=== Trademark approval ===
N/A
=== Alignment with the Fedora Strategy ===
* Fedora will be demonstrating leading / state of the art integration
of SEV-SNP feature into a Linux distribution's virtualization host
stack.
* Fedora will be providing the fully OSS host-to-guest stack for
confidential virtual machines on modern AMD x86_64 EPYC hardware.
== Upgrade/compatibility impact ==
No impact anticipated. Existing users of SEV/SEV-ES technology will
keep using it without changes. The new SEV-SNP technology is strictly
an opt-in for users with suitably new AMD CPUs.
== Early Testing (Optional) ==
Do you require 'QA Blueprint' support? Y (probably useful?)
== How To Test ==
Use of AMD SEV-SNP technology requires an AMD EPYC CPU with the Zen 3
generation micro-architecture, or newer (Milan, Genoa). These are
identified by the 4th digit in the processor model name having the
value '3' or greater. eg EPYC7xx3 or EPYC9xx4. See also
[https://www.amd.com/content/dam/amd/en/documents/epyc-technical-docs/tuning…
SEV User Guide (pdf), Chapter 1] for CPU model support guidance.
* TBD: document process for configuring host bare metal firmware (if
applicable?)
* TBD: document process for deploying host software components. Likely
should not require any special steps, beyond the normal libvirt + KVM
install process.
* TBD: document process for creating a guest. Will update existing
[https://libvirt.org/kbase/launch_security_sev.html libvirt SEV docs]
to cover SEV-SNP
* TBD: document process for attesting a running guest. Again, will
update above libvirt SEV docs.
== User Experience ==
* Virtualization host owners will be launch confidential virtual
machines using AMD SEV-SNP technology
* Virtualization host owners will be able to provide a secure virtual
TPM to confidential guests, replacing the current non-confidential
swtpm solution in the host
* Guest owners will be able to prove that their OS is running in a
Fedora host confidential virtual machine protected by AMD SEV-SNP, by
performing a guest attestation
* Guest owners will be able to measure their guest OS software
environment using the TPM. Caveat: this is an ephemeral TPM initially,
which imposes limits on the ways in which users can take advantage of
it. A persistent TPM will be supported at a later date.
== Dependencies ==
* kernel
* edk2
* coconut-svsm (new package [https://github.com/coconut-svsm/svsm
github upstream])
** rust-intrusive-collections (new package,
[https://bugzilla.redhat.com/show_bug.cgi?id=2290692 Review Request])
** rust-packit (new package or include as vendored)
* igvm (new package [https://github.com/microsoft/igvm/ github upstream])
** rust-hmac-sha512 (new package,
[https://bugzilla.redhat.com/show_bug.cgi?id=2290698 Review Request])
** rust-bitfield-struct (new package,
[https://bugzilla.redhat.com/show_bug.cgi?id=2290696 Review Request])
** rust-open-enum (new package)
** rust-open-enum-derive (new package)
** rust-range-map-vec (new package,
[https://bugzilla.redhat.com/show_bug.cgi?id=2282977 Review Request]:
Approved)
* qemu
* libvirt
* virt-manager (for virt-install tool)
* snphost [https://github.com/virtee/snphost github upstream]
* snpguest (new package, [https://github.com/virtee/snpguest github upstream])
== Contingency Plan ==
* Contingency mechanism: None. All the work is arriving either via
rebases to new upstream versions of existing packages, patches to
existing packages, or via new package reviews. If the deadline is
missed, then whatever has already arrived in Fedora will remain in
Fedora and will not harm other existing Fedora functionality. The
remaining pieces will wait until the next Fedora dev cycle to be
integrated, completing the desired user experience. As such, no
contingency action needs to be taken should the Fedora feature
completion deadline be missed.
* Contingency deadline: N/A
* Blocks release? No
== Documentation ==
* Insert link to kernel docs, when available
* Insert link to QEMU docs, when available
* Insert link to libvirt docs, when available
* Write a Fedora wiki page illustrating end-to-end setup and usage example
== Release Notes ==
--
Aoife Moloney
Fedora Operations Architect
Fedora Project
Matrix: @amoloney:fedora.im
IRC: amoloney