Hi all,
For background: back in January, Neal and I presented a kernel refactoring proposal at CentOS Connect https://cfp.fedoraproject.org/centos-connect-2026/talk/X9JHNV/
This is informed by our work in CentOS' Hyperscale SIG, which builds faster-moving backports (many coming from Fedora) and rebuilds stock CentOS Stream packages with additional features. So - some of our work there could potentially feed back to Fedora; see my previous CentOS Showcase talk for examples - https://cfp.fedoraproject.org/centos-showcase-2024-11/talk/PTV8JM/
For the particular case of perf, we think it might make sense to split out the packaging from the kernel and build it standalone, for the reasons below:
- while perf is mostly stable on x86_64, on aarch64 it is still getting a lot of fixes. e.g. the last update I made incorporated 20 cs-etm fixes: https://gitlab.com/CentOS/Hyperscale/rpms/perf/-/commit/8f8b93b559f9204d1188... - perf build has some BuildRequires that are not needed for building the kernel itself. Isolating the builds help reduce the number of BRs needed on both sides (including, yes, *Java*) - separating the two makes it easier for interested contributors to help out - the kernel.spec is huge and kernel-ark packaging takes a while to get used to. - Also while kernel modules need to be specially signed for Secure Boot, perf does not need to be! - one pain point we have is that perf headers inconsistently show up in kernel-headers. If, say from F46 onwards perf is shipped standalone and perf-devel is the only place that ships perf headers, we can at least stop special-casing this. see e.g. https://gitlab.com/CentOS/Hyperscale/rpms/perf/-/blob/8f8b93b559f9204d1188be... - perf's maintainer (acme) has mentioned in the past that almost nobody consumed the standalone perf tarballs - indeed, 6.15 came out with a bug where some aarch64 headers were left out so the tarball cannot be used to build on that architecture. With a standalone packaging, like in Hyperscale, we can consume the upstream tarball, but fall back to using the full Linux tarball when necessary (or generating our own tarballs from specific Git tags; the makefile targets make this easy)
Hoping to get some discussions going, and especially buy in from Justin Forbes (cc:ed) - if generally this seems like a good idea, we're going to submit a Change Proposal for Fedora 46. Anyone interested is welcome to join, we're hoping to have at least the two of us and Justin onboard.
Best regards,
On Wed, Sep 16, 2026 at 10:43 AM Michel Lind salimma@fedoraproject.org wrote:
Hi all,
For background: back in January, Neal and I presented a kernel refactoring proposal at CentOS Connect https://cfp.fedoraproject.org/centos-connect-2026/talk/X9JHNV/
This is informed by our work in CentOS' Hyperscale SIG, which builds faster-moving backports (many coming from Fedora) and rebuilds stock CentOS Stream packages with additional features. So - some of our work there could potentially feed back to Fedora; see my previous CentOS Showcase talk for examples - https://cfp.fedoraproject.org/centos-showcase-2024-11/talk/PTV8JM/
For the particular case of perf, we think it might make sense to split out the packaging from the kernel and build it standalone, for the reasons below:
- while perf is mostly stable on x86_64, on aarch64 it is still getting
a lot of fixes. e.g. the last update I made incorporated 20 cs-etm fixes: https://gitlab.com/CentOS/Hyperscale/rpms/perf/-/commit/8f8b93b559f9204d1188...
- perf build has some BuildRequires that are not needed for building
the kernel itself. Isolating the builds help reduce the number of BRs needed on both sides (including, yes, *Java*)
- separating the two makes it easier for interested contributors to
help out - the kernel.spec is huge and kernel-ark packaging takes a while to get used to.
- Also while kernel modules need to be specially signed for Secure
Boot, perf does not need to be!
- one pain point we have is that perf headers inconsistently show up in
kernel-headers. If, say from F46 onwards perf is shipped standalone and perf-devel is the only place that ships perf headers, we can at least stop special-casing this. see e.g. https://gitlab.com/CentOS/Hyperscale/rpms/perf/-/blob/8f8b93b559f9204d1188be...
- perf's maintainer (acme) has mentioned in the past that almost nobody
consumed the standalone perf tarballs - indeed, 6.15 came out with a bug where some aarch64 headers were left out so the tarball cannot be used to build on that architecture. With a standalone packaging, like in Hyperscale, we can consume the upstream tarball, but fall back to using the full Linux tarball when necessary (or generating our own tarballs from specific Git tags; the makefile targets make this easy)
Hoping to get some discussions going, and especially buy in from Justin Forbes (cc:ed) - if generally this seems like a good idea, we're going to submit a Change Proposal for Fedora 46. Anyone interested is welcome to join, we're hoping to have at least the two of us and Justin onboard.
So here is my problem with it... You have to convince RHEL. Otherwise I have to maintain it in the kernel spec for ELN and in Fedora as a separate package. We tried for years to get kernel-tools/perf as a separate package inside RHEL, and it was always shot down which is why we eventually moved Fedora back into kernel. You might see if acme is interested in fighting that fight. Perhaps he would have better luck with perf specifically. If RHEL is willing, and we can rip it out of the kernel spec completely, I am fine with this. If not though, we essentially end up with 2 maintenance paths for perf and I am not interested in that.
Justin
Best regards,
-- _o) Michel Lind _( ) https://keyoxide.org/5dce2e7e9c3b1cffd335c1d78b229d2f7ccc04f2 README: https://fedoraproject.org/wiki/User:Salimma#README
[CCing Fedora rarely used kernel list]
On 10/1/26 17:44, Justin Forbes wrote:
On Wed, Sep 16, 2026 at 10:43 AM Michel Lind salimma@fedoraproject.org wrote:
For the particular case of perf, we think it might make sense to split out the packaging from the kernel and build it standalone, for the reasons below:
I understand the reasons from your POV. And I don't care too much (even if what I write below might sound overly again a standalone perf; sorry for that), but I fail somewhat to see how Fedora benefits from the split.
- while perf is mostly stable on x86_64, on aarch64 it is still getting
a lot of fixes. e.g. the last update I made incorporated 20 cs-etm fixes: https://gitlab.com/CentOS/Hyperscale/rpms/perf/-/commit/8f8b93b559f9204d1188...
What does that mean in the end? Will the perf standalone release that is shipped be ahead, round about on-par, or behind the perf in the sources of the kernel that Fedora stable currently ships? If ahead: but it nevertheless is considered stable, yes?
- perf build has some BuildRequires that are not needed for building
the kernel itself. Isolating the builds help reduce the number of BRs needed on both sides (including, yes, *Java*)
So the kernel build might be a bit faster, but two packages instead of one need to be build, which has overhead as well. So there is only a benefit if the separate perf is build a lot less, yes?
- separating the two makes it easier for interested contributors to
help out - the kernel.spec is huge and kernel-ark packaging takes a while to get used to.
+1
OTOH some people might expect that the perf they get from Fedora always matches the kernel Fedora ships, as that is what everybody is doing right now. Wonder if breaking that expectation could lead to confusion, but well, guess I'm overly careful here, so feel free to ignore this.
- Also while kernel modules need to be specially signed for Secure
Boot, perf does not need to be!
Yeah, but that was the case before that, so I wonder if that should be in this list.
- one pain point we have is that perf headers inconsistently show up in
kernel-headers. If, say from F46 onwards perf is shipped standalone and perf-devel is the only place that ships perf headers, we can at least stop special-casing this. see e.g. https://gitlab.com/CentOS/Hyperscale/rpms/perf/-/blob/8f8b93b559f9204d1188be...
Can/should this be fixed with our current packaging?
- perf's maintainer (acme) has mentioned in the past that almost nobody
consumed the standalone perf tarballs - indeed, 6.15 came out with a bug where some aarch64 headers were left out so the tarball cannot be used to build on that architecture. With a standalone packaging, like in Hyperscale, we can consume the upstream tarball, but fall back to using the full Linux tarball when necessary (or generating our own tarballs from specific Git tags; the makefile targets make this easy)
Hoping to get some discussions going, and especially buy in from Justin Forbes (cc:ed) - if generally this seems like a good idea, we're going to submit a Change Proposal for Fedora 46. Anyone interested is welcome to join, we're hoping to have at least the two of us and Justin onboard.
So here is my problem with it... You have to convince RHEL. Otherwise I have to maintain it in the kernel spec for ELN and in Fedora as a separate package. We tried for years to get kernel-tools/perf as a separate package inside RHEL, and it was always shot down which is why we eventually moved Fedora back into kernel. You might see if acme is interested in fighting that fight. Perhaps he would have better luck with perf specifically. If RHEL is willing, and we can rip it out of the kernel spec completely, I am fine with this. If not though, we essentially end up with 2 maintenance paths for perf and I am not interested in that.
The following should not block things, but I'll mention it: right now I regularly build vanilla kernels (including stable-rc and next) for https://copr.fedorainfracloud.org/groups/g/kernel-vanilla/coprs/ -- which often finds build errors in perf/libperf I report upstream, to prevent them from hitting Fedora later. That relies on the perf packaging in kernel.spec, so it would be a shame if that would go (but okay, other people then will likely run into and report that).
Ciao, Thorsten
On 2026-10-01 20:18, Thorsten Leemhuis via devel wrote:
I understand the reasons from your POV. And I don't care too much (even if what I write below might sound overly again a standalone perf; sorry for that), but I fail somewhat to see how Fedora benefits from the split.
Fedora Python maintainer here.
We would love to be able to rebuild perf for Python packaging changes without the need to rebuild kernel (which we cannot rebuild ourselves, due to missing signing permissions). Ever since pytohn3-perf has been included in the kernel component, we need to treat that package specially and ping Justin to rebuild it for us.
On Wed, 7 Oct 2026 at 12:16, Miro Hrončok via devel devel@lists.fedoraproject.org wrote:
On 2026-10-01 20:18, Thorsten Leemhuis via devel wrote:
I understand the reasons from your POV. And I don't care too much (even if what I write below might sound overly again a standalone perf; sorry for that), but I fail somewhat to see how Fedora benefits from the split.
Fedora Python maintainer here.
We would love to be able to rebuild perf for Python packaging changes without the need to rebuild kernel (which we cannot rebuild ourselves, due to missing signing permissions). Ever since pytohn3-perf has been included in the kernel component, we need to treat that package specially and ping Justin to rebuild it for us.
Is that really a major issue since most of the time rawhide kernels get an almost daily build to keep up with Linus's tree?
On 2026-10-07 13:25, Peter Robinson wrote:
On Wed, 7 Oct 2026 at 12:16, Miro Hrončok via devel devel@lists.fedoraproject.org wrote:
On 2026-10-01 20:18, Thorsten Leemhuis via devel wrote:
I understand the reasons from your POV. And I don't care too much (even if what I write below might sound overly again a standalone perf; sorry for that), but I fail somewhat to see how Fedora benefits from the split.
Fedora Python maintainer here.
We would love to be able to rebuild perf for Python packaging changes without the need to rebuild kernel (which we cannot rebuild ourselves, due to missing signing permissions). Ever since pytohn3-perf has been included in the kernel component, we need to treat that package specially and ping Justin to rebuild it for us.
Is that really a major issue since most of the time rawhide kernels get an almost daily build to keep up with Linus's tree?
Yes, because we need to build it in our side tag(s).
Justin is usually very helpful with this, but it requires treating the package specially and introduces a human blocker nevertheless.
-
Justin Forbes -
Michel Lind -
Miro Hrončok -
Peter Robinson -
Thorsten Leemhuis