https://fedoraproject.org/wiki/Changes/PortingToModernC.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.
== Summary == Back in 1999, a new revision of the C standard removed several backwards compatibility features. However, GCC still accepts these obsolete constructs by default. Support for these constructs is confusing to programmers and potentially affect GCC's ability to implement features from future C standards.
== Owner == * Name: [[User:fweimer| Florian Weimer]] * Email: [mailto:fweimer@redhat.com| fweimer@redhat.com]
== Detailed Description == The most important change is the <b>removal of implicit function declarations</b>. This legacy compatibility feature causes the compiler to automatically supply a declaration of type `int function()` if `function` is undeclared, but called as a function. However, the implicit return type of `int` is incompatible with pointer-returning functions, which can lead to difficult debugging sessions if the compiler warning is missed, as described here:
* [https://developers.redhat.com/blog/2019/04/22/implicit-function-declarations... Implicit function declarations: flex's use of "reallocarray"]
On x86-64, functions returning `_Bool` called with an implicit declaration will also not work correctly because the return register value for `_Bool` functions is partially undefined (not all 32 bits in the implicitly declared `int` are defined).
Another change for C99 is the <b>removal of implicit `int` type declarations</b>. For example, in the following fragment, both `i` and `j` are defined as `int` variables:
<pre> static i = 1.0; auto j = 2.0; </pre>
Support for this obsolete constructs is incompatible with adoption of type inference for `auto` definitions (which are part of C++).
Neither change is trivial to implement because introducing errors for these constructs (as required by C99) alters the result of autoconf configure checks. Quite a few such checks use an implicitly declared `exit` function, for instance. These failures are not really related to the feature under test. If the build system is well written, the build still succeeds, the relevant features are automatically disabled in the test suite and removed from reference ABI lists, and it's not immediately apparent that feature is gone. Therefore, some care is needed that no such alterations happen, and packages need to be ported to C99. Various tools for this porting activity are being developed to support this proposal. Cross-distribution collaboration will help as well, sharing patches and insights.
GCC 14 may also make other changes by default, as described below. These changes will also require extensive testing, and some adoption of configure checks.
==== Removal of old-style function definitions ====
An old style function definition looks like this: <pre> int sum (a, b) char *a; int b; { return atoi (a) + b; } </pre> It is equivalent to this definition with a prototype: <pre> int sum (char *a, int b) { return atoi (a) + b; } </pre> This legacy feature is very close to being incompatible with the C2X standard, which introduces unnamed function arguments. But due to a quirk in GCC's implementation, it should be possible to support both at the same time.
==== New keywords <code>bool</code>, <code>true</code>, <code>false</code> ====
Packages which supply their own definitions instead of including `<stdbool.h>` (which remains available) might fail to build.
==== Change of meaning of <code>()</code> in function declarators ====
In earlier C versions, a declaration `int function()` declares `function` as accepting an unspecified number of arguments of unknown type. This means that both `function(1)` and `function("one")` compile, even though they might not work correctly. In a future C standard, `int function()` will mean that `function` does not accept any parameters (like in C++, or as if written as `int function(void)` in current C). Calls that specify parameters will therefore result in compilation errors.
We believe that this change is ABI-compatible, even on ppc64le (with its parameter save area), although it comes very close to an implicit ABI break.
==== Rejecting implicit conversions between integers and pointers as errors ====
Currently, GCC does not treat conversion between integers at pointers as errors, so this compiles:
<pre> int ptr = "string"; </pre>
However, this is very unlikely to work on 64-bit architectures (it may work by accident without PIE/PIC).
== Feedback ==
The transition plan has been discussed at various GNU Tools Cauldrons. Feedback has been generally positive, except a general worry about the work required.
Without a deliberate porting effort, a lot of breakage occurs, [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... seen in 2016 in Fedora with an uncoordinated change], and more recently [https://discourse.llvm.org/t/configure-script-breakage-with-the-new-werror-i... an uncoordinated Clang update].
== Benefit to Fedora == Programmers will no longer waste time tracking down things that look eerily like compiler or ABI bugs because in several cases, builds will fail with a clear error message, instead of producing a warning that is easily missed. Potential blockers to adoption of further C language changes are removed.
== Scope == * Proposal owners: Update rawhide over time to be compatible with strict C99 compilers.
* Other developers: Help out with non-obvious porting issues, and with upstreaming patches in case active upstreams cannot be easily identified. Tolerate early backports of upstream-submitted fixes in Fedora dist-git.
* Release engineering: [https://pagure.io/releng/issues #Releng issue number]
* Policies and guidelines: Fedora compiler policy will likely change due to the adoption of GCC 14 in Fedora 40.
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives: N/A
== Upgrade/compatibility impact == The change is expected to be transparent to those users who do not use C compilers. No features are supposed to be added or removed as a result. In fact, most of the porting effort focuses on avoiding configuration changes.
== How To Test == General regression testing is sufficient.
== User Experience == User experience does not change.
== Dependencies == To avoid regressing the porting effort, GCC as the system compiler needs to reject obsolete constructs by default. This is expected for GCC 14, to be released as part of Fedora 40 in Spring 2024.
== Contingency Plan == * Contingency mechanism: Upstream GCC will probably accept obsolete constructs longer in case distributions like Fedora cannot port to modern C in time. * Blocks release? No if GCC doesn't switch. If GCC switches, we have to complete the port.
== Documentation == This section will be updated once more documentation of the porting effort will become available.
== Release Notes == Probably not needed because no user visible impact is intended.
On Tue, Oct 25, 2022 at 10:05 AM Ben Cotton bcotton@redhat.com wrote:
== Upgrade/compatibility impact == The change is expected to be transparent to those users who do not use C compilers. No features are supposed to be added or removed as a result. In fact, most of the porting effort focuses on avoiding configuration changes.
[deletia]
== Release Notes == Probably not needed because no user visible impact is intended.
These two sections seem in conflict. Wouldn't this Change affect users who compile C software? Or is it only a difference in our build tooling?
* Ben Cotton:
On Tue, Oct 25, 2022 at 10:05 AM Ben Cotton bcotton@redhat.com wrote:
== Upgrade/compatibility impact == The change is expected to be transparent to those users who do not use C compilers. No features are supposed to be added or removed as a result. In fact, most of the porting effort focuses on avoiding configuration changes.
[deletia]
== Release Notes == Probably not needed because no user visible impact is intended.
These two sections seem in conflict. Wouldn't this Change affect users who compile C software? Or is it only a difference in our build tooling?
This aspect will be covered by the GCC change proposal when the time comes (probably with GCC 14/Fedora 40, but it's hard to make predictions so far in the future).
For each individual package, I expect the required changes mostly within the scope of a typical binutils/GCC/glibc update that requires some changes to source code.
Thanks, Florian
On Tue, Oct 25, 2022 at 10:05:52AM -0400, Ben Cotton wrote:
== Owner ==
- Name: [[User:fweimer| Florian Weimer]]
- Email: [mailto:fweimer@redhat.com| fweimer@redhat.com]
== Detailed Description == The most important change is the <b>removal of implicit function declarations</b>. This legacy compatibility feature causes the compiler to automatically supply a declaration of type `int function()` if `function` is undeclared, but called as a function. However, the implicit return type of `int` is incompatible with pointer-returning functions, which can lead to difficult debugging sessions if the compiler warning is missed, as described here:
Implicit function declarations: flex's use of "reallocarray"]
On x86-64, functions returning `_Bool` called with an implicit declaration will also not work correctly because the return register value for `_Bool` functions is partially undefined (not all 32 bits in the implicitly declared `int` are defined).
Another change for C99 is the <b>removal of implicit `int` type declarations</b>. For example, in the following fragment, both `i` and `j` are defined as `int` variables:
<pre> static i = 1.0; auto j = 2.0; </pre>
Support for this obsolete constructs is incompatible with adoption of type inference for `auto` definitions (which are part of C++).
Neither change is trivial to implement because introducing errors for these constructs (as required by C99) alters the result of autoconf configure checks. Quite a few such checks use an implicitly declared `exit` function, for instance. These failures are not really related to the feature under test. If the build system is well written, the build still succeeds, the relevant features are automatically disabled in the test suite and removed from reference ABI lists, and it's not immediately apparent that feature is gone. Therefore, some care is needed that no such alterations happen, and packages need to be ported to C99. Various tools for this porting activity are being developed to support this proposal. Cross-distribution collaboration will help as well, sharing patches and insights.
Good to see it mentions autoconf, as such tests were my immediate concern upon reading the intro.
I presume the same problem will exist in other build systems that probe for features, both well known ones like cmake/meson/etc, but also home grown ones that are package specific (/me glares at QEMU's 'configure' that merely pretends to look like it is from autoconf, but is actually just a hand written shell script).
== Dependencies == To avoid regressing the porting effort, GCC as the system compiler needs to reject obsolete constructs by default. This is expected for GCC 14, to be released as part of Fedora 40 in Spring 2024.
So this change is talking about a new GCC landing in Fedora 40.
To avoid massive disruption to Fedora though, we need to be doing work waaaaay earlier than the Fedora 40 dev cycle though.
Identifying all the places where autoconf tests quietly change their result, and silently toggle on/off alternative code paths is going to be a big job on its own.
Once identified there's the even more work to figure out the right changes needed. I don't know how many packages rely on autoconf, but the idea of carrying patches in Fedora and re-running autotools to re-create configure, isn't that appealing on a large scale.
Obviously the preferred solution is to be able to report the problems upstream, and get the fixes included in the next upstream release and then rebase in Fedora. Even better is if a major information campaign brings this change directly to the attention of upstreams communities, bypassing the distros where there is an active upstream.
I'd be concerned about the timeframe for getting through this dance for a non-negligible number of packages.
IOW, this change is talking about something in Fedora 40, but it feels like the vast majority of work needs to take place in Fedora 38 and Fedora 39. Fedora 40 neeeds to be nothing more than a "flip the switch" scenario, otherwise history suggests the change is likely to end up getting punted to Fedora 41/42.
I feel that this need to start on the prep work in F38/F39 is not very obvious from the F40 change proposal text.
So shouldn't we have explicit distinct 'Fedora 38/39' change proposal(s) to describe the prep work that needs to be done across packages in these two releases as a matter of urgency, in order to make it possible to the F40 change to actually happen ?
Concretely, as an upstream maintainer, what should they do to test the behaviour of their code ? Is there more to it than just setting CFLAGS="-std=gnu99", if they want to validate this in their upstream CI ahead of GCC 14 arriving ?
With regards, Daniel
On Tue, Oct 25, 2022 at 10:51 AM Daniel P. Berrangé berrange@redhat.com wrote:
On Tue, Oct 25, 2022 at 10:05:52AM -0400, Ben Cotton wrote:
== Owner ==
- Name: [[User:fweimer| Florian Weimer]]
- Email: [mailto:fweimer@redhat.com| fweimer@redhat.com]
== Detailed Description == The most important change is the <b>removal of implicit function declarations</b>. This legacy compatibility feature causes the compiler to automatically supply a declaration of type `int function()` if `function` is undeclared, but called as a function. However, the implicit return type of `int` is incompatible with pointer-returning functions, which can lead to difficult debugging sessions if the compiler warning is missed, as described here:
Implicit function declarations: flex's use of "reallocarray"]
On x86-64, functions returning `_Bool` called with an implicit declaration will also not work correctly because the return register value for `_Bool` functions is partially undefined (not all 32 bits in the implicitly declared `int` are defined).
Another change for C99 is the <b>removal of implicit `int` type declarations</b>. For example, in the following fragment, both `i` and `j` are defined as `int` variables:
<pre> static i = 1.0; auto j = 2.0; </pre>
Support for this obsolete constructs is incompatible with adoption of type inference for `auto` definitions (which are part of C++).
Neither change is trivial to implement because introducing errors for these constructs (as required by C99) alters the result of autoconf configure checks. Quite a few such checks use an implicitly declared `exit` function, for instance. These failures are not really related to the feature under test. If the build system is well written, the build still succeeds, the relevant features are automatically disabled in the test suite and removed from reference ABI lists, and it's not immediately apparent that feature is gone. Therefore, some care is needed that no such alterations happen, and packages need to be ported to C99. Various tools for this porting activity are being developed to support this proposal. Cross-distribution collaboration will help as well, sharing patches and insights.
Good to see it mentions autoconf, as such tests were my immediate concern upon reading the intro.
I presume the same problem will exist in other build systems that probe for features, both well known ones like cmake/meson/etc, but also home grown ones that are package specific (/me glares at QEMU's 'configure' that merely pretends to look like it is from autoconf, but is actually just a hand written shell script).
== Dependencies == To avoid regressing the porting effort, GCC as the system compiler needs to reject obsolete constructs by default. This is expected for GCC 14, to be released as part of Fedora 40 in Spring 2024.
So this change is talking about a new GCC landing in Fedora 40.
To avoid massive disruption to Fedora though, we need to be doing work waaaaay earlier than the Fedora 40 dev cycle though.
Identifying all the places where autoconf tests quietly change their result, and silently toggle on/off alternative code paths is going to be a big job on its own.
Once identified there's the even more work to figure out the right changes needed. I don't know how many packages rely on autoconf, but the idea of carrying patches in Fedora and re-running autotools to re-create configure, isn't that appealing on a large scale.
Obviously the preferred solution is to be able to report the problems upstream, and get the fixes included in the next upstream release and then rebase in Fedora. Even better is if a major information campaign brings this change directly to the attention of upstreams communities, bypassing the distros where there is an active upstream.
I'd be concerned about the timeframe for getting through this dance for a non-negligible number of packages.
IOW, this change is talking about something in Fedora 40, but it feels like the vast majority of work needs to take place in Fedora 38 and Fedora 39. Fedora 40 neeeds to be nothing more than a "flip the switch" scenario, otherwise history suggests the change is likely to end up getting punted to Fedora 41/42.
I feel that this need to start on the prep work in F38/F39 is not very obvious from the F40 change proposal text.
So shouldn't we have explicit distinct 'Fedora 38/39' change proposal(s) to describe the prep work that needs to be done across packages in these two releases as a matter of urgency, in order to make it possible to the F40 change to actually happen ?
Concretely, as an upstream maintainer, what should they do to test the behaviour of their code ? Is there more to it than just setting CFLAGS="-std=gnu99", if they want to validate this in their upstream CI ahead of GCC 14 arriving ?
Why C99? Why not C18 instead? If we're going to go through this effort, we should ramp up like we did for C++ and bump all the way up to the latest published standard.
* Neal Gompa:
Why C99? Why not C18 instead? If we're going to go through this effort, we should ramp up like we did for C++ and bump all the way up to the latest published standard.
Getting past C99 is the hardest part because it's not possible (as far as I can see) to do fully automated reporting for implicit function declarations. (Because we don't implement, say, setproctitle, that's going to show up as a legitimate configure check failure.)
The other issues (e.g., implicit ints, true/falls/bool redefined, () for variadic parameter lists) are more tractable because those have to be removed completely. So their presence is always worth reporting.
Thanks, Florian
* Daniel P. Berrangé:
Good to see it mentions autoconf, as such tests were my immediate concern upon reading the intro.
I presume the same problem will exist in other build systems that probe for features, both well known ones like cmake/meson/etc, but also home grown ones that are package specific (/me glares at QEMU's 'configure' that merely pretends to look like it is from autoconf, but is actually just a hand written shell script).
Yes, some distributions use config.log diffing. I've got an instrumented GCC that logs the relevant errors in a way that is impossible to hide. There are pros and cons for each approach.
I found one non-autoconf issue already:
Unclear how CCompiler.has_function works for functions with parameters https://github.com/pypa/setuptools/issues/3648
This is actually quite typical of the things that are papered over by implicit function declarations.
== Dependencies == To avoid regressing the porting effort, GCC as the system compiler needs to reject obsolete constructs by default. This is expected for GCC 14, to be released as part of Fedora 40 in Spring 2024.
So this change is talking about a new GCC landing in Fedora 40.
To avoid massive disruption to Fedora though, we need to be doing work waaaaay earlier than the Fedora 40 dev cycle though.
Yes, I understand that we might be too late already. I wanted to get the announcement out of the door that we can start carrying patches for this in rawhide starting now (particularly where we might have cascading effects on our testing, like the has_function issue mentioned above).
I don't think I can do this work on a branch because keeping that branch up-to-date will be too much work on its own.
Identifying all the places where autoconf tests quietly change their result, and silently toggle on/off alternative code paths is going to be a big job on its own.
Right, but other distributions are doing this as well (Gentoo has already begun working on this from a Clang 16 perspective), and we are trying to come up with ways to share observations and patches.
One tough aspect is that this affects disproportionately legacy software for which there is no upstream around anymore, or the sources are difficult to change because they are supposed to keep building on very, very old systems. We can switch to building in C89 mode in these cases.
I feel that this need to start on the prep work in F38/F39 is not very obvious from the F40 change proposal text.
I mentioned that package maintainers need to tolerate the odd patch to fix issues in rawhide. That's what I'm asking for today.
Concretely, as an upstream maintainer, what should they do to test the behaviour of their code ? Is there more to it than just setting CFLAGS="-std=gnu99", if they want to validate this in their upstream CI ahead of GCC 14 arriving ?
It's -Werror=implicit-int -Werror=implicit-function-declaration. Best to throw in -Werror=int-conversion -Werror=strict-prototypes -Werror=old-style-definition. -Werror=int-conversion should really be a no-brainer, the other two are about the removal of old-style function definitions and declarations that are not prototypes from C2X (also mentioned in the proposal). I can probably spell this out in the proposal.
Some of the planned Clang 16+ and GCC 14+ changes may not be directly testable with current compilers, but they will be less impactful than the two main issues, I think. That's why I'm focusing to get us past the C99 bump first.
Thanks, Florian
...
Concretely, as an upstream maintainer, what should they do to test the behaviour of their code ? Is there more to it than just setting CFLAGS="-std=gnu99", if they want to validate this in their upstream CI ahead of GCC 14 arriving ?
It's -Werror=implicit-int -Werror=implicit-function-declaration. Best to throw in -Werror=int-conversion -Werror=strict-prototypes -Werror=old-style-definition. -Werror=int-conversion should really be a no-brainer, the other two are about the removal of old-style function definitions and declarations that are not prototypes from C2X (also mentioned in the proposal). I can probably spell this out in the proposal.
Even better: could you put together an RPM macro we can use in specfiles to essentially opt-in early to the F40 Change?
* Stephen Gallagher:
...
Concretely, as an upstream maintainer, what should they do to test the behaviour of their code ? Is there more to it than just setting CFLAGS="-std=gnu99", if they want to validate this in their upstream CI ahead of GCC 14 arriving ?
It's -Werror=implicit-int -Werror=implicit-function-declaration. Best to throw in -Werror=int-conversion -Werror=strict-prototypes -Werror=old-style-definition. -Werror=int-conversion should really be a no-brainer, the other two are about the removal of old-style function definitions and declarations that are not prototypes from C2X (also mentioned in the proposal). I can probably spell this out in the proposal.
Even better: could you put together an RPM macro we can use in specfiles to essentially opt-in early to the F40 Change?
Mentioned elsewhere in the thread, repeating in case it got lost:
I think we need a buildroot/side tag with a patched compiler. I filed a request for it a while back:
Long-term side tag for toolchain experiment https://pagure.io/releng/issue/10392
Flag injection isn't reliable and can't be used to model all the expected changes.
Thanks, Florian
On Tue, Oct 25, 2022 at 5:09 PM Daniel P. Berrangé berrange@redhat.com wrote:
So this change is talking about a new GCC landing in Fedora 40.
To avoid massive disruption to Fedora though, we need to be doing work waaaaay earlier than the Fedora 40 dev cycle though.
Identifying all the places where autoconf tests quietly change their result, and silently toggle on/off alternative code paths is going to be a big job on its own.
Once identified there's the even more work to figure out the right changes needed. I don't know how many packages rely on autoconf, but the idea of carrying patches in Fedora and re-running autotools to re-create configure, isn't that appealing on a large scale.
Obviously the preferred solution is to be able to report the problems upstream, and get the fixes included in the next upstream release and then rebase in Fedora. Even better is if a major information campaign brings this change directly to the attention of upstreams communities, bypassing the distros where there is an active upstream.
I'd be concerned about the timeframe for getting through this dance for a non-negligible number of packages.
IOW, this change is talking about something in Fedora 40, but it feels like the vast majority of work needs to take place in Fedora 38 and Fedora 39. Fedora 40 neeeds to be nothing more than a "flip the switch" scenario, otherwise history suggests the change is likely to end up getting punted to Fedora 41/42.
Since it's a build-time-only change, can it be rolled out under controlled pressure like this?
1. every package explicitly opts out (with some macro in specfile, IDK) 2. switch gets flipped, nothing changes 3. bugs get filed to drop that macro and opt into new behaviour 4. opting in gets resolved at a leisurely pace across as many releases as needed?
I feel that this need to start on the prep work in F38/F39 is not very obvious from the F40 change proposal text.
So shouldn't we have explicit distinct 'Fedora 38/39' change proposal(s) to describe the prep work that needs to be done across packages in these two releases as a matter of urgency, in order to make it possible to the F40 change to actually happen ?
Concretely, as an upstream maintainer, what should they do to test the behaviour of their code ? Is there more to it than just setting CFLAGS="-std=gnu99", if they want to validate this in their upstream CI ahead of GCC 14 arriving ?
* Alexander Sosedkin:
Since it's a build-time-only change, can it be rolled out under controlled pressure like this?
- every package explicitly opts out (with some macro in specfile, IDK)
- switch gets flipped, nothing changes
- bugs get filed to drop that macro and opt into new behaviour
- opting in gets resolved at a leisurely pace across as many releases as needed?
Sorry, what's the advantage of doing it this way? We'd have to change all packages that have a C compiler in their buildroots as part of the first step. Then we file thousands and thousands of bugs, and hope that the maintainers take action? I'm not sure that's going to work for those maintain dozens (hundreds?) of packages. It's also kind of difficult for a Fedora developer to predict GCC 14 behavior in 2024 today.
Relying on individual developer action also means that we'd have to teach many more people about C arcana and autoconf corner cases. I don't think that's a good learning investment to be honest. Most of this knowledge was already obsolete in 1999.
I don't really know how much time we have before GCC disabled legacy C89 extensions by default because some of them are incompatible with future C language directions. I'm not convinced things will resolve themselves over time. Obviously there's been some progress in the last 25 years (not everyone uses GCC and Clang with their extensions to build upstream software), but it's been really slow so far. (Back in 2019, I quickly found a bunch of really core packages that needed fixes.) It's also not clear to what degree the Macos efforts that Clemens mentioned have reached upstreams. It doesn't seeem to have helped the Clang 15 release much.
To me, all these things argue against spreading out the porting effort. I wish we could do it this way, but it doesn't look like it's a feasible option.
Thanks, Florian
On Tue, Oct 25, 2022 at 7:42 PM Florian Weimer fweimer@redhat.com wrote:
- Alexander Sosedkin:
Since it's a build-time-only change, can it be rolled out under controlled pressure like this?
- every package explicitly opts out (with some macro in specfile, IDK)
- switch gets flipped, nothing changes
- bugs get filed to drop that macro and opt into new behaviour
- opting in gets resolved at a leisurely pace across as many releases as needed?
Sorry, what's the advantage of doing it this way?
Spreading out the porting effect =)
Maybe I'm scarred by my own unsuccessful attempt to spread out SHA-1 switch flipping across releases, but this sounds like a switch that'll need maintainers to not just react, but also coordinate with upstreams, and Fedora's tight cycle is uncomfortably tight.
We'd have to change all packages that have a C compiler in their buildroots as part of the first step.
OK, maybe just the ones failing to build with C99.
Then we file thousands and thousands of bugs, and hope that the maintainers take action? I'm not sure that's going to work for those maintain dozens (hundreds?) of packages. It's also kind of difficult for a Fedora developer to predict GCC 14 behavior in 2024 today.
Relying on individual developer action also means that we'd have to teach many more people about C arcana and autoconf corner cases. I don't think that's a good learning investment to be honest. Most of this knowledge was already obsolete in 1999.
I don't really know how much time we have before GCC disabled legacy C89 extensions by default because some of them are incompatible with future C language directions. I'm not convinced things will resolve themselves over time. Obviously there's been some progress in the last 25 years (not everyone uses GCC and Clang with their extensions to build upstream software), but it's been really slow so far. (Back in 2019, I quickly found a bunch of really core packages that needed fixes.) It's also not clear to what degree the Macos efforts that Clemens mentioned have reached upstreams. It doesn't seeem to have helped the Clang 15 release much.
To me, all these things argue against spreading out the porting effort.
¯_(ツ)_/¯
I wish we could do it this way, but it doesn't look like it's a feasible option.
One cycle or many, I wish you to find the best way to pull this off.
* Alexander Sosedkin:
On Tue, Oct 25, 2022 at 7:42 PM Florian Weimer fweimer@redhat.com wrote:
- Alexander Sosedkin:
Since it's a build-time-only change, can it be rolled out under controlled pressure like this?
- every package explicitly opts out (with some macro in specfile, IDK)
- switch gets flipped, nothing changes
- bugs get filed to drop that macro and opt into new behaviour
- opting in gets resolved at a leisurely pace across as many releases as needed?
Sorry, what's the advantage of doing it this way?
Spreading out the porting effect =)
Maybe I'm scarred by my own unsuccessful attempt to spread out SHA-1 switch flipping across releases, but this sounds like a switch that'll need maintainers to not just react, but also coordinate with upstreams, and Fedora's tight cycle is uncomfortably tight.
I think the porting changes here are much more mechanical. There's no need to come up with new configuration options or anything like that.
For about an hour, I looked at three random packages (literally, I used Python's random module to pick them) that were flagged by failing rebuilds: kcc, openblas, swift-lang.
kcc has no upstream but is easily fixed. I have something I could push to dist-git. We could also build in C89 mode. There's no obvious upstream anymore. Although now we seem to have some generic Koji issues that causes all builds to fail.
openblas has *very* *recent* upstream work that we'd need to cherry-pick and then test-build. Martin Kroeker seems to have been working on exactly these kind of fixes. It's slightly time consuming because it's necessary to iterate to be sure to have all fixes, and openblas doesn't exactly build quickly.
swift-lang turns out to be a false positive because of failing configure checks that are expected to fail. That's going to be fixed by adjusting the redhat-rpm-config allowed-to-fail function list.
So it seems to amount of per-package changes are fairly manageable.
Thanks, Florian
On Tue, 2022-10-25 at 15:51 +0100, Daniel P. Berrangé wrote:
I feel that this need to start on the prep work in F38/F39 is not very obvious from the F40 change proposal text.
I thought this was kinda very obviously implicit in the decision to file an F40 Change proposal *now*.
Hi everyone,
Ben Cotton bcotton@redhat.com wrote:
Neither change is trivial to implement because introducing errors for these constructs (as required by C99) alters the result of autoconf configure checks. Quite a few such checks use an implicitly declared `exit` function, for instance. These failures are not really related to the feature under test. If the build system is well written, the build still succeeds, the relevant features are automatically disabled in the test suite and removed from reference ABI lists, and it's not immediately apparent that feature is gone. Therefore, some care is needed that no such alterations happen, and packages need to be ported to C99. Various tools for this porting activity are being developed to support this proposal. Cross-distribution collaboration will help as well, sharing patches and insights.
I may have some good news here: When Apple started shipping arm64 hardware, it switched its system compiler to always treat implicit function declarations as errors. Apple engineers did that because the calling ABI for variadic functions on arm64 differs from that of non-variadic functions, so a declaration is required to determine the correct code to generate.
As a consequence, the various macOS package managers have been working on identifying and fixing these autoconf issues for a while now. Obviously, there is a lot of Linux-specific software that nobody will have fixed yet, and the package set may be smaller in the macOS package managers, but commonly used software is probably already fixed.
HTH, Clemens
Am 25.10.22 um 16:05 schrieb Ben Cotton:
https://fedoraproject.org/wiki/Changes/PortingToModernC.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.
IMHO, this proposal doesn't make any sense.
Fedora is a collection of packages, which all obey different sets of language dialects, have various different upstreams and care about Fedora to varying extents.
I.e. all Fedora can do is to switch on "modern" C-dialects as C-defaults, but that's all.
Ralf
* Ralf Corsépius:
Am 25.10.22 um 16:05 schrieb Ben Cotton:
https://fedoraproject.org/wiki/Changes/PortingToModernC. This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.
IMHO, this proposal doesn't make any sense.
Fedora is a collection of packages, which all obey different sets of language dialects, have various different upstreams and care about Fedora to varying extents.
I.e. all Fedora can do is to switch on "modern" C-dialects as C-defaults, but that's all.
Which part doesn't make sense?
Gentoo is doing similar work (and we're going to collaborate) just to be able to build the distribution with Clang 16 and later, which is going to make changes break C89 compatibility in the default configuration.
I think GCC should eventually make this change as well because there are significant usability improvements possible. We won't have much of a choice once C2Y (not next year's release but the one after that) adopts auto for local type inference because that conflicts with a legacy C89 compatibility features still enabled by GCC by default. Doing this now means that “auto copy = strdup(source);” is not valid C99 to C2X, so we can detect it more easily. Once upstreams start using the auto keyword in C sources, this is going to be much harder. So right now is probably the absolutely last point in term when this whole thing is going to be a lot of work, but not *too* complicated.
We will have to build some parts of the distribution in C89 mode potentially indefinitely, but some work is required to actually achieve that. We need to know which packages not to switch, and make sure that whatever mechanism we use actually works for their (often weird, old) build systems.
Thanks, Florian
On Tue, Oct 25, 2022 at 10:05:52AM -0400, Ben Cotton wrote:
Neither change is trivial to implement because introducing errors for these constructs (as required by C99) alters the result of autoconf configure checks. Quite a few such checks use an implicitly declared `exit` function, for instance. These failures are not really related to the feature under test. If the build system is well written, the build still succeeds, the relevant features are automatically disabled in the test suite and removed from reference ABI lists, and it's not immediately apparent that feature is gone. Therefore, some care is needed that no such alterations happen, and packages need to be ported to C99. Various tools for this porting activity are being developed to support this proposal. Cross-distribution collaboration will help as well, sharing patches and insights.
Would it help if we made autoreconf (and the equivalent step for other build systems) mandatory? Debian has declared this as "good practice" since forever[0].
It would mean we could fix any problems in current autoconf concurrently with the changes to GCC and Fedora.
==== Removal of old-style function definitions ====
An old style function definition looks like this:
int sum (a, b) char *a; int b; { return atoi (a) + b; }
<sad-face>
I preferred it when C was like this .. The first real paying job I had used a compiler[1] that only supported K&R C decls.
Rich.
[0] https://wiki.debian.org/Autoreconf [1] Microware C Compiler for OS-9
* Richard W. M. Jones:
On Tue, Oct 25, 2022 at 10:05:52AM -0400, Ben Cotton wrote:
Neither change is trivial to implement because introducing errors for these constructs (as required by C99) alters the result of autoconf configure checks. Quite a few such checks use an implicitly declared `exit` function, for instance. These failures are not really related to the feature under test. If the build system is well written, the build still succeeds, the relevant features are automatically disabled in the test suite and removed from reference ABI lists, and it's not immediately apparent that feature is gone. Therefore, some care is needed that no such alterations happen, and packages need to be ported to C99. Various tools for this porting activity are being developed to support this proposal. Cross-distribution collaboration will help as well, sharing patches and insights.
Would it help if we made autoreconf (and the equivalent step for other build systems) mandatory? Debian has declared this as "good practice" since forever[0].
It would mean we could fix any problems in current autoconf concurrently with the changes to GCC and Fedora.
I'm not aware of any recent-ish core autoconf issues that would be solved by running autoreconf (if that actually worked …). As far as I can tell, autoconf never relied on implicit function declarations for AC_CHECK_FUNC, not since 1993. The sed hack we already have is for LTO compatibility.
We would have to sync aclocal.m4 (or what feeds into aclocal.m4) from autoconf-archive and then run autoreconf, but I don't think that we have any automation for that today. And not everything comes from autoconf-archive, a lot of stuff is just copied around manually. Most of it is unproblematic in this context, fortunately. The few remaining issues we have need to be patched individually.
Thanks, Florian
On 10/26/22 02:58, Florian Weimer wrote:
- Richard W. M. Jones:
On Tue, Oct 25, 2022 at 10:05:52AM -0400, Ben Cotton wrote:
Neither change is trivial to implement because introducing errors for these constructs (as required by C99) alters the result of autoconf configure checks. Quite a few such checks use an implicitly declared `exit` function, for instance. These failures are not really related to the feature under test. If the build system is well written, the build still succeeds, the relevant features are automatically disabled in the test suite and removed from reference ABI lists, and it's not immediately apparent that feature is gone. Therefore, some care is needed that no such alterations happen, and packages need to be ported to C99. Various tools for this porting activity are being developed to support this proposal. Cross-distribution collaboration will help as well, sharing patches and insights.
Would it help if we made autoreconf (and the equivalent step for other build systems) mandatory? Debian has declared this as "good practice" since forever[0].
It would mean we could fix any problems in current autoconf concurrently with the changes to GCC and Fedora.
I'm not aware of any recent-ish core autoconf issues that would be solved by running autoreconf (if that actually worked …). As far as I can tell, autoconf never relied on implicit function declarations for AC_CHECK_FUNC, not since 1993. The sed hack we already have is for LTO compatibility.
Right. I nearly chimed in on this topic. It helps if we can autoreconf as I think that might let us remove some of the LTO insanity, but it's independent of implicit functions.
Jeff
On 25. 10. 22 16:05, Ben Cotton wrote:
== Summary == Back in 1999, a new revision of the C standard removed several backwards compatibility features. However, GCC still accepts these obsolete constructs by default. Support for these constructs is confusing to programmers and potentially affect GCC's ability to implement features from future C standards.
Could you please update the summary with the summary of what's being changed rather than the summary of the status quo? I think that'll be more helpful.
On Tue, Oct 25, 2022 at 10:05:52AM -0400, Ben Cotton wrote:
Line-by-line review:
== Summary == Back in 1999, a new revision of the C standard removed several backwards compatibility features. However, GCC still accepts these obsolete constructs by default. Support for these constructs is confusing to programmers and potentially affect GCC's ability to implement features from future C standards.
Miro already hinted at this, but to be explicit: this Summary does not actually give the information that should be in a summary. It provides some history that maybe belongs in the Detailed Description part.
This part should say something like """gcc compiler will start rejecting some long-deprecated constructs in C and implement some other backwards-incompatible changes. Packages and build systems for user programs must be adjusted to be compatible, or opt-out of the change by specifying ….
The following constructs are now disallowed by default: implicit function declarations, implicit int type declarations, old-style K&R function parameter declarations, and in addition functions declared with no parameters will now accept no instead of any parameters, and new 'bool', 'true', 'false' keywords have been introduced."""
(I assume this is about gcc, not clang. But please be explicit.)
== How To Test ==
As discussed by others, this needs to explain how to opt-in for early testing, and also how to opt-out in packages, how to opt-out as a user (if the default touches users), etc.
== User Experience == User experience does not change.
"The new default for C standard is -c99. Users who want to use an older standard need to specify something like -c89." (???)
== Dependencies == To avoid regressing the porting effort, GCC as the system compiler needs to reject obsolete constructs by default. This is expected for GCC 14, to be released as part of Fedora 40 in Spring 2024.
== Contingency Plan ==
- Contingency mechanism: Upstream GCC will probably accept obsolete
constructs longer in case distributions like Fedora cannot port to modern C in time.
Hmm, this sounds ominous, like if GCC might stop accepting old C syntax altogether. Isn't this just a question of specifying the right standard in compilation options?
== Documentation == This section will be updated once more documentation of the porting effort will become available.
== Release Notes == Probably not needed because no user visible impact is intended.
It's not clear from the description: what about users who compile programs?
Zbyszek
On Wed, Oct 26, 2022 at 07:06:43AM +0000, Zbigniew Jędrzejewski-Szmek wrote:
== User Experience == User experience does not change.
"The new default for C standard is -c99. Users who want to use an older standard need to specify something like -c89." (???)
It is -std=c89/-std=gnu89 actually.
== Dependencies == To avoid regressing the porting effort, GCC as the system compiler needs to reject obsolete constructs by default. This is expected for GCC 14, to be released as part of Fedora 40 in Spring 2024.
== Contingency Plan ==
- Contingency mechanism: Upstream GCC will probably accept obsolete
constructs longer in case distributions like Fedora cannot port to modern C in time.
Hmm, this sounds ominous, like if GCC might stop accepting old C syntax altogether. Isn't this just a question of specifying the right standard in compilation options?
GCC hasn't changed anything yet, the obsolete constructs are accepted for decades with warnings, some of them in -Wall, others in -Wextra. The goal of Florian's work is to determine if it is feasible to reject them in -std={c,gnu}{9x,99,1x,11,17,18,2x} modes by default in GCC 14.
Examples of the obsolescent syntax:
qux (); // implicit int, + () args int baz (a, b) // old style function definition a, b; // even implicit int here { }
foo () // implicit int, + () args { bar (1, 2); // implicitly declared baz (1, 2); }
a; // implicit int
-Werror=implicit -Werror=implicit-function-declaration -Werror=old-style-definition will reject these even now. Though, I think -Wold-style-definition warns even about the no argument case which in C2X is the same as in C++ - () being equivalent to (void).
== Documentation == This section will be updated once more documentation of the porting effort will become available.
== Release Notes == Probably not needed because no user visible impact is intended.
It's not clear from the description: what about users who compile programs?
It would be GCC that rejects those by default, so it would affect users compiling programs as well.
Jakub
* Jakub Jelinek:
-Werror=implicit -Werror=implicit-function-declaration -Werror=old-style-definition will reject these even now. Though, I think -Wold-style-definition warns even about the no argument case which in C2X is the same as in C++ - () being equivalent to (void).
In particular, you can continue to write declarations like this one,
char foo ();
even though -Werror=strict-prototypes rejects such declarations. What will stop working is calling foo ("ignored") because () is no am empty parameter list, not an unspecified parameter list. (-Wstrict-prototypes will become irrelevant because there won't be any functions without prototypes.)
It is really impossible to test this with -Werror=strict-prototypes because the “char foo ();” construct is widely used in configure checks, and will continue to work with the default language mode.
Thanks, Florian
On Wed, Oct 26, 2022 at 09:33:34AM +0200, Jakub Jelinek wrote:
On Wed, Oct 26, 2022 at 07:06:43AM +0000, Zbigniew Jędrzejewski-Szmek wrote:
== User Experience == User experience does not change.
"The new default for C standard is -c99. Users who want to use an older standard need to specify something like -c89." (???)
It is -std=c89/-std=gnu89 actually.
Something like this [with any necessary corrections] should be part of the Change text, in a way that is easy for users to find.
== Dependencies == To avoid regressing the porting effort, GCC as the system compiler needs to reject obsolete constructs by default. This is expected for GCC 14, to be released as part of Fedora 40 in Spring 2024.
== Contingency Plan ==
- Contingency mechanism: Upstream GCC will probably accept obsolete
constructs longer in case distributions like Fedora cannot port to modern C in time.
Hmm, this sounds ominous, like if GCC might stop accepting old C syntax altogether. Isn't this just a question of specifying the right standard in compilation options?
GCC hasn't changed anything yet, the obsolete constructs are accepted for decades with warnings, some of them in -Wall, others in -Wextra. The goal of Florian's work is to determine if it is feasible to reject them in -std={c,gnu}{9x,99,1x,11,17,18,2x} modes by default in GCC 14.
Examples of the obsolescent syntax:
I'm generally understand what is happening. I was complaining that the Change page doesn't do a very good job of providing a quick explanation to a casual reader.
== Release Notes == Probably not needed because no user visible impact is intended.
It's not clear from the description: what about users who compile programs?
It would be GCC that rejects those by default, so it would affect users compiling programs as well.
Right. So both "Upgrade/compatibility impact" and "Release notes" need to be adjusted.
Zbyszek
* Zbigniew Jędrzejewski-Szmek:
On Tue, Oct 25, 2022 at 10:05:52AM -0400, Ben Cotton wrote:
Line-by-line review:
== Summary == Back in 1999, a new revision of the C standard removed several backwards compatibility features. However, GCC still accepts these obsolete constructs by default. Support for these constructs is confusing to programmers and potentially affect GCC's ability to implement features from future C standards.
Miro already hinted at this, but to be explicit: this Summary does not actually give the information that should be in a summary. It provides some history that maybe belongs in the Detailed Description part.
This part should say something like """gcc compiler will start rejecting some long-deprecated constructs in C and implement some other backwards-incompatible changes. Packages and build systems for user programs must be adjusted to be compatible, or opt-out of the change by specifying ….
The following constructs are now disallowed by default: implicit function declarations, implicit int type declarations, old-style K&R function parameter declarations, and in addition functions declared with no parameters will now accept no instead of any parameters, and new 'bool', 'true', 'false' keywords have been introduced."""
(I assume this is about gcc, not clang. But please be explicit.)
I don't know the exact set of changes GCC 14 will bring at this point. I'm going to tweak the text to say that this is done to be compatible with expected future compiler and languages changes.
Clang upstream will likely proceed on a much more aggressive timeline. The LLVM maintainers are discussing how Fedora should deal with that.
== How To Test ==
As discussed by others, this needs to explain how to opt-in for early testing, and also how to opt-out in packages, how to opt-out as a user (if the default touches users), etc.
The challenge with that is that I've been told not to do this work in Fedora proper by release engineering, and a request for a long-living side tag with a suitable compiler has not been approved:
Long-term side tag for toolchain experiment https://pagure.io/releng/issue/10392
More recently, I was explicitly told not to keep the compiler changes on a branch in Fedora dist-git.
It is not really possible to get realistic testing through compiler flag injection because crufty old code that is problematic for these changes often does not inject flags properly. Certain likely changes cannot be modeled through -Werror= options (but can be patched into GCC). Some build systems explicitly filter out -Werror= options during the configure stage (generally a good idea, but not helpful here).
So I'm a bit at a loss what to do here. Maybe releng can reconsider their approach.
User experience does not change.
"The new default for C standard is -c99. Users who want to use an older standard need to specify something like -c89." (???)
For GCC 14, it's likely C23, C24 or even C25 (the calendar year for ISO standards is very unpredictable). GCC and Clang defaults are already at C17/C18, it's just that they still accept obsolete C89 or K&R constructs that have been removed from C99. It's not clear at this point if -std=c99 would enable, say, implicit ints in GCC 14 because they aren't part of C99. Probably not.
== Dependencies == To avoid regressing the porting effort, GCC as the system compiler needs to reject obsolete constructs by default. This is expected for GCC 14, to be released as part of Fedora 40 in Spring 2024.
== Contingency Plan ==
- Contingency mechanism: Upstream GCC will probably accept obsolete
constructs longer in case distributions like Fedora cannot port to modern C in time.
Hmm, this sounds ominous, like if GCC might stop accepting old C syntax altogether. Isn't this just a question of specifying the right standard in compilation options?
GCC will not go the Clang route and implement what are essentially language changes through -Werror= options, it will be controlled primarily via -std=, and maybe there will be separate -f options to enable certain things. Some combinations may not be available at all (e.g., C2X with implicit ints).
Support for some extensions may be gone completely (like support for writable string constants was removed from the compiler a while back).
== Documentation == This section will be updated once more documentation of the porting effort will become available.
== Release Notes == Probably not needed because no user visible impact is intended.
It's not clear from the description: what about users who compile programs?
As discussed before, that falls under general porting for toolchain upgrades. There will separate toolchain Change proposals for Fedora 38 and Fedora 40 at least.
Thanks, Florian
On Wed, Oct 26, 2022 at 10:23:40AM +0200, Florian Weimer wrote:
- Zbigniew Jędrzejewski-Szmek:
On Tue, Oct 25, 2022 at 10:05:52AM -0400, Ben Cotton wrote:
...snip...
== How To Test ==
As discussed by others, this needs to explain how to opt-in for early testing, and also how to opt-out in packages, how to opt-out as a user (if the default touches users), etc.
The challenge with that is that I've been told not to do this work in Fedora proper by release engineering, and a request for a long-living side tag with a suitable compiler has not been approved:
Long-term side tag for toolchain experiment https://pagure.io/releng/issue/10392
Sorry that got dropped...
The thread about it I suggested copr, some other folks thought that might work, and I don't recall ever hearing if that would work for you or not. So, seems communication broke down here somewhere.
I suppose it's too late now for this to matter aside from us trying to not have it happen again? In the future, if a releng issue gets stuck, please do drop by the releng and infra standups (mon/tue/wed/thu) at 18UTC in #fedora-meeting-3 / meeting-3:fedoraproject.org and bring up the issue.
More recently, I was explicitly told not to keep the compiler changes on a branch in Fedora dist-git.
It is not really possible to get realistic testing through compiler flag injection because crufty old code that is problematic for these changes often does not inject flags properly. Certain likely changes cannot be modeled through -Werror= options (but can be patched into GCC). Some build systems explicitly filter out -Werror= options during the configure stage (generally a good idea, but not helpful here).
So I'm a bit at a loss what to do here. Maybe releng can reconsider their approach.
So, can you say that copr definitely will not work for this? If it will, great, use that. If it won't, we can make you a side tag, I don't really have any objection to it other than it seems like copr was better designed for this sort of work...
kevin
* Kevin Fenzi:
On Wed, Oct 26, 2022 at 10:23:40AM +0200, Florian Weimer wrote:
- Zbigniew Jędrzejewski-Szmek:
On Tue, Oct 25, 2022 at 10:05:52AM -0400, Ben Cotton wrote:
...snip...
== How To Test ==
As discussed by others, this needs to explain how to opt-in for early testing, and also how to opt-out in packages, how to opt-out as a user (if the default touches users), etc.
The challenge with that is that I've been told not to do this work in Fedora proper by release engineering, and a request for a long-living side tag with a suitable compiler has not been approved:
Long-term side tag for toolchain experiment https://pagure.io/releng/issue/10392
Sorry that got dropped...
The thread about it I suggested copr, some other folks thought that might work, and I don't recall ever hearing if that would work for you or not. So, seems communication broke down here somewhere.
I suppose it's too late now for this to matter aside from us trying to not have it happen again? In the future, if a releng issue gets stuck, please do drop by the releng and infra standups (mon/tue/wed/thu) at 18UTC in #fedora-meeting-3 / meeting-3:fedoraproject.org and bring up the issue.
Okay, I'll keep this in mind, thanks.
More recently, I was explicitly told not to keep the compiler changes on a branch in Fedora dist-git.
It is not really possible to get realistic testing through compiler flag injection because crufty old code that is problematic for these changes often does not inject flags properly. Certain likely changes cannot be modeled through -Werror= options (but can be patched into GCC). Some build systems explicitly filter out -Werror= options during the configure stage (generally a good idea, but not helpful here).
So I'm a bit at a loss what to do here. Maybe releng can reconsider their approach.
So, can you say that copr definitely will not work for this?
As far as I can tell, COPR doesn't do scratch builds. So if I publish a repository in COPR with everything required in it, Fedora developers still cannot run a single command that uploads some SRPMs and runs a build with the right buildroot configuration. They have to create their own COPR project first and build into that. Maybe it's possible to script this in some way. Maybe it's also possible to provide a reference mock configuration for local builds. But I think it's going to be less convenient for packagers than running a scratch build against Koji.
Thanks, Florian
Dne 27. 10. 22 v 6:57 Florian Weimer napsal(a):
- Kevin Fenzi:
On Wed, Oct 26, 2022 at 10:23:40AM +0200, Florian Weimer wrote:
- Zbigniew Jędrzejewski-Szmek:
More recently, I was explicitly told not to keep the compiler changes on a branch in Fedora dist-git.
It is not really possible to get realistic testing through compiler flag injection because crufty old code that is problematic for these changes often does not inject flags properly. Certain likely changes cannot be modeled through -Werror= options (but can be patched into GCC). Some build systems explicitly filter out -Werror= options during the configure stage (generally a good idea, but not helpful here).
So I'm a bit at a loss what to do here. Maybe releng can reconsider their approach.
So, can you say that copr definitely will not work for this?
As far as I can tell, COPR doesn't do scratch builds. So if I publish a repository in COPR with everything required in it, Fedora developers still cannot run a single command that uploads some SRPMs and runs a build with the right buildroot configuration. They have to create their own COPR project first and build into that.
They have to create their own Copr, righ, but they can specify repositories of your Copr to include everything needed via Settings > Project Details > 2. Build options > External Repositories:
And of course, you can give access to your repository to anybody who asks Settings > Request Permissions
There is also Other Actions > Fork this project option, but I have no experience with this option if this is feasible way or not.
Vít
Maybe it's possible to script this in some way. Maybe it's also possible to provide a reference mock configuration for local builds. But I think it's going to be less convenient for packagers than running a scratch build against Koji.
Thanks, Florian _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
So what is the current status? How many packages are going to be affected by this change? How are we going to track the progress?
Vít
Dne 25. 10. 22 v 16:05 Ben Cotton napsal(a):
https://fedoraproject.org/wiki/Changes/PortingToModernC.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.
== Summary == Back in 1999, a new revision of the C standard removed several backwards compatibility features. However, GCC still accepts these obsolete constructs by default. Support for these constructs is confusing to programmers and potentially affect GCC's ability to implement features from future C standards.
== Owner ==
- Name: [[User:fweimer| Florian Weimer]]
- Email: [mailto:fweimer@redhat.com| fweimer@redhat.com]
== Detailed Description == The most important change is the <b>removal of implicit function declarations</b>. This legacy compatibility feature causes the compiler to automatically supply a declaration of type `int function()` if `function` is undeclared, but called as a function. However, the implicit return type of `int` is incompatible with pointer-returning functions, which can lead to difficult debugging sessions if the compiler warning is missed, as described here:
Implicit function declarations: flex's use of "reallocarray"]
On x86-64, functions returning `_Bool` called with an implicit declaration will also not work correctly because the return register value for `_Bool` functions is partially undefined (not all 32 bits in the implicitly declared `int` are defined).
Another change for C99 is the <b>removal of implicit `int` type declarations</b>. For example, in the following fragment, both `i` and `j` are defined as `int` variables:
<pre> static i = 1.0; auto j = 2.0; </pre>
Support for this obsolete constructs is incompatible with adoption of type inference for `auto` definitions (which are part of C++).
Neither change is trivial to implement because introducing errors for these constructs (as required by C99) alters the result of autoconf configure checks. Quite a few such checks use an implicitly declared `exit` function, for instance. These failures are not really related to the feature under test. If the build system is well written, the build still succeeds, the relevant features are automatically disabled in the test suite and removed from reference ABI lists, and it's not immediately apparent that feature is gone. Therefore, some care is needed that no such alterations happen, and packages need to be ported to C99. Various tools for this porting activity are being developed to support this proposal. Cross-distribution collaboration will help as well, sharing patches and insights.
GCC 14 may also make other changes by default, as described below. These changes will also require extensive testing, and some adoption of configure checks.
==== Removal of old-style function definitions ====
An old style function definition looks like this:
<pre> int sum (a, b) char *a; int b; { return atoi (a) + b; } </pre>
It is equivalent to this definition with a prototype:
<pre> int sum (char *a, int b) { return atoi (a) + b; } </pre>
This legacy feature is very close to being incompatible with the C2X standard, which introduces unnamed function arguments. But due to a quirk in GCC's implementation, it should be possible to support both at the same time.
==== New keywords <code>bool</code>, <code>true</code>, <code>false</code> ====
Packages which supply their own definitions instead of including `<stdbool.h>` (which remains available) might fail to build.
==== Change of meaning of <code>()</code> in function declarators ====
In earlier C versions, a declaration `int function()` declares `function` as accepting an unspecified number of arguments of unknown type. This means that both `function(1)` and `function("one")` compile, even though they might not work correctly. In a future C standard, `int function()` will mean that `function` does not accept any parameters (like in C++, or as if written as `int function(void)` in current C). Calls that specify parameters will therefore result in compilation errors.
We believe that this change is ABI-compatible, even on ppc64le (with its parameter save area), although it comes very close to an implicit ABI break.
==== Rejecting implicit conversions between integers and pointers as errors ====
Currently, GCC does not treat conversion between integers at pointers as errors, so this compiles:
<pre> int ptr = "string"; </pre>
However, this is very unlikely to work on 64-bit architectures (it may work by accident without PIE/PIC).
== Feedback ==
The transition plan has been discussed at various GNU Tools Cauldrons. Feedback has been generally positive, except a general worry about the work required.
Without a deliberate porting effort, a lot of breakage occurs, [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/... seen in 2016 in Fedora with an uncoordinated change], and more recently [https://discourse.llvm.org/t/configure-script-breakage-with-the-new-werror-i... an uncoordinated Clang update].
== Benefit to Fedora == Programmers will no longer waste time tracking down things that look eerily like compiler or ABI bugs because in several cases, builds will fail with a clear error message, instead of producing a warning that is easily missed. Potential blockers to adoption of further C language changes are removed.
== Scope ==
- Proposal owners: Update rawhide over time to be compatible with
strict C99 compilers.
- Other developers: Help out with non-obvious porting issues, and with
upstreaming patches in case active upstreams cannot be easily identified. Tolerate early backports of upstream-submitted fixes in Fedora dist-git.
Release engineering: [https://pagure.io/releng/issues #Releng issue number]
Policies and guidelines: Fedora compiler policy will likely change
due to the adoption of GCC 14 in Fedora 40.
Trademark approval: N/A (not needed for this Change)
Alignment with Objectives: N/A
== Upgrade/compatibility impact == The change is expected to be transparent to those users who do not use C compilers. No features are supposed to be added or removed as a result. In fact, most of the porting effort focuses on avoiding configuration changes.
== How To Test == General regression testing is sufficient.
== User Experience == User experience does not change.
== Dependencies == To avoid regressing the porting effort, GCC as the system compiler needs to reject obsolete constructs by default. This is expected for GCC 14, to be released as part of Fedora 40 in Spring 2024.
== Contingency Plan ==
- Contingency mechanism: Upstream GCC will probably accept obsolete
constructs longer in case distributions like Fedora cannot port to modern C in time.
- Blocks release? No if GCC doesn't switch. If GCC switches, we have
to complete the port.
== Documentation == This section will be updated once more documentation of the porting effort will become available.
== Release Notes == Probably not needed because no user visible impact is intended.
* Vít Ondruch:
So what is the current status? How many packages are going to be affected by this change? How are we going to track the progress?
I see an unaudited rebuild failure rate of about 10% for rebuilds of source packages that produce arch-full binary packages. This number does not include packages which fail to build in rawhide without the compiler change. It includes packages which configure checks for something that we really do not support (like setproctitle or strlcpy). After the first pass completes, I'll have to do a second pass with the expected-to-be-undeclared functions gathered from the first pass filtered out. That should give us better numbers.
There will be a few generic issues in build tools that when address fix multiple packages, like the unfortunate has_function() implementation in Python's setuptools/distutils.
So far, I think 10% is likely an overestimate.
Thanks, Florian
On 26. 10. 22 11:09, Florian Weimer wrote:
There will be a few generic issues in build tools that when address fix multiple packages, like the unfortunate has_function() implementation in Python's setuptools/distutils.
Do you have details for this?
On 26. 10. 22 13:44, Miro Hrončok wrote:
On 26. 10. 22 11:09, Florian Weimer wrote:
There will be a few generic issues in build tools that when address fix multiple packages, like the unfortunate has_function() implementation in Python's setuptools/distutils.
Do you have details for this?
Found it.
https://github.com/python/cpython/issues/98529 https://github.com/pypa/setuptools/issues/3648
On Wed, 2022-10-26 at 11:09 +0200, Florian Weimer wrote:
There will be a few generic issues in build tools that when address fix multiple packages
Hi, I've been interested in a check of a project I maintain and I stopped very soon on Fedora 37 with
gcc-12.2.1-2.fc37.x86_64 glibc-2.36-4.fc37.x86_64 libdb-5.3.28-53.fc37.x86_64
When I try to compile this simple program:
#include <db.h> int main(void) { db_create(NULL, NULL, 0); return 0; }
(which is part of the project's "can build" test to verify the thing is properly installed in the system) using:
gcc -Wall -Wextra test.c -o test -ldb
then it doesn't claim anything and succeeds, but when I compile it as:
gcc -std=c99 -Wall -Wextra test.c -o test -ldb
I get an error:
In file included from test.c:1: /usr/include/db.h:1098:9: error: unknown type name ‘u_int’
and tons of related warning/error lines generated by gcc.
Looking into the db.h file, it has there a comment that it can add the `u_int`, when the system doesn't provide it, but that related block is empty in Fedora. More interestingly, even when I add `#define __USE_MISC 1` at the very top of the program, thus the /usr/include/sys/types.h should declare also `typedef __u_int u_int;`, the type is still unknown and the compilation aborts. Something declares __u_char_defined before the sys/types.h gets to it, it seems.
A fix might be done on the libdb's db.h or elsewhere?
I may face more similar or totally different issues while I progress with my compilation attempt (until I give up), and I'm not going to spam this list with it. I do understand these things need to be addressed individually, which makes perfect sense. Where it would be is harder to guess for me. Bye, Milan
On Wed, 2022-10-26 at 14:18 +0200, Milan Crha wrote:
when I compile it as:
gcc -std=c99 -Wall -Wextra test.c -o test -ldb
I get an error:
In file included from test.c:1: /usr/include/db.h:1098:9: error: unknown type name ‘u_int’
and tons of related warning/error lines generated by gcc.
Hi again, I just figured that compiling with:
gcc -std=c99 -Wall -Wextra -D_DEFAULT_SOURCE=1 test.c -o test -ldb
makes it work. I've no idea where I should add that define, CFLAGS feels too brutal for it. Bye, Milan
On Wed, Oct 26, 2022 at 02:18:02PM +0200, Milan Crha wrote:
When I try to compile this simple program:
#include <db.h> int main(void) { db_create(NULL, NULL, 0); return 0; }
(which is part of the project's "can build" test to verify the thing is properly installed in the system) using:
gcc -Wall -Wextra test.c -o test -ldb
then it doesn't claim anything and succeeds, but when I compile it as:
gcc -std=c99 -Wall -Wextra test.c -o test -ldb
Note that in an earlier message in this thread replying to my question, Florian pointed out that using -std is not actually the way to test this.
All the -std options that exist in GCC today allow the undesired behaviour to be used. So don't try to set '-std' at all, unless you want todo that for other reasons.
For this particular proposal, what's needed is to set warning flags:
-Werror=implicit-int -Werror=implicit-function-declaration -Werror=int-conversion -Werror=strict-prototypes -Werror=old-style-definition
I get an error:
In file included from test.c:1: /usr/include/db.h:1098:9: error: unknown type name ‘u_int’
and tons of related warning/error lines generated by gcc.
GCC in Fedora today actually defaults to -std=gnu18, so apps are already getting the modern C standard out of the box, unless they gave an explikcit -std arg themselves.
The reason for your problem in this particular example is that you requested 'c99' rather than 'gnu99', so you disabled various GCC extensions to the c99 standard, which the code appears to rely on.
With regards, Daniel
On Wed, 2022-10-26 at 13:59 +0100, Daniel P. Berrangé wrote:
Note that in an earlier message in this thread replying to my question, Florian pointed out that using -std is not actually the way to test this.
Hi, I see. I did read several messages in this thread, but definitely not all of them (there are a lot of messages in this thread). My fault.
I used your hints and addressed couple problems in my projects/packages upstream. Thanks and bye, Milan
On Wed, 26 Oct 2022 at 13:24, Milan Crha wrote:
Looking into the db.h file, it has there a comment that it can add the `u_int`, when the system doesn't provide it, but that related block is empty in Fedora. More interestingly, even when I add `#define __USE_MISC 1` at the very top of the program, thus the
Never ever do that. The __USE_xxx macros are internal, for glibc to use. You should not ever check them or define them.
https://lwn.net/Articles/590381/
There are several public macros that you can use to enable additional features, like -D_GNU_SOURCE or _D_DEFAULT_SOURCE, so use those not the __USE_xxx ones.
Hi,
On Mon, 2022-10-31 at 11:46 +0100, Jonathan Wakely wrote:
Never ever do that. The __USE_xxx macros are internal, for glibc to use. You should not ever check them or define them.
right, right, it was only a hackish dirty quick attempt to make it compile. I won't do it, if I knew the right way to test, which I did not in time of writing the previous mail. I know it now.
Thanks and bye, Milan
I frankly love this change. For along time I wanted a way to catch such extensions and non stadard behaviour. For the long term, we could create 3 macros, legacy for old unmainatined code (the -std=c89), standard which will be the result of this porting and a future one with some bonus flags.
standard is the default and the other 2 are opt in.
We need to encourage upstreams to stop using UBs and extension when it makes sense (They for example break when defaults change like this and certain UBs can also prevent putting -O3 and can cause security issues). So I recommend -fsanitize-trap=all to be in future for now (I would like putting -Werror but thats probaly going be too much).
Good to see Gentoo also helping out. We should talk to some of the big upstreams to put these flags in their CIs. That way we will less issues down the line.
Thanks & Regards Marc Pervaz Boocha
-
Adam Williamson -
Alexander Sosedkin -
Ben Cotton -
Clemens Lang -
Daniel P. Berrangé -
Florian Weimer -
Jakub Jelinek -
Jeff Law -
Jonathan Wakely -
Kevin Fenzi -
Marc Pervaz Boocha -
Milan Crha -
Miro Hrončok -
Neal Gompa -
Ralf Corsépius -
Richard W.M. Jones -
Stephen Gallagher -
Vít Ondruch -
Zbigniew Jędrzejewski-Szmek