Fedora 34 Change: GNU Toolchain update (gcc 11, glibc 2.33) (System-Wide Change)

[Ben Cotton]

https://fedoraproject.org/wiki/Changes/GNUToolchain

== Summary ==
Switch the Fedora 33 GNU Toolchain to gcc 11, binutils 2.35, and glibc 2.33.

The binutils 2.35 change is being tracked here:
https://fedoraproject.org/wiki/Changes/BINUTILS235

The gcc 11 and glibc 2.33 change will be tracked in this top-level GNU
Toolchain system-wide update.

== Owner ==
* Name: [[User:codonell|Carlos O'Donell]], [[User:law|Jeff Law]]
* Email: carlos(a)redhat.com, law(a)redhat.com


== Detailed Description ==
The GNU Compiler Collection, GNU C Library, and GNU Binary Utilities
make up the core part of the GNU Toolchain and it is useful to
transition these components as a complete implementation when making a
new release of Fedora.

The GNU Compiler Collection will be releasing version 11 containing
many new features documented here:
https://gcc.gnu.org/gcc-11/changes.html.  Historically pre-releases of
GCC drop into Fedora in Jan/Feb just prior to the mass rebuild.  The
major process change this year is the desire to drop in snapshots of
GCC 11 into rawhide starting in November with updates throughout the
Fedora 34 release process as needed.

The GNU C Library version 2.33 will be released at the beginning of
February 2020; we have started closely tracking the glibc 2.33
development code in Fedora Rawhide and are addressing any issues as
they arise. Given the present schedule Fedora 34 will branch after the
glibc 2.33 upstream release. However, the mass rebuild schedule means
Fedora 34 will mass rebuild (if required) after glibc 2.33 upstream
freezes ABI for release, but before the actual release, so careful
attention must be paid to any last minute ABI changes.

== Benefit to Fedora ==
Stays up to date with latest features, improvements security and bug
fixes from gcc and glibc upstream.

The change to drop GCC 11 snapshots into rawhide earlier is meant to
start more wide scale testing of GCC earlier.  This means that package
maintainers will not be faced with a onslaught on FTBFS issues in
Feb/Mar and the GCC maintainers will not be as stressed trying to fix
all Fedora related issues in a short time frame as well.

== Scope ==
The gcc and glibc teams will need to move their respective upstream
projects to a releasable state.  For GCC this includes correctly
building Fedora rawhide.

* Other developers: Developers need to ensure that gcc, binutils, and
glibc in rawhide is stable and ready for the Fedora 34 branch. Given
that glibc is backwards compatible and we have been testing the new
glibc in rawhide it should make very little impact when updated,
except for the occasional deprecation warnings and removal of legacy
interfaces from public header files.  GCC is currently being tested
weekly against Fedora rawhide and fixes for issues discovered are
continually dropping into rawhide to minimize the impact on package
maintainers.  However, we fully expect some issues to arise,
particularly as the GCC team's tests are limited to x86_64.

* Release engineering:  [https://pagure.io/releng/issue/9858 9858] A
mass rebuild is strongly encouraged.
* Policies and guidelines: The policies and guidelines do not need to
be updated.
* Trademark approval: N/A (not needed for this Change)

== Upgrade/compatibility impact ==
<!-- What happens to systems that have had a previous versions of
Fedora installed and are updated to the version containing this
change? Will anything require manual configuration or data migration?
Will any existing functionality be no longer supported? -->

The compiler, and the the library are backwards compatible with the
previous version of Fedora.

Some packaging changes may be required for the glibc 2.33 rebase:
https://sourceware.org/glibc/wiki/Release/2.33#Packaging_Changes

Some source changes may be required for gcc 11 rebase:
https://gcc.gnu.org/gcc-11/changes.html

We fully expect to fix all packaging changes in Fedora Rawhide without
impact to the release.

== How To Test ==
The GNU C compiler collection has its own testsuite which is run
during the package build and examined by the gcc developers before
being uploaded.  The GCC team also is also doing continuous testing of
GCC 11 snapshots against Fedora rawhide to identify and resolve issues
prior to new versions of GCC landing in rawhide.  This work will
continue, particularly in Nov, Dec, Jan and Feb and we will use it to
help guide decisions about snapshots are stable enough to not cause
major Fedora rawhide disruptions.  We expect that by March the pace of
updates will reduce significantly.

The GCC team will likely need some help addressing some of the new
diagnostics that require package specific knowledge to determine if
the code is valid or not.  This is not new, but the timing will shift
to earlier points in the Fedora release cycle.

The GNU C Library has its own testsuite, which is run during the
package build and examined by the glibc developers before being
uploaded. This test suite has over 6200 tests that run to verify the
correct operation of the library. In the future may also run the
microbenchmark to look for performance regressions.

== User Experience ==
Users will see improved performance, many bugfixes and improvements to
POSIX compliance, additional locales, etc.

== Dependencies ==
All packages do not need to be rebuilt due to backwards compatibility.
However, it is advantageous if a mass rebuild is performed during the
Fedora 34 cycle.

== Contingency Plan ==
* Contingency mechanism: If gcc 11 proves too disruptive to compiling
the distribution, which is not expected, we could revert to gcc 10 for
the release. If glibc 2.33 provides too disruptive to compiling the
distribution we could revert to 2.32, but given that Rawhide has
started tracking glibc 2.33, no show-stopper problems are expected.
At this point, we can still revert to upstream version 2.32 if
insurmountable problems appear, but to do so may require a mass
rebuild to remove new symbols from the ABI/API.
* Contingency deadline: Upstream glibc ABI freeze deadline of 2021-02-01.
* Blocks release? Yes, upgrading to gcc 11 blocks the release. Yes,
upgrading glibc does block the release. We should not ship without a
newer gcc and glibc, there will be gcc and language features that
depend on glibc being upgraded. Thus without the upgrade some features
will be disabled or fall back to less optimal implementations.

== Documentation ==
The gcc manual contains the documentation for the release and doesn't
need any more additional work.

The glibc manual contains the documentation for the release and
doesn't need any more additional work.

== Release Notes ==
The GNU Compiler Collection version 11 will be released shortly. See
https://gcc.gnu.org/gcc-11/changes.html.

The GNU C Library version 2.32 will be released at the beginning of
August 2020. The current NEWS notes can be seen here as they are
added: https://sourceware.org/git/?p=glibc.git;a=blob;f=NEWS;hb=HEAD


-- 
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis

[Simo Sorce]

On Fri, 2020-11-20 at 11:26 -0500, Ben Cotton wrote:
>  https://fedoraproject.org/wiki/Changes/GNUToolchain
>  
>  == Summary ==
>  Switch the Fedora 33 GNU Toolchain to gcc 11, binutils 2.35, and glibc 2.33. 
Hi Ben, shouldn't this ^^ be Fedora 34 ?

>  The binutils 2.35 change is being tracked here:
>  https://fedoraproject.org/wiki/Changes/BINUTILS235
>  
>  The gcc 11 and glibc 2.33 change will be tracked in this top-level GNU
>  Toolchain system-wide update.
>  
>  == Owner ==
>  * Name: [[User:codonell|Carlos O'Donell]], [[User:law|Jeff Law]]
>  * Email: carlos(a)redhat.com, law(a)redhat.com
>  
>  
>  == Detailed Description ==
>  The GNU Compiler Collection, GNU C Library, and GNU Binary Utilities
>  make up the core part of the GNU Toolchain and it is useful to
>  transition these components as a complete implementation when making a
>  new release of Fedora.
>  
>  The GNU Compiler Collection will be releasing version 11 containing
>  many new features documented here:
>  https://gcc.gnu.org/gcc-11/changes.html.  Historically pre-releases of
>  GCC drop into Fedora in Jan/Feb just prior to the mass rebuild.  The
>  major process change this year is the desire to drop in snapshots of
>  GCC 11 into rawhide starting in November with updates throughout the
>  Fedora 34 release process as needed.
>  
>  The GNU C Library version 2.33 will be released at the beginning of
>  February 2020; we have started closely tracking the glibc 2.33
>  development code in Fedora Rawhide and are addressing any issues as
>  they arise. Given the present schedule Fedora 34 will branch after the
>  glibc 2.33 upstream release. However, the mass rebuild schedule means
>  Fedora 34 will mass rebuild (if required) after glibc 2.33 upstream
>  freezes ABI for release, but before the actual release, so careful
>  attention must be paid to any last minute ABI changes.
>  
>  == Benefit to Fedora ==
>  Stays up to date with latest features, improvements security and bug
>  fixes from gcc and glibc upstream.
>  
>  The change to drop GCC 11 snapshots into rawhide earlier is meant to
>  start more wide scale testing of GCC earlier.  This means that package
>  maintainers will not be faced with a onslaught on FTBFS issues in
>  Feb/Mar and the GCC maintainers will not be as stressed trying to fix
>  all Fedora related issues in a short time frame as well.
>  
>  == Scope ==
>  The gcc and glibc teams will need to move their respective upstream
>  projects to a releasable state.  For GCC this includes correctly
>  building Fedora rawhide.
>  
>  * Other developers: Developers need to ensure that gcc, binutils, and
>  glibc in rawhide is stable and ready for the Fedora 34 branch. Given
>  that glibc is backwards compatible and we have been testing the new
>  glibc in rawhide it should make very little impact when updated,
>  except for the occasional deprecation warnings and removal of legacy
>  interfaces from public header files.  GCC is currently being tested
>  weekly against Fedora rawhide and fixes for issues discovered are
>  continually dropping into rawhide to minimize the impact on package
>  maintainers.  However, we fully expect some issues to arise,
>  particularly as the GCC team's tests are limited to x86_64.
>  
>  * Release engineering:  [https://pagure.io/releng/issue/9858 9858] A
>  mass rebuild is strongly encouraged.
>  * Policies and guidelines: The policies and guidelines do not need to
>  be updated.
>  * Trademark approval: N/A (not needed for this Change)
>  
>  == Upgrade/compatibility impact ==
>  <!-- What happens to systems that have had a previous versions of
>  Fedora installed and are updated to the version containing this
>  change? Will anything require manual configuration or data migration?
>  Will any existing functionality be no longer supported? -->
>  
>  The compiler, and the the library are backwards compatible with the
>  previous version of Fedora.
>  
>  Some packaging changes may be required for the glibc 2.33 rebase:
>  https://sourceware.org/glibc/wiki/Release/2.33#Packaging_Changes
>  
>  Some source changes may be required for gcc 11 rebase:
>  https://gcc.gnu.org/gcc-11/changes.html
>  
>  We fully expect to fix all packaging changes in Fedora Rawhide without
>  impact to the release.
>  
>  == How To Test ==
>  The GNU C compiler collection has its own testsuite which is run
>  during the package build and examined by the gcc developers before
>  being uploaded.  The GCC team also is also doing continuous testing of
>  GCC 11 snapshots against Fedora rawhide to identify and resolve issues
>  prior to new versions of GCC landing in rawhide.  This work will
>  continue, particularly in Nov, Dec, Jan and Feb and we will use it to
>  help guide decisions about snapshots are stable enough to not cause
>  major Fedora rawhide disruptions.  We expect that by March the pace of
>  updates will reduce significantly.
>  
>  The GCC team will likely need some help addressing some of the new
>  diagnostics that require package specific knowledge to determine if
>  the code is valid or not.  This is not new, but the timing will shift
>  to earlier points in the Fedora release cycle.
>  
>  The GNU C Library has its own testsuite, which is run during the
>  package build and examined by the glibc developers before being
>  uploaded. This test suite has over 6200 tests that run to verify the
>  correct operation of the library. In the future may also run the
>  microbenchmark to look for performance regressions.
>  
>  == User Experience ==
>  Users will see improved performance, many bugfixes and improvements to
>  POSIX compliance, additional locales, etc.
>  
>  == Dependencies ==
>  All packages do not need to be rebuilt due to backwards compatibility.
>  However, it is advantageous if a mass rebuild is performed during the
>  Fedora 34 cycle.
>  
>  == Contingency Plan ==
>  * Contingency mechanism: If gcc 11 proves too disruptive to compiling
>  the distribution, which is not expected, we could revert to gcc 10 for
>  the release. If glibc 2.33 provides too disruptive to compiling the
>  distribution we could revert to 2.32, but given that Rawhide has
>  started tracking glibc 2.33, no show-stopper problems are expected.
>  At this point, we can still revert to upstream version 2.32 if
>  insurmountable problems appear, but to do so may require a mass
>  rebuild to remove new symbols from the ABI/API.
>  * Contingency deadline: Upstream glibc ABI freeze deadline of 2021-02-01.
>  * Blocks release? Yes, upgrading to gcc 11 blocks the release. Yes,
>  upgrading glibc does block the release. We should not ship without a
>  newer gcc and glibc, there will be gcc and language features that
>  depend on glibc being upgraded. Thus without the upgrade some features
>  will be disabled or fall back to less optimal implementations.
>  
>  == Documentation ==
>  The gcc manual contains the documentation for the release and doesn't
>  need any more additional work.
>  
>  The glibc manual contains the documentation for the release and
>  doesn't need any more additional work.
>  
>  == Release Notes ==
>  The GNU Compiler Collection version 11 will be released shortly. See
>  https://gcc.gnu.org/gcc-11/changes.html.
>  
>  The GNU C Library version 2.32 will be released at the beginning of
>  August 2020. The current NEWS notes can be seen here as they are
>  added: https://sourceware.org/git/?p=glibc.git;a=blob;f=NEWS;hb=HEAD
>  
>  
>  -- 
>  Ben Cotton
>  He / Him / His
>  Senior Program Manager, Fedora & CentOS Stream
>  Red Hat
>  TZ=America/Indiana/Indianapolis
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail] 
-- 
Simo Sorce
RHEL Crypto Team
Red Hat, Inc

[Florian Weimer]

* Simo Sorce:

>  On Fri, 2020-11-20 at 11:26 -0500, Ben Cotton wrote:
> > https://fedoraproject.org/wiki/Changes/GNUToolchain
> > 
> > == Summary ==
> > Switch the Fedora 33 GNU Toolchain to gcc 11, binutils 2.35, and glibc 2.33.
>
>  Hi Ben, shouldn't this ^^ be Fedora 34 ? 
I fixed it on the wiki, and also the wrong glibc version/release date.

Thanks,
Florian
-- 
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

[Fabio Valentini]

On Mon, Nov 23, 2020 at 11:51 AM Florian Weimer <fweimer(a)redhat.com&gt; wrote:
>  * Simo Sorce:
>
>  > On Fri, 2020-11-20 at 11:26 -0500, Ben Cotton wrote:
>  >> https://fedoraproject.org/wiki/Changes/GNUToolchain
>  >>
>  >> == Summary ==
>  >> Switch the Fedora 33 GNU Toolchain to gcc 11, binutils 2.35, and glibc 2.33.
>  >
>  > Hi Ben, shouldn't this ^^ be Fedora 34 ?
>
>  I fixed it on the wiki, and also the wrong glibc version/release date. 
Are there plans to fix the glibc faccessat2 issues with older
systemd-nspawn and docker?
It would be a shame if fedora 34 containers wouldn't be able to run
correctly in most circumstances.

Fabio

[Florian Weimer]

* Fabio Valentini:

>  Are there plans to fix the glibc faccessat2 issues with older
>  systemd-nspawn and docker? 
I'm trying to gather the status on this; I've been out of touch for a
couple of days.

I do not feel comfortable to ship a Fedora-only patch for this.  My hope
is that we can work out something with all the projects involved.

Thanks,
Florian
-- 
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

[Daniel P. Berrangé]

On Mon, Nov 23, 2020 at 12:59:20PM +0100, Florian Weimer wrote:
>  * Fabio Valentini:
>  
>  > Are there plans to fix the glibc faccessat2 issues with older
>  > systemd-nspawn and docker?
>  
>  I'm trying to gather the status on this; I've been out of touch for a
>  couple of days.
>  
>  I do not feel comfortable to ship a Fedora-only patch for this.  My hope
>  is that we can work out something with all the projects involved. 
Even if we got a fix into docker/systemd/runc, etc to not use EPERM,
IMHO we're going to need the workaround present in glibc regardless
for quite some time.

There are many public cloud based services that use containers. For
example majority of public CI services (GitLab, Cirrus, Travis) use
container based infra. While we can encourage them to upgrade, it is
doubtful we can rely on them getting the fix into their versions of
docker anytime soon. We can't leave Fedora rawhide/34 broken
when used in such systems.

Of course this isn't really a Fedora problem - any distro which upgrades
to new glibc will suffer this same bad behaviour. Ubuntu/Debian/RHEL-9
will all hit it and likely want a workaround added in glibc.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Fedora 34, faccessat, and containers (was: Re: Fedora 34 Change: GNU Toolchain update (gcc 11, glibc 2.33) (System-Wide Change))

[Florian Weimer]

* Fabio Valentini:

>  Are there plans to fix the glibc faccessat2 issues with older
>  systemd-nspawn and docker?
>  It would be a shame if fedora 34 containers wouldn't be able to run
>  correctly in most circumstances. 
I've brought the discussion to what I think are the appropriate forums:

  <https://lore.kernel.org/linux-api/[e-mail]...
  <https://groups.google.com/a/opencontainers.org/g/dev/c/8Phfq3VBxtw>

I've also posted a glibc upstream patch to show what it would look like:

  <https://sourceware.org/pipermail/libc-alpha/2020-November/119955.html>

Personally, I find it difficult to support such an approach technically,
and I would like to see some reassurance from kernel developers that
this is okay.

Feedback so far is in the opposite direction, though.

Thanks,
Florian
-- 
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill

Fedora 34, faccessat, and containers (was: Re: Fedora 34 Change: GNU Toolchain update (gcc 11, glibc 2.33) (System-Wide Change))

[Alexander Bokovoy]

On ti, 24 marras 2020, Florian Weimer wrote:
> * Fabio Valentini:
>
> > Are there plans to fix the glibc faccessat2 issues with older
> > systemd-nspawn and docker?
> > It would be a shame if fedora 34 containers wouldn't be able to run
> > correctly in most circumstances.
>
> I've brought the discussion to what I think are the appropriate forums:
>
>   <https://lore.kernel.org/linux-api/[e-mail]...
>   <https://groups.google.com/a/opencontainers.org/g/dev/c/8Phfq3VBxtw>
>
> I've also posted a glibc upstream patch to show what it would look like:
>
>   <https://sourceware.org/pipermail/libc-alpha/2020-November/119955.html>
>
> Personally, I find it difficult to support such an approach technically,
> and I would like to see some reassurance from kernel developers that
> this is okay.
>
> Feedback so far is in the opposite direction, though. 
Thanks, Florian.

For those who need this working now rather when Docker is fixed (as
promised by Aleksa in the linux-api@ thread), I ended up taking
libseccomp 2.5 from Debian Sid and making a PPA with it for Ubuntu
20.04. It seems to help now, so if others have the same need, installing
libseccomp2 from the
https://launchpad.net/~abbra/+archive/ubuntu/freeipa-libseccomp should
help -- you also need to add 'faccessat2' to the Docker profile.

Example use in FreeIPA is
https://pagure.io/freeipa/c/1bf0d628281f33693a1f6c6e156b0c258eee929e?bran... 


-- 
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland

[José Abílio Matos]

On Friday, November 20, 2020 4:26:53 PM WET Ben Cotton wrote:
>  == Release Notes ==
>  The GNU Compiler Collection version 11 will be released shortly. See
>  https://gcc.gnu.org/gcc-11/changes.html.
>  
>  The GNU C Library version 2.32 will be released at the beginning of
>  August 2020. The current NEWS notes can be seen here as they are
>  added: https://sourceware.org/git/?p=glibc.git;a=blob;f=NEWS;hb=HEAD 
Just a small note, glibc was *already* released. :-)
Please update the release notes.

-- 
José Abílio