GCC 13 broke 50 packages requiring libgnat-12.so() and libgnarl-12.so()

[Miro Hrončok]

Hello. GCC was updated to 13 in rawhide while the Fedora change was still being 
voted about by FESCo.

Apparently, the following packages now don't install:

$ repoquery -q --repo=rawhide --whatrequires 'libgnat-12.so()(64bit)' 
--whatrequires 'libgnarl-12.so()(64bit)'
GtkAda3-0:2020-7.fc37.x86_64
GtkAda3-devel-0:2020-7.fc37.x86_64
GtkAda3-gl-0:2020-7.fc37.x86_64
PragmARC-0:20130728-27.fc37.x86_64
ahven-0:2.8-4.fc37.x86_64
anet-0:0.4.1-12.fc37.x86_64
aunit-0:2020-6.fc37.x86_64
aws-0:2020-8.fc37.x86_64
aws-devel-0:2020-8.fc37.x86_64
aws-tools-0:2020-8.fc37.x86_64
florist-2:22.0.0-2.fc38.x86_64
ghdl-0:0.38~dev-16.20201208git83dfd49.fc37.x86_64
ghdl-llvm-0:0.38~dev-16.20201208git83dfd49.fc37.x86_64
ghdl-mcode-0:0.38~dev-16.20201208git83dfd49.fc37.x86_64
gnatcoll-core-2:21.0.0-8.fc38.x86_64
gnatcoll-db-devel-2:21.0.0-5.fc37.x86_64
gnatcoll-gmp-2:21.0.0-9.fc37.x86_64
gnatcoll-iconv-2:21.0.0-9.fc37.x86_64
gnatcoll-postgres-2:21.0.0-5.fc37.x86_64
gnatcoll-readline-2:21.0.0-9.fc37.x86_64
gnatcoll-sql-2:21.0.0-5.fc37.x86_64
gnatcoll-sqlite-2:21.0.0-5.fc37.x86_64
gnatcoll-syslog-2:21.0.0-9.fc37.x86_64
gnatcoll-xref-2:21.0.0-5.fc37.x86_64
matreshka-0:20.1-11.fc37.x86_64
matreshka-amf-0:20.1-11.fc37.x86_64
matreshka-amf-dd-0:20.1-11.fc37.x86_64
matreshka-amf-mofext-0:20.1-11.fc37.x86_64
matreshka-amf-ocl-0:20.1-11.fc37.x86_64
matreshka-amf-uml-0:20.1-11.fc37.x86_64
matreshka-amf-utp-0:20.1-11.fc37.x86_64
matreshka-fastcgi-0:20.1-11.fc37.x86_64
matreshka-servlet-lib-0:20.1-11.fc37.x86_64
matreshka-soap-core-0:20.1-11.fc37.x86_64
matreshka-soap-wsse-0:20.1-11.fc37.x86_64
matreshka-spikedog-api-lib-0:20.1-11.fc37.x86_64
matreshka-spikedog-awsd-0:20.1-11.fc37.x86_64
matreshka-spikedog-core-lib-0:20.1-11.fc37.x86_64
matreshka-sql-core-0:20.1-11.fc37.x86_64
matreshka-sql-mysql-0:20.1-11.fc37.x86_64
matreshka-sql-postgresql-0:20.1-11.fc37.x86_64
matreshka-sql-sqlite-0:20.1-11.fc37.x86_64
matreshka-xml-0:20.1-11.fc37.x86_64
mine_detector-0:6.0-43.fc37.x86_64
plplot-ada-0:5.15.0-47.fc38.x86_64
templates_parser-0:11.8.0-31.fc37.x86_64
templates_parser-tools-0:11.8.0-31.fc37.x86_64
xmlada-0:2020-8.fc37.x86_64
zeromq-ada-0:4.1.5-8.git.fc37.x86_64
zlib-ada-0:1.4-0.31.20120830CVS.fc37.x86_64

Please use side tags for updates that bump soname version.

-- 
Miro Hrončok
-- 
Phone: [phone]
IRC: mhroncok

[Jakub Jelinek]

On Tue, Jan 17, 2023 at 01:24:45PM +0100, Miro Hrončok wrote:
>  Hello. GCC was updated to 13 in rawhide while the Fedora change was
> still
>  being voted about by FESCo.
>  
>  Apparently, the following packages now don't install: 
There is a mass rebuild tomorrow.  The Ada soname changes every year
and we've never rebuilt Ada packages separately for that, just during the
mass rebuild.

	Jakub

[Fabio Valentini]

On Tue, Jan 17, 2023 at 1:39 PM Jakub Jelinek <jakub(a)redhat.com&gt; wrote:>
>  On Tue, Jan 17, 2023 at 01:24:45PM +0100, Miro Hrončok wrote:
>  > Hello. GCC was updated to 13 in rawhide while the Fedora change was still
>  > being voted about by FESCo.
>  >
>  > Apparently, the following packages now don't install:
>
>  There is a mass rebuild tomorrow.  The Ada soname changes every year
>  and we've never rebuilt Ada packages separately for that, just during the
>  mass rebuild. 
"Because we've always done it like that" is usually not a good
argument for something bad to remain the way it is -
(especially since this change hasn't even formally been approved by FESCo).

Fabio

[Pavel Zhukov]

Fabio Valentini <decathorpe(a)gmail.com&gt; writes:

>  On Tue, Jan 17, 2023 at 1:39 PM Jakub Jelinek
> <jakub(a)redhat.com&gt; wrote:>
> > On Tue, Jan 17, 2023 at 01:24:45PM +0100, Miro Hrončok wrote:
> > > Hello. GCC was updated to 13 in rawhide while the Fedora change was still
> > > being voted about by FESCo.
> > >
> > > Apparently, the following packages now don't install:
> >
> > There is a mass rebuild tomorrow.  The Ada soname changes every year
> > and we've never rebuilt Ada packages separately for that, just during the
> > mass rebuild.
>
>  "Because we've always done it like that" is usually not a good
>  argument for something bad to remain the way it is -
>  (especially since this change hasn't even formally been approved by FESCo).
Yes, that happens every year because of koji limitation.
Ada rebuild is not just mass-rebuild thing.
Because of circular dependencies: gprbuild buildrequires xmlada and
xmlada build requires gprbuild and both of them requires
gcc-gnat.so.<version> they were built with
bootstrap of gprbuild should be done which requires manual steps (pretty
straightforward process and documented in xmlada and gprbuild spec files though. thanks to
Bjorn!) 

-- 
Pavel

>  Fabio
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>  Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

[Björn Persson]

Pavel Zhukov wrote:
>  Yes, that happens every year because of koji limitation.
>  Ada rebuild is not just mass-rebuild thing.
>  Because of circular dependencies: gprbuild buildrequires xmlada and
>  xmlada build requires gprbuild and both of them requires
>  gcc-gnat.so.<version> they were built with
>  bootstrap of gprbuild should be done which requires manual steps (pretty
>  straightforward process and documented in xmlada and gprbuild spec files though. thanks
> to Bjorn!)  
The bootstrap is actually needed only if something is seriously broken,
or to add a new arch. GPRbuild is statically linked precisely to remain
functional each time GCC or XMLada is upgraded, so that it can rebuild
itself and the other Ada packages. Thus the dependency loop isn't
normally a problem. A bump and a rebuild per package should be
sufficient, but they must be done in the right order.

A package can't be built if it requires a library that can't be
installed. Therefore those libraries that depend only on Libgnat must be
built first, so they become installable again. Once those packages
appear in the buildroot, the packages that depend on them can be built.
Then those packages in turn must be added to the buildroot before the
third tier can be built. (I have now done all that for all packages I
have commit access to. I rebuilt GPRbuild too, so it's now statically
linked to the new Libgnat. Nobody needs to worry that the static
linking will cause old code to linger.)

For the mass rebuild to handle this automatically, two changes would be
needed:

· It would have to walk the dependency graph and build the packages in
dependency order instead of alphabetical order.

· It would have to make the newly built packages available to packages
that depend on them, not set them aside and then tag them all in at the
end.

So as things stand, these rebuilds need to be done by a human who knows
the dependency graph.

Björn Persson

[John Reiser]

On 1/17/23 14:37, Björn Persson wrote:

>  So as things stand, these rebuilds need to be done by a human who
> knows
>  the dependency graph. 
Requiring "a human who knows the dependency graph" is *severely* broken.
There should be a shell script which computes an acceptable order from the
old installed version, and outputs the order for examination and modification.
Probably the script involves ldd, "readelf --dynamic", and tsort.
For new dependencies the script may need to be re-run during the builds.

[Frederic Berat]

On Wed, Jan 18, 2023 at 4:07 AM John Reiser <jreiser(a)bitwagon.com&gt; wrote:
>  On 1/17/23 14:37, Björn Persson wrote:
>
>  > So as things stand, these rebuilds need to be done by a human who knows
>  > the dependency graph.
>
>  Requiring "a human who knows the dependency graph" is *severely* broken.
>  There should be a shell script which computes an acceptable order from the
>  old installed version, and outputs the order for examination and modification.
>  Probably the script involves ldd, "readelf --dynamic", and tsort.
>  For new dependencies the script may need to be re-run during the builds. 
I'm afraid, as of today, that can't be generalized without human
intervention. This may work with difficulty for binary packages, but
even then not in the general case. Take alone circular dependencies
that are due to documentation being generated, but that would be the
simplest case.

We are trying to solve this kind of problem in the mass pre-builder
project [1], if we succeed to do so, we have hope that the code base
could be used as a basis for all use cases that involve mass building
(whether it is Fedora mass rebuild or CI building reverse dependencies
for stability QE). Any help that would improve the code base and help
us move towards this is more than welcome.

In the current state, the mass prebuilder may try to calculate
building priorities based on the information collected out of the
dependency graph, but this has major flows:
1) Assumptions are made in order to break the circular dependencies
that are unlikely to be generally accurate
2) The calculation is computationally expensive (seems subjectively
exponential), and can't be realistically used for more than 2 or 3
thousand packages. (e.g. GCC and its 10k reverse dependencies is a
no-go)

For some use cases, as of now it looks more realistic to use brute
force (try to build until it works, or a general failure is
subjectively declared) unless someone with proper knowledge specifies
a meaningful list.

[1] https://gitlab.com/fedora/packager-tools/mass-prebuild

[Björn Persson]

Jakub Jelinek wrote:
>  On Tue, Jan 17, 2023 at 01:24:45PM +0100, Miro Hrončok wrote:
>  > Hello. GCC was updated to 13 in rawhide while the Fedora change was still
>  > being voted about by FESCo.
>  > 
>  > Apparently, the following packages now don't install:  
>  
>  There is a mass rebuild tomorrow.  The Ada soname changes every year
>  and we've never rebuilt Ada packages separately for that, just during the
>  mass rebuild. 
Pavel and I have always rebuilt the Ada packages separately for the
yearly soname change. They must be rebuilt in dependency order, and
that's not how the mass rebuild does it.

I'd be willing to cooperate to do the rebuild in a side tag, but I
can't promise to always be available at a moment's notice.

Björn Persson

[Jakub Jelinek]

On Tue, Jan 17, 2023 at 03:24:16PM +0100, Björn Persson wrote:
>  Jakub Jelinek wrote:
>  > On Tue, Jan 17, 2023 at 01:24:45PM +0100, Miro Hrončok wrote:
>  > > Hello. GCC was updated to 13 in rawhide while the Fedora change was still
>  > > being voted about by FESCo.
>  > > 
>  > > Apparently, the following packages now don't install:  
>  > 
>  > There is a mass rebuild tomorrow.  The Ada soname changes every year
>  > and we've never rebuilt Ada packages separately for that, just during the
>  > mass rebuild.
>  
>  Pavel and I have always rebuilt the Ada packages separately for the
>  yearly soname change. They must be rebuilt in dependency order, and
>  that's not how the mass rebuild does it. 
Thanks.

>  I'd be willing to cooperate to do the rebuild in a side tag, but
> I
>  can't promise to always be available at a moment's notice. 
I've been using a side-tag for gcc 13 for a couple of weeks so that annobin
and libtool can be rebuilt against gcc 13.
For trying to rebuild other packages, the problem is to find the right time,
if it is done too early (weeks before it is actually merged into rawhide),
it will make the Ada (or whatever else needs to be rebuilt) packages
buildable either in the side-tag or in rawhide but pain if needed in both,
and if it is too short before the tagging then there won't be enough time to
rebuild those.
E.g. even the annobin and libtool in the side-tag I had to rebuild again
because it was bumped several times in rawhide in between.
It is already pain now to find the right time for this (based on what
critical bugs are already fixed and what is the general state of the
compiler snapshot at that point, plus when is the scheduled mass rebuild
time) and having to do the decision several days or more before it can be
done would be even more painful.

	Jakub