F28 System Wide Change: GCC8

[Jan Kurik]

= System Wide Change: GCC8 =
https://fedoraproject.org/wiki/Changes/GCC8

Change owner(s):
* Jakub Jelínek <jakub AT redhat DOT com>

Switch GCC in Fedora 28 to 8.x.y, rebuild all packages with it, or
optionally rebuild just some packages with it and rebuild all packages
only in Fedora 29.


== Detailed Description ==
GCC 8 is currently in stage3, will move to stage4 on January 14th, in
prerelease state with only regression bugfixes and documentation fixes
allowed. The release will happen probably in the middle of April. We
are working on scratch gcc rpms and will perform a test mass rebuild.

== Scope ==
All packages should be rebuilt with the new gcc once it hits f28, or,
if there is not enough time for that, just all packages built after
the new gcc hits the buildroots.

* Proposal owners:
Build gcc in f28, rebuild packages that have direct dependencies on
exact gcc version (libtool, llvm, gcc-python-plugin, odb).

* Other developers:
First few days/weeks just voluntary rebuilds using the new system gcc,
if things fail, look at http://gcc.gnu.org/gcc-8/porting_to.html and
fix bugs in packages or, if there is a gcc bug or suspected gcc bug,
analyze and report.

* Release engineering:
PR#7249: https://pagure.io/releng/issue/7249
Mass rebuild requested for F28.

* Policies and guidelines:
No policies need to be changed
-- 
Jan Kuřík
Platform & Fedora Program Manager
Red Hat Czech s.r.o., Purkynova 99/71, 612 45 Brno, Czech Republic

[Adam Williamson]

On Tue, 2018-01-09 at 12:11 +0100, Jan Kurik wrote:
>  = System Wide Change: GCC8 =
>  https://fedoraproject.org/wiki/Changes/GCC8
>  
>  Change owner(s):
>  * Jakub Jelínek <jakub AT redhat DOT com>
>  
>  Switch GCC in Fedora 28 to 8.x.y, rebuild all packages with it, or
>  optionally rebuild just some packages with it and rebuild all packages
>  only in Fedora 29.
>  
>  
>  == Detailed Description ==
>  GCC 8 is currently in stage3, will move to stage4 on January 14th, in
>  prerelease state with only regression bugfixes and documentation fixes
>  allowed. The release will happen probably in the middle of April. We
>  are working on scratch gcc rpms and will perform a test mass rebuild. 
The timing on this looks a bit awkward when compared with the current
schedule, which has the Beta going out in March and Final early in May:

https://fedoraproject.org/wiki/Releases/28/Schedule

That would appear to mean we'd have to do a mass rebuild with a pre-
release GCC, or do a mass rebuild between Beta and Final, or suddenly
introduce a new compiler post-Beta, potentially meaning that we need to
rebuild something to fix a blocker bug quite late in the release and
discover that the new GCC which has suddenly appeared causes problems
with building it. None of those sound like great options to me.

Is there significant enough benefit from the new GCC to outweigh all
these potential negative impacts to it going stable between our Beta
and Final releases?
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

[Jakub Jelinek]

On Tue, Jan 09, 2018 at 09:39:41AM -0800, Adam Williamson wrote:
>  The timing on this looks a bit awkward when compared with the
> current
>  schedule, which has the Beta going out in March and Final early in May:
>  
>  https://fedoraproject.org/wiki/Releases/28/Schedule
>  
>  That would appear to mean we'd have to do a mass rebuild with a pre-
>  release GCC, or do a mass rebuild between Beta and Final, or suddenly
>  introduce a new compiler post-Beta, potentially meaning that we need to
>  rebuild something to fix a blocker bug quite late in the release and
>  discover that the new GCC which has suddenly appeared causes problems
>  with building it. None of those sound like great options to me. 
Mass rebuild with a prerelease version of the compiler, like every year at
least in the past 10 years in Fedora.

>  Is there significant enough benefit from the new GCC to outweigh all
>  these potential negative impacts to it going stable between our Beta
>  and Final releases? 
Yes.

	Jakub

[Adam Williamson]

On Tue, 2018-01-09 at 18:45 +0100, Jakub Jelinek wrote:
>  On Tue, Jan 09, 2018 at 09:39:41AM -0800, Adam Williamson wrote:
>  > The timing on this looks a bit awkward when compared with the current
>  > schedule, which has the Beta going out in March and Final early in May:
>  > 
>  > https://fedoraproject.org/wiki/Releases/28/Schedule
>  > 
>  > That would appear to mean we'd have to do a mass rebuild with a pre-
>  > release GCC, or do a mass rebuild between Beta and Final, or suddenly
>  > introduce a new compiler post-Beta, potentially meaning that we need to
>  > rebuild something to fix a blocker bug quite late in the release and
>  > discover that the new GCC which has suddenly appeared causes problems
>  > with building it. None of those sound like great options to me.
>  
>  Mass rebuild with a prerelease version of the compiler, like every year at
>  least in the past 10 years in Fedora. 
Well, true, but then just like every year, we'll wind up doing a lot of
the spadework of fixing things to build with the new GCC. And probably
at first some critical things will fail to build and that'll mess up
the stability of the distro for a couple of weeks. I guess if everyone
else is still loving that grind, hey.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

[Stephen Gallagher]

On Tue, Jan 9, 2018 at 12:58 PM Adam Williamson <adamwill(a)fedoraproject.org&gt;
wrote:

>  On Tue, 2018-01-09 at 18:45 +0100, Jakub Jelinek wrote:
>  > On Tue, Jan 09, 2018 at 09:39:41AM -0800, Adam Williamson wrote:
>  > > The timing on this looks a bit awkward when compared with the current
>  > > schedule, which has the Beta going out in March and Final early in May:
>  > >
>  > > https://fedoraproject.org/wiki/Releases/28/Schedule
>  > >
>  > > That would appear to mean we'd have to do a mass rebuild with a pre-
>  > > release GCC, or do a mass rebuild between Beta and Final, or suddenly
>  > > introduce a new compiler post-Beta, potentially meaning that we need to
>  > > rebuild something to fix a blocker bug quite late in the release and
>  > > discover that the new GCC which has suddenly appeared causes problems
>  > > with building it. None of those sound like great options to me.
>  >
>  > Mass rebuild with a prerelease version of the compiler, like every year
>  at
>  > least in the past 10 years in Fedora.
>
>  Well, true, but then just like every year, we'll wind up doing a lot of
>  the spadework of fixing things to build with the new GCC. And probably
>  at first some critical things will fail to build and that'll mess up
>  the stability of the distro for a couple of weeks. I guess if everyone
>  else is still loving that grind, hey.
>  

This is the cost of being "First". Fedora has long enjoyed a tight coupling
with the GCC upstream. It's a symbiosis: they use our mass-rebuild to help
identify any issues before GCC goes stable and in turn Fedora gets to have
the newest compiler features before anyone else.

Realistically, since Fedora is the first real-world exercise of new GCC, if
we waited for the upstream stable release, it would be exactly as it is
now. Fedora would hit all the same issues and GCC would have to release
updates to fix them for us.

[Jakub Jelinek]

On Tue, Jan 09, 2018 at 07:50:10PM +0000, Stephen Gallagher wrote:
>  > Well, true, but then just like every year, we'll wind up
> doing a lot of
>  > the spadework of fixing things to build with the new GCC. And probably
>  > at first some critical things will fail to build and that'll mess up
>  > the stability of the distro for a couple of weeks. I guess if everyone
>  > else is still loving that grind, hey.
>  >
>  
>  
>  This is the cost of being "First". Fedora has long enjoyed a tight coupling
>  with the GCC upstream. It's a symbiosis: they use our mass-rebuild to help
>  identify any issues before GCC goes stable and in turn Fedora gets to have
>  the newest compiler features before anyone else. 
To be fair, Ubuntu (or Debian or both, dunno) has already performed test mass
rebuilds with GCC 8 prerelease some time ago and OpenSUSE usually performs them
roughly at the same time as we do.  We are likely the first one or one of
the first ones to deploy it as a stable compiler in the distro and it is
mutually beneficial both for the distro and for GCC.

	Jakub

[Adam Williamson]

On Tue, 2018-01-09 at 21:51 +0100, Jakub Jelinek wrote:
>  On Tue, Jan 09, 2018 at 07:50:10PM +0000, Stephen Gallagher wrote:
>  > > Well, true, but then just like every year, we'll wind up doing a lot of
>  > > the spadework of fixing things to build with the new GCC. And probably
>  > > at first some critical things will fail to build and that'll mess up
>  > > the stability of the distro for a couple of weeks. I guess if everyone
>  > > else is still loving that grind, hey.
>  > > 
>  > 
>  > 
>  > This is the cost of being "First". Fedora has long enjoyed a tight
> coupling
>  > with the GCC upstream. It's a symbiosis: they use our mass-rebuild to help
>  > identify any issues before GCC goes stable and in turn Fedora gets to have
>  > the newest compiler features before anyone else.
>  
>  To be fair, Ubuntu (or Debian or both, dunno) has already performed test mass
>  rebuilds with GCC 8 prerelease some time ago and OpenSUSE usually performs them
>  roughly at the same time as we do.  We are likely the first one or one of
>  the first ones to deploy it as a stable compiler in the distro and it is
>  mutually beneficial both for the distro and for GCC. 
Just for the record, it is now 11pm the day before we are supposed to
branch Fedora 28, and I have spent the whole evening fixing
OpenColorIO's Python bindings to build with GCC 8:

https://github.com/imageworks/OpenColorIO/pull/518

only to find that it fails to build on i686 because since pdftex got
rebuilt with GCC 8 (OK, haven't confirmed that yet, but it's the most
obvious suspect), it's segfaulting:

https://koji.fedoraproject.org/koji/taskinfo?taskID=25176488

Also noted by QuLogic trying to build R-htmltools:

https://koji.fedoraproject.org/koji/taskinfo?taskID=25173683

So now I am running the build in an i686 mock so I can shell into the
mock and hopefully get a traceback of the pdftex crash and try to do
*something* about fixing it.

This is the kind of 'spadework' I was talking about. (And I haven't
even mentioned any of the *other* cases we've hit).
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

Dynamically-generated C stored in SVN is breaking our distribution because life is awful (was Re: F28 System Wide Change: GCC8)

[Adam Williamson]

On Mon, 2018-02-19 at 22:57 -0800, Adam Williamson wrote:
>  On Tue, 2018-01-09 at 21:51 +0100, Jakub Jelinek wrote:
>  > On Tue, Jan 09, 2018 at 07:50:10PM +0000, Stephen Gallagher wrote:
>  > > > Well, true, but then just like every year, we'll wind up doing a lot
> of
>  > > > the spadework of fixing things to build with the new GCC. And probably
>  > > > at first some critical things will fail to build and that'll mess up
>  > > > the stability of the distro for a couple of weeks. I guess if everyone
>  > > > else is still loving that grind, hey.
>  > > > 
>  > > 
>  > > 
>  > > This is the cost of being "First". Fedora has long enjoyed a tight
> coupling
>  > > with the GCC upstream. It's a symbiosis: they use our mass-rebuild to help
>  > > identify any issues before GCC goes stable and in turn Fedora gets to have
>  > > the newest compiler features before anyone else.
>  > 
>  > To be fair, Ubuntu (or Debian or both, dunno) has already performed test mass
>  > rebuilds with GCC 8 prerelease some time ago and OpenSUSE usually performs them
>  > roughly at the same time as we do.  We are likely the first one or one of
>  > the first ones to deploy it as a stable compiler in the distro and it is
>  > mutually beneficial both for the distro and for GCC.
>  
>  Just for the record, it is now 11pm the day before we are supposed to
>  branch Fedora 28, and I have spent the whole evening fixing
>  OpenColorIO's Python bindings to build with GCC 8:
>  
>  https://github.com/imageworks/OpenColorIO/pull/518
>  
>  only to find that it fails to build on i686 because since pdftex got
>  rebuilt with GCC 8 (OK, haven't confirmed that yet, but it's the most
>  obvious suspect), it's segfaulting:
>  
>  https://koji.fedoraproject.org/koji/taskinfo?taskID=25176488
>  
>  Also noted by QuLogic trying to build R-htmltools:
>  
>  https://koji.fedoraproject.org/koji/taskinfo?taskID=25173683
>  
>  So now I am running the build in an i686 mock so I can shell into the
>  mock and hopefully get a traceback of the pdftex crash and try to do
>  *something* about fixing it. 
So, here lies a hilarious tale of dynamic generation of entirely
undocumented C code using single-character variable names by a
ridiculously arcane build system tracked in Subversion:

https://bugzilla.redhat.com/show_bug.cgi?id=1546964

Excuse me while I go break some stuff.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

Dynamically-generated C stored in SVN is breaking our distribution because life is awful (was Re: F28 System Wide Change: GCC8)

[Fabio Valentini]

On Feb 20, 2018 08:54, "Adam Williamson" <adamwill(a)fedoraproject.org&gt;
wrote:

On Mon, 2018-02-19 at 22:57 -0800, Adam Williamson wrote:
>  On Tue, 2018-01-09 at 21:51 +0100, Jakub Jelinek wrote:
>  > On Tue, Jan 09, 2018 at 07:50:10PM +0000, Stephen Gallagher wrote:
>  > > > Well, true, but then just like every year, we'll wind up doing a
lot of
>  > > > the spadework of fixing things to build with the new
> GCC. And 
probably
>  > > > at first some critical things will fail to build and
> that'll mess up
>  > > > the stability of the distro for a couple of weeks. I guess if
everyone
>  > > > else is still loving that grind, hey.
>  > > >
>  > >
>  > >
>  > > This is the cost of being "First". Fedora has long enjoyed a tight
coupling
>  > > with the GCC upstream. It's a symbiosis: they use our
> mass-rebuild to 
help
>  > > identify any issues before GCC goes stable and in turn
> Fedora gets to 
have
>  > > the newest compiler features before anyone else.
>  >
>  > To be fair, Ubuntu (or Debian or both, dunno) has already performed 
test mass
>  > rebuilds with GCC 8 prerelease some time ago and OpenSUSE
> usually 
performs them
>  > roughly at the same time as we do.  We are likely the first one
> or one 
of
>  > the first ones to deploy it as a stable compiler in the distro
> and it is
>  > mutually beneficial both for the distro and for GCC.
>
>  Just for the record, it is now 11pm the day before we are supposed to
>  branch Fedora 28, and I have spent the whole evening fixing
>  OpenColorIO's Python bindings to build with GCC 8:
>
>  https://github.com/imageworks/OpenColorIO/pull/518
>
>  only to find that it fails to build on i686 because since pdftex got
>  rebuilt with GCC 8 (OK, haven't confirmed that yet, but it's the most
>  obvious suspect), it's segfaulting:
>
>  https://koji.fedoraproject.org/koji/taskinfo?taskID=25176488
>
>  Also noted by QuLogic trying to build R-htmltools:
>
>  https://koji.fedoraproject.org/koji/taskinfo?taskID=25173683
>
>  So now I am running the build in an i686 mock so I can shell into the
>  mock and hopefully get a traceback of the pdftex crash and try to do
>  *something* about fixing it. 
So, here lies a hilarious tale of dynamic generation of entirely
undocumented C code using single-character variable names by a
ridiculously arcane build system tracked in Subversion:


Wow, that's exceptionally awful.
You're a hero for tracking down and fixing those things. Really. Thank you.

Fabio

https://bugzilla.redhat.com/show_bug.cgi?id=1546964

Excuse me while I go break some stuff.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org

<DKIM> Re: Dynamically-generated C stored in SVN is breaking our distribution because life is awful (was Re: F28 System Wide Change: GCC8)

[Nicolas Mailhot]

BTW dovecot is broken by all the compiler and crypto changes, both for the
two last rebuilds (that failed) and the build before (that succeeded but
has broken auth which is sort of needed for an imap server)

-- 
Nicolas Mailhot

[Sérgio Basto]

On Mon, 2018-02-19 at 22:57 -0800, Adam Williamson wrote:
>  On Tue, 2018-01-09 at 21:51 +0100, Jakub Jelinek wrote:
>  > On Tue, Jan 09, 2018 at 07:50:10PM +0000, Stephen Gallagher wrote:
>  > > > Well, true, but then just like every year, we'll wind up doing
>  > > > a lot of
>  > > > the spadework of fixing things to build with the new GCC. And
>  > > > probably
>  > > > at first some critical things will fail to build and that'll
>  > > > mess up
>  > > > the stability of the distro for a couple of weeks. I guess if
>  > > > everyone
>  > > > else is still loving that grind, hey.
>  > > > 
>  > > 
>  > > 
>  > > This is the cost of being "First". Fedora has long enjoyed a
>  > > tight coupling
>  > > with the GCC upstream. It's a symbiosis: they use our mass-
>  > > rebuild to help
>  > > identify any issues before GCC goes stable and in turn Fedora
>  > > gets to have
>  > > the newest compiler features before anyone else.
>  > 
>  > To be fair, Ubuntu (or Debian or both, dunno) has already performed
>  > test mass
>  > rebuilds with GCC 8 prerelease some time ago and OpenSUSE usually
>  > performs them
>  > roughly at the same time as we do.  We are likely the first one or
>  > one of
>  > the first ones to deploy it as a stable compiler in the distro and
>  > it is
>  > mutually beneficial both for the distro and for GCC.
>  
>  Just for the record, it is now 11pm the day before we are supposed to
>  branch Fedora 28, and I have spent the whole evening fixing
>  OpenColorIO's Python bindings to build with GCC 8: 
pdftex Segmentation fault in arches i686 and amrv7 when build gdal [1] 

gdal build fails and have broken deps, which makes openv broken deps ,
which make mlt broken.  
So I have a big chain of packages with broken deps and I can't do
nothing 

[1] 
https://bugzilla.redhat.com/show_bug.cgi?id=1546913

Best regards, 

>  https://github.com/imageworks/OpenColorIO/pull/518
>  
>  only to find that it fails to build on i686 because since pdftex got
>  rebuilt with GCC 8 (OK, haven't confirmed that yet, but it's the most
>  obvious suspect), it's segfaulting:
>  
>  https://koji.fedoraproject.org/koji/taskinfo?taskID=25176488
>  
>  Also noted by QuLogic trying to build R-htmltools:
>  
>  https://koji.fedoraproject.org/koji/taskinfo?taskID=25173683
>  
>  So now I am running the build in an i686 mock so I can shell into the
>  mock and hopefully get a traceback of the pdftex crash and try to do
>  *something* about fixing it.
>  
>  This is the kind of 'spadework' I was talking about. (And I haven't
>  even mentioned any of the *other* cases we've hit).
>  -- 
>  Adam Williamson
>  Fedora QA Community Monkey
>  IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin .
>  net
>  http://www.happyassassin.net
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org 
-- 
Sérgio M. B.

<DKIM> Re: Dynamically-generated C stored in SVN is breaking our distribution because life is awful (was Re: F28 System Wide Change: GCC8)

[Jason L Tibbitts III]

> >>>> "NM" == Nicolas Mailhot
> <nicolas.mailhot(a)laposte.net&gt; writes: 
NM> BTW dovecot is broken by all the compiler and crypto changes, both
NM> for the two last rebuilds (that failed) and the build before (that
NM> succeeded but has broken auth which is sort of needed for an imap
NM> server)

Interestingly enough, cyrus-imapd is also broken.  Technically it
compiles fine and mostly runs fine, but the extensive test suite (which
we now run at build time) causes the xapian-based search indexing
component to segfault deep down in the bowels of some tests.

In what is surely a plan to make me lose the rest of my hair, the
failures _only_ happen when run in mock.  In a clean rawhide VM or
docker container with enough build deps to run fedpkg local, things
build fine.  So far the only thing resembling a backtrace I've been able
to get out of anything is just this which coredumpctl saves:

    #0  0x0000000000000120 n/a (n/a)
    #1  0x00007ffd8745b720 n/a (n/a)
    #2  0x00007fb063abac80 _ZTVNSt7__cxx1119basic_ostringstreamIcSt11char_traitsIcESaIcEEE
(libstdc++.so.6)

That makes me wonder if I'm hitting a libstdc++ bug.  If I do
coredumpctl gdb and then to a backtrace, I just get:

(gdb) bt
#0  0x0000000000000120 in ?? ()
#1  0x00007fb06352881e in ?? ()
#2  0x00007ffd8745b7b0 in ?? ()
#3  0x00007ffd8745b528 in ?? ()
#4  0x0000000000000000 in ?? ()

which obviously isn't helpful.  And this was obtained by running mock on
a rawhide VM where I've made sure that the environment inside and
outside of the chroot have the same packages, all debuginfo is installed
and the test suite is run in %install instead of %check so that the
unstripped binaries are executed.  I can't figure out a way to get more
useful information out.

 - J<

<DKIM> Re: Dynamically-generated C stored in SVN is breaking our distribution because life is awful (was Re: F28 System Wide Change: GCC8)

[Nicolas Mailhot]

Le 2018-02-20 19:51, Jason L Tibbitts III a écrit :
> >>>>> "NM" == Nicolas Mailhot
> <nicolas.mailhot(a)laposte.net&gt; writes:
>  
>  NM> BTW dovecot is broken by all the compiler and crypto changes, both
>  NM> for the two last rebuilds (that failed) and the build before (that
>  NM> succeeded but has broken auth which is sort of needed for an imap
>  NM> server)
>  
>  Interestingly enough, cyrus-imapd is also broken.  Technically it
>  compiles fine and mostly runs fine, but the extensive test suite (which
>  we now run at build time) causes the xapian-based search indexing
>  component to segfault deep down in the bowels of some tests. 
Yes, the failure for the latest dovecot that built is weird, and for 
some reason abrt refuses to collect traces on the process. In the end I 
had to downgrade (I need my mail) and thankfully the previous devel 
binary build in koji does not bomb. I'll try to at least post the stack 
bits journalctl captured (investigating more would be nice but I doubt 
I’ll find the time, I’m kind of over-over-over-booked right now)

Regards,

-- 
Nicolas Mailhot