F31 System-Wide Change proposal: Enable Compiler Security hardening flags by default in G

[Ben Cotton]

https://fedoraproject.org/wiki/Changes/HardenedCompiler

== Summary ==
By Default enable a few security hardening flags which are used with GCC.

== Owner ==
* Name: [[User:huzaifas|Huzaifa Sidhpurwala]]
* Email: huzaifas(a)redhat.com
* Release notes owner: huzaifas(a)redhat.com


== Detailed Description ==
Currently GCC does not enable any security hardening flags by default.
They have to be explicitly enabled by the developers one-by-one.
Ubuntu (https://wiki.ubuntu.com/ToolChain/CompilerFlags) however
enables them and therefore has a hardened compiler by default. Each of
these options can be explicitly disabled if required by the developer
via a GCC command line flag.  I am currently proposing the following
flags be enabled by default.

'''-Wformat -Wformat-security -fstack-protector-strong
--param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -O'''''

{| class="wikitable"
|-
! No !! Flag !! Use !! How to disable
|-
| 1 || -Wformat || Check calls to "printf" and "scanf", etc., to make
sure that the arguments supplied have types appropriate to the format
string specified, and that the conversions specified in the  format
string make sense. || -Wno-format
|-
| 2 || -Wformat-security || If -Wformat is specified, also warn about
uses of format functions that represent possible security problems.
|| -Wno-format should disable this as well
|-
| 3 || -fstack-protector-strong || Like -fstack-protector but includes
additional functions to be protected --- those that have local array
definitions, or have references to local frame addresses.
|| -fno-stack-protector
|}


== Benefit to Fedora ==
We provide better security both for our packages and for
applications/programs which users are building.

== Scope ==
* Proposal owners: Patch gcc to enable these options by default. Patch
should be very simple, since the compile/link code isnt actually
touched.
* Other developers: Developers need to ensure that Fedora package is
built and if any build failures they are corrected
* Release engineering: [https://pagure.io/releng/issue/8204 #8204]
* Policies and guidelines: The policies and guidelines do not need to
be updated.
* Trademark approval: Not needed for this change

== Upgrade/compatibility impact ==
None

== How To Test ==
Run "gcc -Q -v <foo.c>" to check if these flags are enabled by default

== User Experience ==
Fedora is more secure because the entire distribution is compiled with
the correct security technologies enabled. Developers dont have to
worry about enabling the right flags when they compile their
application in Fedora because the compiler has them enabled by
default.

== Dependencies ==
All packages will be rebuild with new GCC options.

== Contingency Plan ==
* Contingency mechanism: Roll back the GCC options and use the default ones.
* Contingency deadline: Beta Feeze
* Blocks release? No




-- 
Ben Cotton
Fedora Program Manager
TZ=America/Indiana/Indianapolis

[Florian Weimer]

* Ben Cotton:

>  '''-Wformat -Wformat-security -fstack-protector-strong
>  --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -O''''' 
--param=ssp-buffer-size=4 will not affect anything because
-fstack-protector-strong uses a completely different heuristic.

>  == Benefit to Fedora ==
>  We provide better security both for our packages and for
>  applications/programs which users are building. 
We can check using annocheck if there are packages missing hardening and
fix them.  What's the current level of coverage we have?

Have the Red Hat Enterprise Linux 8 packaging changes been upstreamed?
We were aiming for nearly-complete coverage there.

>  == Scope ==
>  * Proposal owners: Patch gcc to enable these options by default. Patch
>  should be very simple, since the compile/link code isnt actually
>  touched. 
-D_FORTIFY_SOURCE=2 by default needs patching of glibc because of the
pesky warning it prints without optimization.

What about PIE by defauld and non-lazy binding by default?  These two
are probably the two hardest to get right with CFLAGS/LDFLAGS injection.
(PIE is the reason for the -specs= hack.)

PIE-by-default compilers are very common already, although there are
many StackOverflow questions from peopel who use them and follow older
training material.

Thanks,
Florian

[Huzaifa Sidhpurwala]

On 3/11/19 11:48 PM, Florian Weimer wrote:
>  * Ben Cotton:
>  
> > '''-Wformat -Wformat-security -fstack-protector-strong
> > --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -O'''''
>  
>  --param=ssp-buffer-size=4 will not affect anything because
>  -fstack-protector-strong uses a completely different heuristic.
>  
> > == Benefit to Fedora ==
> > We provide better security both for our packages and for
> > applications/programs which users are building.
>  
>  We can check using annocheck if there are packages missing hardening and
>  fix them.  What's the current level of coverage we have?
>   
The intention is to have a secure compiler defaults. I have not taken a
survey of what is the current coverage, but that is not really the goal
here. Also having secure defaults help with user applications and not
only fedora packages.

>  Have the Red Hat Enterprise Linux 8 packaging changes been
> upstreamed?
>  We were aiming for nearly-complete coverage there.
>   
Not sure about this, but again this is not the aim.
> > == Scope ==
> > * Proposal owners: Patch gcc to enable these options by default. Patch
> > should be very simple, since the compile/link code isnt actually
> > touched.
>  
>  -D_FORTIFY_SOURCE=2 by default needs patching of glibc because of the
>  pesky warning it prints without optimization. 
What is the amount of work this
requires? Can we aim this for F31? Also
these are warnings and not errors i assume?
>  
>  What about PIE by defauld and non-lazy binding by default?  These two
>  are probably the two hardest to get right with CFLAGS/LDFLAGS injection.
>  (PIE is the reason for the -specs= hack.)
>   
I am aiming this for F32, The intention is to add small number of secure
defaults to GCC for each release. I am open to add PIE by default
though, if you feel its not going to break large number of packages.


>  PIE-by-default compilers are very common already, although there are
>  many StackOverflow questions from peopel who use them and follow older
>  training material.
>  
>  Thanks,
>  Florian
>   

-- 
Huzaifa Sidhpurwala / Red Hat Product Security Team

[Vít Ondruch]

Will it help to mitigate issues such as:

https://bugzilla.redhat.com/show_bug.cgi?id=1284684

and mitigate workarounds such as:

https://bugzilla.redhat.com/show_bug.cgi?id=1543394

That would be wonderful.


Also, while OT to this specific change, I would love to have ability to
have some compiler flags tailored to my environment. E.g. enabled
optimizations specific to my CPU. That could enable potential of JIT
compilation in Ruby and possibly everywhere else where compiler is
involved in installation some extensions from 3rd party repositories.


Vít


Dne 11. 03. 19 v 18:56 Ben Cotton napsal(a):
> https://fedoraproject.org/wiki/Changes/HardenedCompiler
>
> == Summary ==
> By Default enable a few security hardening flags which are used with GCC.
>
> == Owner ==
> * Name: [[User:huzaifas|Huzaifa Sidhpurwala]]
> * Email: huzaifas(a)redhat.com
> * Release notes owner: huzaifas(a)redhat.com
>
>
> == Detailed Description ==
> Currently GCC does not enable any security hardening flags by default.
> They have to be explicitly enabled by the developers one-by-one.
> Ubuntu (https://wiki.ubuntu.com/ToolChain/CompilerFlags) however
> enables them and therefore has a hardened compiler by default. Each of
> these options can be explicitly disabled if required by the developer
> via a GCC command line flag.  I am currently proposing the following
> flags be enabled by default.
>
> '''-Wformat -Wformat-security -fstack-protector-strong
> --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -O'''''
>
> {| class="wikitable"
> |-
> ! No !! Flag !! Use !! How to disable
> |-
> | 1 || -Wformat || Check calls to "printf" and "scanf", etc., to
make
> sure that the arguments supplied have types appropriate to the format
> string specified, and that the conversions specified in the  format
> string make sense. || -Wno-format
> |-
> | 2 || -Wformat-security || If -Wformat is specified, also warn about
> uses of format functions that represent possible security problems.
> || -Wno-format should disable this as well
> |-
> | 3 || -fstack-protector-strong || Like -fstack-protector but includes
> additional functions to be protected --- those that have local array
> definitions, or have references to local frame addresses.
> || -fno-stack-protector
> |}
>
>
> == Benefit to Fedora ==
> We provide better security both for our packages and for
> applications/programs which users are building.
>
> == Scope ==
> * Proposal owners: Patch gcc to enable these options by default. Patch
> should be very simple, since the compile/link code isnt actually
> touched.
> * Other developers: Developers need to ensure that Fedora package is
> built and if any build failures they are corrected
> * Release engineering: [https://pagure.io/releng/issue/8204 #8204]
> * Policies and guidelines: The policies and guidelines do not need to
> be updated.
> * Trademark approval: Not needed for this change
>
> == Upgrade/compatibility impact ==
> None
>
> == How To Test ==
> Run "gcc -Q -v <foo.c>" to check if these flags are enabled by
default
>
> == User Experience ==
> Fedora is more secure because the entire distribution is compiled with
> the correct security technologies enabled. Developers dont have to
> worry about enabling the right flags when they compile their
> application in Fedora because the compiler has them enabled by
> default.
>
> == Dependencies ==
> All packages will be rebuild with new GCC options.
>
> == Contingency Plan ==
> * Contingency mechanism: Roll back the GCC options and use the default ones.
> * Contingency deadline: Beta Feeze
> * Blocks release? No
>
>
>
>

[Huzaifa Sidhpurwala]

Hi Vit,

On 3/12/19 5:40 PM, Vít Ondruch wrote:
>  Will it help to mitigate issues such as:
>  
>  https://bugzilla.redhat.com/show_bug.cgi?id=1284684
>  
This is related to the following change which was made in Fedora 23:
https://fedoraproject.org/wiki/Changes/Harden_All_Packages.

My proposal does not touch PIE or RELRO at all, but is related to
compiling code with protections which mitigate, format string attacks
and stack-based buffer overflows. It is pretty common to enable these
flags while compiling, its just strange that we dont enable these by
default.

>  and mitigate workarounds such as:
>  
>  https://bugzilla.redhat.com/show_bug.cgi?id=1543394
>  
>  That would be wonderful.
>  
>  
>  Also, while OT to this specific change, I would love to have ability to
>  have some compiler flags tailored to my environment. E.g. enabled
>  optimizations specific to my CPU. That could enable potential of JIT
>  compilation in Ruby and possibly everywhere else where compiler is
>  involved in installation some extensions from 3rd party repositories.
>  
>  
>  Vít
>  
>  
>  Dne 11. 03. 19 v 18:56 Ben Cotton napsal(a):
> > https://fedoraproject.org/wiki/Changes/HardenedCompiler
> >
> > == Summary ==
> > By Default enable a few security hardening flags which are used with GCC.
> >
> > == Owner ==
> > * Name: [[User:huzaifas|Huzaifa Sidhpurwala]]
> > * Email: huzaifas(a)redhat.com
> > * Release notes owner: huzaifas(a)redhat.com
> >
> >
> > == Detailed Description ==
> > Currently GCC does not enable any security hardening flags by default.
> > They have to be explicitly enabled by the developers one-by-one.
> > Ubuntu (https://wiki.ubuntu.com/ToolChain/CompilerFlags) however
> > enables them and therefore has a hardened compiler by default. Each of
> > these options can be explicitly disabled if required by the developer
> > via a GCC command line flag.  I am currently proposing the following
> > flags be enabled by default.
> >
> > '''-Wformat -Wformat-security -fstack-protector-strong
> > --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -O'''''
> >
> > {| class="wikitable"
> > |-
> > ! No !! Flag !! Use !! How to disable
> > |-
> > | 1 || -Wformat || Check calls to "printf" and "scanf", etc., to
> make
> > sure that the arguments supplied have types appropriate to the format
> > string specified, and that the conversions specified in the  format
> > string make sense. || -Wno-format
> > |-
> > | 2 || -Wformat-security || If -Wformat is specified, also warn about
> > uses of format functions that represent possible security problems.
> > || -Wno-format should disable this as well
> > |-
> > | 3 || -fstack-protector-strong || Like -fstack-protector but includes
> > additional functions to be protected --- those that have local array
> > definitions, or have references to local frame addresses.
> > || -fno-stack-protector
> > |}
> >
> >
> > == Benefit to Fedora ==
> > We provide better security both for our packages and for
> > applications/programs which users are building.
> >
> > == Scope ==
> > * Proposal owners: Patch gcc to enable these options by default. Patch
> > should be very simple, since the compile/link code isnt actually
> > touched.
> > * Other developers: Developers need to ensure that Fedora package is
> > built and if any build failures they are corrected
> > * Release engineering: [https://pagure.io/releng/issue/8204 #8204]
> > * Policies and guidelines: The policies and guidelines do not need to
> > be updated.
> > * Trademark approval: Not needed for this change
> >
> > == Upgrade/compatibility impact ==
> > None
> >
> > == How To Test ==
> > Run "gcc -Q -v <foo.c>" to check if these flags are enabled by
> default
> >
> > == User Experience ==
> > Fedora is more secure because the entire distribution is compiled with
> > the correct security technologies enabled. Developers dont have to
> > worry about enabling the right flags when they compile their
> > application in Fedora because the compiler has them enabled by
> > default.
> >
> > == Dependencies ==
> > All packages will be rebuild with new GCC options.
> >
> > == Contingency Plan ==
> > * Contingency mechanism: Roll back the GCC options and use the default ones.
> > * Contingency deadline: Beta Feeze
> > * Blocks release? No
> >
> >
> >
> >
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>   

-- 
Huzaifa Sidhpurwala / Red Hat Product Security Team

[Tom Hughes]

On 13/03/2019 03:27, Huzaifa Sidhpurwala wrote:

>  On 3/12/19 5:40 PM, Vít Ondruch wrote:
> > Will it help to mitigate issues such as:
> >
> > https://bugzilla.redhat.com/show_bug.cgi?id=1284684
> >
>  This is related to the following change which was made in Fedora 23:
>  https://fedoraproject.org/wiki/Changes/Harden_All_Packages.
>  
>  My proposal does not touch PIE or RELRO at all, but is related to
>  compiling code with protections which mitigate, format string attacks
>  and stack-based buffer overflows. It is pretty common to enable these
>  flags while compiling, its just strange that we dont enable these by
>  default. 
We do, just not by changing the compiler defaults.

Instead they are in %{optflags} which all packages are expected
to use for their compiler flags:

https://docs.fedoraproject.org/en-US/packaging-guidelines/#_compiler_flags

Here's what %optflags looks like for F29:

-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 
-Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong 
-grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic 
-fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection

Tom

-- 
Tom Hughes (tom(a)compton.nu)
http://compton.nu/

[Florian Weimer]

* Tom Hughes:

>  On 13/03/2019 03:27, Huzaifa Sidhpurwala wrote:
>
> > On 3/12/19 5:40 PM, Vít Ondruch wrote:
> >> Will it help to mitigate issues such as:
> >>
> >> https://bugzilla.redhat.com/show_bug.cgi?id=1284684
> >>
> > This is related to the following change which was made in Fedora 23:
> > https://fedoraproject.org/wiki/Changes/Harden_All_Packages.
> >
> > My proposal does not touch PIE or RELRO at all, but is related to
> > compiling code with protections which mitigate, format string attacks
> > and stack-based buffer overflows. It is pretty common to enable these
> > flags while compiling, its just strange that we dont enable these by
> > default.
>
>  We do, just not by changing the compiler defaults.
>
>  Instead they are in %{optflags} which all packages are expected
>  to use for their compiler flags:
>
>  https://docs.fedoraproject.org/en-US/packaging-guidelines/#_compiler_flags
>
>  Here's what %optflags looks like for F29:
>
>  -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2
>  -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong
>  -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
>  -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
>  -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection 
I think Huzaifa knows that. 8-)

But I'm wondering to what extent this is not working.  Previous guidance
from the Red Hat Platform Tools team was changing the compiler defaults
was not a good idea.  If the data show that changing the defaults is the
only way to achieve decent coverage, then we will need to reevaluate
what we are doing.

However, starting out with -D_FORTIFY_SOURCE=2 (and not things like PIE
or -fstack-clash-protection) seems odd in any case because that's one of
the most difficult changes.

Thanks,
Florian

[Tom Hughes]

On 13/03/2019 11:11, Florian Weimer wrote:
>  * Tom Hughes:
>  
> > On 13/03/2019 03:27, Huzaifa Sidhpurwala wrote:
> >
> >> On 3/12/19 5:40 PM, Vít Ondruch wrote:
> >>> Will it help to mitigate issues such as:
> >>>
> >>> https://bugzilla.redhat.com/show_bug.cgi?id=1284684
> >>>
> >> This is related to the following change which was made in Fedora 23:
> >> https://fedoraproject.org/wiki/Changes/Harden_All_Packages.
> >>
> >> My proposal does not touch PIE or RELRO at all, but is related to
> >> compiling code with protections which mitigate, format string attacks
> >> and stack-based buffer overflows. It is pretty common to enable these
> >> flags while compiling, its just strange that we dont enable these by
> >> default.
> >
> > We do, just not by changing the compiler defaults.
> >
> > Instead they are in %{optflags} which all packages are expected
> > to use for their compiler flags:
> >
> > https://docs.fedoraproject.org/en-US/packaging-guidelines/#_compiler_flags
> >
> > Here's what %optflags looks like for F29:
> >
> > -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2
> > -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong
> > -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
> > -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
> > -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
>  
>  I think Huzaifa knows that. 8-) 
Well that hasn't been at all clear in this thread as he keeps
talking like we're not building packages with these options at
the moment.

Tom

-- 
Tom Hughes (tom(a)compton.nu)
http://compton.nu/

[Jakub Jelinek]

On Mon, Mar 11, 2019 at 01:56:14PM -0400, Ben Cotton wrote:
>  https://fedoraproject.org/wiki/Changes/HardenedCompiler
>  
>  == Summary ==
>  By Default enable a few security hardening flags which are used with GCC. 
I'm strongly against this, the reasons have been explained multiple times.

We have annobin and easy way to determine what misses to propagate the flags
down.

	Jakub

[Dridi Boukelmoune]

On Wed, Mar 13, 2019 at 12:19 PM Jakub Jelinek <jakub(a)redhat.com&gt; wrote:
>  On Mon, Mar 11, 2019 at 01:56:14PM -0400, Ben Cotton wrote:
>  > https://fedoraproject.org/wiki/Changes/HardenedCompiler
>  >
>  > == Summary ==
>  > By Default enable a few security hardening flags which are used with GCC.
>
>  I'm strongly against this, the reasons have been explained multiple times.
>
>  We have annobin and easy way to determine what misses to propagate the flags
>  down. 
I think the key sentence here is this one:

>  == Benefit to Fedora ==
>  We provide better security both for our packages and for
>  applications/programs which users are building. 
IMHO this should have nothing to do with our packages since we already
have guidelines regarding hardening and in most cases it should be the
case without package maintainer intervention (exotic build systems or
misuse or misconfiguration do exist).

To me this change should only be meant for end-users of GCC, not the
Fedora build infrastructure itself.

Dridi

[Jakub Jelinek]

On Wed, Mar 13, 2019 at 12:38:02PM +0100, Dridi Boukelmoune wrote:
>  On Wed, Mar 13, 2019 at 12:19 PM Jakub Jelinek
> <jakub(a)redhat.com&gt; wrote:
>  >
>  > On Mon, Mar 11, 2019 at 01:56:14PM -0400, Ben Cotton wrote:
>  > > https://fedoraproject.org/wiki/Changes/HardenedCompiler
>  > >
>  > > == Summary ==
>  > > By Default enable a few security hardening flags which are used with GCC.
>  >
>  > I'm strongly against this, the reasons have been explained multiple times.
>  >
>  > We have annobin and easy way to determine what misses to propagate the flags
>  > down.
>  
>  I think the key sentence here is this one:
>  
>  > == Benefit to Fedora ==
>  > We provide better security both for our packages and for
>  > applications/programs which users are building.
>  
>  IMHO this should have nothing to do with our packages since we already
>  have guidelines regarding hardening and in most cases it should be the
>  case without package maintainer intervention (exotic build systems or
>  misuse or misconfiguration do exist).
>  
>  To me this change should only be meant for end-users of GCC, not the
>  Fedora build infrastructure itself. 
I'm all for making it easier for users, say by adding
hardened-gcc/hardened-g++ wrappers or some dir with gcc/g++ wrappers users
can prepend in PATH if they want certain behavior, but changing the defaults
of what gcc does is a huge mistake.  I know some distros have done it for
certain options, that doesn't change my opinion about it.
The thing is, when the defaults change, then people using the compiler need
to start using -fno-pie, -U__FORTIFY_SOURCE, -fno-stack-protector and the
like whenever they do want normal behavior, and as cross environments you
can't rely on the same defaults you need to stick those or the hardening
flags everywhere because you don't know what the compiler of the day will
do.  Not to mention that -D__FORTIFY_SOURCE=2 rejects some valid C programs,
so gcc would be no longer standard compliant (and e.g. glibc headers warn
about it when used with -O0).
It is a similar reason why gcc doesn't change all of sudden -O0 default to
-O2.

	Jakub

[Michael Cronenworth]

On 3/13/19 6:52 AM, Jakub Jelinek wrote:
>  Not to mention that -D__FORTIFY_SOURCE=2 rejects some valid C
> programs,
>  so gcc would be no longer standard compliant (and e.g. glibc headers warn
>  about it when used with -O0). 
+1, Wine breaks with fortify set to 2.

I hope Huzaifa takes your opinion to heart, Jakub.

Huzaifa, please work with upstream if you want to change the defaults.

[Huzaifa Sidhpurwala]

On 3/13/19 5:22 PM, Jakub Jelinek wrote:
>  On Wed, Mar 13, 2019 at 12:38:02PM +0100, Dridi Boukelmoune wrote:
> > On Wed, Mar 13, 2019 at 12:19 PM Jakub Jelinek <jakub(a)redhat.com&gt; wrote:
> >>
> >> On Mon, Mar 11, 2019 at 01:56:14PM -0400, Ben Cotton wrote:
> >>> https://fedoraproject.org/wiki/Changes/HardenedCompiler
> >>>
> >>> == Summary ==
> >>> By Default enable a few security hardening flags which are used with GCC.
> >>
> >> I'm strongly against this, the reasons have been explained multiple times.
> >>
> >> We have annobin and easy way to determine what misses to propagate the flags
> >> down.
> >
> > I think the key sentence here is this one:
> >
> >> == Benefit to Fedora ==
> >> We provide better security both for our packages and for
> >> applications/programs which users are building.
> >
> > IMHO this should have nothing to do with our packages since we already
> > have guidelines regarding hardening and in most cases it should be the
> > case without package maintainer intervention (exotic build systems or
> > misuse or misconfiguration do exist).
> >
> > To me this change should only be meant for end-users of GCC, not the
> > Fedora build infrastructure itself.
>  
>  I'm all for making it easier for users, say by adding
>  hardened-gcc/hardened-g++ wrappers or some dir with gcc/g++ wrappers users
>  can prepend in PATH if they want certain behavior, but changing the defaults
>  of what gcc does is a huge mistake.  I know some distros have done it for
>  certain options, that doesn't change my opinion about it.
>  The thing is, when the defaults change, then people using the compiler need
>  to start using -fno-pie, -U__FORTIFY_SOURCE, -fno-stack-protector and the
>  like whenever they do want normal behavior, and as cross environments you
>  can't rely on the same defaults you need to stick those or the hardening
>  flags everywhere because you don't know what the compiler of the day will
>  do.  Not to mention that -D__FORTIFY_SOURCE=2 rejects some valid C programs,
>  so gcc would be no longer standard compliant (and e.g. glibc headers warn
>  about it when used with -O0).
>  It is a similar reason why gcc doesn't change all of sudden -O0 default to
>  -O2. 
If -D__FORTIFY_SOURCE=2 breaks applications we can exclude that from the
default flags. However i still think we should keep the other flags. Its
upto developers to disable security flags if they want to, but that does
not mean we should have default secure options.

We assume that developers know what they are doing when they disable
these flags. Also if it breaks on some arches, we can probably only
enable it on x86_64 to start with.


>  
>  	Jakub
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]
>   

-- 
Huzaifa Sidhpurwala / Red Hat Product Security Team

[Daniel P. Berrangé]

On Mon, Mar 11, 2019 at 01:56:14PM -0400, Ben Cotton wrote:
>  https://fedoraproject.org/wiki/Changes/HardenedCompiler
>  
>  == Summary ==
>  By Default enable a few security hardening flags which are used with GCC.
>  
>  == Owner ==
>  * Name: [[User:huzaifas|Huzaifa Sidhpurwala]]
>  * Email: huzaifas(a)redhat.com
>  * Release notes owner: huzaifas(a)redhat.com
>  
>  
>  == Detailed Description ==
>  Currently GCC does not enable any security hardening flags by default.
>  They have to be explicitly enabled by the developers one-by-one.
>  Ubuntu (https://wiki.ubuntu.com/ToolChain/CompilerFlags) however
>  enables them and therefore has a hardened compiler by default. Each of
>  these options can be explicitly disabled if required by the developer
>  via a GCC command line flag.  I am currently proposing the following
>  flags be enabled by default.
>  
>  '''-Wformat -Wformat-security -fstack-protector-strong
>  --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -O''''' 
>  | 1 || -Wformat || Check calls to "printf" and
> "scanf", etc., to make
>  sure that the arguments supplied have types appropriate to the format
>  string specified, and that the conversions specified in the  format
>  string make sense. || -Wno-format
>  |-
>  | 2 || -Wformat-security || If -Wformat is specified, also warn about
>  uses of format functions that represent possible security problems.
>  || -Wno-format should disable this as well 
These two are very valuable warnings. If a C application's existing
build process has not already enabled them by default, I would expect
they'll trigger a great number of warnings.

We're not using -Werror in Fedora though, so these will not cause a
build failure.

Are we expecting Fedora maintainers to read the build logs & look for
these new warnings & report them upstream for fixing ? I'm sceptical
that many maintainers are going to put effort into that kind of thing
if it isn't blocking their builds.

IOW what is the real benefit of enabling them ? Emitting more warnings
doesn't make Fedora more secure as the change claims. To be more secure
would require using -Werror=format-security which would be a harder sell
as a default policy for Fedora.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[Tom Hughes]

On 13/03/2019 12:00, Daniel P. Berrangé wrote:
>  On Mon, Mar 11, 2019 at 01:56:14PM -0400, Ben Cotton wrote:
>
> > | 1 || -Wformat || Check calls to "printf" and "scanf", etc., to
> make
> > sure that the arguments supplied have types appropriate to the format
> > string specified, and that the conversions specified in the  format
> > string make sense. || -Wno-format
> > |-
> > | 2 || -Wformat-security || If -Wformat is specified, also warn about
> > uses of format functions that represent possible security problems.
> > || -Wno-format should disable this as well
>  
>  These two are very valuable warnings. If a C application's existing
>  build process has not already enabled them by default, I would expect
>  they'll trigger a great number of warnings.
>  
>  We're not using -Werror in Fedora though, so these will not cause a
>  build failure. 
Actually the default optflags already has -Wall (which includes
-Wformat) and -Werror=format-security which enables that warning
and turns on -Werror for it.

So format-security does actually cause build failures already.

Tom

-- 
Tom Hughes (tom(a)compton.nu)
http://compton.nu/

[Daniel P. Berrangé]

On Wed, Mar 13, 2019 at 12:21:29PM +0000, Tom Hughes wrote:
>  On 13/03/2019 12:00, Daniel P. Berrangé wrote:
>  > On Mon, Mar 11, 2019 at 01:56:14PM -0400, Ben Cotton wrote:
>  > 
>  > > | 1 || -Wformat || Check calls to "printf" and "scanf",
> etc., to make
>  > > sure that the arguments supplied have types appropriate to the format
>  > > string specified, and that the conversions specified in the  format
>  > > string make sense. || -Wno-format
>  > > |-
>  > > | 2 || -Wformat-security || If -Wformat is specified, also warn about
>  > > uses of format functions that represent possible security problems.
>  > > || -Wno-format should disable this as well
>  > 
>  > These two are very valuable warnings. If a C application's existing
>  > build process has not already enabled them by default, I would expect
>  > they'll trigger a great number of warnings.
>  > 
>  > We're not using -Werror in Fedora though, so these will not cause a
>  > build failure.
>  
>  Actually the default optflags already has -Wall (which includes
>  -Wformat) and -Werror=format-security which enables that warning
>  and turns on -Werror for it. 
I wonder why this change is suggesting to add the flags if they
are already present in our current optflags ?

>  So format-security does actually cause build failures already.
That's good !

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[Björn Persson]

Daniel P. Berrangé wrote:
>  I wonder why this change is suggesting to add the flags if they
>  are already present in our current optflags ? 
Obviously because Fedora's packaged GCC is used in more use cases than
just building Fedora packages, and Fedora's RPM macros aren't typically
applied to the users' own, non-RPM-packaged software.

Björn Persson

[Huzaifa Sidhpurwala]

On 3/13/19 5:55 PM, Daniel P. Berrangé wrote:
>  On Wed, Mar 13, 2019 at 12:21:29PM +0000, Tom Hughes wrote:
> > On 13/03/2019 12:00, Daniel P. Berrangé wrote:
> >> On Mon, Mar 11, 2019 at 01:56:14PM -0400, Ben Cotton wrote:
> >>
> >>> | 1 || -Wformat || Check calls to "printf" and "scanf",
> etc., to make
> >>> sure that the arguments supplied have types appropriate to the format
> >>> string specified, and that the conversions specified in the  format
> >>> string make sense. || -Wno-format
> >>> |-
> >>> | 2 || -Wformat-security || If -Wformat is specified, also warn about
> >>> uses of format functions that represent possible security problems.
> >>> || -Wno-format should disable this as well
> >>
> >> These two are very valuable warnings. If a C application's existing
> >> build process has not already enabled them by default, I would expect
> >> they'll trigger a great number of warnings.
> >>
> >> We're not using -Werror in Fedora though, so these will not cause a
> >> build failure.
> >
> > Actually the default optflags already has -Wall (which includes
> > -Wformat) and -Werror=format-security which enables that warning
> > and turns on -Werror for it.
>  
>  I wonder why this change is suggesting to add the flags if they
>  are already present in our current optflags ?
>   
These gets enabled when you build packages via koji, not when you use
gcc to build packages. Having them enabled by default in gcc, ensures
that user applications get them by default.

-- 
Huzaifa Sidhpurwala / Red Hat Product Security Team

[Huzaifa Sidhpurwala]

On 3/13/19 5:30 PM, Daniel P. Berrangé wrote:
>  On Mon, Mar 11, 2019 at 01:56:14PM -0400, Ben Cotton wrote:
> > https://fedoraproject.org/wiki/Changes/HardenedCompiler
> >
> > == Summary ==
> > By Default enable a few security hardening flags which are used with GCC.
> >
> > == Owner ==
> > * Name: [[User:huzaifas|Huzaifa Sidhpurwala]]
> > * Email: huzaifas(a)redhat.com
> > * Release notes owner: huzaifas(a)redhat.com
> >
> >
> > == Detailed Description ==
> > Currently GCC does not enable any security hardening flags by default.
> > They have to be explicitly enabled by the developers one-by-one.
> > Ubuntu (https://wiki.ubuntu.com/ToolChain/CompilerFlags) however
> > enables them and therefore has a hardened compiler by default. Each of
> > these options can be explicitly disabled if required by the developer
> > via a GCC command line flag.  I am currently proposing the following
> > flags be enabled by default.
> >
> > '''-Wformat -Wformat-security -fstack-protector-strong
> > --param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2 -O'''''
>  
> > | 1 || -Wformat || Check calls to "printf" and "scanf", etc., to
> make
> > sure that the arguments supplied have types appropriate to the format
> > string specified, and that the conversions specified in the  format
> > string make sense. || -Wno-format
> > |-
> > | 2 || -Wformat-security || If -Wformat is specified, also warn about
> > uses of format functions that represent possible security problems.
> > || -Wno-format should disable this as well
>  
>  These two are very valuable warnings. If a C application's existing
>  build process has not already enabled them by default, I would expect
>  they'll trigger a great number of warnings.
>  
>  We're not using -Werror in Fedora though, so these will not cause a
>  build failure.
>  
>  Are we expecting Fedora maintainers to read the build logs & look for
>  these new warnings & report them upstream for fixing ? I'm sceptical
>  that many maintainers are going to put effort into that kind of thing
>  if it isn't blocking their builds.
>   
Its upto the package maintainer to use -Werror to block builds which
show these warnings. I dont think we should enforce at this point. But
maybe at some time later we could:

1. Do an automated scan of build logs to figure out which packages show
these warnings and figure out how to handle them.

2. For critical packages like network daemons etc, we could actively
block packages which show these warnings.


Overall as i mentioned before, i would like to start somewhere :)

-- 
Huzaifa Sidhpurwala / Red Hat Product Security Team

[Richard W.M. Jones]

On Mon, Mar 11, 2019 at 01:56:14PM -0400, Ben Cotton wrote:
>  https://fedoraproject.org/wiki/Changes/HardenedCompiler 
I'm not opposing this, but is it possible we could do this without
breaking clang at the same time?

In the past (and currently) the Fedora compiler flags need some hairy
editing so they work with clang, eg:

https://src.fedoraproject.org/rpms/american-fuzzy-lop/blob/master/f/ameri...

(Actually this is not the latest iteration - latest clang 7 and gcc 9
and Fedora 30+ needs even more editing, but I didn't push it yet since
there are other issues with this package.)

It would be nice if there was a way we could avoid this.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

[Richard W.M. Jones]

On Fri, Mar 15, 2019 at 04:15:58PM +0000, Richard W.M. Jones wrote:
>  On Mon, Mar 11, 2019 at 01:56:14PM -0400, Ben Cotton wrote:
>  > https://fedoraproject.org/wiki/Changes/HardenedCompiler
>  
>  I'm not opposing this, but is it possible we could do this without
>  breaking clang at the same time?
>  
>  In the past (and currently) the Fedora compiler flags need some hairy
>  editing so they work with clang, eg:
>  
>
> https://src.fedoraproject.org/rpms/american-fuzzy-lop/blob/master/f/ameri...
>  
>  (Actually this is not the latest iteration - latest clang 7 and gcc 9
>  and Fedora 30+ needs even more editing, but I didn't push it yet since
>  there are other issues with this package.)
>  
>  It would be nice if there was a way we could avoid this. 
So after rereading the proposal more carefully it seems as if the
proposal is to change the defaults in GCC so no flags would need to be
specified.  Would we consequently remove those flags from the command
line (which would solve my problem above)?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

[Huzaifa Sidhpurwala]

On 3/15/19 9:49 PM, Richard W.M. Jones wrote:
>  On Fri, Mar 15, 2019 at 04:15:58PM +0000, Richard W.M. Jones wrote:
> > On Mon, Mar 11, 2019 at 01:56:14PM -0400, Ben Cotton wrote:
> >> https://fedoraproject.org/wiki/Changes/HardenedCompiler
> >
> > I'm not opposing this, but is it possible we could do this without
> > breaking clang at the same time?
> >
> > In the past (and currently) the Fedora compiler flags need some hairy
> > editing so they work with clang, eg:
> >
> >
> https://src.fedoraproject.org/rpms/american-fuzzy-lop/blob/master/f/ameri...
> >
> > (Actually this is not the latest iteration - latest clang 7 and gcc 9
> > and Fedora 30+ needs even more editing, but I didn't push it yet since
> > there are other issues with this package.)
> >
> > It would be nice if there was a way we could avoid this.
>  
>  So after rereading the proposal more carefully it seems as if the
>  proposal is to change the defaults in GCC so no flags would need to be
>  specified.  Would we consequently remove those flags from the command
>  line (which would solve my problem above)? 
The flags in my proposal will be removed from the command line during
the Fedora build process, since they are now default. Only people who
dont want to use these flags due to some reason will need to unset them
(I am assuming there are not a lot of packages like that)

Currently based on Jakub's suggestion i am also planning to remove to
fortify_source flag and keep others.

The plan is to start some where and each release work with glibc and
other teams so that we make more such security flags as default and also
work with packages which break due to inclusion of such flags.


-- 
Huzaifa Sidhpurwala / Red Hat Product Security Team

[Zbigniew Jędrzejewski-Szmek]

On Wed, Mar 20, 2019 at 09:52:02AM +0530, Huzaifa Sidhpurwala wrote:
>  On 3/15/19 9:49 PM, Richard W.M. Jones wrote:
>  > On Fri, Mar 15, 2019 at 04:15:58PM +0000, Richard W.M. Jones wrote:
>  >> On Mon, Mar 11, 2019 at 01:56:14PM -0400, Ben Cotton wrote:
>  >>> https://fedoraproject.org/wiki/Changes/HardenedCompiler
>  >>
>  >> I'm not opposing this, but is it possible we could do this without
>  >> breaking clang at the same time?
>  >>
>  >> In the past (and currently) the Fedora compiler flags need some hairy
>  >> editing so they work with clang, eg:
>  >>
>  >>
> https://src.fedoraproject.org/rpms/american-fuzzy-lop/blob/master/f/ameri...
>  >>
>  >> (Actually this is not the latest iteration - latest clang 7 and gcc 9
>  >> and Fedora 30+ needs even more editing, but I didn't push it yet since
>  >> there are other issues with this package.)
>  >>
>  >> It would be nice if there was a way we could avoid this.
>  > 
>  > So after rereading the proposal more carefully it seems as if the
>  > proposal is to change the defaults in GCC so no flags would need to be
>  > specified.  Would we consequently remove those flags from the command
>  > line (which would solve my problem above)?
>  
>  The flags in my proposal will be removed from the command line during
>  the Fedora build process, since they are now default. Only people who
>  dont want to use these flags due to some reason will need to unset them
>  (I am assuming there are not a lot of packages like that)
>  
>  Currently based on Jakub's suggestion i am also planning to remove to
>  fortify_source flag and keep others.
>  
>  The plan is to start some where and each release work with glibc and
>  other teams so that we make more such security flags as default and also
>  work with packages which break due to inclusion of such flags. 
           
^^^^^^^^

The original proposal had:
-Wformat -Wformat-security -fstack-protector-strong --param=ssp-buffer-size=4
-D_FORTIFY_SOURCE=2 -O

IIUC, the proposal now would be:
-Wformat -Wformat-security -fstack-protector-strong -O

As was stated multiple times in this thread, this change does not
apply to *packages* which follow the packaging guidelines, because various
flags are already set in %{optflags}. This change would only affect
mispackaged packages and locally compiled programs.

The effect of "-Wformat -Wformat-security" without -Werror is only more
warnings.
Unfortunately -Wformat will generate spurious warnings if the code is
not careful to give additional information to the compiler with
__attribute__((__format__(printf))) and friends. And even that sometimes
not enough, and explicit #pragma GCC diagnostic ignored "-Wformat-nonliteral"
is needed. So all in all, it is totally expected that code which is not
written with recent gcc in mind will generate spurious format warnings, even
if the code is completely OK. So turning this on will make builds more
noisy, and possibly break projects which use -Werror.

The effect of "-O" is not nice, because it's often useful to compile
without
any optimization whatsoever for debugging, so that the binary code
corresponds as closely as possible to the source. I'm sure people will
not know that '-O' has been made the default and will be confused.

"-fstack-protector-strong" is the only one that has a clearly
beneficial effect.

But then there's the overall counterargument from Jakub that we start
deviating from upstream defaults and some users will need to add counter-options
to go back to the compiler defaults. I feel like the possible benefits
from enabling "-fstack-protector-strong" are not big enough to justify
the change. For serious hardening, one would enable way more flags,
and just turning on one or two is enough for the downsides to kick in, but
not enough to have serious benefits.

I think that instead of playing with compiler defaults like this, it
would be more productive to investigate *packaged* software and to
check if it really follows the packaging guidelines and is hardened
effectively. With annobin enabled, it should be possible to do this
programatically.

Zbyszek

[Tomasz Kłoczko]

On Thu, 21 Mar 2019 at 09:07, Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl&gt; wrote:
[..]
>  The effect of "-Wformat -Wformat-security" without -Werror
> is only more warnings.
>  Unfortunately -Wformat will generate spurious warnings if the code is
>  not careful to give additional information to the compiler with
>  __attribute__((__format__(printf))) and friends. And even that sometimes
>  not enough, and explicit #pragma GCC diagnostic ignored "-Wformat-nonliteral"
>  is needed. So all in all, it is totally expected that code which is not
>  written with recent gcc in mind will generate spurious format warnings, even
>  if the code is completely OK. So turning this on will make builds more
>  noisy, and possibly break projects which use -Werror. 
Even gcc themselves "is not written with recent gcc in mind".

$ grep '\[\-W' gcc.log| awk -F\[ '{print $2}'|awk -F\] '{print
$1}'|sort | uniq -c | sort -nr| head -n 20
    485 -Wmissing-profile
    106 -Wformat-security
     81 -Wmaybe-uninitialized
     44 -Wimplicit-fallthrough=
     24 -Wunused-function
     20 -Wpointer-sign
     20 -Wimplicit-function-declaration
     19 -Wstringop-truncation
      8 -Wformat-truncation=
      8 -Wcast-qual
      7 -Wcast-function-type
      4 -Wcpp
      4 -Wbuiltin-declaration-mismatch
      3 -Wparentheses
      2 -Wunused-value
      2 -Wunused-parameter
      2 -Wmissing-prototypes
      2 -Wmisleading-indentation
      2 -Wint-to-pointer-cast
      2 -Wdiscarded-qualifiers

BTW: each Fedora package build should have as part of the build report
something like above.

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

[Stephen John Smoogen]

On Thu, 21 Mar 2019 at 07:23, Tomasz Kłoczko <kloczko.tomasz(a)gmail.com&gt; wrote:
>  On Thu, 21 Mar 2019 at 09:07, Zbigniew Jędrzejewski-Szmek
>  <zbyszek(a)in.waw.pl&gt; wrote:
>  [..]
>  > The effect of "-Wformat -Wformat-security" without -Werror is only more
> warnings.
>  > Unfortunately -Wformat will generate spurious warnings if the code is
>  > not careful to give additional information to the compiler with
>  > __attribute__((__format__(printf))) and friends. And even that sometimes
>  > not enough, and explicit #pragma GCC diagnostic ignored
> "-Wformat-nonliteral"
>  > is needed. So all in all, it is totally expected that code which is not
>  > written with recent gcc in mind will generate spurious format warnings, even
>  > if the code is completely OK. So turning this on will make builds more
>  > noisy, and possibly break projects which use -Werror.
>
>  Even gcc themselves "is not written with recent gcc in mind".
>
>  $ grep '\[\-W' gcc.log| awk -F\[ '{print $2}'|awk -F\] '{print
>  $1}'|sort | uniq -c | sort -nr| head -n 20
>      485 -Wmissing-profile
>      106 -Wformat-security
>       81 -Wmaybe-uninitialized
>       44 -Wimplicit-fallthrough=
>       24 -Wunused-function
>       20 -Wpointer-sign
>       20 -Wimplicit-function-declaration
>       19 -Wstringop-truncation
>        8 -Wformat-truncation=
>        8 -Wcast-qual
>        7 -Wcast-function-type
>        4 -Wcpp
>        4 -Wbuiltin-declaration-mismatch
>        3 -Wparentheses
>        2 -Wunused-value
>        2 -Wunused-parameter
>        2 -Wmissing-prototypes
>        2 -Wmisleading-indentation
>        2 -Wint-to-pointer-cast
>        2 -Wdiscarded-qualifiers
>
>  BTW: each Fedora package build should have as part of the build report
>  something like above.
>  
Could you explain why it should? I am not sure what those flags
actually mean and why it would tell me anything about a package build.
If upstream decides that libX needs to be compiled with
-Wmissing-prototypes but nothing else.. what is it to me?


>  kloczek
>  --
>  Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail] 


-- 
Stephen J Smoogen.

[Tomasz Kłoczko]

On Thu, 21 Mar 2019 at 11:37, Stephen John Smoogen <smooge(a)gmail.com&gt; wrote:
[..]
>  > Even gcc themselves "is not written with recent gcc in
> mind".
>  >
>  > $ grep '\[\-W' gcc.log| awk -F\[ '{print $2}'|awk -F\] '{print
>  > $1}'|sort | uniq -c | sort -nr| head -n 20
>  >     485 -Wmissing-profile
>  >     106 -Wformat-security
>  >      81 -Wmaybe-uninitialized
>  >      44 -Wimplicit-fallthrough=
>  >      24 -Wunused-function
>  >      20 -Wpointer-sign
>  >      20 -Wimplicit-function-declaration
>  >      19 -Wstringop-truncation
>  >       8 -Wformat-truncation=
>  >       8 -Wcast-qual
>  >       7 -Wcast-function-type
>  >       4 -Wcpp
>  >       4 -Wbuiltin-declaration-mismatch
>  >       3 -Wparentheses
>  >       2 -Wunused-value
>  >       2 -Wunused-parameter
>  >       2 -Wmissing-prototypes
>  >       2 -Wmisleading-indentation
>  >       2 -Wint-to-pointer-cast
>  >       2 -Wdiscarded-qualifiers
>  >
>  > BTW: each Fedora package build should have as part of the build report
>  > something like above.
>  >
>
>  Could you explain why it should? I am not sure what those flags
>  actually mean and why it would tell me anything about a package build.
>  If upstream decides that libX needs to be compiled with
>  -Wmissing-prototypes but nothing else.. what is it to me? 
That list is not in order of importance but how often some warning
happened, and I think that you are fully aware that on that list are
things far more important than missing prototype.

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

[Stephen John Smoogen]

On Thu, 21 Mar 2019 at 07:46, Tomasz Kłoczko <kloczko.tomasz(a)gmail.com&gt; wrote:
>  On Thu, 21 Mar 2019 at 11:37, Stephen John Smoogen <smooge(a)gmail.com&gt; wrote:
>  [..]
>  > > Even gcc themselves "is not written with recent gcc in mind".
>  > >
>  > > $ grep '\[\-W' gcc.log| awk -F\[ '{print $2}'|awk -F\]
> '{print
>  > > $1}'|sort | uniq -c | sort -nr| head -n 20
>  > >     485 -Wmissing-profile
>  > >     106 -Wformat-security
>  > >      81 -Wmaybe-uninitialized
>  > >      44 -Wimplicit-fallthrough=
>  > >      24 -Wunused-function
>  > >      20 -Wpointer-sign
>  > >      20 -Wimplicit-function-declaration
>  > >      19 -Wstringop-truncation
>  > >       8 -Wformat-truncation=
>  > >       8 -Wcast-qual
>  > >       7 -Wcast-function-type
>  > >       4 -Wcpp
>  > >       4 -Wbuiltin-declaration-mismatch
>  > >       3 -Wparentheses
>  > >       2 -Wunused-value
>  > >       2 -Wunused-parameter
>  > >       2 -Wmissing-prototypes
>  > >       2 -Wmisleading-indentation
>  > >       2 -Wint-to-pointer-cast
>  > >       2 -Wdiscarded-qualifiers
>  > >
>  > > BTW: each Fedora package build should have as part of the build report
>  > > something like above.
>  > >
>  >
>  > Could you explain why it should? I am not sure what those flags
>  > actually mean and why it would tell me anything about a package build.
>  > If upstream decides that libX needs to be compiled with
>  > -Wmissing-prototypes but nothing else.. what is it to me?
>
>  That list is not in order of importance but how often some warning
>  happened, and I think that you are fully aware that on that list are
>  things far more important than missing prototype.
>  

When people see lists like this they are going to assume it is order
of importance because if X is used N many more times, it must be much
more important than Y. Most packagers are not because most packages
are just things they do so they can get what they really want done.
That may be a sad state to some, but the majority of packages in
Fedora are the commons on which the cattle (packages people do things
with) graze on. If I am a perl/python/erlang/nodejs/ghc/R packager and
I get a report that something down in my stack has 2
-Wmisleading-indentations and 485 -Wmissing-profiles.. I am going to
assume that the upstream wanted it that way since all I did was copy
this spec file from another one and use the defaults. If I get
something with the tool I actually am familiar with
(perl/python/erlang/etc) I might be better atuned to knowing that flag
X or dependency Y is important.. but in the end I might really just
want emacs-nethack.el to be a package and I won't have a clue if any
of the above was important. So we need to make sure it is clear when
we add something to a report why it is important.


>  kloczek
>  --
>  Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail] 


-- 
Stephen J Smoogen.

[Stephen John Smoogen]

On Thu, 21 Mar 2019 at 07:53, Stephen John Smoogen <smooge(a)gmail.com&gt; wrote:
>  
>  
> When people see lists like this they are going to assume
it is order
> of importance because if X is used N many more times, it must be much
> more important than Y. Most packagers are not because most packages

wow.. I need a better editor (and probably author).

Most packagers are not aware of the importance of flags because most
packages are just things they put up with for the thing they really
want to get done.

[Which is an ugly run-on sentence and probably in the passive voice
and a bunch of other things.]


> are just things they do so they can get what they really want done.
> That may be a sad state to some, but the majority of packages in
> Fedora are the commons on which the cattle (packages people do things
> with) graze on. If I am a perl/python/erlang/nodejs/ghc/R packager and
> I get a report that something down in my stack has 2
> -Wmisleading-indentations and 485 -Wmissing-profiles.. I am going to
> assume that the upstream wanted it that way since all I did was copy
> this spec file from another one and use the defaults. If I get
> something with the tool I actually am familiar with
> (perl/python/erlang/etc) I might be better atuned to knowing that flag
> X or dependency Y is important.. but in the end I might really just
> want emacs-nethack.el to be a package and I won't have a clue if any
> of the above was important. So we need to make sure it is clear when
> we add something to a report why it is important.
>  
>  
> > kloczek
> > --
> > Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
> > _______________________________________________
> > devel mailing list -- devel(a)lists.fedoraproject.org
> > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]
>  
>  
>  
> --
> Stephen J Smoogen.



-- 
Stephen J Smoogen.

[Jakub Jelinek]

On Thu, Mar 21, 2019 at 11:43:49AM +0000, Tomasz Kłoczko wrote:
>  On Thu, 21 Mar 2019 at 11:37, Stephen John Smoogen
> <smooge(a)gmail.com&gt; wrote:
>  [..]
>  > > Even gcc themselves "is not written with recent gcc in mind".
>  > >
>  > > $ grep '\[\-W' gcc.log| awk -F\[ '{print $2}'|awk -F\]
> '{print
>  > > $1}'|sort | uniq -c | sort -nr| head -n 20
>  > >     485 -Wmissing-profile
>  > >     106 -Wformat-security
>  > >      81 -Wmaybe-uninitialized
>  > >      44 -Wimplicit-fallthrough=
>  > >      24 -Wunused-function
>  > >      20 -Wpointer-sign
>  > >      20 -Wimplicit-function-declaration
>  > >      19 -Wstringop-truncation
>  > >       8 -Wformat-truncation=
>  > >       8 -Wcast-qual
>  > >       7 -Wcast-function-type
>  > >       4 -Wcpp
>  > >       4 -Wbuiltin-declaration-mismatch
>  > >       3 -Wparentheses
>  > >       2 -Wunused-value
>  > >       2 -Wunused-parameter
>  > >       2 -Wmissing-prototypes
>  > >       2 -Wmisleading-indentation
>  > >       2 -Wint-to-pointer-cast
>  > >       2 -Wdiscarded-qualifiers
>  > >
>  > > BTW: each Fedora package build should have as part of the build report
>  > > something like above.
>  > >
>  >
>  > Could you explain why it should? I am not sure what those flags
>  > actually mean and why it would tell me anything about a package build.
>  > If upstream decides that libX needs to be compiled with
>  > -Wmissing-prototypes but nothing else.. what is it to me?
>  
>  That list is not in order of importance but how often some warning
>  happened, and I think that you are fully aware that on that list are
>  things far more important than missing prototype. 
Like what exactly?  E.g. all the -Wformat-security warnings are about cases
where the format strings are constructed shortly before they are passed to
the format attribute functions, either unmodified or through gettext.
In no case any of that is from user provided strings.
I've tried several times to change that but it turned to be too ugly for no
real benefit.
-Wmissing-profile is just a matter of not 100% code coverage during
bootstrap when using FDO, not a bug.  What else?

GCC carefully uses -Werror with some -Wno- options in some parts of the
bootstrap on some parts of the codebases and not others for various reasons.

	Jakub

[Tomasz Kłoczko]

On Thu, 21 Mar 2019 at 12:07, Jakub Jelinek <jakub(a)redhat.com&gt; wrote:
[..]
>  Like what exactly?  E.g. all the -Wformat-security warnings are about
> cases
>  where the format strings are constructed shortly before they are passed to
>  the format attribute functions, either unmodified or through gettext. 
KISS principle ..
Maybe it is time to remove gettext support from gcc/binutils it is
really so hard to maintain?
Who needs this?
I'm not trying to give any answers on those questions but people like
you who are directly maintaining gcc should be able to balance between
some priorities like what is most important and what is only
decoration on the Christmas Tree.

People are writing the code with i18n support which compiles without
single warning. This is not a rocket science.
I can understand that it may be difficult for some people but not
everything needs to be done by single person. Isn't it?

Just try to execute "update-po" target in gcc/ and you will see that
only by this single command is possible to reduce gcc/po/ directory
content by ~2MB (which is about 5% and that is only in one directory
with translations) which says that for quite long time no one cares
that most of the translations are not up-to-date.
Maybe telling other people that if all compile time warnings related
to gettex will be not sorted out before final gcc 9 i18n support will
be dropped? Let's see how many people cares about that support (?).
Chess players sometimes says that in this game more dangerous than
chess mat is knowing that in next move you will be under pressure of
that state.

Every source code maintainer doesn't need to do everything and I'm not
asking you why all those problems are or sort out every even minor
issue.
At least such person needs take care of some problem management by
identify the problem and making aware of them people around, and
assigning priority to each point of the list problems.
It is really good to resent some statistics on each release.
Statistics about up-to-date i18n or number of stats about warnings are
only two I'm sure that you
Am I could be right or not?


Going back to main subject about default compile options. I have
question for you Jakub :)
Quote from redhat-rpm-config package buildflags.md:

* `-fexceptions`: Provide exception unwinding support for C programs.
  See the [`-fexceptions` option in the GCC
  manual](https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html#index-fe...
  and the [`cleanup` variable
 
attribute](https://gcc.gnu.org/onlinedocs/gcc/Common-Variable-Attributes.....
  This also hardens cancellation handling in C programs because
  it is not required to use an on-stack jump buffer to install
  a cancellation handler with `pthread_cleanup_push`.  It also makes
  it possible to unwind the stack (using C++ `throw` or Rust panics)
  from C callback functions if a C library supports non-local exits
  from them (e.g., via `longjmp`).

Is it still correct? If it is why that kind of issues are solved that
way? Someone maybe remember in which one project this caused some
issue?
Building all code with exceptions when many people spent many hours to
have code which does not uses exceptions is kind waste .. and still
compile many programs with exceptions only causes that executable code
is puffier. As you know even gcc is not compliant with above :)

What about use -fno-rtti if it is possible to use it?

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

[Jakub Jelinek]

On Thu, Mar 21, 2019 at 03:19:48PM +0000, Tomasz Kłoczko wrote:
>  On Thu, 21 Mar 2019 at 12:07, Jakub Jelinek <jakub(a)redhat.com&gt;
> wrote:
>  [..]
>  > Like what exactly?  E.g. all the -Wformat-security warnings are about cases
>  > where the format strings are constructed shortly before they are passed to
>  > the format attribute functions, either unmodified or through gettext.
>  
>  Maybe it is time to remove gettext support from gcc/binutils it is
>  really so hard to maintain? 
i18n and l10n is an important goal of the upstream project.
It is not hard to maintain and I didn't say above the problem is caused
by gettext, if I have error ("jump to label %qD", label); then there
will not be any -Wformat-security warnings about it, the attributes
on gettext functions take care of everything.
The thing is that no -Wformat-security warnings on gcc source is not a
goal of the upstream project, because none of the cases actually take
user-controllable input, but there are many cases where some code sets
some const char * variable/field/whatever to say
G_("whatever %<something%>") and similar, those can't be printed with
%s in whatever code 5 frames up in the stack trace finally emits
in diagnostics, because we do want % processing in those strings.

If we in Fedora made -Werror=format-security warning default, then
one couldn't out of box build gcc nor many other projects.  Not to mention
that it is extremely nasty, because if e.g. some project uses -Wno-format
for selected source(s), then if -Werror=format-security is the default
(or just present in flags), then it causes errors as well.

>  Just try to execute "update-po" target in gcc/ and you will
> see that
>  only by this single command is possible to reduce gcc/po/ directory
>  content by ~2MB (which is about 5% and that is only in one directory
>  with translations) which says that for quite long time no one cares
>  that most of the translations are not up-to-date. 
The translations are provided to gcc by a different project -
translationproject.org - some translations are in very good shape, others
are lacking, but that doesn't mean translations aren't useful.
gcc has around 13000 translatable messages and quite a lot of that changes
from release to release.

>  Going back to main subject about default compile options. I have
>  question for you Jakub :)
>  Quote from redhat-rpm-config package buildflags.md:
>  
>  * `-fexceptions`: Provide exception unwinding support for C programs.
>    See the [`-fexceptions` option in the GCC
>    manual](https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html#index-fe...
>    and the [`cleanup` variable
>   
> attribute](https://gcc.gnu.org/onlinedocs/gcc/Common-Variable-Attributes.....
>    This also hardens cancellation handling in C programs because
>    it is not required to use an on-stack jump buffer to install
>    a cancellation handler with `pthread_cleanup_push`.  It also makes
>    it possible to unwind the stack (using C++ `throw` or Rust panics)
>    from C callback functions if a C library supports non-local exits
>    from them (e.g., via `longjmp`).
>  
>  Is it still correct? If it is why that kind of issues are solved that
>  way? Someone maybe remember in which one project this caused some
>  issue? 
This is to make sure pthread_cancel works properly, as the comment says. 
-fasynchronous-unwind-tables is the default (upstream default; on most
targets Fedora cares about, otherwise added by %optflags anyway) and that is
desirable to be able to debug properly, as well as for pthread_cancel,
backtrace etc.  -fexceptions for C code doesn't really change pretty much
anything for most of the code, all it changes is how __attribute__((cleanup
(...))) behaves and tells glibc headers to use that attribute for
pthread_cleanup*.

So, I don't really understand your objection to -fexceptions for C code,
it just changes cases where it is actually highly desirable that it works
that way for pthread_cancel, C++ exception compatibility etc.
And for C++ it is the default.

Of course if you have some package that you know will not use exceptions,
nor pthread_cancel etc., you can build with -fno-exceptions -fno-rtti,
as I said for C it will not really make a difference, for C++ surely will.
A more difficult question is if you have a library, because then it is not
just about whether the library itself uses exceptions/pthread_cancel etc.,
but also whether any users of that library use those.

	Jakub

[Kamil Dudka]

On Friday, March 22, 2019 10:41:23 AM CET Jakub Jelinek wrote:
>  If we in Fedora made -Werror=format-security warning default, then
>  one couldn't out of box build gcc nor many other projects.  Not to mention
>  that it is extremely nasty, because if e.g. some project uses -Wno-format
>  for selected source(s), then if -Werror=format-security is the default
>  (or just present in flags), then it causes errors as well. 
Enforcing -Werror=format-security in Fedora build system was a mistake.
It made developers introduce new bugs while trying to fix nonsense build 
failures, as in this example:

https://bugzilla.redhat.com/1025257#c5

Kamil

[Vitaly Zaitsev]

Hello, Kamil Dudka.

Fri, 22 Mar 2019 12:03:26 +0100 you wrote:

>  Enforcing -Werror=format-security in Fedora build system was a
> mistake. 
No. Enforcing -Werror=format-security is good choice. This helped
maintainers to fix lots of potential security vulnerabilities in packages.

--
Sincerely,
 Vitaly Zaitsev (vitaly(a)easycoding.org)

[Kamil Dudka]

On Friday, March 22, 2019 12:25:28 PM CET Vitaly Zaitsev wrote:
>  Hello, Kamil Dudka.
>  
>  Fri, 22 Mar 2019 12:03:26 +0100 you wrote:
>  
>  
>  > Enforcing -Werror=format-security in Fedora build system was a mistake.
>  
>  
>  No. Enforcing -Werror=format-security is good choice. This helped
>  maintainers to fix lots of potential security vulnerabilities in packages. 
There are more effective ways to capture and process compiler warnings.

Using -Werror in production builds can in fact be counterproductive when 
someone is trying to release a security update for a real security issue
and has to deal with unexpected build failures.

Another problem is that -Werror aborts the build prematurely.  When building 
in parallel, it may be even non-deterministic how many errors are reported 
before the build stops.  So people have to apply fixes (or workarounds) in 
iterations, which is not fun with packages like libreoffice.

Needless to say that build.log does not contain any machine-readable 
information about the warnings detected during the build, because compiler 
diagnostic may be suppressed by the build system, diagnostic messages may
be interleaved with each other when building in parallel, etc.

We have better tooling to reliably capture compiler warnings of your choice,
fully automatically, and in a machine readable format:

$ csmock ${pkg}.src.rpm -t gcc

You can also transparently inject compiler flags without changing anything
in the buildroot (and unnecessarily breaking production builds of others):

$ csmock ${pkg}.src.rpm -t gcc --gcc-add-flag=...

Kamil

[Tomasz Kłoczko]

On Fri, 22 Mar 2019 at 09:47, Jakub Jelinek <jakub(a)redhat.com&gt; wrote:
[..]

>  > Just try to execute "update-po" target in gcc/ and you
> will see that
>  > only by this single command is possible to reduce gcc/po/ directory
>  > content by ~2MB (which is about 5% and that is only in one directory
>  > with translations) which says that for quite long time no one cares
>  > that most of the translations are not up-to-date.
>
>  The translations are provided to gcc by a different project -
>  translationproject.org - some translations are in very good shape, others
>  are lacking, but that doesn't mean translations aren't useful.
>  gcc has around 13000 translatable messages and quite a lot of that changes
>  from release to release.
>  
Sorry to say that but you just told that that you don't know enough about
maintaining translation :P
According to my personal observations this is the case in probably +90% of
all open source projects so you are not the only one :)

People maintaining translations are not looking for the C or other language
code changes. They don't know that something has changed in set of strings
until new .po files will be generated.
Really .. they are looking only on .po files changes!!!
As there is no changes in those .po files -> translators are thinking that
none of translations needs to be updated. That how it works. In other words
you have only impression that some translations are in good shape.
Here is the proof:

[tkloczko@barrel gcc-9.0.1-20190312]$ (make -C obj-x86_64-redhat-linux/gcc
update-po 2>&1 >/dev/null; update-po; diff -u <(cd gcc/po; ls -l *po|awk
'{print $5 " " $9 "x"}') <(cd
obj-x86_64-redhat-linux/gcc/po; ls -l
*pox|awk '{print $5 " " $9}'))
--- /dev/fd/63 2019-03-22 12:15:31.161415161 +0000
+++ /dev/fd/62 2019-03-22 12:15:31.165424556 +0000
@@ -1,19 +1,19 @@
-1819975 be.pox
-2619276 da.pox
-2532199 de.pox
-1983130 el.pox
-3206943 es.pox
-2245685 fi.pox
-2650499 fr.pox
-1641431 hr.pox
-2705049 id.pox
-2369903 ja.pox
-2005289 nl.pox
-3184976 ru.pox
-2938716 sr.pox
-2426131 sv.pox
-2561562 tr.pox
-1688950 uk.pox
-2059085 vi.pox
-2406450 zh_CN.pox
-2466373 zh_TW.pox
+1791091 be.pox
+2370766 da.pox
+2522956 de.pox
+1987854 el.pox
+3072641 es.pox
+2260403 fi.pox
+2690109 fr.pox
+1643615 hr.pox
+2338720 id.pox
+2193995 ja.pox
+1995932 nl.pox
+3176529 ru.pox
+2592373 sr.pox
+2410896 sv.pox
+2210663 tr.pox
+1700251 uk.pox
+2017485 vi.pox
+2258622 zh_CN.pox
+2364447 zh_TW.pox

As you see 100% of .po file has been changed/updated!!!
Above output only rises other question: why the <censored> gcc is not using
standard gettext am/ac support in build trees? but this is is less
important :)

*What generally above tells?*

That after every change in the code in the strings wrapped by _() or
wrapped by other macros should cause instant  "make -C po update-po" or
"ninja -C . <package>-update-po" to let know those guys that something
has
been changes in translations and give them chance to they work properly.
All this mess with translations is result of *assumptions* made by many
code maintainers that it-work-somehow-and-I-don-need-to-know-how(tm). No ..
everything has its own facts context and as long as you are not aware of
those facts you only yet another small "cut" in "death by thousands
cuts"
result.

That is really shame that only few maintainer are fully aware of that and
they are regularly after each change in the code  testing update .po files
update result and even instantly committing those .po files changes with
with actual code change.
Some maintainers at least remember to do that final .po files update as one
of the last steps before make/tag new release.
Few (those who are able to plan better) are doing that in week or two in
advance before the make release to send translators signal that something
needs to be updated, and to have as much as possible updated all
translation when finally release will be made.
As automake/autoconf/libtool tooling IMO is generally death (and GNU
maintainers are even acting like ostriches not replying on any calls to
make new automake/autoconf/libtool releases or pass control of the repos) I
think that maybe at least meson could take care of keeping up to date
translations by taking care of warning that .po files are not updated on
executing "dist" target .. because all those updates should be done
manually but by build framework/tooling automatically.

However as tolling still lacks of some features as I told in mean time
*MAJORITY of ALL* OSS projects maintainers don't know about above details
and they are thinking (like you) that every change in the code
instantly/magically propagates to all translators ..
No .. that is not how it should works.

So please Jakub make some readjustments in your procedures and please keep
all possible .po files up-to-date because without that all translations
will be lagging BY DEFINITION .. ALWAYS.

And as I'me telling about general methodology of maintaining .po files it
is yet another "small" detail related to encoding of .po files. I've just
checked that and gcc is affected by other issue as well.

[tkloczko@barrel gcc-9.0.1-20190312]$ find . -name \*.po | xargs grep
Content-Type | grep -v UTF-8
./libstdc++-v3/po/de.po:"Content-Type: text/plain; charset=ISO-8859-1\n"
./libstdc++-v3/po/fr.po:"Content-Type: text/plain; charset=ISO-8859-1\n"
./libcpp/po/ca.po:"Content-Type: text/plain; charset=ISO-8859-1\n"
./libcpp/po/be.po:"Content-Type: text/plain; charset=utf-8\n"
./libcpp/po/eo.po:"Content-Type: text/plain; charset=utf-8\n"
./libcpp/po/id.po:"Content-Type: text/plain; charset=ISO-8859-1\n"
./libcpp/po/zh_CN.po:"Content-Type: text/plain; charset=utf-8\n"
./libcpp/po/el.po:"Content-Type: text/plain; charset=iso-8859-7\n"
./gcc/po/be.po:"Content-Type: text/plain; charset=utf-8\n"
./gcc/po/id.po:"Content-Type: text/plain; charset=ISO-8859-1\n"
./gcc/po/el.po:"Content-Type: text/plain; charset=utf-8\n"

Why above is wrong? Simple .. gettext tools on generate binary .mo files
are preserving original encoding used in .po files. What it means? It means
that on displaying those translated messages in case of using UTF-8 based
locale settings (which is now *common*) glibc needs to load additional
translation table to convert those messages to UTF-8 before they will be
displayed.

So please do "for i in libstdc++-v3 libcpp gcc; do make -C $i/po update-po"
then convert all above files to UTF-8 using iconv -> change in all those
.po file lines to "Content-Type: text/plain; charset=UTF-8\n" and .. THAN
commit all those updates.

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

[Jakub Jelinek]

On Fri, Mar 22, 2019 at 01:03:19PM +0000, Tomasz Kłoczko wrote:
>  > > Just try to execute "update-po" target in gcc/
> and you will see that
>  > > only by this single command is possible to reduce gcc/po/ directory
>  > > content by ~2MB (which is about 5% and that is only in one directory
>  > > with translations) which says that for quite long time no one cares
>  > > that most of the translations are not up-to-date.
>  >
>  > The translations are provided to gcc by a different project -
>  > translationproject.org - some translations are in very good shape, others
>  > are lacking, but that doesn't mean translations aren't useful.
>  > gcc has around 13000 translatable messages and quite a lot of that changes
>  > from release to release.
>  >
>  
>  Sorry to say that but you just told that that you don't know enough about
>  maintaining translation :P 
Sorry but you are wrong, it is unclear from what you are judging this.  We as
the GCC project have procedures to maintain translations, new gcc.pot is not
regenerated daily but usually twice before a major release (usually at the
start of stage4 development phase and shortly before release) and on major/minor
releases, as you can see in https://translationproject.org/domain/gcc.html
It isn't worth bothering our translators with new changes every day.
The latest gcc.pot regeneration/upload was February 2nd, so there are almost
two months of further changes.  While I'm not a translation maintainer, I've
spent quite a lot of time including this year working with the translators
to improve the diagnostics, so that it is better translatable or easier to
handle for the translators, see e.g. http://gcc.gnu.org/PR40883 meta-bug
and the recent changes.
In our project rules the *.po files are maintained by the
translationproject.org, we never make changes to those, just regenerate the
pot and upload it and when updates are available from translators merge
those back.  I'm not going to violate this stuff with a GCC downstream
Fedora hat.

	Jakub

[Tomasz Kłoczko]

On Fri, 22 Mar 2019 at 13:18, Jakub Jelinek <jakub(a)redhat.com&gt; wrote:
[..]

>  > Sorry to say that but you just told that that you don't know
> enough about
>  > maintaining translation :P
>
>  Sorry but you are wrong, it is unclear from what you are judging this.  We
>  as
>  the GCC project have procedures to maintain translations, new gcc.pot is
>  not
>  regenerated daily but usually twice before a major release (usually at the
>  start of stage4 development phase and shortly before release) and on
>  major/minor
>  releases, as you can see in https://translationproject.org/domain/gcc.html
>  It isn't worth bothering our translators with new changes every day.
>  The latest gcc.pot regeneration/upload was February 2nd, so there are
>  almost
>  two months of further changes.  While I'm not a translation maintainer,
>  I've
>  spent quite a lot of time including this year working with the translators
>  to improve the diagnostics, so that it is better translatable or easier to
>  handle for the translators, see e.g. http://gcc.gnu.org/PR40883 meta-bug
>  and the recent changes.
>  In our project rules the *.po files are maintained by the
>  translationproject.org, we never make changes to those, just regenerate
>  the
>  pot and upload it and when updates are available from translators merge
>  those back.  I'm not going to violate this stuff with a GCC downstream
>  Fedora hat.
>  
1) I don't see even single fact in what you just wrote which
contradicts/undermines what I told.
You just wrote that gcc has its "own procedures" and those procedures are
not the same as what I've described. In other words using some analogy you
did not show that I presented apples but those apples are unhealthy and
shpuld not be eaten. You said "Look Tomasz you are wrong because I have
plumbs". You just described some facts on axis oriented kind of
orthogonality .. only. I'm fully aware of that and this is why I wrote
about readjusting some gcc procedures.
You simple ignored what I wrote and that's it.

2) It looks like even people from translationproject.org are somehow wrong
or they are so polite they dpn't want to hurt gcc maintainers fillings
because in the well-maintained project with i18n resources no one keeps
.pot files in VCS!!!!
If you don't believe me go to for example https://gitlab.gnome.org/GNOME/
and take some random project with po/ directory and try to find in that
directory .pot file. Even if you would be able to find such file in po/ it
will be nothing more than the exception. Do you see this now that ggc is
wasting time on maintaining .pot files content which no one is asking for?

3) gcc has a long history, and already many things are behind how some
technologies/tooling should be used. I would accept that gettext support is
not used as it should be used as long as .po files would be up-to-date but
as I proved they are not .. You may only ask "why?" but that is all what
you really should do.
I wrote that gettext support in gcc is implemented in kind of DYI way. Here
is the proof:

[tkloczko@barrel gcc-9.0.1-20190312]$ find . -name configure.ac |xargs grep
GETTEXT
./libcpp/configure.ac:ZW_GNU_GETTEXT_SISTER_DIR
./intl/configure.ac:AM_GNU_GETTEXT_VERSION(0.12.1)
./intl/configure.ac:AM_GNU_GETTEXT([], [need-ngettext])
./gcc/configure.ac:ZW_GNU_GETTEXT_SISTER_DIR

Only in intl/configure.ac are used AM_GNU_GETTEXT and
AM_GNU_GETTEXT_VERSION macros however that directory is just copy of the
gettext files. In gcc/ and cpp/ is used kind of own version aclocal macros,
and in libstdc++/ gettext support is done completely DIY method.
Why? Why someone is wasting time on reinventing the wheel?
You can just remove from VCS few m4 and even some Makefile.* or Makefile
files content and just enjoy use AM_GNU_GETTEXT and AM_GNU_GETTEXT_VERSION
macros.
If this will be done that way whole gcc build framework would be one huge
step closer to have whole build framework initialised after VCS checkout by
execution of "autoreconf -fiv" in gcc root directory.

Whoever as translator would be trying to update gcc .po file will be stuck
with a simple thing that there is no update-po make target in any of the
gcc po/ directory. That is probably the main factor why all .po files are
not up to date. Whoever would be trying to do regular maintenance of .po
files will be forced to decipher first DIY gcc own .po files updated
procedures (not seen in any other project).

You may still try to convince me that I'm wrong and you are right, but
whatever you would be able to add cannot not be able to change basic fact
that translations in gcc/binutins are not well maintained (even one file).
In other words, you will be trying talking to convince those files .. not
me.

People who like helping translate text resources in some projects are
familiar with some exact procedures.
gcc is not using those well-known i18n maintenance procedures.
This is really end of that story. Have a good weekend :)

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

[Stephan Bergmann]

On 21/03/2019 09:59, Zbigniew Jędrzejewski-Szmek wrote:
>  "-fstack-protector-strong" is the only one that has a
> clearly
>  beneficial effect.
>  
>  But then there's the overall counterargument from Jakub that we start
>  deviating from upstream defaults and some users will need to add counter-options
>  to go back to the compiler defaults. I feel like the possible benefits
>  from enabling "-fstack-protector-strong" are not big enough to justify
>  the change. For serious hardening, one would enable way more flags,
>  and just turning on one or two is enough for the downsides to kick in, but
>  not enough to have serious benefits. 
...and if any of the suggested changes to default options are deemed to 
be of value to users of Fedora, wouldn't they also be of value to users 
of upstream GCC, and should be implemented there?

(I share the sentiment that deviating defaults in distros are a pain for 
users.  It already bites me often enough when a distro unhelpfully 
sneaks in ccache behind my back, let alone something like adding -O.)