https://bugzilla.redhat.com/show_bug.cgi?id=2043092
This is not about the feature itself but about the way it has been implemented.
During builds LDFLAGS is modified so it contains a build path, something like:
-Wl,-dT,/builddir/build/BUILD/.package_note-rubygem-nio4r-2.5.2-6.fc36.x86_64.ld
Many builds embed/store LDFLAGS somewhere. For OCaml it gets embedded in the ocamlopt binary, and in *.cma files. Similar sort of thing happening in Ruby, Perl, Haskell, Python, ...
But the problem is more general than this too. It also turns up in some *.pc (pkgconf) files.
I think this change should be reverted until a cleaner way can be found to implement it.
Rich.
(Sent too soon)
On Thu, Jan 27, 2022 at 09:05:32AM +0000, Richard W.M. Jones wrote:
I think this change should be reverted until a cleaner way can be found to implement it.
And do we have a way to scan all binary packages looking for the build path so we can find out which ones will need to be rebuilt once the feature has been disabled?
Rich.
On Thu, 27 Jan 2022 at 10:13, Richard W.M. Jones rjones@redhat.com wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=2043092
This is not about the feature itself but about the way it has been implemented.
During builds LDFLAGS is modified so it contains a build path, something like:
-Wl,-dT,/builddir/build/BUILD/.package_note-rubygem-nio4r-2.5.2-6.fc36.x86_64.ld
Many builds embed/store LDFLAGS somewhere. For OCaml it gets embedded in the ocamlopt binary, and in *.cma files. Similar sort of thing happening in Ruby, Perl, Haskell, Python, ...
Also R, which breaks R packages. Tom disabled this feature in the spec for the time being.
But the problem is more general than this too. It also turns up in some *.pc (pkgconf) files.
I think this change should be reverted until a cleaner way can be found to implement it.
I agree. Switching to %extension_*flags has been proposed, as Python does, but there was a system-wide change for that [1], and now we are asked to switch everything to using this without pondering the consequences because this change overlooked its consequences.
[1] https://fedoraproject.org/wiki/Changes/Python_Extension_Flags
On Thu, Jan 27, 2022 at 09:05:32AM +0000, Richard W.M. Jones wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=2043092
This is not about the feature itself but about the way it has been implemented.
During builds LDFLAGS is modified so it contains a build path, something like:
-Wl,-dT,/builddir/build/BUILD/.package_note-rubygem-nio4r-2.5.2-6.fc36.x86_64.ld
Many builds embed/store LDFLAGS somewhere. For OCaml it gets embedded in the ocamlopt binary, and in *.cma files. Similar sort of thing happening in Ruby, Perl, Haskell, Python, ...
This isn't actually true, AFAIK. Python at least exports flags to extensions, and it uses %extension_ldflags for this purpose. It is been well known that exporting the flags that the interpreter was compiled with itself causes problems, so %extension_cflags/%extension_cxxflags/%extension_ldflags were introduced for this purpose back in 2018 (rhbz#1543394).
So the best solution would be to update ocamlopt to use %extension_*flags. (If it cannot be fixed like this, opting out of the package note for ocaml is also an option, but it's not a nice solution. Using %extension_*flags is good for other reasons too.)
But the problem is more general than this too. It also turns up in some *.pc (pkgconf) files.
That's a bug too.
I think this change should be reverted until a cleaner way can be found to implement it.
I'm all for making better, but please make concrete proposals. Just saying "revert, revert" because some corner case is not satisfied, when a simple opt-out exists, is IMO not useful.
Zbyszek
On Thu, Jan 27, 2022 at 09:46:59AM +0000, Zbigniew Jędrzejewski-Szmek wrote:
On Thu, Jan 27, 2022 at 09:05:32AM +0000, Richard W.M. Jones wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=2043092
This is not about the feature itself but about the way it has been implemented.
During builds LDFLAGS is modified so it contains a build path, something like:
-Wl,-dT,/builddir/build/BUILD/.package_note-rubygem-nio4r-2.5.2-6.fc36.x86_64.ld
Many builds embed/store LDFLAGS somewhere. For OCaml it gets embedded in the ocamlopt binary, and in *.cma files. Similar sort of thing happening in Ruby, Perl, Haskell, Python, ...
This isn't actually true, AFAIK. Python at least exports flags to extensions, and it uses %extension_ldflags for this purpose. It is been well known that exporting the flags that the interpreter was compiled with itself causes problems, so %extension_cflags/%extension_cxxflags/%extension_ldflags were introduced for this purpose back in 2018 (rhbz#1543394).
So the best solution would be to update ocamlopt to use %extension_*flags. (If it cannot be fixed like this, opting out of the package note for ocaml is also an option, but it's not a nice solution. Using %extension_*flags is good for other reasons too.)
But the problem is more general than this too. It also turns up in some *.pc (pkgconf) files.
That's a bug too.
I think this change should be reverted until a cleaner way can be found to implement it.
I'm all for making better, but please make concrete proposals. Just saying "revert, revert" because some corner case is not satisfied, when a simple opt-out exists, is IMO not useful.
The "simple opt-out" involves me modifying and rebuilding every single OCaml package.
Rich.
On Thu, 27 Jan 2022 at 10:58, Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl wrote:
I think this change should be reverted until a cleaner way can be found to implement it.
I'm all for making better, but please make concrete proposals.
Here's a concrete proposal:
- Copy %build_*flags to another private macro, let's say %_build_*flags. - Add there the stuff that is not valid once the current build finishes (e.g., the flag that this change adds). - Use that in %set_build_flags and leave %build_*flags as it was.
Another concrete proposal:
Is there a way to scan all binary packages in Fedora to get either a count or list of those packages that contain strings like "-Wl,-dT,/builddir/". This will give us numbers on how bad the problem is. I suspect it's quite widespread.
Rich.
On Thu, 27 Jan 2022 at 11:34, Richard W.M. Jones rjones@redhat.com wrote:
Another concrete proposal:
Is there a way to scan all binary packages in Fedora to get either a count or list of those packages that contain strings like "-Wl,-dT,/builddir/". This will give us numbers on how bad the problem is. I suspect it's quite widespread.
Rich.
And another concrete proposal:
This is not reflected *at all* in the change proposal. Modify the change proposal to acknowledge this issue and discuss it. If changes are needed with respect to how things are done in other packages, then required modifications should be in the scope of the proposal owners.
On Thu, Jan 27, 2022 at 11:51:54AM +0100, Iñaki Ucar wrote:
On Thu, 27 Jan 2022 at 11:34, Richard W.M. Jones rjones@redhat.com wrote:
Another concrete proposal:
Is there a way to scan all binary packages in Fedora to get either a count or list of those packages that contain strings like "-Wl,-dT,/builddir/". This will give us numbers on how bad the problem is. I suspect it's quite widespread.
Rich.
And another concrete proposal:
This is not reflected *at all* in the change proposal. Modify the change proposal to acknowledge this issue and discuss it. If changes are needed with respect to how things are done in other packages, then required modifications should be in the scope of the proposal owners.
This all came up in the mass rebuild, so it wasn't known at the time the proposal was written. I'll add more information to the Change page.
Zbyszek
On Thu, Jan 27, 2022 at 2:55 PM Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl wrote:
On Thu, Jan 27, 2022 at 11:51:54AM +0100, Iñaki Ucar wrote:
On Thu, 27 Jan 2022 at 11:34, Richard W.M. Jones rjones@redhat.com wrote:
Another concrete proposal:
Is there a way to scan all binary packages in Fedora to get either a count or list of those packages that contain strings like "-Wl,-dT,/builddir/". This will give us numbers on how bad the problem is. I suspect it's quite widespread.
Rich.
And another concrete proposal:
This is not reflected *at all* in the change proposal. Modify the change proposal to acknowledge this issue and discuss it. If changes are needed with respect to how things are done in other packages, then required modifications should be in the scope of the proposal owners.
This all came up in the mass rebuild, so it wasn't known at the time the proposal was written. I'll add more information to the Change page.
I think it was rather unfortunate that this change was pushed to rawhide only a very short time before the mass rebuild. This way there was neither a "testing" period in which issues could have been found, nor was there opportunity to fix them *before* the mass rebuild. All this could probably have been avoided with a less tight schedule ... (same goes for a lot of the GCC 12 issues or %set_build_flags ...)
Fabio
Dne 27. 01. 22 v 11:28 Richard W.M. Jones napsal(a):
Another concrete proposal:
Is there a way to scan all binary packages in Fedora to get either a count or list of those packages that contain strings like "-Wl,-dT,/builddir/". This will give us numbers on how bad the problem is.
If this was embedded in Ruby, all binary rubygem- packages would be FTBFS, therefore you would not be able to obtain the number. But you could measure how many FTBFS you have, probably not state we want.
Vít
On Thu, 27 Jan 2022 at 11:19, Iñaki Ucar iucar@fedoraproject.org wrote:
On Thu, 27 Jan 2022 at 10:58, Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl wrote:
I think this change should be reverted until a cleaner way can be found to implement it.
I'm all for making better, but please make concrete proposals.
Here's a concrete proposal:
- Copy %build_*flags to another private macro, let's say %_build_*flags.
- Add there the stuff that is not valid once the current build
finishes (e.g., the flag that this change adds).
- Use that in %set_build_flags and leave %build_*flags as it was.
Or even better:
- Define a new set of build-specific build flags, let's say %build_specific_*flags - Add the stuff there. - Add them to %set_build_flags along %build_*flags
On Thu, Jan 27, 2022 at 11:29:13AM +0100, Iñaki Ucar wrote:
On Thu, 27 Jan 2022 at 11:19, Iñaki Ucar iucar@fedoraproject.org wrote:
On Thu, 27 Jan 2022 at 10:58, Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl wrote:
I think this change should be reverted until a cleaner way can be found to implement it.
I'm all for making better, but please make concrete proposals.
Here's a concrete proposal:
- Copy %build_*flags to another private macro, let's say %_build_*flags.
- Add there the stuff that is not valid once the current build
finishes (e.g., the flag that this change adds).
- Use that in %set_build_flags and leave %build_*flags as it was.
This wouldn't help: CFLAGS/CXXFLAGS/LDFLAGS would still contain the problematic flags. So packages which embed those flags in a place where they shouldn't would still have a problem.
We *do* have a macro that has build flags with "problematic" flags removed: %extension_{c,cxx,ld}flags.
Or even better:
- Define a new set of build-specific build flags, let's say
%build_specific_*flags
- Add the stuff there.
- Add them to %set_build_flags along %build_*flags
This has the same problem as above.
Zbyszek
Dne 27. 01. 22 v 11:19 Iñaki Ucar napsal(a):
On Thu, 27 Jan 2022 at 10:58, Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl wrote:
I think this change should be reverted until a cleaner way can be found to implement it.
I'm all for making better, but please make concrete proposals.
Here's a concrete proposal:
- Copy %build_*flags to another private macro, let's say %_build_*flags.
- Add there the stuff that is not valid once the current build
finishes (e.g., the flag that this change adds).
- Use that in %set_build_flags and leave %build_*flags as it was.
That sounds similar to "%extension_cflags/%extension_cxxflags/%extension_ldflags" Zbyszek already mentioned elsewhere. Or I don't get the point.
Here is concrete proposal for Ruby:
https://src.fedoraproject.org/rpms/ruby/pull-request/110
Vít
On Thu, 27 Jan 2022 at 12:09, Vít Ondruch vondruch@redhat.com wrote:
Dne 27. 01. 22 v 11:19 Iñaki Ucar napsal(a):
On Thu, 27 Jan 2022 at 10:58, Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl wrote:
I think this change should be reverted until a cleaner way can be found to implement it.
I'm all for making better, but please make concrete proposals.
Here's a concrete proposal:
- Copy %build_*flags to another private macro, let's say %_build_*flags.
- Add there the stuff that is not valid once the current build
finishes (e.g., the flag that this change adds).
- Use that in %set_build_flags and leave %build_*flags as it was.
That sounds similar to "%extension_cflags/%extension_cxxflags/%extension_ldflags" Zbyszek already mentioned elsewhere. Or I don't get the point.
Here is concrete proposal for Ruby:
There's a *big* difference: switching to %extension_*flags means that many packages have to be modified, and we have to ensure that stuff does not end up in places like pkgconfig files. This is not discussed in the change proposal. Are proposal owners willing to do that work? Instead, what I propose leaves %build_*flags as they are, so there should be no impact. Still, proposal owners would need to identify which packages are currently broken and rebuild them.
And then there are other minor differences: - Are %extension_*flags present in EPEL? - Do we really want to remove hardening flags from extensions? Why should we have a different standard for them? Apparently they caused trouble in the past in Python packages. Not sure what kind of trouble, but we have no issues in the R ecosystem, for example. And I suspect other ecosystems do not either, since they are using %build_*flags and not extension flags.
Dne 27. 01. 22 v 12:18 Iñaki Ucar napsal(a):
On Thu, 27 Jan 2022 at 12:09, Vít Ondruch vondruch@redhat.com wrote:
Dne 27. 01. 22 v 11:19 Iñaki Ucar napsal(a):
On Thu, 27 Jan 2022 at 10:58, Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl wrote:
I think this change should be reverted until a cleaner way can be found to implement it.
I'm all for making better, but please make concrete proposals.
Here's a concrete proposal:
- Copy %build_*flags to another private macro, let's say %_build_*flags.
- Add there the stuff that is not valid once the current build
finishes (e.g., the flag that this change adds).
- Use that in %set_build_flags and leave %build_*flags as it was.
That sounds similar to "%extension_cflags/%extension_cxxflags/%extension_ldflags" Zbyszek already mentioned elsewhere. Or I don't get the point.
Here is concrete proposal for Ruby:
There's a *big* difference: switching to %extension_*flags means that many packages have to be modified
But why? In Ruby, it should be enough to change Ruby itself and all the other Ruby packages should be fine. Why is it different for R/OCAML?
Vít
, and we have to ensure that stuff does not end up in places like pkgconfig files. This is not discussed in the change proposal. Are proposal owners willing to do that work? Instead, what I propose leaves %build_*flags as they are, so there should be no impact. Still, proposal owners would need to identify which packages are currently broken and rebuild them.
And then there are other minor differences:
- Are %extension_*flags present in EPEL?
- Do we really want to remove hardening flags from extensions? Why
should we have a different standard for them? Apparently they caused trouble in the past in Python packages. Not sure what kind of trouble, but we have no issues in the R ecosystem, for example. And I suspect other ecosystems do not either, since they are using %build_*flags and not extension flags.
On Thu, Jan 27, 2022 at 12:50:33PM +0100, Vít Ondruch wrote:
Dne 27. 01. 22 v 12:18 Iñaki Ucar napsal(a):
On Thu, 27 Jan 2022 at 12:09, Vít Ondruch vondruch@redhat.com wrote:
Dne 27. 01. 22 v 11:19 Iñaki Ucar napsal(a):
On Thu, 27 Jan 2022 at 10:58, Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl wrote:
I think this change should be reverted until a cleaner way can be found to implement it.
I'm all for making better, but please make concrete proposals.
Here's a concrete proposal:
- Copy %build_*flags to another private macro, let's say %_build_*flags.
- Add there the stuff that is not valid once the current build
finishes (e.g., the flag that this change adds).
- Use that in %set_build_flags and leave %build_*flags as it was.
That sounds similar to "%extension_cflags/%extension_cxxflags/%extension_ldflags" Zbyszek already mentioned elsewhere. Or I don't get the point.
Here is concrete proposal for Ruby:
There's a *big* difference: switching to %extension_*flags means that many packages have to be modified
But why? In Ruby, it should be enough to change Ruby itself and all the other Ruby packages should be fine. Why is it different for R/OCAML?
OCaml is a compiled language, not interpreted. It ships a compiler which embeds CFLAGS / LDFLAGS (for hardening). But additionally it ships many libraries that contain link instructions (so they can link to C libraries), and those also embed LDFLAGS.
Just about every OCaml package is negatively affected by this change and they all have to be rebuilt.
Rich.
Vít
, and we have to ensure that stuff does not end up in places like pkgconfig files. This is not discussed in the change proposal. Are proposal owners willing to do that work? Instead, what I propose leaves %build_*flags as they are, so there should be no impact. Still, proposal owners would need to identify which packages are currently broken and rebuild them.
And then there are other minor differences:
- Are %extension_*flags present in EPEL?
- Do we really want to remove hardening flags from extensions? Why
should we have a different standard for them? Apparently they caused trouble in the past in Python packages. Not sure what kind of trouble, but we have no issues in the R ecosystem, for example. And I suspect other ecosystems do not either, since they are using %build_*flags and not extension flags.
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Dne 27. 01. 22 v 13:04 Richard W.M. Jones napsal(a):
On Thu, Jan 27, 2022 at 12:50:33PM +0100, Vít Ondruch wrote:
Dne 27. 01. 22 v 12:18 Iñaki Ucar napsal(a):
On Thu, 27 Jan 2022 at 12:09, Vít Ondruch vondruch@redhat.com wrote:
Dne 27. 01. 22 v 11:19 Iñaki Ucar napsal(a):
On Thu, 27 Jan 2022 at 10:58, Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl wrote:
> I think this change should be reverted until a cleaner way can be > found to implement it. I'm all for making better, but please make concrete proposals.
Here's a concrete proposal:
- Copy %build_*flags to another private macro, let's say %_build_*flags.
- Add there the stuff that is not valid once the current build
finishes (e.g., the flag that this change adds).
- Use that in %set_build_flags and leave %build_*flags as it was.
That sounds similar to "%extension_cflags/%extension_cxxflags/%extension_ldflags" Zbyszek already mentioned elsewhere. Or I don't get the point.
Here is concrete proposal for Ruby:
There's a *big* difference: switching to %extension_*flags means that many packages have to be modified
But why? In Ruby, it should be enough to change Ruby itself and all the other Ruby packages should be fine. Why is it different for R/OCAML?
OCaml is a compiled language, not interpreted. It ships a compiler which embeds CFLAGS / LDFLAGS (for hardening). But additionally it ships many libraries that contain link instructions (so they can link to C libraries), and those also embed LDFLAGS.
Just about every OCaml package is negatively affected by this change and they all have to be rebuilt.
Ah, so they are embedded into binary. Where in Ruby, they are embedded in Ruby library, which is a piece of cake to change. Got it. Thanks for being patient with me ;)
I should have insisted on the revert more, because that was my initial gut feeling ....
Vít
Rich.
Vít
, and we have to ensure that stuff does not end up in places like pkgconfig files. This is not discussed in the change proposal. Are proposal owners willing to do that work? Instead, what I propose leaves %build_*flags as they are, so there should be no impact. Still, proposal owners would need to identify which packages are currently broken and rebuild them.
And then there are other minor differences:
- Are %extension_*flags present in EPEL?
- Do we really want to remove hardening flags from extensions? Why
should we have a different standard for them? Apparently they caused trouble in the past in Python packages. Not sure what kind of trouble, but we have no issues in the R ecosystem, for example. And I suspect other ecosystems do not either, since they are using %build_*flags and not extension flags.
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On Thu, Jan 27, 2022 at 12:04:02PM +0000, Richard W.M. Jones wrote:
On Thu, Jan 27, 2022 at 12:50:33PM +0100, Vít Ondruch wrote:
Dne 27. 01. 22 v 12:18 Iñaki Ucar napsal(a):
On Thu, 27 Jan 2022 at 12:09, Vít Ondruch vondruch@redhat.com wrote:
Dne 27. 01. 22 v 11:19 Iñaki Ucar napsal(a):
On Thu, 27 Jan 2022 at 10:58, Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl wrote:
>I think this change should be reverted until a cleaner way can be >found to implement it. I'm all for making better, but please make concrete proposals.
Here's a concrete proposal:
- Copy %build_*flags to another private macro, let's say %_build_*flags.
- Add there the stuff that is not valid once the current build
finishes (e.g., the flag that this change adds).
- Use that in %set_build_flags and leave %build_*flags as it was.
That sounds similar to "%extension_cflags/%extension_cxxflags/%extension_ldflags" Zbyszek already mentioned elsewhere. Or I don't get the point.
Here is concrete proposal for Ruby:
There's a *big* difference: switching to %extension_*flags means that many packages have to be modified
But why? In Ruby, it should be enough to change Ruby itself and all the other Ruby packages should be fine. Why is it different for R/OCAML?
OCaml is a compiled language, not interpreted. It ships a compiler which embeds CFLAGS / LDFLAGS (for hardening). But additionally it ships many libraries that contain link instructions (so they can link to C libraries), and those also embed LDFLAGS.
Hmm, so it sounds like OCaml was actually broken already. For example, if OCaml is linked with -flto or -use-ld=lld, then this should not "leak" into extensions. The split between flags which are appropriate to propagate to extensions and flags which are not is very important, and if it is not followed, extensions would be broken too.
Just about every OCaml package is negatively affected by this change and they all have to be rebuilt.
That doesn't sound good, but if it turns out that they need to be rebuild, I can do that myself.
Zbyszek
On Thu, Jan 27, 2022 at 01:58:34PM +0000, Zbigniew Jędrzejewski-Szmek wrote:
Hmm, so it sounds like OCaml was actually broken already. For example, if OCaml is linked with -flto or -use-ld=lld, then this should not "leak" into extensions. The split between flags which are appropriate to propagate to extensions and flags which are not is very important, and if it is not followed, extensions would be broken too.
I checked and it does not include those.
Before the change (ocaml-csexp-devel-1.5.1-3.fc36.x86_64):
$ ocamlobjinfo /usr/lib64/ocaml/csexp/csexp.cmxa | grep ^Extra Extra C object files: -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 Extra C options: -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
After the change (ocaml-csexp-devel-1.5.1-4.fc36.x86_64):
$ ocamlobjinfo /usr/lib64/ocaml/csexp/csexp.cmxa | grep ^Extra Extra C object files: -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -Wl,-dT,/builddir/build/BUILD/.package_note-ocaml-csexp-1.5.1-4.fc36.x86_64.ld Extra C options: -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
These flags are used when building and linking OCaml native code that uses the library (csexp in this case).
Rich.
Just about every OCaml package is negatively affected by this change and they all have to be rebuilt.
That doesn't sound good, but if it turns out that they need to be rebuild, I can do that myself.
Zbyszek _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On Thu, 27 Jan 2022 at 12:59, Vít Ondruch vondruch@redhat.com wrote:
Dne 27. 01. 22 v 12:18 Iñaki Ucar napsal(a):
On Thu, 27 Jan 2022 at 12:09, Vít Ondruch vondruch@redhat.com wrote:
Dne 27. 01. 22 v 11:19 Iñaki Ucar napsal(a):
On Thu, 27 Jan 2022 at 10:58, Zbigniew Jędrzejewski-Szmek zbyszek@in.waw.pl wrote:
I think this change should be reverted until a cleaner way can be found to implement it.
I'm all for making better, but please make concrete proposals.
Here's a concrete proposal:
- Copy %build_*flags to another private macro, let's say %_build_*flags.
- Add there the stuff that is not valid once the current build
finishes (e.g., the flag that this change adds).
- Use that in %set_build_flags and leave %build_*flags as it was.
That sounds similar to "%extension_cflags/%extension_cxxflags/%extension_ldflags" Zbyszek already mentioned elsewhere. Or I don't get the point.
Here is concrete proposal for Ruby:
There's a *big* difference: switching to %extension_*flags means that many packages have to be modified
But why? In Ruby, it should be enough to change Ruby itself and all the other Ruby packages should be fine. Why is it different for R/OCAML?
In R, this could be enough for most packages too. But e.g. there are R packages that are template builders for user programs, and probably suffer from the same issue. The point is that we don't even know how many packages are affected.
Am 27.01.22 um 10:46 schrieb Zbigniew Jędrzejewski-Szmek:
But the problem is more general than this too. It also turns up in some *.pc (pkgconf) files.
That's a bug too.
I think this change should be reverted until a cleaner way can be found to implement it.
I'm all for making better, but please make concrete proposals.
Revert?
Just saying "revert, revert" because some corner case is not satisfied, when a simple opt-out exists, is IMO not useful.
IMO, we are not talking about corner cases, about about a fundamental design flaw, which you, as I perceive it, are trying to play down.
Ralf
Thinking about this a bit more, the implementation of this feature simply seems to be wrong. RPM already has a final stage where it strips ELF files and builds debuginfo. Why wasn't the addition of package notes done there?
So my concrete proposal:
- Revert this change now
- Move the change to F37
- Work with us to help list all of the packages that have been broken by this change and help us to rebuild them all
- Change the way it works so it doesn't affect build flags
Rich.
* Richard W. M. Jones:
Thinking about this a bit more, the implementation of this feature simply seems to be wrong. RPM already has a final stage where it strips ELF files and builds debuginfo. Why wasn't the addition of package notes done there?
The package notes are in an allocatable section, to be mapped at run time, so that they end up in core files. As far as I know, it's not reliably possible to add such data to an ELF file after the final (non-relocatable) link.
We would have to pre-allocate some fixed space and fill it in later.
Cleaner approaches are possible if we teach the core dumper how to copy select data from non-allocated sections. I think we would then need just a placeholder program header.
Thanks, Florian
On Thu, 2022-02-03 at 10:09 +0100, Florian Weimer wrote:
- Richard W. M. Jones:
Thinking about this a bit more, the implementation of this feature simply seems to be wrong. RPM already has a final stage where it strips ELF files and builds debuginfo. Why wasn't the addition of package notes done there?
The package notes are in an allocatable section, to be mapped at run time, so that they end up in core files. As far as I know, it's not reliably possible to add such data to an ELF file after the final (non-relocatable) link.
We would have to pre-allocate some fixed space and fill it in later.
Cleaner approaches are possible if we teach the core dumper how to copy select data from non-allocated sections. I think we would then need just a placeholder program header.
While it is nice to discuss future options, do we have a way to fix FTBFS's in rawhide _now_ ?
My time is limited and I want to upgrade one of my packages and this is blocking me.
Is opening a FESCO ticket the only way ?
Simo.
V Thu, Feb 03, 2022 at 08:56:20AM -0500, Simo Sorce napsal(a):
On Thu, 2022-02-03 at 10:09 +0100, Florian Weimer wrote:
- Richard W. M. Jones:
Thinking about this a bit more, the implementation of this feature simply seems to be wrong. RPM already has a final stage where it strips ELF files and builds debuginfo. Why wasn't the addition of package notes done there?
The package notes are in an allocatable section, to be mapped at run time, so that they end up in core files. As far as I know, it's not reliably possible to add such data to an ELF file after the final (non-relocatable) link.
We would have to pre-allocate some fixed space and fill it in later.
Cleaner approaches are possible if we teach the core dumper how to copy select data from non-allocated sections. I think we would then need just a placeholder program header.
While it is nice to discuss future options, do we have a way to fix FTBFS's in rawhide _now_ ?
You can disable embedding the package notes by undefining _package_note_file macro in the package which builds in the linker flags. See https://src.fedoraproject.org/rpms/perl/c/4751b01e52fad1ef9c3012675791d979436ff8fe.patch for an example. Kudos to Jitka.
-- Petr
On Thu, 2022-02-03 at 15:15 +0100, Petr Pisar wrote:
V Thu, Feb 03, 2022 at 08:56:20AM -0500, Simo Sorce napsal(a):
On Thu, 2022-02-03 at 10:09 +0100, Florian Weimer wrote:
- Richard W. M. Jones:
Thinking about this a bit more, the implementation of this feature simply seems to be wrong. RPM already has a final stage where it strips ELF files and builds debuginfo. Why wasn't the addition of package notes done there?
The package notes are in an allocatable section, to be mapped at run time, so that they end up in core files. As far as I know, it's not reliably possible to add such data to an ELF file after the final (non-relocatable) link.
We would have to pre-allocate some fixed space and fill it in later.
Cleaner approaches are possible if we teach the core dumper how to copy select data from non-allocated sections. I think we would then need just a placeholder program header.
While it is nice to discuss future options, do we have a way to fix FTBFS's in rawhide _now_ ?
You can disable embedding the package notes by undefining _package_note_file macro in the package which builds in the linker flags. See https://src.fedoraproject.org/rpms/perl/c/4751b01e52fad1ef9c3012675791d979436ff8fe.patch for an example. Kudos to Jitka.
No I could not, because I still got the dependent krb5 package to bring in another unavailable linker script.
FTR we resolved this by rebuilding krb5-libs *without* notes, and then I could rebuild python-gssapi also without notes.
However I resent a bit that I had to chase down this problem myself, days after it had already been exposed, and basically manually disable this feature for a large part of Fedora (anything that links to krb5 now is missing these notes, rights?) only relying on hearsay and some brave soul giving tips.
It is also now littering spec files with disablement of this feature, which will make it harder to re-introduce it once these problems are solved (or will cause abandonment of the feature, with fragments of unused spec files all over the place).
Simo.
V Thu, Feb 03, 2022 at 09:26:09AM -0500, Simo Sorce napsal(a):
On Thu, 2022-02-03 at 15:15 +0100, Petr Pisar wrote:
V Thu, Feb 03, 2022 at 08:56:20AM -0500, Simo Sorce napsal(a):
On Thu, 2022-02-03 at 10:09 +0100, Florian Weimer wrote:
- Richard W. M. Jones:
Thinking about this a bit more, the implementation of this feature simply seems to be wrong. RPM already has a final stage where it strips ELF files and builds debuginfo. Why wasn't the addition of package notes done there?
The package notes are in an allocatable section, to be mapped at run time, so that they end up in core files. As far as I know, it's not reliably possible to add such data to an ELF file after the final (non-relocatable) link.
We would have to pre-allocate some fixed space and fill it in later.
Cleaner approaches are possible if we teach the core dumper how to copy select data from non-allocated sections. I think we would then need just a placeholder program header.
While it is nice to discuss future options, do we have a way to fix FTBFS's in rawhide _now_ ?
You can disable embedding the package notes by undefining _package_note_file macro in the package which builds in the linker flags. See https://src.fedoraproject.org/rpms/perl/c/4751b01e52fad1ef9c3012675791d979436ff8fe.patch for an example. Kudos to Jitka.
No I could not, because I still got the dependent krb5 package to bring in another unavailable linker script.
FTR we resolved this by rebuilding krb5-libs *without* notes, and then I could rebuild python-gssapi also without notes.
However I resent a bit that I had to chase down this problem myself, days after it had already been exposed, and basically manually disable this feature for a large part of Fedora (anything that links to krb5 now is missing these notes, rights?)
No. krb5-libs will miss its notes. But python-gssapi will contain its correct notes. (Provided python-gssapi links to krb5-libs dynamically. I don't know whether the notes only record a source package name the ELF file belongs to, or whether they try to track origin of all object files the ELF consists of.)
-- Petr
On Thu, 2022-02-03 at 16:22 +0100, Petr Pisar wrote:
V Thu, Feb 03, 2022 at 09:26:09AM -0500, Simo Sorce napsal(a):
On Thu, 2022-02-03 at 15:15 +0100, Petr Pisar wrote:
V Thu, Feb 03, 2022 at 08:56:20AM -0500, Simo Sorce napsal(a):
On Thu, 2022-02-03 at 10:09 +0100, Florian Weimer wrote:
- Richard W. M. Jones:
Thinking about this a bit more, the implementation of this feature simply seems to be wrong. RPM already has a final stage where it strips ELF files and builds debuginfo. Why wasn't the addition of package notes done there?
The package notes are in an allocatable section, to be mapped at run time, so that they end up in core files. As far as I know, it's not reliably possible to add such data to an ELF file after the final (non-relocatable) link.
We would have to pre-allocate some fixed space and fill it in later.
Cleaner approaches are possible if we teach the core dumper how to copy select data from non-allocated sections. I think we would then need just a placeholder program header.
While it is nice to discuss future options, do we have a way to fix FTBFS's in rawhide _now_ ?
You can disable embedding the package notes by undefining _package_note_file macro in the package which builds in the linker flags. See https://src.fedoraproject.org/rpms/perl/c/4751b01e52fad1ef9c3012675791d979436ff8fe.patch for an example. Kudos to Jitka.
No I could not, because I still got the dependent krb5 package to bring in another unavailable linker script.
FTR we resolved this by rebuilding krb5-libs *without* notes, and then I could rebuild python-gssapi also without notes.
However I resent a bit that I had to chase down this problem myself, days after it had already been exposed, and basically manually disable this feature for a large part of Fedora (anything that links to krb5 now is missing these notes, rights?)
No. krb5-libs will miss its notes. But python-gssapi will contain its correct notes. (Provided python-gssapi links to krb5-libs dynamically. I don't know whether the notes only record a source package name the ELF file belongs to, or whether they try to track origin of all object files the ELF consists of.)
I had to undefine notes on python-gssapi as well, so not notes, period.
On 03. 02. 22 16:36, Simo Sorce wrote:
No. krb5-libs will miss its notes. But python-gssapi will contain its correct notes. (Provided python-gssapi links to krb5-libs dynamically. I don't know whether the notes only record a source package name the ELF file belongs to, or whether they try to track origin of all object files the ELF consists of.)
I had to undefine notes on python-gssapi as well, so not notes, period.
I've just tried to build python-gssapi with notes enabled after krb5 was fixed and it builds fine.
See https://src.fedoraproject.org/rpms/python-gssapi/pull-request/4
On 03. 02. 22 16:36, Simo Sorce wrote:
I've just tried to build python-gssapi with notes enabled after krb5 was fixed and it builds fine.
See https://src.fedoraproject.org/rpms/python-gssapi/pull-request/4
It looks like it's not the first time that krb5-config causes issues because it picks up state from the build env:
https://bugzilla.redhat.com/show_bug.cgi?id=1997021 https://bugzilla.redhat.com/show_bug.cgi?id=1204646 https://krbdev.mit.edu/rt/Ticket/Display.html?id=8159
We had similar problems long ago with libpcap's own tool, before the project made available a pkg-config file, which is much simpler and nicer (and doesn't fall apart when cross-compiling). Is there any reason to keep using these specialized binaries over pkg-config these days? Or is it just a matter of doing the work to gradually switch over?
On Thu, 2022-02-03 at 22:02 +0000, Luca Boccassi wrote:
On 03. 02. 22 16:36, Simo Sorce wrote:
I've just tried to build python-gssapi with notes enabled after krb5 was fixed and it builds fine.
See https://src.fedoraproject.org/rpms/python-gssapi/pull-request/4
Nice! Especially the auto build deps, which I tried to enable but was failing for me, so I ended up reverting that change.
It looks like it's not the first time that krb5-config causes issues because it picks up state from the build env:
https://bugzilla.redhat.com/show_bug.cgi?id=1997021 https://bugzilla.redhat.com/show_bug.cgi?id=1204646 https://krbdev.mit.edu/rt/Ticket/Display.html?id=8159
We had similar problems long ago with libpcap's own tool, before the project made available a pkg-config file, which is much simpler and nicer (and doesn't fall apart when cross-compiling). Is there any reason to keep using these specialized binaries over pkg-config these days? Or is it just a matter of doing the work to gradually switch over?
The latter, but it needs to happen in upstream and needs to take in account that pkg-config is not available in all architectures, so it is hard to sell, as it makes he code more complicate to preserve both mechanism.
Simo.
On Fri, Feb 4, 2022 at 9:07 AM Simo Sorce simo@redhat.com wrote:
On Thu, 2022-02-03 at 22:02 +0000, Luca Boccassi wrote:
On 03. 02. 22 16:36, Simo Sorce wrote:
I've just tried to build python-gssapi with notes enabled after krb5 was fixed and it builds fine.
See https://src.fedoraproject.org/rpms/python-gssapi/pull-request/4
Nice! Especially the auto build deps, which I tried to enable but was failing for me, so I ended up reverting that change.
It looks like it's not the first time that krb5-config causes issues because it picks up state from the build env:
https://bugzilla.redhat.com/show_bug.cgi?id=1997021 https://bugzilla.redhat.com/show_bug.cgi?id=1204646 https://krbdev.mit.edu/rt/Ticket/Display.html?id=8159
We had similar problems long ago with libpcap's own tool, before the project made available a pkg-config file, which is much simpler and nicer (and doesn't fall apart when cross-compiling). Is there any reason to keep using these specialized binaries over pkg-config these days? Or is it just a matter of doing the work to gradually switch over?
The latter, but it needs to happen in upstream and needs to take in account that pkg-config is not available in all architectures, so it is hard to sell, as it makes he code more complicate to preserve both mechanism.
Wait what? pkgconf (our pkg-config implementation) builds on even the weird esoteric architectures like ppc32 and m68k. What are we missing here?
On Fri, Feb 4, 2022 at 9:07 AM Simo Sorce <simo(a)redhat.com> wrote:
Wait what? pkgconf (our pkg-config implementation) builds on even the weird esoteric architectures like ppc32 and m68k. What are we missing here?
Yeah, and the older implementation in Debian builds and runs fine even on architectures that have more letters in their abbreviated names than users: https://buildd.debian.org/status/package.php?p=pkg-config I mean, if it can run on a Motorola CPU, I'm pretty sure it can run anywhere :-)
On Thu, Feb 03, 2022 at 10:36:02AM -0500, Simo Sorce wrote:
On Thu, 2022-02-03 at 16:22 +0100, Petr Pisar wrote:
V Thu, Feb 03, 2022 at 09:26:09AM -0500, Simo Sorce napsal(a):
On Thu, 2022-02-03 at 15:15 +0100, Petr Pisar wrote:
V Thu, Feb 03, 2022 at 08:56:20AM -0500, Simo Sorce napsal(a):
On Thu, 2022-02-03 at 10:09 +0100, Florian Weimer wrote:
- Richard W. M. Jones:
> Thinking about this a bit more, the implementation of this feature > simply seems to be wrong. RPM already has a final stage where it > strips ELF files and builds debuginfo. Why wasn't the addition of > package notes done there?
The package notes are in an allocatable section, to be mapped at run time, so that they end up in core files. As far as I know, it's not reliably possible to add such data to an ELF file after the final (non-relocatable) link.
We would have to pre-allocate some fixed space and fill it in later.
Cleaner approaches are possible if we teach the core dumper how to copy select data from non-allocated sections. I think we would then need just a placeholder program header.
While it is nice to discuss future options, do we have a way to fix FTBFS's in rawhide _now_ ?
You can disable embedding the package notes by undefining _package_note_file macro in the package which builds in the linker flags. See https://src.fedoraproject.org/rpms/perl/c/4751b01e52fad1ef9c3012675791d979436ff8fe.patch for an example. Kudos to Jitka.
No I could not, because I still got the dependent krb5 package to bring in another unavailable linker script.
FTR we resolved this by rebuilding krb5-libs *without* notes, and then I could rebuild python-gssapi also without notes.
However I resent a bit that I had to chase down this problem myself, days after it had already been exposed, and basically manually disable this feature for a large part of Fedora (anything that links to krb5 now is missing these notes, rights?)
No. krb5-libs will miss its notes. But python-gssapi will contain its correct notes. (Provided python-gssapi links to krb5-libs dynamically. I don't know whether the notes only record a source package name the ELF file belongs to, or whether they try to track origin of all object files the ELF consists of.)
I had to undefine notes on python-gssapi as well, so not notes, period.
Similarly I added the %undefine to every OCaml package and every package built using OCaml today.
Rich.
-
Fabio Valentini -
Florian Weimer -
Iñaki Ucar -
Luca Boccassi -
Miro Hrončok -
Neal Gompa -
Petr Pisar -
Ralf Corsépius -
Richard W.M. Jones -
Simo Sorce -
Vitaly Zaitsev -
Vít Ondruch -
Zbigniew Jędrzejewski-Szmek