On Mon, Jan 16, 2023 at 1:40 PM Miro HronĨok <mhroncok(a)redhat.com> wrote:
Hello folks,
this recent Fedora change:
https://fedoraproject.org/wiki/Changes/Add_FORTIFY_SOURCE%3D3_to_distribu...
Made me think:
Which compiler flags we need to store in Python and which can we omit?
In order to make Python extension modules binary compatible with Python, Python
saves the compiler flags at compile-time and reuses them when building
extension modules.
Historically, we had troubles with this approach because some of the flags are
unusable without redhat-rpm-config, annobin etc.
As a result, there are now 2 compiler flags macros available for RPM:
%{build_cflags} and %{extension_cflags} (same for ldflags etc.).
While Python itself is built with %{build_cflags}, it saves %{extension_cflags}
in sysconfig.
The flags differ like this:
$ diff -u <(rpm --eval '%build_cflags' | tr ' ' '\n') <(rpm
--eval
'%extension_cflags' | tr ' ' '\n') | grep ^-
--flto=auto
--ffat-lto-objects
--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
--specs=/usr/lib/rpm/redhat/redhat-annobin-cc1
$ diff -u <(rpm --eval '%build_ldflags' | tr ' ' '\n')
<(rpm --eval
'%extension_ldflags' | tr ' ' '\n') | grep ^-
--specs=/usr/lib/rpm/redhat/redhat-hardened-ld
--specs=/usr/lib/rpm/redhat/redhat-annobin-cc1
(There are also some other differences wrt
https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects but
those are apparently harder to get from outside of a real build.)
The current set of flags from Python can be obtained by:
>>> sysconfig.get_config_var('CFLAGS')
'-Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g
-grecord-gcc-switches -pipe -Wall -Werror=format-security -U_FORTIFY_SOURCE
-Wp,-U_FORTIFY_SOURCE -Wp,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS
-fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables
-fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2
-fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security
-U_FORTIFY_SOURCE -Wp,-U_FORTIFY_SOURCE -Wp,-D_FORTIFY_SOURCE=3
-Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic
-fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
-D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe
-Wall -Werror=format-security -U_FORTIFY_SOURCE -Wp,-U_FORTIFY_SOURCE
-Wp,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong
-m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection
-fcf-protection -D_GNU_SOURCE -fPIC -fwrapv'
>>> sysconfig.get_config_var('LDFLAGS')
'-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -Wl,--build-id=sha1 -g
-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -Wl,--build-id=sha1 -g'
I wonder if other flags should be removed as well.
Isn't Python built e.g. with -Werror=format-security or -Wsign-compare binary
compatible with extension modules built without it?
None of the warning flags should impact ABI or for that matter, even codegen.
What about FORTIFY_SOURCE and others?
It won't impact ABI, i.e. python could be built with _FORTIFY_SOURCE
and modules without, or vice versa.
Is there a compiler flags expert here who could help me determine
what flags
could (or even should) be removed from %{extension_*flags}?
If it is for distribution packages then I reckon the flags should be
as close as possible for the mere reason of consistency within the
distribution. If they're used for custom built modules (e.g. through
a tool that python auto-generates to build modules), then a very small
subset of the above flags would be necessary to retain binary
compatibility; you should be able to remove most of it.
Sid