From: "Victor Stinner" <vstinner(a)redhat.com>
To: devel(a)lists.fedoraproject.org
Sent: Friday, November 15, 2019 10:21:44 AM
Subject: Re: Fedora 32 System-Wide Change proposal: Build Python 3 to statically link
with libpython3.8.a for better
performance
Hi Jan,
With the helper of Florian Weimer and Charalampos Stratakis, we also agreed
to test this flag in priority. I understood that it disables the LD_PRELOAD
feature: it's no longer possible to override symbols in libpython with
LD_PRELOAD. Thanks to that, the compiler can avoid PLT indirection for
function calls and can inline more function functions in libpython. I'm
talking about a function call from libpython to libpython: something which
is very common in python. Basically, almost all function calls are calls
from libpython to libpython.
I'm impressed. Thanks to -fno-semantic-interposition, I get the same speedup
on a dynamically linked Python (libpython) compared to statically linked
Python!
Yesterday, I tried on a vanilla Python compiled manually:
./configure --enable-optimizations --with-lto --enable-shared
CFLAGS="-fno-semantic-interposition"
LDFLAGS="-fno-semantic-interposition"
I saw the same speed up than avoiding --enable-shared. Today I validated this
result using the RPM generated by Charalampos's PR:
https://src.fedoraproject.org/rpms/python38/pull-request/53
In short,
https://fedoraproject.org/wiki/Changes/PythonStaticSpeedup is
useless: there is no need to modify Python to statically link it to
libpython. We can keep the dynamically library libpython and keep Python
dynamically linked to it. We only need to pass -fno-semantic-interposition
to compiler and linker flags when building Python!
I'm not sure if we need a Fedora change just for a compiler flag. Again, the
only drawback is that we will no longer be able to override a symbol using
LD_PRELOAD. Honestly, I never did that. I don't see any use case for that.
But I used LD_PRELOAD on the libc multiple times to mock the system clock
for example.
If someone really needs LD_PRELOAD, it's quite easy to build a custom Python
without -fno-semantic-interposition.
Victor
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Thanks Victor for running the benchmarks.
The change will be withdrawn and another self-contained one will be created. I think not
being able to override symbols on the system python is a better tradeoff than the
size/speed and possible incompatibilities.
Side note: the list of packages that still link to libpython without embedding the
interpreter will be used to mass file bugs in order to unlink them.
--
Regards,
Charalampos Stratakis
Software Engineer
Python Maintenance Team, Red Hat