Python 3.10 in Fedora 36+: sysconfig's "posix_prefix" install scheme now has /usr/local

[Miro Hrončok]

Hello Pythonistas,
since Fedora 27, we have been patching Python to install packages to 
/usr/local/lib(64)/python3.X/site-packages if not in rpmbuild:

https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe

The patch was a pragmatic hack in the distutils module that proved to be 
working quite reliably over the years. Unfortunately, the distutils module is 
deprecated and is going to be removed from Python 3.12:

https://www.python.org/dev/peps/pep-0632/

Working with the upstreams of setuptools, pip, and Python, as well as other 
Linux distros Python maintainers (Arch, Debian), we have now changed the way 
the patch works to patch the sysconfig module instead, changing the 
posix_prefix install scheme to use /usr/local paths.

See https://discuss.python.org/t/pep-632-deprecate-distutils-module/5134/104 
for details.

This change is aligned with the upstream plan for Python 3.11 to offer 
distributors a way to supply their installation schemes to sysconfig:

https://bugs.python.org/issue43976

Nothing should change for Fedora packages. If you encounter problems, do let us 
know and we'll try to help solve them.

Software that reads the posix_prefix install_scheme on runtime to figure out 
where to install stuff should behave the right way by default. OTOH software 
that reads it to figure out where to load stuff from might need changes, see 
e.g. the change we did in dnf: 
https://github.com/rpm-software-management/dnf/pull/1782

Relevant commit: https://src.fedoraproject.org/rpms/python3.10/c/47935cfb9870

Bodhi update: https://bodhi.fedoraproject.org/updates/FEDORA-2021-31c7a2fca8

We might later introduce this to older Python versions as well, but we are not 
planning to backport this to older Fedora releases.

[Miro Hrončok]

On 05. 10. 21 16:02, Miro Hrončok wrote:
> Hello Pythonistas,
> since Fedora 27, we have been patching Python to install packages to 
> /usr/local/lib(64)/python3.X/site-packages if not in rpmbuild:
>
> https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe
>
> The patch was a pragmatic hack in the distutils module that proved to be 
> working quite reliably over the years. Unfortunately, the distutils module is 
> deprecated and is going to be removed from Python 3.12:
>
> https://www.python.org/dev/peps/pep-0632/
>
> Working with the upstreams of setuptools, pip, and Python, as well as other 
> Linux distros Python maintainers (Arch, Debian), we have now changed the way 
> the patch works to patch the sysconfig module instead, changing the 
> posix_prefix install scheme to use /usr/local paths.
>
> See https://discuss.python.org/t/pep-632-deprecate-distutils-module/5134/104 
> for details.
>
> This change is aligned with the upstream plan for Python 3.11 to offer 
> distributors a way to supply their installation schemes to sysconfig:
>
> https://bugs.python.org/issue43976
>
> Nothing should change for Fedora packages. If you encounter problems, do let us 
> know and we'll try to help solve them.
>
> Software that reads the posix_prefix install_scheme on runtime to figure out 
> where to install stuff should behave the right way by default. OTOH software 
> that reads it to figure out where to load stuff from might need changes, see 
> e.g. the change we did in dnf: 
> https://github.com/rpm-software-management/dnf/pull/1782
>
> Relevant commit: https://src.fedoraproject.org/rpms/python3.10/c/47935cfb9870

This change apparently breaks virtualenv and pipenv:

https://bugzilla.redhat.com/2011455 (virtualenv)

https://bugzilla.redhat.com/2011475 (pipenv)

I'll try to fix that ASAP and if I am unable, will consider temporarily 
reverting the change.

[Miro Hrončok]

On 05. 10. 21 16:02, Miro Hrončok wrote:
> Hello Pythonistas,
> since Fedora 27, we have been patching Python to install packages to 
> /usr/local/lib(64)/python3.X/site-packages if not in rpmbuild:
>
> https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe
>
> The patch was a pragmatic hack in the distutils module that proved to be 
> working quite reliably over the years. Unfortunately, the distutils module is 
> deprecated and is going to be removed from Python 3.12:
>
> https://www.python.org/dev/peps/pep-0632/
>
> Working with the upstreams of setuptools, pip, and Python, as well as other 
> Linux distros Python maintainers (Arch, Debian), we have now changed the way 
> the patch works to patch the sysconfig module instead, changing the 
> posix_prefix install scheme to use /usr/local paths.
>
> See https://discuss.python.org/t/pep-632-deprecate-distutils-module/5134/104 
> for details.
>
> This change is aligned with the upstream plan for Python 3.11 to offer 
> distributors a way to supply their installation schemes to sysconfig:
>
> https://bugs.python.org/issue43976
>
> Nothing should change for Fedora packages. If you encounter problems, do let us 
> know and we'll try to help solve them.
>
> Software that reads the posix_prefix install_scheme on runtime to figure out 
> where to install stuff should behave the right way by default. OTOH software 
> that reads it to figure out where to load stuff from might need changes, see 
> e.g. the change we did in dnf: 
> https://github.com/rpm-software-management/dnf/pull/1782
>
> Relevant commit: https://src.fedoraproject.org/rpms/python3.10/c/47935cfb9870
>
> Bodhi update: https://bodhi.fedoraproject.org/updates/FEDORA-2021-31c7a2fca8
>
> We might later introduce this to older Python versions as well, but we are not 
> planning to backport this to older Fedora releases.

This also apparently breaks the compose:

https://bugzilla.redhat.com/show_bug.cgi?id=2012513

The situation seem to be the same as dnf. syconfig used to figure out where to 
find Python modules, not where to install them.