On 27 April 2017 at 11:47, Nico Kadel-Garcia <nkadel(a)gmail.com> wrote:
On Wed, Apr 26, 2017 at 7:13 AM, Charalampos Stratakis
> At the present time, running sudo pip3 in Fedora is not safe.
> Pip shares its installation directory with dnf, can remove
> dnf-managed files and generally break the Python 3 interpreter.
This is true of every version of pip, not merely pip3. It was also
true of CPAN, and of many gradle, maven, and ant working environments
that semi-randomly collate the very latest versions of indeterminate
components and spray them on top of your system workspace with their
own quite distinct ideas about packaging and versioning.
There is no solution to this problem, except to use tools like
"pyvenv" to set aside secluded workspaces for Python modules and their
dependencies assembly. So, for most developers, they *should not* use
"sudo pip". They should remain in user space, or possibly in shared
workspaces, to avoid overwriting system components.
Nothing is changing in terms of *recommended* practices. This change
proposal is entirely about harm mitigation for the cases where users
*do* run "sudo pip ...", either because that's their instinctive
reaction to a permissions error, or because some misguided
installation instructions for a 3rd party package advised them to do
Debian and derivatives already mitigate the potential harm for these
cases by requiring the "--install-layout=deb" option to be passed to
get distutils to install into the system directories rather than doing
it by default: https://wiki.debian.org/Python#Deviations_from_upstream
Their approach means that any harm caused by "sudo pip install X" can
subsequently be fully reversed by doing "sudo pip uninstall X".
At this moment, this is NOT true for Fedora and derivatives. Instead,
the remediation step here is "sudo pip uninstall X && sudo dnf
reinstall <something>" where you have to:
1. Figure out what "<something>" needs to be
2. Hope that whatever you broke didn't affect your ability to run
"sudo dnf reinstall"
> Our first attempt to make sudo pip safe on Fedora  was
> based on using a different binary (/usr/libexec/system-python)
> for RPM packaging purposes and changing the behavior
> of /usr/bin/python3 (and pip that uses its settings) to install
> under /usr/local by default. Switching all the Python 3 packages
Oh, Oh my. The extent to which this unworkable is... fascinating.
Nico, the contemptuous tone you have adopted in this post is entirely
inappropriate. Please give other list participants the courtesy of
assuming expertise at least equivalent to your own, rather than
engaging in attempted displays of assumed superiority.
will inevitably install incompatible future versions of critical
packages which are incompatible with the Fedora system packages. And
as soon as you install one that breaks RPM, if /usr/local/ if ahead of
/usr in your search path for modules, you are *screwed*.
This isn't true, as long as the fix is as simple as using pip to
uninstall whatever problematic package you just installed.
At the moment, that isn't the case on Fedora and derivatives, since
"sudo pip install ..." can actually *corrupt* a system package
installation, rather than merely shadowing it at import time the way
it does on Debian et al.
/usr/local is first, well, an RPM update may introduce a component
that is incompatible with the pip installed modules at arbitrary
The goal of this change isn't to allow co-existence of arbitrary
upstream and downstream versions as supported combinations - it's to
make sure that forced installation of an incompatible upstream version
is an easily reversible error, rather than a potentially
> We decided to try a different approach. The main idea is to
> an ongoing RPM build and modify the behavior of the Python 3
> executable only when in normal user environment so that RPM builds
> won't be affected at all.
. You're inventing a workflow that isn't
supported by *anyone*.
The problem is that the default behaviour needs to change (to mitigate
the potential harm caused by a relatively common end user error), but
unlike Debian package builds (which know they need to set
"--install-layout=deb"), most RPM builds for Python packages currently
rely on that default behaviour as part of the build process. Changing
the default indiscriminately would thus be a breaking change that not
only affects Fedora RPM builds, but also COPR and other third party
So we need a solution that retains the existing behaviour for RPM
builds, but changes the behaviour to be less potentially destructive
when run interactively at a shell prompt, or as part of a
configuration management script.
> The adjustment of the behavior can be done on different levels.
> The first option is to set the sys.prefix variable to /usr/local
> when the interpreter is initialized. This will affect
> all the install methods, but the solution can cause some
> problems in applications that rely on the value of sys.prefix.
> A prototype of this implementation can be seen here .
I'm sorry, but.... ouch. Please stop trying to implement "pip install"
as a system installation tool.
That is not the goal of the change.
The effort would be better spent
upgrading and enhancing "py2pack".
Improving the level of commonality and collaboration between Fedora &
Mageia's pyp2rpm and openSUSE's py2pack would indeed be desirable, but
it's entirely orthogonal to the harm mitigation exercise being
> The other possibility is to limit the pip install location
> to distutils and pip . This is the "safer" option, but does
> not cover all corner cases. For example, Python software built
> locally using cmake or similar tools will be installed into
> /usr/lib and can conflict with system tools. Debian chose to
> implement similar solution.
It's local environment specific, unstable software sensitive to
arbitrary and unmanaged upstream dependency updates, and extremely
sensitive to local system pip modules. It *does not belong* in
/usr/lib. Because the components are modular and bundled in a non-RPM
compatible fashion, it behooves developers to use a tool to segregate
the tools as much as feasible from the Fedora underlying
infrastructure. i.e., use pyvenv to build them in a contained
environment segregated from the system files.
Helping users that are already doing the right thing isn't the problem
at hand - this is about making it easier for users to recover from
doing the *wrong* thing.
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia