serious conflicts between python pks installed via yum vs pip

[Toshio Kuratomi]

On Fri, Feb 10, 2012 at 11:38:20AM -0500, Neal Becker wrote:
> 80 wrote:
> 
> > 2012/2/10 Neal Becker <ndbecker2 at gmail.com>:
> >> I've seen this repeatedly, with often very serious consequences (complete
> >> failure of update from f15->f16 for example).
> >>
> >> Between packages installed via pip, and packages installed via yum, some
> >> packages seem to switch between
> >>
> >> e.g.,
> >> numpy-1.6.1-py2.7.egg-info
> >>
> >> being installed as a file and the same name being installed as a directory.
> >> This can cause subsequent rpm update (via cpio) to abort.
> >>
> >> I don't know what's causing this, but IMO this is a serious problem.
> >>
> >> --
> >> devel mailing list
> >> devel at lists.fedoraproject.org
> >> https://admin.fedoraproject.org/mailman/listinfo/devel
> > 
> > Never use pip outside an isolated environment (use virtualenv)
> > 
> > H.
> 
> Really?  This is the only answer?  Can't we tweek rpm/yum to accomodate this?  
> Does anyone understand what is causing it?  Why would pip install the egg-info 
> differently than rpm?
>
If a package uses distutils (from the stdlib) to build, it will create an
egg-info file with the metadata.  If the package is installed with
setuptools (from python-setuptools), it will create and egg-info directory
with the metadata.

Some upstream packages allow you to use either distutils or setuptools with
code like:

try:
    from setuptools import setup
except ImportError:
    from distutils.core import setup

(numpy is more complex than this.... looks like with numpy, pip imports
setuptools and therefore the numpy setup.py uses setuptools' setup whereas
python setup.py [COMMAND] does not import setuptools so it uses distutils)

This has the unhappy side effect that sometimes the package is installed
with the egg-info as a file adn sometimes as a directory.

rpm/yum cannot accomodate this easily.  I thought that Panu had posted
a good explanation to the mailing lists many years ago but my search skills
are failing me right now.  IIRC, it's basically that in your rpm
transaction, rpm must install the new archive before removing the old one.
If the archive includes a file that has to replace a directory, this has no
way of succeeding.  (What if the directory contains files, for instance).

Now, there is a way out in this case that doesn't involve the rpm program.
In this case, the numpy rpm could contain either a file or a directory for
the egg-info.  It presently contains a file and thus the failure.  If you
can figure out how to make numpy create directory egg-info in this case,
you'll be able to make an rpm that will install over a pip local install.

Another option would be to configure python (via the PYTHONPATH environment
variable probably) to read modules from
/usr/local/lib{64,}/python2.7/site-packages/ or another directory that isn't
for the OS vendor.  Then have pip install numpy into there.

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20120210/12caa5db/attachment.sig>