Python 3.4, ensurepip and wheels

[Bohuslav Kabrda]

----- Original Message -----
> On 12/11/2013 04:52 PM, Bohuslav Kabrda wrote:
> >> So, here's a crazy thought: what if, rather than copying the installed
> >> files directly into the virtual environment, we reverse engineered a
> >> wheel archive *dynamically* from the system install and then installed
> >> from that? That would avoid the problems with trying to bypass pip's
> >> script generation, while still bootstrapping new virtual environments
> >> based on the installed versions of pip and setuptools.

My attempt on recreating wheels: https://github.com/bkabrda/rewheel
This implementation doesn't handle included scripts, although it should work fine with autogenerated entrypoints.

> > There is a slight problem here:
> > In a stable Fedora release, we typically won't change pip's version, but
> > rather backport security patches and the version will stay the same. So if
> > you reconstruct the wheel from that, running "ensurepip --upgrade" in
> > virtualenvs won't actually do anything, since pip will see the same
> > version in virtualenv as is in the system and do nothing. There are few
> > possible solutions to this:
> > - if version in system is the same as in virtualenv, check file checksums
> > and if some differ, reinstall
> > - if version in system is the same as in virtualenv, uninstall pip in
> > virtualenv and then reinstall the one from the system
> > - (ugly, not upstream acceptable) do "rpm -q python3-pip" and use the
> > Fedora release as a "build tag" for reconstructed wheel (similar to what I
> > proposed in the "bundled wheels" solution)
> > If upstream (you :)) accepts any of these types of behaviour (the second
> > one seems the easiest and should work fine) in the PEP, I'll work on that.
> > Quite literally this time, we would reinvent the wheel :)
> 
> Ah, true, this is the thing we're planning to fix by formally adding
> local version numbering as part of the upstream data model in metadata
> 2.0 (that is, pip, setuptools, etc would actually be *required* to cope
> properly if the version metadata contained a number like "1.5-2" rather
> than just "1.5", so that patched downstream rebuilds can have different
> version numbers in the metadata than vanilla upstream versions). (This
> is already in the draft metadata spec, but I haven't quite worked
> through all the implications for version pinning yet)
> 
> You may even want to try that out (patching the pip and setuptools
> version numbers to include a release suffix) and see if it already
> works, since the scheme we're making official in metadata 2.0 is based
> on the way pkg_resources.parse_version() already behaves:
> 
> >>> pkg_resources.parse_version("1.5") < pkg_resources.parse_version("1.5-1")
> >>> < pkg_resources.parse_version("1.5-2")
> True
> 
> Failing that, I think the uninstall/reinstall trick (preferably combined
> with some kind of marker so it only happens when there *is* an upgrade
> to be installed) sounds like a reasonable workaround until we get
> metadata 2.0 formalised upstream (which will hopefully happen before pip
> 1.6 is released mid next year).

Well yeah, my point is that there is no upstream-acceptable way other than checking the file hashes by ensurepip, is there? If I wouldn't want to check file hashes, I'd have to query RPM for release - or is there some other way you're thinking of?
So I'm basically back at the original idea, you got me :) Although the wheel reconstruction feels a lot cleaner than just copying files and has some pros like the entrypoint generation.

> Cheers,
> Nick.
> 
> --
> Nick Coghlan
> Red Hat Hosted & Shared Services
> Software Engineering & Development, Brisbane
> 
> Testing Solutions Team Lead
> Beaker Development Lead (http://beaker-project.org/)

-- 
Regards,
Bohuslav "Slavek" Kabrda.