Packaging Python 3.4.0

[Nick Coghlan]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/06/2013 12:50 AM, Toshio Kuratomi wrote:
> On Tue, Nov 05, 2013 at 07:29:35AM -0500, Bohuslav Kabrda wrote:
>> -- Maybe remove the bundled pip (although this goes against the
>> pep) and use python3-pip. E.g. everything would work as expected,
>> but under the wraps, python3-pip package would be used. So when
>> doing security updates, we'd just fix python3-pip. It is however
>> true that if Fedora's pip would be different from the upstream
>> bundled one, users may experience some behaviour differencies.
>> I'd like to hear your opinions on how we should approach this.
>> 
> We should definitely do this.  pip itself bundles code that we've
> had to unbundle locally -- maintaining that morass of software
> again (it's also bundled inside of virtualenv...) is going to lead
> to a lot of work anytime an update needs to happen.
> 
> However, talking with ncoghlan it may not be simple.  The case that
> makes things tricky is pyvenv.  Upstream python wants to protect
> venv's from changes to code on the system.  ie: if a user installs
> a backwards incompatible pip on the system, that should not affect
> the pip inside the existing venvs.  Equally, if there are security
> issues found with pip, those changes would not be propogated to the
> pip inside the venvs.

Right - the wording in the PEP is a little stronger than the real
requirement of "don't break pyvenv installing pip into virtual
environments", because we wanted to make it clear that providing an
appropriate wheel file is the only upstream supported way to make
pyvenv work as expected in 3.4+.

There are actually other ways to make it work, the approach in the PEP
is just the one that we know for sure will work properly and will
cover in the upstream test suite. (Leaving room for those alternative
approaches is also why the PEP explicitly states that it is OK for
system pip upgrades to affect later venv creation)

> ncoghlan proposed that we create wheels in the pip package (and all
> the packages that pip bundles that we have to unbundle) and then
> when ensurepip installs into a venv, it copies those wheels in
> there.

Technically, ensurepip uses the wheel to install pip into the venv,
rather than copying the wheel anywhere.

> I'm wondering if we could do a little better.  Instead of keeping
> around two copies of all this software on the system, if we can
> reconstruct a wheel zip (or possibly just copy the files in) from
> the wheel metadata and files on the system.   I know that sysadmins
> and developers will hotfix software on the system for issues that
> they care about.  Having those changes propogate to newly installed
> venvs seems like a better plan than having the venvs get the old
> copies that were built at rpm build time.

Yep, that should be entirely feasible - wheel installation is just a
matter of copying files, so reversing that process should work fine.
You could also potentially skip the interim wheel step and just copy
files into the venv based on the pip and setuptools RECORD files. The
one thing to watch out for will be the files *generated* at install
time - you won't want to copy those back into the wheel file or
virtual environment (that should just be the script files in /usr/bin,
though).

If you get this working for Fedora, I could definitely see us
accepting an upstream patch to make pyvenv work that way by default in
3.5+ (i.e. copying a system installed pip if available, and only fall
back to using the bundled wheel if there's no system pip). We just
didn't want to make blazing that trail (since it's not officially
supported by pip at this point in time) a precondition for the PEP 453
implementation :)

Cheers,
Nick.

- -- 
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane

Testing Solutions Team Lead
Beaker Development Lead (http://beaker-project.org/)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJSec6HAAoJEHEkJo9fMO/L4wsH/iIdm6FVrMGwhEgYLOWn1RnA
qHDy5pJ4o5QHXdSW6LXrU8/W9BTg3DwsqyWs0gThKOmzXh7neol018EYzmbK1mVY
BRTEclqVGwXllPsnOK5VMZV+i3gVDF35zHjEX62u0JpYXB16An2OwYIbk05hsUnG
J+mE21ot1g1NdNSWL9rw4ftBA/k9Rc6n7Hn6qyf39VMls1AWq3QKNYj/bvDvNoaJ
J0ZVZ5Yg42Fh5zYAq/+JYPVtfNqCcW3wkCrLOpvimcnAwCJBKh+qUIM0EQefs1v6
putQFO6VnwycuM3VgXO3hjPkxgba54KPCg4crd5iR8BTvxRwmL5xzeUCr7JS1SI=
=exZP
-----END PGP SIGNATURE-----