Python 3 as a Default - Status

Tue Jan 20 13:22:25 UTC 2015

Hi all,
since the "Python 3 as a Default" change [1] has been accepted a while ago and is scheduled for F22, I'd like to share with you the status.

The proposed change [1] mentions several goals that should be reached to pronounce python3 the "default":
1) DNF is the default package manager instead of Yum, which only works with Python 2
2) Python 3 is the only Python implementation in the minimal buildroot
3) Python 3 is the only Python implementation on the LiveCD
4) Anaconda and all of its dependencies run on Python 3
5) cloud-init and all of its dependencies run on Python 3

Before I go through the 5 goals, let me explain the approach that has been taken so far:
There are basically two types of packages (from Fedora's point of view) that depend on Python: "libraries" and "applications" (note that the distinction is intentionally not very clear, there are packages that are both)
"Libraries" are, simply put, Python extension modules, those that live in %{python2_sitelib} and %{python2_sitearch} in python2 builds. "Libraries" were receiving both upstream and downstream care from the people working on the change - upstream porting and downstream python3- subpackage additions, to also put files in %{python3_sitelib} and %{python3_sitearch}. We could afford to do the subpackage additions downstream during F21 lifecycle, since this doesn't change anything, it's just one more binary RPM that's spit out of the SRPM build process.
"Applications" are stuff that users run (an important characteristic is that users don't care under which Python the application runs) - like Anaconda. "Applications" have been receiving some upstream patches, but weren't rebuilt with Python 3 yet. The reason for not rebuilding yet is that we first wanted to make sure that we've ported all "applications" upstream to be able to switch them all to Python 3 in Fedora at once - we wanted to be sure we wouldn't introduce both Python runtimes to livemedia, cloudimages, etc. As of now, Python 2 is still the suggested Python runtime to build "applications" against. We will start filing bugs to rebuild against Python 3 once we're sure that the switch can safely happen (which I think is now according to the points below).

Let's go through the 5 goals:
1) DNF will be the default package manager for F22 [2], so everything is ok here.
2) One of our goals was to not make minimal buildroot larger. The only package from minimal buildroot requiring python (python-libs to be precise) is gdb, which is compatible with python3 in upstream (but it's considered to be an "application", so it hasn't been rebuilt yet). So minimal buildroot is fine.
3) As for LiveCD, we now have more of these - Workstation, Server and all the various flavours and spins. Let me go through Workstation and Server:
Workstation:
Fedora LiveCDs have, since at least Fedora 20, included both Python 2 and Python 3 runtimes (the primary reason for having Python 3 back then was AFAICS speech-dispatcher, which is Python 3 only in upstream). Although we may not be able to port all libraries and applications (but we may, there's still chance!) from workstation livecd to Python 3, the fact that it already ships both runtimes makes me think that we should build as much apps as possible with Python 3.
Server:
Because of giants like samba and freeipa, I think we won't be able to achieve python2-free server livecd for F22. But as it is with Workstation, I think we should proceed and build as much with Python 3 as possible.
4) Anaconda is work in progress. I'm communicating with Anaconda devs quite regularly and everything looks promising.
5) cloud-init is a problem. Basically, Python 3 cloud-init is the only thing blocking the cloud images (*). Other packages are ready or being worked on. The problem here is that cloud-init upstream is really unresponsive about Python 3 porting (patch is submitted in their bug tracker [3]) - if someone knows these people, please help us by pinging them.

Attached to this mail, you'll find 3 files (fedora-cloud-base.txt, fedora-install-server.txt, fedora-live-workstation.txt). Each one of them is a result of automated script that tells with quite a good certainty what Python dependent packages are on a livecd/image generated by a kickstart with the same name.
The files have two sections: "Good" and "Bad". "Good" packages are either "libraries" that have python3- subpackage or "applications" that have already been built with Python 3. "Bad" packages are either "libraries" not having python3- subpackage or "applications" built with Python 2. Note that lots of applications are now "bad" even though their code is Python 3 compatible, they just haven't been built with Python 3 yet. Also, "Bad" contains packages that will not be ported at all and will disappear from livecd, e.g. pyliblzma or yum.
We're tracking all packages needed for Workstation and Cloud images at [4] - feel free to grab anything unassigned there or help us pushing our patches upstream. All help is welcome!

All in all, I think we're in a good shape and I suggest that we move on by building all current "applications" (those that are Python 3 compatbile in upstream) with Python 3. I already suggested a change to Python guidelines that all *new* "applications" should be built with Python 3 if possible [5].

(*) Assuming cloud images are created out of fedora-cloud-base.ks, which I'd like someone to confirm.

-- 
Regards,
Slavek Kabrda

[1] http://fedoraproject.org/wiki/Changes/Python_3_as_Default
[2] http://fedoraproject.org/wiki/Changes/ReplaceYumWithDNF
[3] https://code.launchpad.net/~harlowja/cloud-init/py2-3/+merge/225240
[4] https://fedoraproject.org/wiki/User:Churchyard/python3
[5] https://fedorahosted.org/fpc/ticket/475
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: fedora-cloud-base.txt
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20150120/7741d763/attachment-0003.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: fedora-install-server.txt
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20150120/7741d763/attachment-0004.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: fedora-live-workstation.txt
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20150120/7741d763/attachment-0005.txt>