PyPy gains Python 3 support!
by David Malcolm
Anyone want to package this for Fedora? It would be a different srpm to
the pypy.srpm since the sources are different, but presumably the
pypy.spec file could be used as a base.
10 years, 2 months
Re: Multirelease effort: Moving to Python 3
by Toshio Kuratomi
Hey, I'm forwarding this on from Barry Warsaw. He's a contributor to
Debian and Ubuntu and has been very active in managing porting to python3
both within those communities and as an upstream python committer. So he's
experienced in a lot of the work that we're going to be embarking upon to
move this forward.
-Toshio
Begin forwarded message:
Date: Mon, 22 Jul 2013 18:34:01 -0400
From: Barry Warsaw <barry(a)python.org>
To: devel(a)lists.fedoraproject.org
Cc: Python-devel(a)lists.fedoraproject.org
Subject: Re: Multirelease effort: Moving to Python 3
Organization: Damn Crazy Followers of the Horn
X-Newsreader: Claws Mail 3.8.1 (GTK+ 2.24.20; x86_64-pc-linux-gnu)
On Jul 22, 2013, at 03:31 PM, Miloslav Trmač wrote:
>IMHO this is precisely the kind of integration where a distribution is
>perfectly justified to carry local patches, even against explicit
>wishes of upstream if necessary (though cooperating with upstream and
>finding a way to integrate the patch upstream is much better). We
>shouldn't block the transition just because a one or two upstreams
>refuse to port (or, more likely, a dozen or two upstreams do not exist
>any more).
In my own experience, most upstreams are very grateful for contributions which
improve/add the Python 3 support. They will often work with you to craft your
changes so they are more easily accepted upstream. I've encountered only one
or two cases of "unfriendly" upstreams who either can't or won't cooperate
with a distro's contributions here. Most know that we're past the Python 3
tipping point; you either support it or risk being irrelevant.
As for abandoned upstreams, it's almost always better to find a replacement
and port those packages which depend on the abandonware. A good example is
oauth, which hasn't been updated since 2009. oauthlib is a better library
*anyway* and the fact that it is Python 3 compatible is definitely a plus.
The APIs are different, but not so much so that porting isn't possible.
Tougher are things like pygtk -> gobject introspection, but a lot of that work
has been done, so helping those projects along benefits the entire Python
ecosystem. You also have to consider big frameworks, but some of the really
key ones like Twisted and Django are already well on their way to Python 3.
One tip for carrying distro-specific patches. You sometimes have to do this
because while upstream may accept your changes, they can often be slower to do
releases than what you need. Try to at least get upstream to commit the
changes to their VCS so you know the patches are blessed, and that once the
upstream releases, you can drop your deltas. Carrying a VCS derived patch is
way better than carrying one that upstream disagrees with. (Sometimes
upstream even knows better, as was the case with my first dbus Python 3 port
:).
You should really view distro-specific patches as temporary if absolutely
required, and avoid them if at all possible.
Cheers,
-Barry
----- End forwarded message -----
10 years, 2 months
Multirelease effort: Moving to Python 3
by Bohuslav Kabrda
Hi all,
as a new Fedora Python maintainer, I have set myself a goal of moving Fedora to Python 3 as a default. This is going to be a multirelease effort that is going to affect lots of Fedora parts. Since we will need to switch default package manager from Yum to DNF (which is supposed to work with Python 3), we will need to wait for that. I've been told that DNF should be default in F22, so that's my target, too. That should also give everyone else plenty of time to work on other essential packages to make this happen.
Here is my analysis/proposal:
Before switching, we need to make sure that everything "important" (*) is Python 3 compatible. There are three steps I see in this transition:
1) Getting rid of Python 2 in mock minimal buildroot.
2) Porting Anaconda to Python 3.
3) Making all livecd packages depend on Python 3 by default (and eventually getting rid of Python 2 from livecd) - this will also require switching from Yum to DNF as a default, that is supposed to support Python 3.
( 4) Making as much of the remaining packages Python 3 compatible )
In past few days, I've been going through packages that are part of the above steps. I have reported numerous bugs asking upstream and/or Fedora maintainers for help with porting to Python 3. We have some spare cycles in our small Python packaging team, that we will try to provide to whoever needs them most, but we're limited and we'll have to rely on the upstreams to do most of the work. I'm attaching a document with list of packages that need porting with some notes/links to opened bugs. Sometime soon, I'll open a tracking bug for this, so that everyone can see where we are quickly.
(*) I call these "important" packages (in terms of being important for the Python 3 switch)
>From packaging point of view, this will probably require:
1) Renaming python package to python2
2) Renaming python3 package to python
3) Switching the %{?with_python3} conditionals in specfiles to %{?with_python2} (we will probably create a script to automate this, at least partially)
FAQ:
Q: Why do we need to switch to Python 3?
A: Because Python 2 is old, slower, less pythonic, doesn't get any more functionality and it won't be that long before the official upstream support ends [1]
Q: How do I port to Python 3?
A: There are tons of tutorials and howtos about porting and the differencies in general. E.g. [2] (general), [3] (c-extensions)
Q: What about Python 2?
A: We will maintain that at least as long as upstream supports it. After that, I'd prefer dropping it, but since I know there will be people wanting to keep it around, I'll gladly give the maintenance to someone else.
I'll be glad to answer all your questions and discuss the above points. Nothing is set into stone and I'd love to hear your ideas and comments.
Thanks for reading this through!
Slavek.
--
Regards,
Bohuslav "Slavek" Kabrda.
[1] http://www.python.org/dev/peps/pep-0373/
[2] http://docs.python.org/dev/howto/pyporting.html
[3] http://docs.python.org/3/howto/cporting.html
10 years, 2 months
Re: Multirelease effort: Moving to Python 3
by Bohuslav Kabrda
----- Original Message -----
> On Thu, Jul 18, 2013 at 11:24:22AM -0400, Bohuslav Kabrda wrote:
> > 3) Making all livecd packages depend on Python 3 by default (and
> > eventually getting rid of Python 2 from livecd) - this will also require
> > switching from Yum to DNF as a default, that is supposed to support Python
>
> I have a concern about bloating @core and by extension the cloud image.
> Right now, python is about 5% of the total on-disk usage. I'd hate to see
> that go to 10%. Therefore, I'd like to see a goal of making the transition
> for usage in the base cloud image go entirely from python2 to python3 in
> one release cycle.
>
> (Roughly, that's @core + cloud-init, which isn't currently on your list.)
>
Doing everything in one shot sounds reasonable from this POV, I'll try to put this into my plan (I'll go through cloud-init the same way I went through livecd and post my findings here).
>
> --
> Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ <mattdm(a)fedoraproject.org>
--
Regards,
Bohuslav "Slavek" Kabrda.
10 years, 2 months
Re: Multirelease effort: Moving to Python 3
by Toshio Kuratomi
On Fri, Jul 19, 2013 at 05:30:36PM -0400, Matthew Miller wrote:
> On Fri, Jul 19, 2013 at 01:45:41PM -0700, Toshio Kuratomi wrote:
> > * python-cheetah -- Development slowed way down after they made their last
> > release in 2010 and announced that the next release cheetah-3.0 would
> > include python3 support. Probably need to contact upstream about this and
> > may need to prepare the patch to do the port.
>
> What I'd _really_ like to do is get cheetah factored out of cloud-init.
> (https://bugzilla.redhat.com/show_bug.cgi?id=974327). It brings in a whole
> dependency chain of which python2 vs. python3 is the least of the troubles.
>
If your needs are very minimal, python3-tempita might be a good choice.
If you actually do need more features than that, python-mako and
python-jinja2 are popular. Note that both of those have a few deps (but
hopefully not as bad as cheetah). (Also -- the python3 version of mako has
less deps than the python2 version... I think that it just because those
deps haven't been ported to python3 yet and the package can operate with
reduced functionality without them. the deps fo the python3 version might
expand i nthe future).
-Toshio
10 years, 2 months
Re: Multirelease effort: Moving to Python 3
by Toshio Kuratomi
On Fri, Jul 19, 2013 at 03:29:57PM -0400, Matthew Miller wrote:
> On Thu, Jul 18, 2013 at 11:24:22AM -0400, Bohuslav Kabrda wrote:
> > 3) Making all livecd packages depend on Python 3 by default (and
> > eventually getting rid of Python 2 from livecd) - this will also require
> > switching from Yum to DNF as a default, that is supposed to support Python
>
> I have a concern about bloating @core and by extension the cloud image.
> Right now, python is about 5% of the total on-disk usage. I'd hate to see
> that go to 10%. Therefore, I'd like to see a goal of making the transition
> for usage in the base cloud image go entirely from python2 to python3 in
> one release cycle.
>
> (Roughly, that's @core + cloud-init, which isn't currently on your list.)
>
This seems like a very very worthy goal. I took a brief look at @core
(I didn't recurse down the whole dep chain so hopefully I didn't miss an
iceberg) and that looks doable. The switch from yum to DNF is the big
thing.
cloud-init is python so it might need to be ported in and of itself. Some
of its dependencies also don't have python3 ports:
* configobj -- no upstream python3 port and the third-party port never made
it to a release (the code may work but it's unmaintained). If someone
is ready to become the upstream for this, indications are that it would be
welcomed by both the port's original author and the original ConfigObj
maintainer.
* policycoreutils-python -- not sure if this interfaces with the C libraries
or is about implementing higher level functionality. Nearly anytime you
interface with C you have to make choices about bytes vs text and problems
with encoding. That could slow up this work.
* python-boto -- Looks like there's a badly out-of-date branch in the
upstream SCM repository to add python3 support. That may be based on an
API incompatible update, though (I see that there's work on boto3...)
boto is going to need to talk over the wire to AWS so there's going to be
a lot of places where bytes vs text decisions have to be made. All in
all, this package feels like it might be the hardest piece of the
cloud-init porting.
* python-cheetah -- Development slowed way down after they made their last
release in 2010 and announced that the next release cheetah-3.0 would
include python3 support. Probably need to contact upstream about this and
may need to prepare the patch to do the port.
-Toshio
10 years, 2 months
Re: Multirelease effort: Moving to Python 3
by Bohuslav Kabrda
----- Original Message -----
> On Thu, Jul 18, 2013 at 11:24:22AM -0400, Bohuslav Kabrda wrote:
> > Hi all,
> > as a new Fedora Python maintainer, I have set myself a goal of
> > moving Fedora to Python 3 as a default.
> That's a worthy goal!
>
> > From packaging point of view, this will probably require:
> > 1) Renaming python package to python2
> > 2) Renaming python3 package to python
> > 3) Switching the %{?with_python3} conditionals in specfiles to
> > %{?with_python2} (we will probably create a script to automate this, at
> > least partially)
>
> But that seems like pointless busywork and churn. Sure, when python3
> is ubiquitous, those checks can be dropped or reversed... But right
> now it would be better to focus on actual python3 versions of all
> packages — we're not there yet.
>
Yes, current efforts should be directed to porting for Python 3, the packaging points I mentioned will really be the very last step.
Thanks,
Slavek.
> Zbyszek
> --
> they are not broken. they are refucktored
> -- alxchk
--
Regards,
Bohuslav "Slavek" Kabrda.
10 years, 2 months
Issue packaging python lib into RPM due to conflicting __init__.py
by Braddock
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi folks,
I'm attempting to make an RPM of the Python package backports.lzma for
a One Laptop Per Child project using bdist_rpm.
tar xzf backports.lzma-0.0.2.tar.gz
cd backports.lzma-0.0.2
python setup.py bdist_rpm
My problem is that more than one python library wants to live under
the "backports" directory, resulting in a backports/__init__.py file
conflict on SOME platforms as each RPM attempts to create the
backports/__init__.py file in its parent directory.
For example, backports.lzma RPM conflicts with
python-backports-ssl_match_hostname
file /usr/lib/python2.7/site-packages/backports/__init__.py from
install of backports.lzma-0.0.2-1.armv7hl conflicts with file from
package python-backports-ssl_match_hostname-3.2-0.3.a3.fc18.noarch
I am uncertain how to resolve this. Is there a way for an RPM to only
create the backports/__init__.py file if it does not already exist?
There are a number of packages which would want to live under the
backports/ module.
Any advice appreciated.
Thanks,
Braddock Gaskill
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQEcBAEBAgAGBQJR2ZXJAAoJEHWLR/DQzlZuEJQH/2gWjk8F/tAiPBG1lmKFLcsR
Sbmv77RUWiFOHt7Kti+C1uzkig1Ub9sLXdcFXdI6klGkovo0WunMVBsFD5iS80pT
Z7QYVsXP9csDFkywHr49WEP6p5YlIey7ypj7JYJVXYVfSLZde3h9ooc2PVElOCyJ
eVyZzdZ/FBz/7Erpgv+iqp5oUHuP9LvTwCn4cw3eytAiH6CPs6DG6RfRBvlkyeYy
Q9Qw44coJLnoMVKUuXdkfbvRA72oaZcgmRJv7PMf74t3Ii0yVsIOHGYCJA002zLS
CNwSeelZm2w2A21WE2mwt8C3DiA6MZpotj8CMRLGH/zhUAUXxWjk/i4GmwCxzmw=
=eU0m
-----END PGP SIGNATURE-----
10 years, 3 months
python-setuptools-0.7.7 built for rawhide; hold off on mass rebuilds
by Toshio Kuratomi
I've just built python-setuptools-0.7.7 for rawhide. As mentioned in the
Changes Planning page:
https://fedoraproject.org/wiki/Changes/Python_setuptools_0.7#Summary
This is a switch of code base so there is the possibility of failures
however it isn't a major change of API so it should hopefully be mostly
transparent.
*Do* go about rebuilding python packages as usual and let me know if you see
any failures that you think could be attributed to the new setuptools.
*Do not* start mass rebuilding all your python packages with the expectation
that this is the last major change for this release cycle -- there's
a setuptools-0.8 package that is due out in a week or two and I would like
to talk with ncoghlan about whether we should push that into Fedora 20. The
changes from 0.7 to 0.8 seem to be similar in scope as the 0.6 to 0.7 switch
(little to no API changes, large changes to the code base itself to make it
more maintainable) so it probably makes sense to switch over to that for F20
(I'll build the beta2 for rawhide if Nick agrees that that is a good plan).
*Note*: python packages do not need to rebuild against this package.
However, I hope we can include python packages in any mass rebuild for F20
to make sure that there aren't any major new bugs in the new package that
would cause other packages to unexpectedly FTBFS later on.
-Toshio
10 years, 3 months
