4:51 a.m.
Hello,
Here is a message I want to post to devel-announce later. Let me know if
you see anything wrong (or would like to take over python2 maintainership).
---
Intent to orphan Python 2
tl;dr: Unless someone steps up to Python 2 after 2020, we need to start
dropping python2 packages now.
Python 2.7 will reach end of upstream support on 1st of January, 2020,
after almost 10 years (!) of volunteer maintenance.
Fedora still has more than 3000 packages depending on python2 – many
more than we can support without upstream help.
We (rightly) don't have the authority to say "please drop your unneeded
python2 subpackages, or let us drop them for you" [0].
The next best thing we *can* say is: "if Fedora is to keep python2
alive, we won't be the ones doing it – at least not at the current
magnitude".
Here are the details.
The current maintainers of python2 would like to "orphan" the python2
package in 2020 (~ Fedora 32/33):
- Charalampos Stratakis (cstratak)
- Tomáš Orsava (torsava)
- Miro Hrnočok (churchyard)
- Petr Viktorin (pviktori)
- Iryna Schcherbina (ishcherb)
- Michal Cyprian (mcyprian)
- Bohuslav Kabrda (bkabrda)
- David Malcolm (dmalcolm)
- Thomas Spura (tomspur)
As with any orphaning, that leaves two options:
- someone else agrees now to take over in 2020 (keeping in mind this is
a security-critical package and will be abandoned upstream), or
- dependent packages drop support for Python 2.
Unlike most other orphanings, we have some thousands of dependent
packages, so a lot of time and care is required.
In case no one steps up, we'd like to start dropping Python 2 support
from dependent packages *now*, starting with ported libraries on whose
python2 version nothing in Fedora depends. (We keep a list of those at [1].)
Of course, we're ready to make various compromises with interested
packagers, as long as there's an understanding that we won't just
support python2 forever.
If you are a maintainer of anything at [1] we ask you kindly to consider
removing the python2 subpackages.
If no one steps up to maintain python2 after 2020, we're prepared to
package a "legacy" python27 package, similar to what we do for e.g.
python33 [2], to:
- help developers that still need to test against this version
- support exceptionally important non–security critical applications, if
their upstreams don't manage to port to Python 3 in time
[0] https://pagure.io/packaging-committee/issue/753
[1] http://fedora.portingdb.xyz/#legacy-leaf
[2] https://src.fedoraproject.org/rpms/python33/
5:25 a.m.
On Tue, Mar 20, 2018 at 10:51:30AM +0100, Petr Viktorin wrote:
Hello,
Here is a message I want to post to devel-announce later. Let me
know if you see anything wrong (or would like to take over python2
maintainership).
---
Intent to orphan Python 2
First a disclaimer: I'm all for moving to python3, so
I'm not
disagreeing with the general idea, but with the transition plan.
tl;dr: Unless someone steps up to Python 2 after 2020, we need to
start dropping python2 packages now.
I think this is a contentious statement.
"need" is the wrong word.
"One of the ways to prepare for the declared EOS from upsteam is to"
would be more accurate.
And it's just one of the possible ways. Another would be prepare a
conditional in all spec files and simply flip it over before a mass
rebuild for F32 (or whatever).
Python 2.7 will reach end of upstream support on 1st of January,
2020, after almost 10 years (!) of volunteer maintenance.
Fedora still has more than 3000 packages depending on python2 – many
more than we can support without upstream help.
This is also misleading. Those 3000
packages have different upstreams,
and many of those upstreams will support python2 even after python-eol.
We (rightly) don't have the authority to say "please drop
your
unneeded python2 subpackages, or let us drop them for you" [0].
The next best thing we *can* say is: "if Fedora is to keep python2
alive, we won't be the ones doing it – at least not at the current
magnitude".
The way you propose is essentially a mirror of how python3 support
was introduced. For a long time it was hard to use python3 for anything
serious because there was _always_ one or two more unported dependencies.
It took many years to reach this critical mass where everything
"important" was available. This period was harsh on users, because python3
was nice and shiny but people were unable to switch and unable to
_influence_ this state in any significant way, the way that it is hard
to push the whole ecosystem. The proposed model of nipping python2 support
at the edges is the same thing, in reverse. First some leaf packages
are dropped, and then some somewhat more important packages, and then
suddenly it becomes hard to use python2 for anything "serious", even
though it's still "available". I think that's a bad way to proceed for
our users. Let them have a single moment where python2 support goes away.
Announce this moment at least a year in advance. Let them keep running
the last release before that for as long as they need.
If we want to prepare for the python2-eol, IMHO we should first finish
all the steps that don't impact users: convert our tooling, make sure
python2 is not used in build for anything where python3 can be used,
rename packages, fix provides and requires, update she-bangs, add
conditionals to make dropping python2 support easy. In particular,
let's first finish Changes/Avoid_usr_bin_python_in_RPM_Build [1].
[1] https://fedoraproject.org/wiki/Changes/Avoid_usr_bin_python_in_RPM_Build
Zbyszek
6:23 a.m.
On 20.3.2018 11:25, Zbigniew Jędrzejewski-Szmek wrote:
to push the whole ecosystem. The proposed model of nipping python2
support
at the edges is the same thing, in reverse. First some leaf packages
are dropped, and then some somewhat more important packages, and then
suddenly it becomes hard to use python2 for anything "serious", even
though it's still "available". I think that's a bad way to proceed for
our users. Let them have a single moment where python2 support goes away.
Announce this moment at least a year in advance. Let them keep running
the last release before that for as long as they need.
I think there is a fundamental difference of how I see python RPMs in
Fedora and how you do. In my POV, purpose of pythonX-foo packages that
only bring libraries is to provide dependency for an application that
happens to be written in python, runs on pythonX and need the foo
library. If there is no such application packaged in Fedora, I see no
purpose of such package. (Except maybe if such app is packaged in
another repository used by our users. That's a corner case and if the
maintainer of pythonX-foo is aware of that, it can always be used as
argument to keep pythonX-foo.)
We live in an era of pip (and rubygems, and cpan, and npm etc.).
Trying to mirror those systems with RPM packages just because it's
possible is tedious, incomplete and mostly pointless IMHO. An era when
upstream projects were eager to get their tool into the distros has
passed. Having pythonX-foo packaged where no other tool/app/service
needs it is not beneficial (if foo is a library only thing, not a tool
itself). I have this opinion even with python3 packages.
Developers who wants to use python2 for whatever reasons (rational or
emotional ones) can do that on Fedora. We encourage developers to use
virtual environments and pip [1]. Removing leaf python2 packages does
not block them here at all.
Let them have a single moment where python2 support goes away.
Removing all python2 things at one point is impossible. We've tried to
test this with Django [2] on a much smaller scale. (Tens instead of
thousands packages.) It took us too much time and too much energy. At
the end, too much packages were just retired because their maintainers
are "dead". I cannot imagine doing this on current scale in 2020.
We can just retire python2 then and see what breaks. Honestly,
everything would break.
We need to communicate loudly that Python 2 is dead. And this is a way
to do it. If the maintainrs add conditionals for Fedora 33, let them
have it. But every single python2 package that goes away is a good thing
longterm both for Fedora and for the Python ecosystem.
If we want to prepare for the python2-eol, IMHO we should first
finish
all the steps that don't impact users: convert our tooling, make sure
python2 is not used in build for anything where python3 can be used,
rename packages, fix provides and requires, update she-bangs, add
conditionals to make dropping python2 support easy. In particular,
let's first finish Changes/Avoid_usr_bin_python_in_RPM_Build.
Yes, we should focus on that. But if we don't do things in parallel, we
have very little chance to move forward fast enough.
[1]
https://developer.fedoraproject.org/tech/languages/python/python-installa...
[2] https://fedoraproject.org/wiki/Changes/Django20
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
8:45 a.m.
On Tue, Mar 20, 2018 at 12:23:06PM +0100, Miro Hrončok wrote:
On 20.3.2018 11:25, Zbigniew Jędrzejewski-Szmek wrote:
>to push the whole ecosystem. The proposed model of nipping python2 support
>at the edges is the same thing, in reverse. First some leaf packages
>are dropped, and then some somewhat more important packages, and then
>suddenly it becomes hard to use python2 for anything "serious", even
>though it's still "available". I think that's a bad way to proceed
for
>our users. Let them have a single moment where python2 support goes away.
>Announce this moment at least a year in advance. Let them keep running
>the last release before that for as long as they need.
I think there is a fundamental difference of how I see python RPMs
in Fedora and how you do. In my POV, purpose of pythonX-foo packages
that only bring libraries is to provide dependency for an
application that happens to be written in python, runs on pythonX
and need the foo library. If there is no such application packaged
in Fedora, I see no purpose of such package. (Except maybe if such
app is packaged in another repository used by our users. That's a
corner case and if the maintainer of pythonX-foo is aware of that,
it can always be used as argument to keep pythonX-foo.)
Indeed, I'm using those python packages like a dinosaur ;)
But more seriously, I think the distinction between library and app in
the python world is neither clear nor particularly significant. Distro
packaging provides clear benefits in both cases: easy and quick
install, easy and quick uninstall, testing, provenance history,
license checks, general reliability.
So instead of losing packages, I'd like to see more automatic
packaging and updates of packages from pypi and other "forges".
We are making steps in that direction, but not quickly enough.
We live in an era of pip (and rubygems, and cpan, and npm etc.).
Trying to mirror those systems with RPM packages just because it's
possible is tedious, incomplete and mostly pointless IMHO. An era
when upstream projects were eager to get their tool into the distros
has passed. Having pythonX-foo packaged where no other
tool/app/service needs it is not beneficial (if foo is a library
only thing, not a tool itself). I have this opinion even with
python3 packages.
Developers who wants to use python2 for whatever reasons (rational
or emotional ones) can do that on Fedora. We encourage developers to
use virtual environments and pip [1]. Removing leaf python2 packages
does not block them here at all.
> Let them have a single moment where python2 support goes away.
Removing all python2 things at one point is impossible. We've tried
to test this with Django [2] on a much smaller scale. (Tens instead
of thousands packages.) It took us too much time and too much
energy. At the end, too much packages were just retired because
their maintainers are "dead". I cannot imagine doing this on current
scale in 2020.
We can just retire python2 then and see what breaks. Honestly,
everything would break.
I don't think this is entirely comparable. Django20 was
about updating
packages and switching support from python2 to python3. Just dropping
existing support for python2 is something much easier.
Nevertheless, I agree that this still is a very significant chunk of
work at this scale. So spreading this out makes a lot of sense, but
imho python2 subpackages don't need to go away, they can just be
conditionalized.
What about adding conditionals like
%if 0%{?rhel} > 7 || 0%{?fedora} > 31
# Disable python2 build by default
%bcond_with python2
%else
%bcond_without python2
%endif
starting now?
Zbyszek
9:28 a.m.
On 20.3.2018 14:45, Zbigniew Jędrzejewski-Szmek wrote:
Indeed, I'm using those python packages like a dinosaur ;)
:D
What about adding conditionals like
%if 0%{?rhel} > 7 || 0%{?fedora} > 31
# Disable python2 build by default
%bcond_with python2
%else
%bcond_without python2
%endif
starting now?
I am not against that. However currently that number (31) is somewhat
artificial. it's up to the maintainer to choose if it's 28 or 32 (or
anything in between). Should we somehow recommend a specific Fedora
release here? Why 31?
Should a Fedora Change for that release be crafted that says "Mass
python2 packages removal"?
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
10:12 a.m.
On 03/20/18 14:45, Zbigniew Jędrzejewski-Szmek wrote:
On Tue, Mar 20, 2018 at 12:23:06PM +0100, Miro Hrončok wrote:
> On 20.3.2018 11:25, Zbigniew Jędrzejewski-Szmek wrote:
>> to push the whole ecosystem. The proposed model of nipping python2 support
>> at the edges is the same thing, in reverse. First some leaf packages
>> are dropped, and then some somewhat more important packages, and then
>> suddenly it becomes hard to use python2 for anything "serious", even
>> though it's still "available". I think that's a bad way to
proceed for
>> our users. Let them have a single moment where python2 support goes away.
>> Announce this moment at least a year in advance. Let them keep running
>> the last release before that for as long as they need.
>
> I think there is a fundamental difference of how I see python RPMs
> in Fedora and how you do. In my POV, purpose of pythonX-foo packages
> that only bring libraries is to provide dependency for an
> application that happens to be written in python, runs on pythonX
> and need the foo library. If there is no such application packaged
> in Fedora, I see no purpose of such package. (Except maybe if such
> app is packaged in another repository used by our users. That's a
> corner case and if the maintainer of pythonX-foo is aware of that,
> it can always be used as argument to keep pythonX-foo.)
Indeed, I'm using those python packages like a dinosaur ;)
But more seriously, I think the distinction between library and app in
the python world is neither clear nor particularly significant. Distro
packaging provides clear benefits in both cases: easy and quick
install, easy and quick uninstall, testing, provenance history,
license checks, general reliability.
So instead of losing packages, I'd like to see more automatic
packaging and updates of packages from pypi and other "forges".
We are making steps in that direction, but not quickly enough.
> We live in an era of pip (and rubygems, and cpan, and npm etc.).
> Trying to mirror those systems with RPM packages just because it's
> possible is tedious, incomplete and mostly pointless IMHO. An era
> when upstream projects were eager to get their tool into the distros
> has passed. Having pythonX-foo packaged where no other
> tool/app/service needs it is not beneficial (if foo is a library
> only thing, not a tool itself). I have this opinion even with
> python3 packages.
>
> Developers who wants to use python2 for whatever reasons (rational
> or emotional ones) can do that on Fedora. We encourage developers to
> use virtual environments and pip [1]. Removing leaf python2 packages
> does not block them here at all.
>
>> Let them have a single moment where python2 support goes away.
>
> Removing all python2 things at one point is impossible. We've tried
> to test this with Django [2] on a much smaller scale. (Tens instead
> of thousands packages.) It took us too much time and too much
> energy. At the end, too much packages were just retired because
> their maintainers are "dead". I cannot imagine doing this on current
> scale in 2020.
> We can just retire python2 then and see what breaks. Honestly,
> everything would break.
I don't think this is entirely comparable. Django20 was about updating
packages and switching support from python2 to python3. Just dropping
existing support for python2 is something much easier.
Maybe it's much easier, but it's still not trivial.
I just saw a package that uses epydoc to generate its documentation,
from its py2 code. Since epydoc is not ported, that package would lose
its documentation if python2 was just suddenly dropped – and we wouldn't
know in advance, because the package was in the "dual py2/py3 support"
bucket.
But I'm not complaining about epydoc in general. I'm afraid there are
many more cases like that – cases where we won't know about the problem
until we try to remove the py2 packages. Especially those with
unresponsive maintainers, which tend to use the most interesting ancient
technology.
I don't see a good way to uncover such problems other than to start
removing stuff.
Nevertheless, I agree that this still is a very significant chunk of
work at this scale. So spreading this out makes a lot of sense, but
imho python2 subpackages don't need to go away, they can just be
conditionalized.
What about adding conditionals like
%if 0%{?rhel} > 7 || 0%{?fedora} > 31
# Disable python2 build by default
%bcond_with python2
%else
%bcond_without python2
%endif
starting now?
Agreed, we just need to bikeshed the "31" number there.
11:11 a.m.
On Tue, Mar 20, 2018 at 03:28:23PM +0100, Miro Hrončok wrote:
On 20.3.2018 14:45, Zbigniew Jędrzejewski-Szmek wrote:
>Indeed, I'm using those python packages like a dinosaur ;)
:D
>What about adding conditionals like
>
>%if 0%{?rhel} > 7 || 0%{?fedora} > 31
># Disable python2 build by default
>%bcond_with python2
>%else
>%bcond_without python2
>%endif
>
>starting now?
I am not against that. However currently that number (31) is
somewhat artificial. it's up to the maintainer to choose if it's 28
or 32 (or anything in between). Should we somehow recommend a
specific Fedora release here? Why 31?
Re 31: my thinking was that python2-eos is at
2010-01-01. If we keep
up current biannual release schedule, we'll have 28 and 29 this year,
30 and 31 in 2019, and 32 in early 2020. But this is of course wrong,
because we need python2 support for the lifetime of the release, not
just at the start. So actually 29 which will be supported until
2018-10-30 + 13 months ≈ 2020-01-01, is the last release overlapped
by python2 support. This right conditional would be:
%if 0%{?rhel} > 7 || 0%{?fedora} > 28
but that round around the corner. So it's not even necessary to add
the conditional, as long as the patch to drop support is only added
in rawhide, not F28.
OK, so I withdraw my objections. Removing python2 support in rawhide
is fine.
Should a Fedora Change for that release be crafted that says
"Mass
python2 packages removal"?
Yeah, I think it should be filed as a F29 change.
Zbyszek
12:05 p.m.
On 20.3.2018 17:11, Zbigniew Jędrzejewski-Szmek wrote:
> Should a Fedora Change for that release be crafted that says
"Mass
> python2 packages removal"?
Yeah, I think it should be filed as a F29 change.
I think we should send the e-mail first and do that after, if nobody
takes python2.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
3:47 p.m.
On Tue, Mar 20, 2018 at 04:11:46PM +0000, Zbigniew Jędrzejewski-Szmek wrote:
On Tue, Mar 20, 2018 at 03:28:23PM +0100, Miro Hrončok wrote:
> On 20.3.2018 14:45, Zbigniew Jędrzejewski-Szmek wrote:
> >Indeed, I'm using those python packages like a dinosaur ;)
>
> :D
>
> >What about adding conditionals like
> >
> >%if 0%{?rhel} > 7 || 0%{?fedora} > 31
> ># Disable python2 build by default
> >%bcond_with python2
> >%else
> >%bcond_without python2
> >%endif
> >
> >starting now?
>
> I am not against that. However currently that number (31) is
> somewhat artificial. it's up to the maintainer to choose if it's 28
> or 32 (or anything in between). Should we somehow recommend a
> specific Fedora release here? Why 31?
Re 31: my thinking was that python2-eos is at 2010-01-01. If we keep
up current biannual release schedule, we'll have 28 and 29 this year,
30 and 31 in 2019, and 32 in early 2020. But this is of course wrong,
because we need python2 support for the lifetime of the release, not
just at the start. So actually 29 which will be supported until
2018-10-30 + 13 months ≈ 2020-01-01, is the last release overlapped
by python2 support. This right conditional would be:
%if 0%{?rhel} > 7 || 0%{?fedora} > 28
but that round around the corner. So it's not even necessary to add
the conditional, as long as the patch to drop support is only added
in rawhide, not F28.
OK, so I withdraw my objections. Removing python2 support in rawhide
is fine.
Pff, sorry I can't count. The right conditional would be
%if 0%{?rhel} > 7 || 0%{?fedora} > 29
so dropping support in rawhide right now would be premature. It seems
that adding a conditional like that is the way to go for now.
> Should a Fedora Change for that release be crafted that says
"Mass
> python2 packages removal"?
Yeah, I think it should be filed as a F29 change.
F30 might be more appropriate.
Zbyszek
9:49 a.m.
On 03/20/18 21:47, Zbigniew Jędrzejewski-Szmek wrote:
On Tue, Mar 20, 2018 at 04:11:46PM +0000, Zbigniew Jędrzejewski-Szmek
wrote:
> On Tue, Mar 20, 2018 at 03:28:23PM +0100, Miro Hrončok wrote:
>> On 20.3.2018 14:45, Zbigniew Jędrzejewski-Szmek wrote:
>>> Indeed, I'm using those python packages like a dinosaur ;)
>>
>> :D
>>
>>> What about adding conditionals like
>>>
>>> %if 0%{?rhel} > 7 || 0%{?fedora} > 31
>>> # Disable python2 build by default
>>> %bcond_with python2
>>> %else
>>> %bcond_without python2
>>> %endif
>>>
>>> starting now?
>>
>> I am not against that. However currently that number (31) is
>> somewhat artificial. it's up to the maintainer to choose if it's 28
>> or 32 (or anything in between). Should we somehow recommend a
>> specific Fedora release here? Why 31?
> Re 31: my thinking was that python2-eos is at 2010-01-01. If we keep
> up current biannual release schedule, we'll have 28 and 29 this year,
> 30 and 31 in 2019, and 32 in early 2020. But this is of course wrong,
> because we need python2 support for the lifetime of the release, not
> just at the start. So actually 29 which will be supported until
> 2018-10-30 + 13 months ≈ 2020-01-01, is the last release overlapped
> by python2 support. This right conditional would be:
>
> %if 0%{?rhel} > 7 || 0%{?fedora} > 28
>
> but that round around the corner. So it's not even necessary to add
> the conditional, as long as the patch to drop support is only added
> in rawhide, not F28.
>
> OK, so I withdraw my objections. Removing python2 support in rawhide
> is fine.
Pff, sorry I can't count. The right conditional would be
%if 0%{?rhel} > 7 || 0%{?fedora} > 29
so dropping support in rawhide right now would be premature. It seems
that adding a conditional like that is the way to go for now.
Compared to just removing the subpackage, it's a difference of just one
release. It may not be worth the complication of conditionalizing
everything.
I'd say maintainers can either drop the subpackages now in Rawhide, or
add a conditional for Fedora > 29 -- up to the maintainer.
2227
days inactive
2228
days old
python-devel@lists.fedoraproject.org
9 comments
3 participants
participants (3)
-
Miro Hrončok -
Petr Viktorin -
Zbigniew Jędrzejewski-Szmek