11:07 a.m.
Hello fellow Fedora contributors,
We've created a page on Fedora wiki [1] that's something like a Change
proposal*.
The document is called "Finalizing Fedora's Switch to Python 3" and it
describes steps needed in order to:
* Switch /usr/bin/python to Python 3 in cooperation with Python upstream.
* Switch `dnf install python-foo` to install python3-foo.
* Hopefully eventually get rid of Python 2.
This is all planned to line with the fact that Python 2 will be dead
upstream by 2020.
The first phase of the changes is planned for Fedora 30, which is pretty
far away today, but the proposal involves changing thousands of
packages. Hence we'd like to have something we can use to support our
efforts.
I'd like to gather feedback for the document: whether the changes are
understandable and make sense, whether you think we should do things
differently, etc.
[1] https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3
* It involves steps in multiple releases, so we were advised not to
create a multi-release Change proposal, but instead split it into more.
This page summarizes it all an individual Change proposals will follow.
Thank you,
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
12:15 p.m.
Do you think it'd be possible to script the python-foo to python2-foo
renaming? If yes, then maybe it'd make sense to just get some pps to
do it in rawhide now and get the "easy" part done with? That should
significantly cut down on the number of misnamed packages and let
packagers spend their times on the ones where the automatic way is not
obvious.
Zbyszek
2:49 a.m.
On 27.7.2017 19:15, Zbigniew Jędrzejewski-Szmek wrote:
Do you think it'd be possible to script the python-foo to
python2-foo
renaming?
May be possible but given the variety in the spec files, we didn't think
it would be easy. IMHO taking a bunch of provenpackagers and updating
the packages manually without asking the owners will be less time
consuming than creating an automated tool.
However, in both scenarios (people or tools), I consider that approach
aggressive and would rather only do it when the deadlines are close.
We'd rather not change other people packages and make them mad about
what have we done. The transition to Python 3 is unfortunately
considered a failure in eyes of some Fedora contributors.
I consider communicating our efforts with the package owners and
explaining them what and how to do as more friendly approach. However,
definitively more time expensive.
On the other hand, maybe we can start updating the packages co-owned by
python-sig, as I would say those are considered to be maintained by the
group rather than a person. In case you'd like to try to write an
automated tool for that, please go ahead (but please let us know).
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
2:51 a.m.
On 28 July 2017 at 03:15, Zbigniew Jędrzejewski-Szmek <zbyszek(a)in.waw.pl> wrote:
Do you think it'd be possible to script the python-foo to
python2-foo
renaming? If yes, then maybe it'd make sense to just get some pps to
do it in rawhide now and get the "easy" part done with? That should
significantly cut down on the number of misnamed packages and let
packagers spend their times on the ones where the automatic way is not
obvious.
I was going to ask whether it might be possible to tweak
http://fedora.portingdb.xyz/ to also report on compliance with the
naming policy, but then I went and saw that it *does* already report
on that: http://fedora.portingdb.xyz/namingpolicy/
While it also turns out the wiki page already links to that page, it
may be good to call it out a second time in a "How can I help?"
section.
Checking an initial sampling of spec files (python-d2to1,
python-BeautfulSoup, python-amqplib) suggests to me that a script
implementing the following rules might offer a reasonably start point,
at least for Python-2-only modules that are remaining Python-2-only:
- immediately before the first BuildRequires or Requires entry, add a
%package section header for "-n python2-<name>" (where
"<name>" is the
lowercased package source name with any "python-" prefix stripped)
- add a %python_provides entry after the new package header in
accordance with the current guidelines
- if the original package provided a non-lowercase "python-*"
provides, remote it and add a second %python_provides with the
original capitalisation
- if the source package lacks the "python-" prefix, add a virtual
provides for the unqualified package name
- add a "-n python2-<name>" qualifier to any currently unqualified
description and files sections
A script like that may even do a tolerable job for packages that *do*
offer Python 3 subpackages (since those will already have qualifiers,
and will necessarily appear after any unqualified runtime and build
requirements for the default subpackage).
Cheers,
Nick.
--
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
5:01 a.m.
On Fri, Jul 28, 2017 at 09:49:42AM +0200, Miro Hrončok wrote:
On 27.7.2017 19:15, Zbigniew Jędrzejewski-Szmek wrote:
>Do you think it'd be possible to script the python-foo to python2-foo
>renaming?
May be possible but given the variety in the spec files, we didn't
think it would be easy. IMHO taking a bunch of provenpackagers and
updating the packages manually without asking the owners will be
less time consuming than creating an automated tool.
However, in both scenarios (people or tools), I consider that
approach aggressive and would rather only do it when the deadlines
are close.
We'd rather not change other people packages and make them mad about
what have we done. The transition to Python 3 is unfortunately
considered a failure in eyes of some Fedora contributors.
[My original comment here
has been deleted to protect the innocent ;) ]
I consider communicating our efforts with the package owners and
explaining them what and how to do as more friendly approach.
However, definitively more time expensive.
On the other hand, maybe we can start updating the packages co-owned
by python-sig, as I would say those are considered to be maintained
by the group rather than a person. In case you'd like to try to
write an automated tool for that, please go ahead (but please let us
know).
I don't know if delaying this will help. Actually stretching out the
transition might make things more painful, because it'll increase the
window where you don't know if any given python package has python2-
or python- prefix, so you need to check all deps. I'm sure getting rid
of this ambiguity would be welcome.
So instead of taking this slowly, I'd rather go through FESCo
and public announcement on fedora-devel, and then just apply the change
to as many packages as possible at once. Technically, I think it would
be reasonable to try to script this, w/o pushing to dist git, then put
up the diffs for review somewhere, and then finally push at once, at get
a few pps to do the rebuilds (some stuff is bound to fail for related
or unrelated reasons, so some debugging is expected).
In the other mail Nick Coghlan listed various steps: it's far from
trivial, but OTOH doing 800+ packages by hand ain't gonna be fun either.
I'll try to look into this over the weekend, it sounds like a fun
challenge.
Zbyszek
5:48 a.m.
On Thu, Jul 27, 2017, at 12:07 PM, Miro Hrončok wrote:
* Switch /usr/bin/python to Python 3 in cooperation with Python
upstream.
That again? That really seems like a nonstarter; previous
discussion specifically around Atomic Host + Ansible:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
Plus just breaking everyone's random old scripts. Now, I
don't know if this has been discussed upstream, but one
thing to do possibly would be to have an interactive shell
alias for `python` = `ipython3` or something like that, and
switch `ipython` to `ipython3`.
6:47 a.m.
On Fri, Jul 28, 2017 at 11:48 AM, Colin Walters <walters(a)verbum.org> wrote:
On Thu, Jul 27, 2017, at 12:07 PM, Miro Hrončok wrote:
> * Switch /usr/bin/python to Python 3 in cooperation with Python upstream.
That again? That really seems like a nonstarter; previous
discussion specifically around Atomic Host + Ansible:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
Ansible now supports clients that are py3 only and there's active work
[1] to enable py3 for the controller as well.
[1] http://docs.ansible.com/ansible/latest/python_3_support.html
7:29 a.m.
On Fri, Jul 28, 2017, at 12:48 PM, Colin Walters wrote:
On Thu, Jul 27, 2017, at 12:07 PM, Miro Hrončok wrote:
> * Switch /usr/bin/python to Python 3 in cooperation with Python upstream.
That again? That really seems like a nonstarter; previous
discussion specifically around Atomic Host + Ansible:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
Plus just breaking everyone's random old scripts. Now, I
don't know if this has been discussed upstream
This is a statement born of ignorance, but ...
Would it be possible to switch the default from python2 to python3 and
provide an RPM that would switch it back? This way people who needed to
stay on python2 for some time period to clean up legacy components
before teh 2020 EOL could.
I realize that package renaming and things like that will move forward
regardless and should.
regards,
bex
7:29 a.m.
On 28 July 2017 at 20:48, Colin Walters <walters(a)verbum.org> wrote:
On Thu, Jul 27, 2017, at 12:07 PM, Miro Hrončok wrote:
> * Switch /usr/bin/python to Python 3 in cooperation with Python upstream.
That again? That really seems like a nonstarter; previous
discussion specifically around Atomic Host + Ansible:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
Plus just breaking everyone's random old scripts.
Option 1: *guarantee* breaking old scripts by removing
"/usr/bin/python" entirely (current status quo on Py3-only Fedora
systems)
Option 2: keep such scripts working *if* somewhere in the *decade*
since Python 3 was released, folks accepted that Python 2 really was
going to go away, and modernised their code to run in the common
subset of Python 2.7 and 3.6+ (which covers a *lot* of Python 2.7's
capabilities, especially if you do "from __future__ import
print_function" rather than relying on the old print statement syntax)
Corollary to option 2: for as long Fedora ships a Python 2.7 stack, we
should provide a supported mechanism (building on either modularity,
xdg-alternatives, or both) that makes it straightforward for system
administrators to switch "/usr/bin/python" back to referring to Python
2.7
Now, I
don't know if this has been discussed upstream, but one
thing to do possibly would be to have an interactive shell
alias for `python` = `ipython3` or something like that, and
switch `ipython` to `ipython3`.
User level aliasing isn't the issue - there are already several
solutions for that, and users are free to pick whichever one works
best for them.
It's only /usr/bin/python itself that still presents an unsolved
problem, since the status quo (not providing it at all) is even more
user hostile than pointing it at a modern version of Python 3 that
includes the various changes aimed at increasing the size of the
common subset of Python 2 & 3 (e.g. explicit unicode literals in 3.3,
binary codecs in 3.4, binary mod-formatting in 3.5, Fedora's backport
of implicit locale coercion to 3.6).
The most common Py2-induced syntax errors (print without parentheses,
print redirects using the old syntax) have also gained or are gaining
much improved error messages in 3.6.x (such that the error also
suggests directly how to fix it), and there are various other things
we can consider to help smooth the transition (like providing the
"six" and "future" compatibility modules by default in both our
Python
2 and Python 3 stacks in order to make portable hybrid code easier to
write).
Cheers,
Nick.
--
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
8:38 a.m.
On 07/28/2017 11:01 AM, Zbigniew Jędrzejewski-Szmek wrote:
So instead of taking this slowly, I'd rather go through FESCo
and public announcement on fedora-devel, and then just apply the change
to as many packages as possible at once.
I concur. I think this is one of the cases
where we should use
provenpackagers / scripting and get all packages updated as quickly as
possible.
--
Kalev
9:10 a.m.
On 28 July 2017 at 22:29, Brian Exelbierd <bex(a)pobox.com> wrote:
On Fri, Jul 28, 2017, at 12:48 PM, Colin Walters wrote:
> Plus just breaking everyone's random old scripts. Now, I
> don't know if this has been discussed upstream
This is a statement born of ignorance, but ...
Would it be possible to switch the default from python2 to python3 and
provide an RPM that would switch it back? This way people who needed to
stay on python2 for some time period to clean up legacy components
before teh 2020 EOL could.
Aye, while it's still a bit "and then some magic happens" at this
point, I actually think it may be possible to do something like this
through a combination of:
1. All Fedora packages switching to explicitly depending on either the
Python 2 stack or the Python 3 stack
2. Using the modularity tooling to make at least the target of
/usr/bin/python and potentially even the "python-*" dependency
resolution switchable
It's part 2 that counts as hand-wavey right now, since Boltron was
only just released, and we haven't even defined what we think the
Python 2 & Python 3 modules for the F27 Modular Server should look
like yet, let alone a "default-python" module that controls what
"/usr/bin/python" refers to. However, it seems to me that an approach
like that *should* work, so it's an idea I'm going to pursue until we
either having a working "Change it back!" switch, or else I've been
persuaded that it isn't feasible after all :)
Cheers,
Nick.
--
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
9:17 a.m.
On Fri, Jul 28, 2017, at 07:47 AM, Peter Robinson wrote:
On Fri, Jul 28, 2017 at 11:48 AM, Colin Walters
<walters(a)verbum.org> wrote:
> On Thu, Jul 27, 2017, at 12:07 PM, Miro Hrončok wrote:
>
>> * Switch /usr/bin/python to Python 3 in cooperation with Python upstream.
>
> That again? That really seems like a nonstarter; previous
> discussion specifically around Atomic Host + Ansible:
>
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
Ansible now supports clients that are py3 only and there's active work
[1] to enable py3 for the controller as well.
Yes. But basically, I don't think we can rely on everyone porting their Ansible
code to Python3 anytime soon. Among other reasons, many, many organizations
will need their Ansible (and in general, "config mgmt/scripting") to work
across
RHEL7 and Fedora hosts. For Fedora Atomic Host (which may be different from
Workstation!) I don't see dropping /usr/bin/python as python2 until RHEL7 is
near EOL. Maybe we don't do security updates for it etc., but still.
9:25 a.m.
I made this comment previously but because I think it's important I'm
going to repeat it.
Fedora's Python version migration needs to be coordinated with RHEL.
Yes I know Fedora is independent of both Red Hat and RHEL but the real
world reality is spec files are shared between both. At the moment you
cannot easily share a spec file between the two, this leads to
maintaining two independent spec files for something that ought to be
nearly identical and increases the burden on package maintainers and
increases the opportunity for errors.
The transition is already going to be painful, let's not make it more
painful than it needs to be.
At a minimum the set of RPM macros to manage the difference between
Python versions needs to be identically supported in both Fedora and
RHEL. I'd like to see this as an explicit goal of the transition strategy.
--
John
10:02 a.m.
On 29 July 2017 at 00:25, John Dennis <jdennis(a)redhat.com> wrote:
I made this comment previously but because I think it's important
I'm going
to repeat it.
Fedora's Python version migration needs to be coordinated with RHEL.
Yes I know Fedora is independent of both Red Hat and RHEL but the real world
reality is spec files are shared between both. At the moment you cannot
easily share a spec file between the two, this leads to maintaining two
independent spec files for something that ought to be nearly identical and
increases the burden on package maintainers and increases the opportunity
for errors.
Aye, I agree we should be actively seeking to make single-spec
feasible across Fedora/RHEL/CentOS, at least in combination with EPEL
(without the EPEL dependency, it's hard to consistently make new
Fedora level macro definitions available to older EL releases).
Cheers,
Nick.
--
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
10:17 a.m.
On 28.7.2017 09:51, Nick Coghlan wrote:
On 28 July 2017 at 03:15, Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl> wrote:
> Do you think it'd be possible to script the python-foo to python2-foo
> renaming? If yes, then maybe it'd make sense to just get some pps to
> do it in rawhide now and get the "easy" part done with? That should
> significantly cut down on the number of misnamed packages and let
> packagers spend their times on the ones where the automatic way is not
> obvious.
I was going to ask whether it might be possible to tweak
http://fedora.portingdb.xyz/ to also report on compliance with the
naming policy, but then I went and saw that it *does* already report
on that: http://fedora.portingdb.xyz/namingpolicy/
While it also turns out the wiki page already links to that page, it
may be good to call it out a second time in a "How can I help?"
section.
I've just added that section. However, feel free to reword it or add
stuff. Thanks for the suggestion.
Checking an initial sampling of spec files (python-d2to1,
python-BeautfulSoup, python-amqplib) suggests to me that a script
implementing the following rules might offer a reasonably start point,
at least for Python-2-only modules that are remaining Python-2-only:
- immediately before the first BuildRequires or Requires entry, add a
%package section header for "-n python2-<name>" (where
"<name>" is the
lowercased package source name with any "python-" prefix stripped)
- add a %python_provides entry after the new package header in
accordance with the current guidelines
- if the original package provided a non-lowercase "python-*"
provides, remote it and add a second %python_provides with the
original capitalisation
- if the source package lacks the "python-" prefix, add a virtual
provides for the unqualified package name
- add a "-n python2-<name>" qualifier to any currently unqualified
description and files sections
This get's problematic once there spec files are full of macros.
However, feel free to work with Zbyszek, maybe you'll find it less
complicating than we did.
A script like that may even do a tolerable job for packages that
*do*
offer Python 3 subpackages (since those will already have qualifiers,
and will necessarily appear after any unqualified runtime and build
requirements for the default subpackage).
Cheers,
Nick.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
10:19 a.m.
On 28.7.2017 12:01, Zbigniew Jędrzejewski-Szmek wrote:
On Fri, Jul 28, 2017 at 09:49:42AM +0200, Miro Hrončok wrote:
> On 27.7.2017 19:15, Zbigniew Jędrzejewski-Szmek wrote:
>> Do you think it'd be possible to script the python-foo to python2-foo
>> renaming?
>
> May be possible but given the variety in the spec files, we didn't
> think it would be easy. IMHO taking a bunch of provenpackagers and
> updating the packages manually without asking the owners will be
> less time consuming than creating an automated tool.
>
> However, in both scenarios (people or tools), I consider that
> approach aggressive and would rather only do it when the deadlines
> are close.
> We'd rather not change other people packages and make them mad about
> what have we done. The transition to Python 3 is unfortunately
> considered a failure in eyes of some Fedora contributors.
[My original comment here has been deleted to protect the innocent ;) ]
> I consider communicating our efforts with the package owners and
> explaining them what and how to do as more friendly approach.
> However, definitively more time expensive.
>
>
> On the other hand, maybe we can start updating the packages co-owned
> by python-sig, as I would say those are considered to be maintained
> by the group rather than a person. In case you'd like to try to
> write an automated tool for that, please go ahead (but please let us
> know).
I don't know if delaying this will help. Actually stretching out the
transition might make things more painful, because it'll increase the
window where you don't know if any given python package has python2-
or python- prefix, so you need to check all deps. I'm sure getting rid
of this ambiguity would be welcome.
So instead of taking this slowly, I'd rather go through FESCo
and public announcement on fedora-devel, and then just apply the change
to as many packages as possible at once. Technically, I think it would
be reasonable to try to script this, w/o pushing to dist git, then put
up the diffs for review somewhere, and then finally push at once, at get
a few pps to do the rebuilds (some stuff is bound to fail for related
or unrelated reasons, so some debugging is expected).
In the other mail Nick Coghlan listed various steps: it's far from
trivial, but OTOH doing 800+ packages by hand ain't gonna be fun either.
I'll try to look into this over the weekend, it sounds like a fun
challenge.
Thanks, feel free to reach out any time for help.
Zbyszek
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
10:25 a.m.
On 28.7.2017 12:48, Colin Walters wrote:
On Thu, Jul 27, 2017, at 12:07 PM, Miro Hrončok wrote:
> * Switch /usr/bin/python to Python 3 in cooperation with Python upstream.
That again?
Seems like a never ending story, right? :(
That really seems like a nonstarter; previous> discussion
specifically around Atomic Host + Ansible:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
Plus just breaking everyone's random old scripts.
Actually, we are trying to avoid that. In the corresponding part of the
document [1], we actually say:
"Note that after Phase 1, nothing in Fedora uses /usr/bin/python any
more, so it should be safe for sysadmins to flip the symblink back to
python2 and we should provide a documented and supported way of doing so."
Now, I> don't know if this has been discussed upstream,
It is being discussed upstream right now and the discussion is linked
from the relevant part of the document [2][3].
but one> thing to do possibly would be to have an interactive
shell
alias for `python` = `ipython3` or something like that, and
switch `ipython` to `ipython3`.
[1]
https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3#Phase_2:_S...
[2]
https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3#Interlude:...
[3] https://mail.python.org/pipermail/linux-sig/2017-July/000029.html
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
10:28 a.m.
On 28.7.2017 14:29, Brian Exelbierd wrote:
On Fri, Jul 28, 2017, at 12:48 PM, Colin Walters wrote:
> On Thu, Jul 27, 2017, at 12:07 PM, Miro Hrončok wrote:
>
>> * Switch /usr/bin/python to Python 3 in cooperation with Python upstream.
>
> That again? That really seems like a nonstarter; previous
> discussion specifically around Atomic Host + Ansible:
>
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
>
> Plus just breaking everyone's random old scripts. Now, I
> don't know if this has been discussed upstream
This is a statement born of ignorance, but ...
Would it be possible to switch the default from python2 to python3 and
provide an RPM that would switch it back? This way people who needed to
stay on python2 for some time period to clean up legacy components
before teh 2020 EOL could.
Not yet sure about the actual technical solution here, but we are
planning for this use case, see [1]:
"Note that after Phase 1, nothing in Fedora uses /usr/bin/python any
more, so it should be safe for sysadmins to flip the symblink back to
python2 and we should provide a documented and supported way of doing so."
[1]
https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3#Phase_2:_S...
I realize that package renaming and things like that will move forward
regardless and should.
regards,
bex
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
10:34 a.m.
On 28.7.2017 16:17, Colin Walters wrote:
On Fri, Jul 28, 2017, at 07:47 AM, Peter Robinson wrote:
> On Fri, Jul 28, 2017 at 11:48 AM, Colin Walters <walters(a)verbum.org> wrote:
>> On Thu, Jul 27, 2017, at 12:07 PM, Miro Hrončok wrote:
>>
>>> * Switch /usr/bin/python to Python 3 in cooperation with Python upstream.
>>
>> That again? That really seems like a nonstarter; previous
>> discussion specifically around Atomic Host + Ansible:
>>
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
>
> Ansible now supports clients that are py3 only and there's active work
> [1] to enable py3 for the controller as well.
Yes. But basically, I don't think we can rely on everyone porting their Ansible
code to Python3 anytime soon. Among other reasons, many, many organizations
will need their Ansible (and in general, "config mgmt/scripting") to work
across
RHEL7 and Fedora hosts. For Fedora Atomic Host (which may be different from
Workstation!) I don't see dropping /usr/bin/python as python2 until RHEL7 is
near EOL. Maybe we don't do security updates for it etc., but still.
If we don't switch it in Fedora until RHEL7 is near EOL, it will neither
be changed in any hypothetical future versions of RHEL, because RHEL
tends to be based on Fedora. And than you can say that we cannot do the
switch in Fedora until that hypothetical future version of RHEL is near
EOL and you will end up never changing it at all. The change has to
start somewhere and even though I cannot really speak about RHEL,
I would think that changing it in Fedora first makes more sense.
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
10:42 a.m.
On 28.7.2017 16:25, John Dennis wrote:
I made this comment previously but because I think it's important
I'm
going to repeat it.
Yes, we hear you.
Fedora's Python version migration needs to be coordinated with
RHEL.
It is. However it is hard to coordinate a future change with RHEL7 that
is kinda designed not to change much.
Yes I know Fedora is independent of both Red Hat and RHEL but the
real
world reality is spec files are shared between both. At the moment you
cannot easily share a spec file between the two, this leads to
maintaining two independent spec files for something that ought to be
nearly identical and increases the burden on package maintainers and
increases the opportunity for errors.
We added a Q&A section that says something about sharing a specfile
across Fedora and EPEL [1].
It doesn't speak about RHEL, because AFAIK adding macros to RHEL7 (or
even RHEL6) may be a bit complicated. However changing one's packages
form RHEL to use those new macros might be as complicated as that.
If you have packages in RHEL and Fedora that have identical spec files
and you update them in RHEL regularly when you update them in Fedora,
please contact us trough Red Hat channels so we can craft a solution
that works for you.
If you share specfiles across EPEL and Fedora, use the proposed solution
from the Q&A section.
[1] https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3#Q.26A
The transition is already going to be painful, let's not make it
more
painful than it needs to be.
Believe me, we are trying hard not to.
At a minimum the set of RPM macros to manage the difference between
Python versions needs to be identically supported in both Fedora and
RHEL. I'd like to see this as an explicit goal of the transition strategy.
I don't want to discuss RHEL related strategies on a public ML, again
feel free to contact us trough Red Hat channels, if you want to hear more.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
10:51 a.m.
On Fri, Jul 28, 2017, at 11:34 AM, Miro Hrončok wrote:
The change has to
start somewhere and even though I cannot really speak about RHEL,
I would think that changing it in Fedora first makes more sense.
There hasn't been a single "Fedora" since we landed the editions,
although that's somewhat superficial. With Modularity there will
*really* no longer be one "Fedora". Want to drop python2 out of
the container base image? Actually, that's done already! I'm
all for that. Anyone who wants python2 in their containers
is probably going to be choosing centos/rhel anyways.
Dropping python2 from Workstation? Sure, sounds fine to me, I
don't care much personally. I'm just pushing back against that change
landing for Atomic Host.
11:16 a.m.
On 28.7.2017 17:51, Colin Walters wrote:
On Fri, Jul 28, 2017, at 11:34 AM, Miro Hrončok wrote:
> The change has to
> start somewhere and even though I cannot really speak about RHEL,
> I would think that changing it in Fedora first makes more sense.
There hasn't been a single "Fedora" since we landed the editions,
although that's somewhat superficial. With Modularity there will
*really* no longer be one "Fedora". Want to drop python2 out of
the container base image? Actually, that's done already! I'm
all for that. Anyone who wants python2 in their containers
is probably going to be choosing centos/rhel anyways.
Dropping python2 from Workstation? Sure, sounds fine to me, I
don't care much personally. I'm just pushing back against that change
landing for Atomic Host.
It's up to the maintainers of Atomic Host to decide what's there. I
don't see how that relates to this thread. We have never said anything
about what is supposed to be or not to be in the Atomic Host.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
11:28 a.m.
On 28 July 2017 at 08:29, Nick Coghlan <ncoghlan(a)gmail.com> wrote:
--
Stephen J Smoogen.
On 28 July 2017 at 20:48, Colin Walters <walters(a)verbum.org>
wrote:
> On Thu, Jul 27, 2017, at 12:07 PM, Miro Hrončok wrote:
>
>> * Switch /usr/bin/python to Python 3 in cooperation with Python
upstream.
>
> That again? That really seems like a nonstarter; previous
> discussion specifically around Atomic Host + Ansible:
> https://lists.fedoraproject.org/archives/list/devel@lists.
fedoraproject.org/thread/T4PGIEUYXWXSDRVWYQ2BJXCECL45XWVW/
>
> Plus just breaking everyone's random old scripts.
Option 1: *guarantee* breaking old scripts by removing
"/usr/bin/python" entirely (current status quo on Py3-only Fedora
systems)
From having dealt with this from a support side during the python1 ->
python2 changes .. this is probably preferable. There are going to be the
scripts which work well enough but not the same. And there are the scripts
which won't work until it hits a part that isn't python3 compatible.
Having python set up as an alternatives would probably be the Debian-ish
like solution in that if you want to have your box running python3 only you
can switch using that.. and if not you can switch it to the other. And if
for some reason we decided that Jython was better than the Cpython.. you
can do that.
11:38 a.m.
On 28 July 2017 at 11:42, Miro Hrončok <mhroncok(a)redhat.com> wrote:
On 28.7.2017 16:25, John Dennis wrote:
> I made this comment previously but because I think it's important I'm
> going to repeat it.
>
Yes, we hear you.
Fedora's Python version migration needs to be coordinated with RHEL.
>
It is. However it is hard to coordinate a future change with RHEL7 that is
kinda designed not to change much.
It isn't with just RHEL7. Customers of RHEL are probably going to want to
run their python2 code for years after it is EOL. If something has been
written as part of a regulatory compliance (say a plane/car/rocket
simulation).. that code is locked for 20 years whether the language below
is alive or not. If the code was written for infrastructure it might be 60
years. So as Python became the goto language for science and engineering..
it has also become the language of regulations and other things. RHELn may
still need to have python be able to point to python2 because customers
need it for reasons we in Fedora or the Python community do not care about.
From the conversations, this is the part that seems less explored.
--
Stephen J Smoogen.
11:41 a.m.
On Fri, Jul 28, 2017, at 12:16 PM, Miro Hrončok wrote:
It's up to the maintainers of Atomic Host to decide what's
there. I
don't see how that relates to this thread. We have never said anything
about what is supposed to be or not to be in the Atomic Host.
I'm opposed to switching the meaning of `/usr/bin/python` for AH
anytime soon. It's just going to break stuff, and to me the gain is quite
low. If you still propose to make that change for other editions and in the Everything
repository,
we'll likely need to override it.
Regardless, I can certainly take care of updating the Change proposal to mention
that the `/usr/bin/python` portion doesn't apply to AH, and we'll continue
to have `python2` installed by default, if that's OK with you?
12:53 p.m.
On Fri, 2017-07-28 at 12:41 -0400, Colin Walters wrote:
I'm opposed to switching the meaning of `/usr/bin/python` for AH
anytime soon. It's just going to break stuff, and to me the gain is
quite
low.
One of Fedora's stated goals is to remain close to the upstream.
Upstream Python is going to change /usr/bin/python to mean Python 3. If
AH keeps it on Python 2, it will be confusing for Python programmers
using AH. It will be especially confusing for programmers who want to
write software that works on "normal" Fedora and AH. Also, as software
begins to switch AH will have to patch its Python packages, diverging
from the rest of Fedora.
Regardless, I can certainly take care of updating the Change
proposal
to mention
that the `/usr/bin/python` portion doesn't apply to AH, and we'll
continue
to have `python2` installed by default, if that's OK with you?
I think this will cause a poor experience for Fedora users. Python
packages would work one way in AH, but work differently in the rest of
the Fedora editions.
I don't see a practical advantage to divergence. I believe the
divergence will break stuff in a more confusing manner than the change
will.
12:55 p.m.
On 28.7.2017 18:41, Colin Walters wrote:
On Fri, Jul 28, 2017, at 12:16 PM, Miro Hrončok wrote:
> It's up to the maintainers of Atomic Host to decide what's there. I
> don't see how that relates to this thread. We have never said anything
> about what is supposed to be or not to be in the Atomic Host.
I'm opposed to switching the meaning of `/usr/bin/python` for AH
anytime soon. It's just going to break stuff, and to me the gain is quite
low. If you still propose to make that change for other editions and in the Everything
repository,
we'll likely need to override it.
Regardless, I can certainly take care of updating the Change proposal to mention
that the `/usr/bin/python` portion doesn't apply to AH, and we'll continue
to have `python2` installed by default, if that's OK with you?
As long as you figure out how to do it properly and as long as you
represent the AH maintainers and can make that decision for AH, I think
it's probably OK to put it there (I still don't like it personally, but
that's not good enough reason to not do it).
What about this:
Note that after Phase 1, nothing in Fedora uses /usr/bin/python any
more, so it should be safe for sysadmins to flip the symblink back to
python2 and we should provide a documented and supported way of doing
so. In the same way, the Atomic Host will keep /usr/bin/python pointing
to Python 2 regardless of rest of Fedora and possibly against upstream
recommendation in PEP 394, because of [reasons].
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
12:58 p.m.
On 28.7.2017 18:38, Stephen John Smoogen wrote:
On 28 July 2017 at 11:42, Miro Hrončok <mhroncok(a)redhat.com
<mailto:mhroncok@redhat.com>> wrote:
On 28.7.2017 16:25, John Dennis wrote:
I made this comment previously but because I think it's
important I'm going to repeat it.
Yes, we hear you.
Fedora's Python version migration needs to be coordinated with RHEL.
It is. However it is hard to coordinate a future change with RHEL7
that is kinda designed not to change much.
It isn't with just RHEL7. Customers of RHEL are probably going to want
to run their python2 code for years after it is EOL. If something has
been written as part of a regulatory compliance (say a plane/car/rocket
simulation).. that code is locked for 20 years whether the language
below is alive or not. If the code was written for infrastructure it
might be 60 years. So as Python became the goto language for science and
engineering.. it has also become the language of regulations and other
things. RHELn may still need to have python be able to point to python2
because customers need it for reasons we in Fedora or the Python
community do not care about.
From the conversations, this is the part that seems less explored.
Because we discuss those things trough Red Hat channels.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
1:11 p.m.
On Fri, Jul 28, 2017, at 01:53 PM, Randy Barlow wrote:
On Fri, 2017-07-28 at 12:41 -0400, Colin Walters wrote:
> I'm opposed to switching the meaning of `/usr/bin/python` for AH
> anytime soon. It's just going to break stuff, and to me the gain is
> quite
> low.
One of Fedora's stated goals is to remain close to the upstream.
Upstream Python is going to change /usr/bin/python to mean Python 3. If
AH keeps it on Python 2, it will be confusing for Python programmers
using AH. It will be especially confusing for programmers who want to
write software that works on "normal" Fedora and AH.
First, I don't see a need for even mild pejoratives; instead of "normal" you
could say
for example "other editions".
Now, we don't really expect people "program" AH directly in that sense;
most development should go in containers, except for development of
the various Python things we carry in the host, but that's not a huge set.
Also, as software
begins to switch AH will have to patch its Python packages, diverging
from the rest of Fedora.
That's actually something I hadn't even considered; is it proposed
here to actually have a flag day where all (or even some) of the python3 executables
start using /usr/bin/python and everything else is /usr/bin/python2?
The scope of that would appear to me to be quite staggering.
Are we really at the point even where everything that's python2 only
explicitly references /usr/bin/python2? I'm doubtful.
I don't see a practical advantage to divergence. I believe the
divergence will break stuff in a more confusing manner than the change
will.
I don't want to break Ansible users or in general random sysadmin scripts;
the value of replacing the meaning of /usr/bin/python feels quite low relative
to that.
1:21 p.m.
On 28.7.2017 20:11, Colin Walters wrote:
On Fri, Jul 28, 2017, at 01:53 PM, Randy Barlow wrote:
> On Fri, 2017-07-28 at 12:41 -0400, Colin Walters wrote:
>> I'm opposed to switching the meaning of `/usr/bin/python` for AH
>> anytime soon. It's just going to break stuff, and to me the gain is
>> quite
>> low.
>
> One of Fedora's stated goals is to remain close to the upstream.
> Upstream Python is going to change /usr/bin/python to mean Python 3. If
> AH keeps it on Python 2, it will be confusing for Python programmers
> using AH. It will be especially confusing for programmers who want to
> write software that works on "normal" Fedora and AH.
First, I don't see a need for even mild pejoratives; instead of "normal"
you could say
for example "other editions".
Now, we don't really expect people "program" AH directly in that sense;
most development should go in containers, except for development of
the various Python things we carry in the host, but that's not a huge set.
> Also, as software
> begins to switch AH will have to patch its Python packages, diverging
> from the rest of Fedora.
That's actually something I hadn't even considered; is it proposed
here to actually have a flag day where all (or even some) of the python3 executables
start using /usr/bin/python and everything else is /usr/bin/python2?
No. We'd like to keep it the way that everything in Fedora either uses
/usr/bin/python3 or /usr/bin/python2. To keep the possibility for the
sysadmins to change /usr/bin/python to whatever crazy target they want.
Patching packages would not be required here.
The scope of that would appear to me to be quite staggering.
Are we really at the point even where everything that's python2 only
explicitly references /usr/bin/python2? I'm doubtful.
> I don't see a practical advantage to divergence. I believe the
> divergence will break stuff in a more confusing manner than the change
> will.
I don't want to break Ansible users or in general random sysadmin scripts;
the value of replacing the meaning of /usr/bin/python feels quite low relative
to that.
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
2:06 p.m.
On Fri, Jul 28, 2017 at 10:29:12PM +1000, Nick Coghlan wrote:
It's only /usr/bin/python itself that still presents an unsolved
problem, since the status quo (not providing it at all) is even more
user hostile than pointing it at a modern version of Python 3 that
Why is not providing /usr/bin/python more use hostile? Why is it better
for any script to use /usr/bin/python instead of /usr/bin/python3?
Kind regards
Till
3:44 p.m.
On Fri, Jul 28, 2017 at 05:51:39PM +1000, Nick Coghlan wrote:
On 28 July 2017 at 03:15, Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl> wrote:
> Do you think it'd be possible to script the python-foo to python2-foo
> renaming? If yes, then maybe it'd make sense to just get some pps to
> do it in rawhide now and get the "easy" part done with? That should
> significantly cut down on the number of misnamed packages and let
> packagers spend their times on the ones where the automatic way is not
> obvious.
I was going to ask whether it might be possible to tweak
http://fedora.portingdb.xyz/ to also report on compliance with the
naming policy, but then I went and saw that it *does* already report
on that: http://fedora.portingdb.xyz/namingpolicy/
While it also turns out the wiki page already links to that page, it
may be good to call it out a second time in a "How can I help?"
section.
Checking an initial sampling of spec files (python-d2to1,
python-BeautfulSoup, python-amqplib) suggests to me that a script
implementing the following rules might offer a reasonably start point,
at least for Python-2-only modules that are remaining Python-2-only:
- immediately before the first BuildRequires or Requires entry, add a
%package section header for "-n python2-<name>" (where
"<name>" is the
lowercased package source name with any "python-" prefix stripped)
- add a %python_provides entry after the new package header in
accordance with the current guidelines
- if the original package provided a non-lowercase "python-*"
provides, remote it and add a second %python_provides with the
original capitalisation
- if the source package lacks the "python-" prefix, add a virtual
provides for the unqualified package name
- add a "-n python2-<name>" qualifier to any currently unqualified
description and files sections
A script like that may even do a tolerable job for packages that *do*
offer Python 3 subpackages (since those will already have qualifiers,
and will necessarily appear after any unqualified runtime and build
requirements for the default subpackage).
I hacked up a script, it's on pagure now [1].
See logs [2] and the diff [3] (unfortunately no highlighting on paste.fp.o.).
This does 561 python-* packages, out of 645.
I took the list from portingdb, but it seems partially outdated, 32
packages already have %python_provide python2-*.
One more by hand (patch in repo).
That leaves 51 packages to convert by hand (mostly because of conditionals
which are too complicated to do automatically).
So, please take a look at the diff [3]. If you want to adapt the
script, ping me, and I'll add you to the repo.
After I started working on this, I limited it to python-* packages,
because those are easiest. There's 237 other packages on the portingdb
list. I *would* be possible to make the script work for those packages
which are pure python and just have an old name. I don't think it makes
sense to do other packages which just have a python subpackage this way.
This would be a good start, I think. Before being pushed anywhere,
it'd need to be checked carefully of course and extended to some of
the remaining packages. I'd apply this as one step, and rebuild everything.
After that, it'd be easier to convert *dependencies*, because all or most
python2- names should be available. Some time after current mass rebuild
is done?
Zbyszek
[1] https://pagure.io/pyrenamer/tree/master
[2] https://pagure.io/pyrenamer/blob/master/f/results.txt
[3] https://paste.fedoraproject.org/paste/57oqggmOwvZFQjDbndRfcg
4:11 p.m.
On 28.7.2017 22:44, Zbigniew Jędrzejewski-Szmek wrote:
On Fri, Jul 28, 2017 at 05:51:39PM +1000, Nick Coghlan wrote:
> On 28 July 2017 at 03:15, Zbigniew Jędrzejewski-Szmek <zbyszek(a)in.waw.pl>
wrote:
>> Do you think it'd be possible to script the python-foo to python2-foo
>> renaming? If yes, then maybe it'd make sense to just get some pps to
>> do it in rawhide now and get the "easy" part done with? That should
>> significantly cut down on the number of misnamed packages and let
>> packagers spend their times on the ones where the automatic way is not
>> obvious.
>
> I was going to ask whether it might be possible to tweak
> http://fedora.portingdb.xyz/ to also report on compliance with the
> naming policy, but then I went and saw that it *does* already report
> on that: http://fedora.portingdb.xyz/namingpolicy/
>
> While it also turns out the wiki page already links to that page, it
> may be good to call it out a second time in a "How can I help?"
> section.
>
> Checking an initial sampling of spec files (python-d2to1,
> python-BeautfulSoup, python-amqplib) suggests to me that a script
> implementing the following rules might offer a reasonably start point,
> at least for Python-2-only modules that are remaining Python-2-only:
>
> - immediately before the first BuildRequires or Requires entry, add a
> %package section header for "-n python2-<name>" (where
"<name>" is the
> lowercased package source name with any "python-" prefix stripped)
> - add a %python_provides entry after the new package header in
> accordance with the current guidelines
> - if the original package provided a non-lowercase "python-*"
> provides, remote it and add a second %python_provides with the
> original capitalisation
> - if the source package lacks the "python-" prefix, add a virtual
> provides for the unqualified package name
> - add a "-n python2-<name>" qualifier to any currently unqualified
> description and files sections
>
> A script like that may even do a tolerable job for packages that *do*
> offer Python 3 subpackages (since those will already have qualifiers,
> and will necessarily appear after any unqualified runtime and build
> requirements for the default subpackage).
I hacked up a script, it's on pagure now [1].
See logs [2] and the diff [3] (unfortunately no highlighting on paste.fp.o.).
This does 561 python-* packages, out of 645.
Wow, nice job!
I took the list from portingdb, but it seems partially outdated, 32
packages already have %python_provide python2-*.
One more by hand (patch in repo).
That leaves 51 packages to convert by hand (mostly because of conditionals
which are too complicated to do automatically).
So, please take a look at the diff [3]. If you want to adapt the
script, ping me, and I'll add you to the repo.
After I started working on this, I limited it to python-* packages,
because those are easiest. There's 237 other packages on the portingdb
list. I *would* be possible to make the script work for those packages
which are pure python and just have an old name. I don't think it makes
sense to do other packages which just have a python subpackage this way.
This would be a good start, I think. Before being pushed anywhere,
it'd need to be checked carefully of course and extended to some of
the remaining packages. I'd apply this as one step, and rebuild everything.
After that, it'd be easier to convert *dependencies*, because all or most
python2- names should be available. Some time after current mass rebuild
is done?
That sounds like something that would need to be approved by FESCo,
isn't it?
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
5:59 p.m.
On Fri, 2017-07-28 at 21:06 +0200, Till Maas wrote:
On Fri, Jul 28, 2017 at 10:29:12PM +1000, Nick Coghlan wrote:
> It's only /usr/bin/python itself that still presents an unsolved
> problem, since the status quo (not providing it at all) is even
> more user hostile than pointing it at a modern version of Python 3
> that
Why is not providing /usr/bin/python more use hostile? Why is it
better for any script to use /usr/bin/python instead of
/usr/bin/python3?
Exactly.
By default, any script using `#!/usr/bin/zsh` won't work on Fedora,
until the admin installs zsh.
We could very well consider that /usr/bin/python remains python2, and
not have it installed by default.
To be honest, given how much energy is spent on this migration for a
very low gain, it makes me feel like having an unversioned "python"
(whether as package or executable names) was a mistake we should let
disappear with Python 2.
If everything is a versioned pythonX in the future, moving to a
hypothetical Python 4 would only imply building new python4-* packages
and changing the (Build)Requires, no gymnastic with unversioned things.
--
Mathieu
6:27 p.m.
On Fri, Jul 28, 2017 at 11:11:05PM +0200, Miro Hrončok wrote:
On 28.7.2017 22:44, Zbigniew Jędrzejewski-Szmek wrote:
>On Fri, Jul 28, 2017 at 05:51:39PM +1000, Nick Coghlan wrote:
>>On 28 July 2017 at 03:15, Zbigniew Jędrzejewski-Szmek <zbyszek(a)in.waw.pl>
wrote:
>>>Do you think it'd be possible to script the python-foo to python2-foo
>>>renaming? If yes, then maybe it'd make sense to just get some pps to
>>>do it in rawhide now and get the "easy" part done with? That
should
>>>significantly cut down on the number of misnamed packages and let
>>>packagers spend their times on the ones where the automatic way is not
>>>obvious.
>>
>>I was going to ask whether it might be possible to tweak
>>http://fedora.portingdb.xyz/ to also report on compliance with the
>>naming policy, but then I went and saw that it *does* already report
>>on that: http://fedora.portingdb.xyz/namingpolicy/
>>
>>While it also turns out the wiki page already links to that page, it
>>may be good to call it out a second time in a "How can I help?"
>>section.
>>
>>Checking an initial sampling of spec files (python-d2to1,
>>python-BeautfulSoup, python-amqplib) suggests to me that a script
>>implementing the following rules might offer a reasonably start point,
>>at least for Python-2-only modules that are remaining Python-2-only:
>>
>>- immediately before the first BuildRequires or Requires entry, add a
>>%package section header for "-n python2-<name>" (where
"<name>" is the
>>lowercased package source name with any "python-" prefix stripped)
>>- add a %python_provides entry after the new package header in
>>accordance with the current guidelines
>>- if the original package provided a non-lowercase "python-*"
>>provides, remote it and add a second %python_provides with the
>>original capitalisation
>>- if the source package lacks the "python-" prefix, add a virtual
>>provides for the unqualified package name
>>- add a "-n python2-<name>" qualifier to any currently
unqualified
>>description and files sections
>>
>>A script like that may even do a tolerable job for packages that *do*
>>offer Python 3 subpackages (since those will already have qualifiers,
>>and will necessarily appear after any unqualified runtime and build
>>requirements for the default subpackage).
>
>I hacked up a script, it's on pagure now [1].
>See logs [2] and the diff [3] (unfortunately no highlighting on paste.fp.o.).
>This does 561 python-* packages, out of 645.
Wow, nice job!
>I took the list from portingdb, but it seems partially outdated, 32
>packages already have %python_provide python2-*.
>One more by hand (patch in repo).
>That leaves 51 packages to convert by hand (mostly because of conditionals
>which are too complicated to do automatically).
>
>So, please take a look at the diff [3]. If you want to adapt the
>script, ping me, and I'll add you to the repo.
>
>After I started working on this, I limited it to python-* packages,
>because those are easiest. There's 237 other packages on the portingdb
>list. I *would* be possible to make the script work for those packages
>which are pure python and just have an old name. I don't think it makes
>sense to do other packages which just have a python subpackage this way.
>
>This would be a good start, I think. Before being pushed anywhere,
>it'd need to be checked carefully of course and extended to some of
>the remaining packages. I'd apply this as one step, and rebuild everything.
>After that, it'd be easier to convert *dependencies*, because all or most
>python2- names should be available. Some time after current mass rebuild
>is done?
That sounds like something that would need to be approved by FESCo,
isn't it?
Yes. That's what I said before too. I would like to see buy-in from
you and Nick and other python gurus before submitting it anywhere else
though.
Zbyszek
12:27 p.m.
On Fri, Jul 28, 2017 at 08:44:37PM +0000, Zbigniew Jędrzejewski-Szmek wrote:
On Fri, Jul 28, 2017 at 05:51:39PM +1000, Nick Coghlan wrote:
> On 28 July 2017 at 03:15, Zbigniew Jędrzejewski-Szmek <zbyszek(a)in.waw.pl>
wrote:
> > Do you think it'd be possible to script the python-foo to python2-foo
> > renaming? If yes, then maybe it'd make sense to just get some pps to
> > do it in rawhide now and get the "easy" part done with? That should
> > significantly cut down on the number of misnamed packages and let
> > packagers spend their times on the ones where the automatic way is not
> > obvious.
>
> I was going to ask whether it might be possible to tweak
> http://fedora.portingdb.xyz/ to also report on compliance with the
> naming policy, but then I went and saw that it *does* already report
> on that: http://fedora.portingdb.xyz/namingpolicy/
>
> While it also turns out the wiki page already links to that page, it
> may be good to call it out a second time in a "How can I help?"
> section.
>
> Checking an initial sampling of spec files (python-d2to1,
> python-BeautfulSoup, python-amqplib) suggests to me that a script
> implementing the following rules might offer a reasonably start point,
> at least for Python-2-only modules that are remaining Python-2-only:
>
> - immediately before the first BuildRequires or Requires entry, add a
> %package section header for "-n python2-<name>" (where
"<name>" is the
> lowercased package source name with any "python-" prefix stripped)
> - add a %python_provides entry after the new package header in
> accordance with the current guidelines
> - if the original package provided a non-lowercase "python-*"
> provides, remote it and add a second %python_provides with the
> original capitalisation
> - if the source package lacks the "python-" prefix, add a virtual
> provides for the unqualified package name
> - add a "-n python2-<name>" qualifier to any currently unqualified
> description and files sections
>
> A script like that may even do a tolerable job for packages that *do*
> offer Python 3 subpackages (since those will already have qualifiers,
> and will necessarily appear after any unqualified runtime and build
> requirements for the default subpackage).
I hacked up a script, it's on pagure now [1].
See logs [2] and the diff [3] (unfortunately no highlighting on paste.fp.o.).
This does 561 python-* packages, out of 645.
I took the list from portingdb, but it seems partially outdated, 32
packages already have %python_provide python2-*.
One more by hand (patch in repo).
That leaves 51 packages to convert by hand (mostly because of conditionals
which are too complicated to do automatically).
So, please take a look at the diff [3]. If you want to adapt the
script, ping me, and I'll add you to the repo.
After I started working on this, I limited it to python-* packages,
because those are easiest. There's 237 other packages on the portingdb
list. I *would* be possible to make the script work for those packages
which are pure python and just have an old name. I don't think it makes
sense to do other packages which just have a python subpackage this way.
This would be a good start, I think. Before being pushed anywhere,
it'd need to be checked carefully of course and extended to some of
the remaining packages. I'd apply this as one step, and rebuild everything.
After that, it'd be easier to convert *dependencies*, because all or most
python2- names should be available. Some time after current mass rebuild
is done?
Zbyszek
[1] https://pagure.io/pyrenamer/tree/master
[2] https://pagure.io/pyrenamer/blob/master/f/results.txt
[3] https://paste.fedoraproject.org/paste/57oqggmOwvZFQjDbndRfcg
I made some fixes and updates. Some packages were converted
incorrectly, and are now skipped, and I updated the script to work for
source packages without the python- prefix.
Current stats:
'DONE': 584 (OK)
'MANY-SUBPACKAGES': 148 (not a python package)
'NEEDS-RENAME': 52, (already has a python- package, needs manual handling)
'CONDITIONALS': 47, (conditional provides/requires, needs manual handling)
'LOOKS-GOOD': 38, (already has %python_provide, probably got converted)
'DEAD': 10,
'SKIP-FILE': 4 (to complicated, like vimsyntastic)
'EXECUTABLES': 88 — just a warning that the package has stuff in /usr/bin,
and users might be confused that the package is suddenly python2-foobar.
The real solution would be to convert the executable to python3.
log: https://paste.fedoraproject.org/paste/OmhyHFGRGAOXRZf7ZVLH3g/
diff: https://paste.fedoraproject.org/paste/EaBgJRpNokY1BJc~4PtGmg/
Zbyszek
4:02 p.m.
Fedora's Python version migration needs to be coordinated with
RHEL.
The most important step would be to *NOT* change /usr/bin/python to /usr/bin/python3, but
have it provided by a separate (non-default) package. For Fedora and RHEL, this would mean
the separate package does not get installed and all scripts still requiring
/usr/bin/python will fail. Sysadmins then have to choose whether to install the compat
package or port their scripts.
4:57 p.m.
Mathieu Bridon <bochecha(a)daitauha.fr> wrote:
To be honest, given how much energy is spent on this migration for a
very low gain, it makes me feel like having an unversioned "python"
(whether as package or executable names) was a mistake we should let
disappear with Python 2.
I agree.
From an emotional point of view I fully understand the Python folks'
desire to have "Python" mean Python 3, and that's fine in marketing
contexts such as websites. But filenames and package names are technical
identifiers. They must be chosen so that they'll fulfil their functions,
and with consideration for forward and backward compatibility. Providing
an unversioned "python" serves only to lure incautious programmers into
using it where they should use a versioned name.
Björn Persson
9:02 a.m.
On 30 July 2017 at 07:57, Björn Persson <Bjorn(a)xn--rombobjrn-67a.se> wrote:
Mathieu Bridon <bochecha(a)daitauha.fr> wrote:
> To be honest, given how much energy is spent on this migration for a
> very low gain, it makes me feel like having an unversioned "python"
> (whether as package or executable names) was a mistake we should let
> disappear with Python 2.
I agree.
From an emotional point of view I fully understand the Python folks'
desire to have "Python" mean Python 3, and that's fine in marketing
contexts such as websites.
While I'm a CPython core dev in addition to being a RHEL developer,
it's mainly the latter role that leads me to consider the status quo
in Fedora as being rather problematic :)
But filenames and package names are technical
identifiers. They must be chosen so that they'll fulfil their functions,
and with consideration for forward and backward compatibility.
Right. Unfortunately, the folks saying "just use /usr/bin/python3 in
your shebang lines" are choosing to ignore systems that don't have,
and *aren't going to get* a Python 3 installation. Even if Red Hat
were to start making a native Python 3 stack available, there would
still be an awful lot of EL6 and EL7 systems around where that stack
isn't going to be installed.
So that's effectively a hard design constraint for me: folks targeting
EL6 and EL7 *are* going to have to use "/usr/bin/python" in their
shebang lines (since they can't even assume "/usr/bin/python2" will be
present, let alone "/usr/bin/python3"), so I have to assume that "just
tell Red Hat's customers to change all their Python shebang lines to
require Python 3" isn't one of the design options I have available to
me.
That means the only degrees of freedom I/we potentially have are
related to what "/usr/bin/python" means in Fedora. The status quo is
that by default such scripts simply break, and if you want them to
work, you have to install the full Python 2.7 stack. If you want to do
anything else, you have to resort to creating symlinks manually, and
as soon as you do, your system is in an unsupported configuration, so
if it breaks, any bugs you file are just going to be closed as "not a
bug".
Accordingly, my proposal is that we support *3* admin selectable
configurations (probably using the modularity tooling as the config
switching mechanism) instead of the current two:
1. Absent (the current default)
2. Python 2.7 (a legacy Python stack gets installed in parallel with
the default one)
3. Python 3.x (such scripts are run in the default Python 3 stack)
The third option is the new possibility I'm proposing, and it
acknowledges that there's an awful lot of Python 2 code that will
actually run under Python 3 once you run python-modernize over it to
drop the use of ancient idioms (especially if you allow use of the
"six" runtime compatibility library).
If we introduce the 3rd option via the modularity tooling as a new
opt-in behaviour for F27 while retaining the "absent" default
indefinitely, then when the upstream end of maintenance for Python 2.7
rolls around in 2020, we won't need to change the default, we'd just
look at dropping the entire legacy Python 2.7 stack.
Providing
an unversioned "python" serves only to lure incautious programmers into
using it where they should use a versioned name.
My aim is for folks to start thinking of a "/usr/bin/python" shebang
as being akin to writing "/bin/sh" instead of "/bin/bash": as the
author of the script, by deciding not to explicitly qualify your
shebang line you're saying "I'm not writing in either Python 2 *or*
Python 3, I'm writing in the hybrid subset of both of them". There
aren't many good reasons for a script in a distro package to do that
(they should just depend on the stack they want), but there are plenty
of good reasons for multi-distro scripts to be written that way.
As time goes by, I'd expect the meaning of the unqualified variant to
shift slightly such that "/usr/bin/python" is taken to mean "written
in the oldest Python dialect that is still actively supported by
upstream and/or commercial vendors (or potentially even older than
that)" while "/usr/bin/python3" retains its current meaning of "run
in
the default Python 3 stack" (with the two links thus becoming
functionally equivalent for systems that only have one Python stack
installed).
Cheers,
Nick.
--
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
9:28 a.m.
On Mon, 2017-07-31 at 00:02 +1000, Nick Coghlan wrote:
On 30 July 2017 at 07:57, Björn Persson <Bjorn(a)xn--rombobjrn-67a.se>
wrote:
> Mathieu Bridon <bochecha(a)daitauha.fr> wrote:
> > To be honest, given how much energy is spent on this migration
> > for a very low gain, it makes me feel like having an unversioned
> > "python" (whether as package or executable names) was a mistake
> > we should let disappear with Python 2.
>
> I agree.
>
> From an emotional point of view I fully understand the Python
> folks' desire to have "Python" mean Python 3, and that's fine in
> marketing contexts such as websites.
While I'm a CPython core dev in addition to being a RHEL developer,
it's mainly the latter role that leads me to consider the status quo
in Fedora as being rather problematic :)
> But filenames and package names are technical
> identifiers. They must be chosen so that they'll fulfil their
> functions, and with consideration for forward and backward
> compatibility.
Right. Unfortunately, the folks saying "just use /usr/bin/python3 in
your shebang lines" are choosing to ignore systems that don't have,
and *aren't going to get* a Python 3 installation. Even if Red Hat
were to start making a native Python 3 stack available, there would
still be an awful lot of EL6 and EL7 systems around where that stack
isn't going to be installed.
Sure, but nobody is telling everybody to use /usr/bin/python3 in their
shebang line.
I started this subthread saying everybody should just use the versioned
executable in their shebang line. Whether that's Python 2 or 3 depends
on what's available on their system and what they are targetting.
So that's effectively a hard design constraint for me: folks
targeting EL6 and EL7 *are* going to have to use "/usr/bin/python" in
their shebang lines (since they can't even assume "/usr/bin/python2"
will be present,
Wait, what?
I see /usr/bin/python2 in the EL7 spec file:
https://git.centos.org/blob/rpms!python.git/c7/SPECS!python.spec#L2041
I'm pretty sure I had /usr/bin/python2 back in the days when I was
using EL6. (but I have no way to check that now)
let alone "/usr/bin/python3"), so I have to assume that
"just tell
Red Hat's customers to change all their Python shebang lines to
require Python 3" isn't one of the design options I have available to
me.
And again, nobody in this subthread suggested moving things to Python
3.
That means the only degrees of freedom I/we potentially have are
related to what "/usr/bin/python" means in Fedora. The status quo is
that by default such scripts simply break, and if you want them to
work, you have to install the full Python 2.7 stack.
Which, again, is what happens for other shebang lines like /usr/bin/zsh
or /usr/bin/ruby.
Having to install the stack needed by your custom-built tools seem like
a perfectly normal thing to me.
If you want to do anything else, you have to resort to creating
symlinks manually, and as soon as you do, your system is in an
unsupported configuration, so if it breaks, any bugs you file are
just going to be closed as "not a bug".
Accordingly, my proposal is that we support *3* admin selectable
configurations
That seems like a recipe for confusion. How does one know what
/usr/bin/python actually is if it can change from one machine to
another?
> Providing an unversioned "python" serves only to lure
incautious
> programmers into using it where they should use a versioned name.
My aim is for folks to start thinking of a "/usr/bin/python" shebang
as being akin to writing "/bin/sh" instead of "/bin/bash": as the
author of the script, by deciding not to explicitly qualify your
shebang line you're saying "I'm not writing in either Python 2 *or*
Python 3, I'm writing in the hybrid subset of both of them". There
aren't many good reasons for a script in a distro package to do that
(they should just depend on the stack they want), but there are
plenty of good reasons for multi-distro scripts to be written that
way.
As time goes by, I'd expect the meaning of the unqualified variant to
shift slightly such that "/usr/bin/python" is taken to mean "written
in the oldest Python dialect that is still actively supported by
upstream and/or commercial vendors (or potentially even older than
that)" while "/usr/bin/python3" retains its current meaning of "run
in the default Python 3 stack" (with the two links thus becoming
functionally equivalent for systems that only have one Python stack
installed).
I can't see admins testing their scripts on multiple Python version. A
lot of us Python developers already don't want to bother with that, and
just target one or another. :) (or something like Python >= 3.4)
In case I wasn't clear, my proposal when I began this subthread was:
* have the versioned /usr/bin/pythonX for each major version of Python
we support;
* /usr/bin/python will remain a symlink to /usr/bin/python2 forever, as
long as we decide to keep a Python 2 stack in Fedora, and after that
it would just disappear;
This means that existing scripts using /usr/bin/python as a shebang
will keep working as long as we have Python 2 in Fedora, we won't break
them by switching what Python version is pointed to by the symlink.
It also means people wanting to move their script to Python 3 will just
opt-in to it by using the /usr/bin/python3 shebang, after which their
script will always run on Python 3.
When Python 4 comes in, we'd introduce /usr/bin/python4 and people
would have their script keep working on Python 3, until they are ready
to port them and opt-in to Python 4 by switching their shebang line.
After all, "explicit is better than implicit". ;)
--
Mathieu
9:34 a.m.
On Fri, 28 Jul 2017 21:06:45 +0200, you wrote:
On Fri, Jul 28, 2017 at 10:29:12PM +1000, Nick Coghlan wrote:
> It's only /usr/bin/python itself that still presents an unsolved
> problem, since the status quo (not providing it at all) is even more
> user hostile than pointing it at a modern version of Python 3 that
Why is not providing /usr/bin/python more use hostile? Why is it better
for any script to use /usr/bin/python instead of /usr/bin/python3?
Because any user of Fedora trying to learn Python using either books
or online sources will be told to type "python" and said user will
expect it to work.
Or they know python, use it, but don't care about the details of
versions, and so just expect to type "python" and it works because
thats what upstream decided?
And when it doesn't work, a percentage of said users will then try
another distribution or *bsd that does provide said standard of just
typing "python"
9:37 a.m.
On Fri, 28 Jul 2017 14:11:56 -0400, you wrote:
> One of Fedora's stated goals is to remain close to the
upstream.
> Upstream Python is going to change /usr/bin/python to mean Python 3. If
> AH keeps it on Python 2, it will be confusing for Python programmers
> using AH. It will be especially confusing for programmers who want to
> write software that works on "normal" Fedora and AH.
I don't want to break Ansible users or in general random sysadmin
scripts;
the value of replacing the meaning of /usr/bin/python feels quite low relative
to that.
Then perhaps someone should take this up with upstream and see if
upstream agrees with the issues and changes their plans?
2:10 p.m.
On 30 July 2017 at 10:02, Nick Coghlan <ncoghlan(a)gmail.com> wrote:
> Providing
> an unversioned "python" serves only to lure incautious programmers into
> using it where they should use a versioned name.
My aim is for folks to start thinking of a "/usr/bin/python" shebang
as being akin to writing "/bin/sh" instead of "/bin/bash": as the
author of the script, by deciding not to explicitly qualify your
shebang line you're saying "I'm not writing in either Python 2 *or*
Python 3, I'm writing in the hybrid subset of both of them". There
aren't many good reasons for a script in a distro package to do that
(they should just depend on the stack they want), but there are plenty
of good reasons for multi-distro scripts to be written that way.
If I am writing it as /bin/sh I am expecting that bash, zsh, etc are going
to fall into POSIX conformant mode outlined many years ago. Then the script
I run should work on everything from that SunOS-4.1.1 box needed for a 30
year old experiment or the brand new RHEL box.
When that doesn't happen we normally see some person opening up bugs,
complaining on lists and various other flames. And the counter flames from
people who are enjoying all the benefits of a newer shell that completely
broke when /bin/sh was used versus /bin/bash or /bin/zsh. When a developer
is working on bash, zsh internals they know that the /bin/sh must pass a
set of standards set by some committee years ago.
That last part is the issue missing here. Unless that gets decided, this
problem is getting kicked down the road every time a major 'update' in
syntax happens.
As time goes by, I'd expect the meaning of the unqualified variant to
shift slightly such that "/usr/bin/python" is taken to mean "written
in the oldest Python dialect that is still actively supported by
upstream and/or commercial vendors (or potentially even older than
that)" while "/usr/bin/python3" retains its current meaning of "run
in
the default Python 3 stack" (with the two links thus becoming
functionally equivalent for systems that only have one Python stack
installed).
Cheers,
Nick.
--
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
--
Stephen J Smoogen.
2:19 p.m.
Mathieu Bridon <bochecha(a)daitauha.fr> wrote:
On Mon, 2017-07-31 at 00:02 +1000, Nick Coghlan wrote:
> So that's effectively a hard design constraint for me: folks
> targeting EL6 and EL7 *are* going to have to use "/usr/bin/python" in
> their shebang lines (since they can't even assume "/usr/bin/python2"
> will be present,
Wait, what?
I see /usr/bin/python2 in the EL7 spec file:
https://git.centos.org/blob/rpms!python.git/c7/SPECS!python.spec#L2041
I'm pretty sure I had /usr/bin/python2 back in the days when I was
using EL6. (but I have no way to check that now)
I just checked:
$ cat /etc/redhat-release
CentOS release 6.9 (Final)
$ ll /usr/bin/python*
-rwxr-xr-x 2 root root 4864 18 aug 2016 /usr/bin/python
lrwxrwxrwx 1 root root 6 25 aug 2016 /usr/bin/python2 -> python
-rwxr-xr-x 2 root root 4864 18 aug 2016 /usr/bin/python2.6
$ rpm -q --file /usr/bin/python*
python-2.6.6-66.el6_8.x86_64
python-2.6.6-66.el6_8.x86_64
python-2.6.6-66.el6_8.x86_64
Unless this is different in RHEL than in CentOS I don't see
how /usr/bin/python2 could be missing if /usr/bin/python is present.
Björn Persson
2:40 p.m.
Gerald Henriksen <ghenriks(a)gmail.com> wrote:
On Fri, 28 Jul 2017 21:06:45 +0200, you wrote:
>Why is not providing /usr/bin/python more use hostile? Why is it better
>for any script to use /usr/bin/python instead of /usr/bin/python3?
Because any user of Fedora trying to learn Python using either books
or online sources will be told to type "python" and said user will
expect it to work.
Those instructions are indeed part of the problem.
Or they know python, use it, but don't care about the details of
versions, and so just expect to type "python" and it works because
thats what upstream decided?
If they don't care about the details of versions, then their code can't
be expected to work on other versions than the one they have on their
workstation, and their shebang line should be versioned to reflect this.
Björn Persson
6:35 a.m.
On 31 July 2017 at 05:19, Björn Persson <Bjorn(a)xn--rombobjrn-67a.se> wrote:
Mathieu Bridon <bochecha(a)daitauha.fr> wrote:
> On Mon, 2017-07-31 at 00:02 +1000, Nick Coghlan wrote:
> > So that's effectively a hard design constraint for me: folks
> > targeting EL6 and EL7 *are* going to have to use "/usr/bin/python" in
> > their shebang lines (since they can't even assume
"/usr/bin/python2"
> > will be present,
>
> Wait, what?
>
> I see /usr/bin/python2 in the EL7 spec file:
>
> https://git.centos.org/blob/rpms!python.git/c7/SPECS!python.spec#L2041
>
> I'm pretty sure I had /usr/bin/python2 back in the days when I was
> using EL6. (but I have no way to check that now)
I just checked:
$ cat /etc/redhat-release
CentOS release 6.9 (Final)
$ ll /usr/bin/python*
-rwxr-xr-x 2 root root 4864 18 aug 2016 /usr/bin/python
lrwxrwxrwx 1 root root 6 25 aug 2016 /usr/bin/python2 -> python
-rwxr-xr-x 2 root root 4864 18 aug 2016 /usr/bin/python2.6
$ rpm -q --file /usr/bin/python*
python-2.6.6-66.el6_8.x86_64
python-2.6.6-66.el6_8.x86_64
python-2.6.6-66.el6_8.x86_64
Unless this is different in RHEL than in CentOS I don't see
how /usr/bin/python2 could be missing if /usr/bin/python is present.
Yeah, I was just straight up wrong on that front - while we only added
this as far back as 2.7 upstream, it looks like Red Hat took care of
backporting the addition to both EL5 & EL6. So mea culpa for failing
to double check how portable explicit python2 references actually were
- that part of my post can be dismissed as being based on a faulty
assumption.
Cheers,
Nick.
--
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
7:07 a.m.
On 31 July 2017 at 05:40, Björn Persson <Bjorn(a)xn--rombobjrn-67a.se> wrote:
Gerald Henriksen <ghenriks(a)gmail.com> wrote:
> On Fri, 28 Jul 2017 21:06:45 +0200, you wrote:
> >Why is not providing /usr/bin/python more use hostile? Why is it better
> >for any script to use /usr/bin/python instead of /usr/bin/python3?
>
> Because any user of Fedora trying to learn Python using either books
> or online sources will be told to type "python" and said user will
> expect it to work.
Those instructions are indeed part of the problem.
Those instructions work everywhere *except* on platforms that are
currently reserving "python" to specifically mean Python 2.
On Windows? "python" may refer to either Python 2 or 3, whichever you
set up via PATH
In a Python virtual environment? "python" may refer to either Python 2
or 3, whichever is currently activated
Using environment modules, Software Collections, or nix environments?
As for Python level virtual environments
Using a cross-platform environment manager like conda? As for Python
level virtual environments
In a shebang line as "/usr/bin/env python"? As for Python level
virtual environments
System Python installations on Linux and other *nix systems are well
and truly the odd ones out at this point (and even there, a number of
distros have already decided that having "/usr/bin/python" mean
"/usr/bin/python3" is the right choice for their user base).
That said, having "python" be absent by default in Fedora Server makes
a lot of sense (for simple outright failures when running old unported
scripts), as does making it easy to point it at Python 2 (for full
backwards compatibility with RHEL 7 and derivatives, and near-full
compatibility with the Python 2.6 stack in RHEL 6 and derivatives).
Having it be absent by default in Fedora Workstation makes markedly
less sense, though, and there it's more desirable to make it point to
Python 3 (for the benefit of folks just starting to learn Python, and
for developers that prefer to use Py3 ourselves, and only need Python
2.7 for cases where we're working on an existing 2.7 application, or
else targeting RHEL or a derivative and can't use Software Collections
or container images for some reason).
Thus the idea of moving "/usr/bin/python" *out* of the python2
package, and into its own "python2-as-default" package: that allows
the decision of "What does '/usr/bin/python/ run?" to be cleanly
separated from "Which versions of Python do I have installed?". That
particular approach then wouldn't even need to rely on the new
modularity system or xdg-alternatives - it could just be a plain old
pair of mutually conflicting packages, where "python2-as-default"
(built as a subpackage of the python2 SRPM) pointed the symlink at
python2 and "python3-as-default" (built as a subpackage of the python3
SRPM) pointed it at python3.
Cheers,
Nick.
P.S. After talking to Petr Viktorin about it off-list, I *don't* think
we should do anything similar for the unqualified "python-foo" RPM
dependencies - once the Python 2.7 stack goes away, those unqualified
provides can disappear along with it. If it turns out that a future
Python 4.0 is judged to be a drop-in replacement for the last Python
3.x release, then that will be a nice problem to have, and presumably
one we'll have the appropriate tools to handle.
--
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
8:32 a.m.
On 31 July 2017 at 05:10, Stephen John Smoogen <smooge(a)gmail.com> wrote:
If I am writing it as /bin/sh I am expecting that bash, zsh, etc are
going
to fall into POSIX conformant mode outlined many years ago. Then the script
I run should work on everything from that SunOS-4.1.1 box needed for a 30
year old experiment or the brand new RHEL box.
When that doesn't happen we normally see some person opening up bugs,
complaining on lists and various other flames. And the counter flames from
people who are enjoying all the benefits of a newer shell that completely
broke when /bin/sh was used versus /bin/bash or /bin/zsh. When a developer
is working on bash, zsh internals they know that the /bin/sh must pass a set
of standards set by some committee years ago.
That last part is the issue missing here. Unless that gets decided, this
problem is getting kicked down the road every time a major 'update' in
syntax happens.
Syntactically, the de facto equivalent for Python now is code that
meets the following criteria:
- runs without warnings under "python2 -3"
- passes "pylint --py3k" without warnings
Even "async" and "await" haven't graduated to full keyword status
yet
(while they're expected to do so at some point, it's also something we
can readily postpone if doing so is seen as helpful)
Full standard library compatibility is somewhat trickier, as that's
closer to the question of "compatibility of both /bin/sh *and* the
default contents of PATH" than it is pure syntactic compatibility, but
the six compatibility library provides most of the necessary pieces.
Cheers,
Nick.
--
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
7:18 a.m.
On 29.7.2017 01:27, Zbigniew Jędrzejewski-Szmek wrote:
On Fri, Jul 28, 2017 at 11:11:05PM +0200, Miro Hrončok wrote:
> On 28.7.2017 22:44, Zbigniew Jędrzejewski-Szmek wrote:
>> On Fri, Jul 28, 2017 at 05:51:39PM +1000, Nick Coghlan wrote:
>>> On 28 July 2017 at 03:15, Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl> wrote:
>>>> Do you think it'd be possible to script the python-foo to
python2-foo
>>>> renaming? If yes, then maybe it'd make sense to just get some pps to
>>>> do it in rawhide now and get the "easy" part done with? That
should
>>>> significantly cut down on the number of misnamed packages and let
>>>> packagers spend their times on the ones where the automatic way is not
>>>> obvious.
>>>
>>> I was going to ask whether it might be possible to tweak
>>> http://fedora.portingdb.xyz/ to also report on compliance with the
>>> naming policy, but then I went and saw that it *does* already report
>>> on that: http://fedora.portingdb.xyz/namingpolicy/
>>>
>>> While it also turns out the wiki page already links to that page, it
>>> may be good to call it out a second time in a "How can I help?"
>>> section.
>>>
>>> Checking an initial sampling of spec files (python-d2to1,
>>> python-BeautfulSoup, python-amqplib) suggests to me that a script
>>> implementing the following rules might offer a reasonably start point,
>>> at least for Python-2-only modules that are remaining Python-2-only:
>>>
>>> - immediately before the first BuildRequires or Requires entry, add a
>>> %package section header for "-n python2-<name>" (where
"<name>" is the
>>> lowercased package source name with any "python-" prefix stripped)
>>> - add a %python_provides entry after the new package header in
>>> accordance with the current guidelines
>>> - if the original package provided a non-lowercase "python-*"
>>> provides, remote it and add a second %python_provides with the
>>> original capitalisation
>>> - if the source package lacks the "python-" prefix, add a virtual
>>> provides for the unqualified package name
>>> - add a "-n python2-<name>" qualifier to any currently
unqualified
>>> description and files sections
>>>
>>> A script like that may even do a tolerable job for packages that *do*
>>> offer Python 3 subpackages (since those will already have qualifiers,
>>> and will necessarily appear after any unqualified runtime and build
>>> requirements for the default subpackage).
>>
>> I hacked up a script, it's on pagure now [1].
>> See logs [2] and the diff [3] (unfortunately no highlighting on paste.fp.o.).
>> This does 561 python-* packages, out of 645.
>
> Wow, nice job!
>
>> I took the list from portingdb, but it seems partially outdated, 32
>> packages already have %python_provide python2-*.
>> One more by hand (patch in repo).
>> That leaves 51 packages to convert by hand (mostly because of conditionals
>> which are too complicated to do automatically).
>>
>> So, please take a look at the diff [3]. If you want to adapt the
>> script, ping me, and I'll add you to the repo.
>>
>> After I started working on this, I limited it to python-* packages,
>> because those are easiest. There's 237 other packages on the portingdb
>> list. I *would* be possible to make the script work for those packages
>> which are pure python and just have an old name. I don't think it makes
>> sense to do other packages which just have a python subpackage this way.
>>
>> This would be a good start, I think. Before being pushed anywhere,
>> it'd need to be checked carefully of course and extended to some of
>> the remaining packages. I'd apply this as one step, and rebuild everything.
>> After that, it'd be easier to convert *dependencies*, because all or most
>> python2- names should be available. Some time after current mass rebuild
>> is done?
>
> That sounds like something that would need to be approved by FESCo,
> isn't it?
Yes. That's what I said before too. I would like to see buy-in from
you and Nick and other python gurus before submitting it anywhere else
though.
I just had a discussion with Tomáš Orsava and Petr Viktorin on
#fedora-python. Rather than asking FESCo now to allow mass
fully-automated spec changing, we'll open bugs as planned, but we'll
attach patches generated by your script to them. We'll see if people
tend to accept them and gather any feedback. If those patched are
generally accepted, we can then use this to support our request t FESCo.
If not, we might want to change the script or abandon the idea.
Since Fedora 27 is already at the testable deadline, we would do this in
rawhide after F27 branching at a very soonest, so I guess we can spare
some time and proceed slowly.
We can revisit this discussion (whether to ask FESCo for permission to
automatically fix the specfiles) before the Proposal submission deadline
for Fedora 28 (that's January 2018). Until then, we should already have
some feedback from the open bugzillas.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
11:29 a.m.
>>>> "MH" == Miro Hrončok
<mhroncok(a)redhat.com> writes:
MH> I just had a discussion with Tomáš Orsava and Petr Viktorin on
MH> #fedora-python. Rather than asking FESCo now to allow mass
MH> fully-automated spec changing, we'll open bugs as planned, but we'll
MH> attach patches generated by your script to them.
It's not really required that you get an ACK from FESCo for automated
spec changes as long as you have a clear plan, people on hand to fix any
breakage, and let the devel list know about the changes you're making.
The packaging guidelines are pretty clear about maintainers needing to
be prepared for automated changes to their specs. But of course you can
run it by either FESCo or the packaging committee if you really want to
cover your bases.
The relevant guideline section is
https://fedoraproject.org/wiki/Packaging:Guidelines#Spec_Maintenance_and_...
Of course, if you wait just a bit you'll be able to submit pull
requests to pagure, but that could get kind of messy.
To be honest, though, just fixing things for people where you can tell
those fixes are clean and easy is far preferable to opening bugs. In
fact, it's specifically encouraged. In any case, before you get to that
please do follow the policy for pass package changes:
https://fedoraproject.org/wiki/Mass_package_changes
- J<
12:29 p.m.
On Tue, Aug 01, 2017 at 11:29:30AM -0500, Jason L Tibbitts III wrote:
>>>>> "MH" == Miro Hrončok
<mhroncok(a)redhat.com> writes:
MH> I just had a discussion with Tomáš Orsava and Petr Viktorin on
MH> #fedora-python. Rather than asking FESCo now to allow mass
MH> fully-automated spec changing, we'll open bugs as planned, but we'll
MH> attach patches generated by your script to them.
It's not really required that you get an ACK from FESCo for automated
spec changes as long as you have a clear plan, people on hand to fix any
breakage, and let the devel list know about the changes you're making.
The packaging guidelines are pretty clear about maintainers needing to
be prepared for automated changes to their specs. But of course you can
run it by either FESCo or the packaging committee if you really want to
cover your bases.
The relevant guideline section is
https://fedoraproject.org/wiki/Packaging:Guidelines#Spec_Maintenance_and_...
Of course, if you wait just a bit you'll be able to submit pull
requests to pagure, but that could get kind of messy.
To be honest, though, just fixing things for people where you can tell
those fixes are clean and easy is far preferable to opening bugs. In
fact, it's specifically encouraged. In any case, before you get to that
please do follow the policy for pass package changes:
https://fedoraproject.org/wiki/Mass_package_changes
Thank you! That was my feeling too, but I was missing the formal text
to back it up. I think the best way would indeed be to follow the
"Mass package changes" procedure, in the "automated cleanup" flavour,
but
also file a FESCo ticket to "cover all bases". I don't think it's
necessary
to file an FPC ticket because in a sense FPC has already spoken:
proposed change is to make the packages follow current guidelines,
so I don't think FPC could be reasonably opposed.
~580 can be done automatically with the current version of the script.
~60 packages already have a subpackage with a bad name. It is possible
that those could be done automatically too.
That leaves maybe 150 packages that would require further manual
changes, and for those it might make sense to file bugs. Further research
would be needed which packages are left to convert.
Zbyszek
1:55 p.m.
On 07/28/2017 02:21 PM, Miro Hrončok wrote:
On 28.7.2017 20:11, Colin Walters wrote:
> That's actually something I hadn't even considered; is it proposed
> here to actually have a flag day where all (or even some) of the
> python3 executables
> start using /usr/bin/python and everything else is /usr/bin/python2?
No. We'd like to keep it the way that everything in Fedora either uses
/usr/bin/python3 or /usr/bin/python2. To keep the possibility for the
sysadmins to change /usr/bin/python to whatever crazy target they want.
This looks
like a trend: sqlite started doing this and now python :). I
dislike being forced to specify versions after I've been on my best
behavior and made sure that my tools work with up-to-date software. If
python3 is becoming the default python interpreter, what exactly is the
argument for NOT providing an alias 'python' that runs python3?
In case of sqlite they even kept the alias that still points to sqlite2:
$ sqlite
bash: sqlite: command not found...
Install package 'sqlite2' to provide command 'sqlite'? [N/y]
You can of course override it by something like
ln -sf /bin/sqlite3 ~/bin/sqlite
but it seems like a bad practice given that it conflicts with the sqlite
v2 package. Come to think of it, this should be reported:
https://bugzilla.redhat.com/show_bug.cgi?id=1477740
12:07 a.m.
On 3 August 2017 at 04:55, Przemek Klosowski <przemek.klosowski(a)nist.gov> wrote:
This looks like a trend: sqlite started doing this and now python :).
I
dislike being forced to specify versions after I've been on my best behavior
and made sure that my tools work with up-to-date software. If python3 is
becoming the default python interpreter, what exactly is the argument for
NOT providing an alias 'python' that runs python3?
The proposed outcome is for that alias to be switchable by end users
between 3 configurations (always reports an error, always runs Python
2, always runs Python 3).
For that to be supportable, we need *distro* packages to always
specify either python2 or python3 (otherwise they'll break in the "not
specified" configuration, or when the alias points to the "wrong"
version).
Cheers,
Nick.
--
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
2452
days inactive
2463
days old
53 comments
17 participants
participants (17)
-
Björn Persson -
Brian Exelbierd -
Christian Stadelmann -
Colin Walters -
Gerald Henriksen -
Jason L Tibbitts III -
John Dennis -
Kalev Lember -
Mathieu Bridon -
Miro Hrončok -
Nick Coghlan -
Peter Robinson -
Przemek Klosowski -
Randy Barlow -
Stephen John Smoogen -
Till Maas -
Zbigniew Jędrzejewski-Szmek