Fedora 34 Change: Remove Python2 RPM Macros (Self-Contained Change proposal)

[Ben Cotton]

https://fedoraproject.org/wiki/Changes/Remove_Python2_RPM_Macros


== Summary ==
The {{package|python2-rpm-macros}} package (containing
`/usr/lib/rpm/macros.d/macros.python2`) will be removed from Fedora 34
and newer. The python2 RPM macros will ceases to exist. Python 2
packages are already not allowed. Around a dozen of packages use the
macros in Fedora and will need to be adjusted either by expanding them
or by no longer using Python 2. The `python2.7dist()` and
`python2dist()` automatic provides/requires will no longer be
automatically generated.

== Owner ==
* Name: [[User:Churchyard|Miro Hrončok]]
* Email: mhroncok(a)redhat.com


== Detailed Description ==
Python 2 is upstream dead, usage in Fedora packages is forbidden,
except for several package with an explicit FESCo approved exception.
The Python Maintenance team no longer wishes to support the python2
RPM macros from the {{package|python2-rpm-macros}} package (located at
`/usr/lib/rpm/macros.d/macros.python2`) and hence decided to remove
the package. The following macros will be undefined:

 %{python2_sitelib}
 %{python2_sitearch}
 %{python2_version}
 %{python2_version_nodots}
 %{python2_platform}
 %{py2_shbang_opts}
 %{py2_shbang_opts_nodash}
 %{py2_shebang_flags}
 %py2_shebang_fix
 %py2_build
 %py2_build_egg
 %py2_build_wheel
 %py2_install
 %py2_install_egg
 %py2_install_wheel

The following macros will remain defined for the time being to not
break packages that are using Python 2 as build-time only dependency
(which is also not allowed, but happens more often):

 %{__python2}
 %{python2}

The
[https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix...
Python 2 section of Python packaging guidelines] will be removed.

{{package|python2.7}} will no longer require
{{package|python2-rpm-macros}} and {{package|python3-rpm-generators}}.

Provides/requires like this will no longer be automatically generated:

 python2dist(...)
 python2.7dist(...)

The changes will happen after the Fedora 34 mass rebuild and before branching.

Packages that used to use those macros in Fedora will need to be
adapted to expand the macros or switch to Python 3. In January 2021,
we have:

* several Python 2 trac plugins, but {{package|trac}} uses Python 3 now
** only those require `python2dist(...)`
* several Python 2 sugar packages, but they already don't build and/or
install as {{package|sugar}} uses Python 3 now
* 13 other affected packages in Fedora (some of them co-owned by Python Maint):
** avahi
** gimp-layer-via-copy-cut
** gimp-resynthesizer
** chromium
** NFStest
** offlineimap
** pygobject2
** pygtk2
** python-psutil
** python-six
** python2-cairo
** python2-dns
** python2-setuptools

Note: Many other packages use the macros in a disabled `%if` section.
Some of the listed ones might as well (and hence are not really
impacted), the list is an intersection of packages that (Build)Require
Python 2 and are greppable for the macros.

The removed macros and migration plan:

=== %{python2_sitelib} ===

Affected Fedora packages:

* avahi
* NFStest
* offlineimap
* python2-dns
* python2-setuptools
* python-six

Migration plan if not possible to switch to Python 3 / retire: Use
`%{_prefix}/lib/python2.7/site-packages`.

=== %{python2_sitearch} ===

Affected Fedora packages:

* chromium
* offlineimap
* pygobject2
* pygtk2
* python2-cairo
* python-psutil

Migration plan if not possible to switch to Python 3 / retire: Use
`%{_libdir}/python2.7/site-packages`.

=== %{python2_version} ===

Affected Fedora packages:

* offlineimap
* pygtk2
* python2-setuptools

Migration plan if not possible to switch to Python 3 / retire: Use `2.7`.

=== %{python2_version_nodots} ===

No affected Fedora packages.

Migration plan: Use `27`.

=== %{python2_platform} ===

No affected Fedora packages.

Migration plan: Use a glob like `linux-*`.

=== %{py2_shbang_opts}, %{py2_shbang_opts_nodash},
%{py2_shebang_flags}, %py2_shebang_fix ===

Affected Fedora packages:

* gimp-layer-via-copy-cut
* gimp-resynthesizer
* offlineimap

All use `pathfix.py -pni "%{__python2} %{py2_shbang_opts}" ...`

Migration plan: `pathfix.py -pni "%{_bindir}/python2" -kas ...`

=== %py2_build, %py2_install ===

Affected Fedora packages:

* NFStest
* offlineimap
* python2-cairo
* python2-dns
* python2-setuptools
* python-psutil
* python-six

Migration plan:

 %build
 %set_build_flags
 python2 setup.py build

 %install
 python2 setup.py install --skip-build --root %{buildroot}

Note: Add additional `sleep 1` after/before the build if you also
build for Python 3.

=== %py2_build_egg, %py2_build_wheel, %py2_install_egg, %py2_install_wheel ===

No affected Fedora packages. There are no build dependencies in Fedora
to build a Python 2 wheel or install it.

Migration plan for egg: Please don't use this!

 %build
 %set_build_flags
 python2 setup.py bdist_egg

 %install
 python2 -m easy_install -m --prefix %{buildroot}%{_prefix} -Z <path_to_egg>


== Benefit to Fedora ==
We tried to keep the macros as similar to their Python 3 counterparts
as possible, but we have decided that it is not worth it for 13
packages. We want to discourage building Python 2 packages. We want to
remove the package from RHEL 9 as well.

== Scope ==
* Proposal owners:
** Drop the macros. Disable the `python2dist()`/`python2.7dist()`
dependency generators.
** No longer require {{package|python2-rpm-macros}} and
{{package|python3-rpm-generators}} from {{package|python2.7}}.
** Fix packages with FESCo exceptions (except already broken trac
plugins), leave the other affected packages broken for the maintainers
to fix.

* Other developers:
** Fix their packages by migrating to Python 3 or expanding the macros.
* Release engineering: no impact on Release Engineering is anticipated
* Policies and guidelines: the Python 2 guidelines will be deleted
* Trademark approval: Not neccessary.
* Alignment with Objectives: Not really.


== Upgrade/compatibility impact ==
None.

== How To Test ==
Check the macros are not defined.

== Packager Experience ==
Mildly annoying (for the affected packagers), but expected to be
temporary as the number of Python 2 packages constantly decreases.

== Dependencies ==
None.

== Contingency Plan ==
* Contingency mechanism: If this fails, we'll introduce the macros as
a separate component and orphan it.
* Contingency deadline: Beta freeze
* Blocks release? No
* Blocks product? No

== Documentation ==
This page is the documentation for the affected packagers.


-- 
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis

[Neal Gompa]

On Fri, Jan 15, 2021 at 3:57 PM Ben Cotton <bcotton(a)redhat.com&gt; wrote:
>  https://fedoraproject.org/wiki/Changes/Remove_Python2_RPM_Macros
>
>
>  == Summary ==
>  The {{package|python2-rpm-macros}} package (containing
>  `/usr/lib/rpm/macros.d/macros.python2`) will be removed from Fedora 34
>  and newer. The python2 RPM macros will ceases to exist. Python 2
>  packages are already not allowed. Around a dozen of packages use the
>  macros in Fedora and will need to be adjusted either by expanding them
>  or by no longer using Python 2. The `python2.7dist()` and
>  `python2dist()` automatic provides/requires will no longer be
>  automatically generated.
>
>  == Owner ==
>  * Name: [[User:Churchyard|Miro Hrončok]]
>  * Email: mhroncok(a)redhat.com
>
>
>  == Detailed Description ==
>  Python 2 is upstream dead, usage in Fedora packages is forbidden,
>  except for several package with an explicit FESCo approved exception.
>  The Python Maintenance team no longer wishes to support the python2
>  RPM macros from the {{package|python2-rpm-macros}} package (located at
>  `/usr/lib/rpm/macros.d/macros.python2`) and hence decided to remove
>  the package. The following macros will be undefined:
>
>   %{python2_sitelib}
>   %{python2_sitearch}
>   %{python2_version}
>   %{python2_version_nodots}
>   %{python2_platform}
>   %{py2_shbang_opts}
>   %{py2_shbang_opts_nodash}
>   %{py2_shebang_flags}
>   %py2_shebang_fix
>   %py2_build
>   %py2_build_egg
>   %py2_build_wheel
>   %py2_install
>   %py2_install_egg
>   %py2_install_wheel
>
>  The following macros will remain defined for the time being to not
>  break packages that are using Python 2 as build-time only dependency
>  (which is also not allowed, but happens more often):
>
>   %{__python2}
>   %{python2}
>
>  The
> [https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix...
>  Python 2 section of Python packaging guidelines] will be removed.
>
>  {{package|python2.7}} will no longer require
>  {{package|python2-rpm-macros}} and {{package|python3-rpm-generators}}.
>
>  Provides/requires like this will no longer be automatically generated:
>
>   python2dist(...)
>   python2.7dist(...)
>
>  The changes will happen after the Fedora 34 mass rebuild and before branching.
>
>  Packages that used to use those macros in Fedora will need to be
>  adapted to expand the macros or switch to Python 3. In January 2021,
>  we have:
>
>  * several Python 2 trac plugins, but {{package|trac}} uses Python 3 now
>  ** only those require `python2dist(...)`
>  * several Python 2 sugar packages, but they already don't build and/or
>  install as {{package|sugar}} uses Python 3 now
>  * 13 other affected packages in Fedora (some of them co-owned by Python Maint):
>  ** avahi
>  ** gimp-layer-via-copy-cut
>  ** gimp-resynthesizer
>  ** chromium
>  ** NFStest
>  ** offlineimap
>  ** pygobject2
>  ** pygtk2
>  ** python-psutil
>  ** python-six
>  ** python2-cairo
>  ** python2-dns
>  ** python2-setuptools
>
>  Note: Many other packages use the macros in a disabled `%if` section.
>  Some of the listed ones might as well (and hence are not really
>  impacted), the list is an intersection of packages that (Build)Require
>  Python 2 and are greppable for the macros.
>
>  The removed macros and migration plan:
>
>  === %{python2_sitelib} ===
>
>  Affected Fedora packages:
>
>  * avahi
>  * NFStest
>  * offlineimap
>  * python2-dns
>  * python2-setuptools
>  * python-six
>
>  Migration plan if not possible to switch to Python 3 / retire: Use
>  `%{_prefix}/lib/python2.7/site-packages`.
>
>  === %{python2_sitearch} ===
>
>  Affected Fedora packages:
>
>  * chromium
>  * offlineimap
>  * pygobject2
>  * pygtk2
>  * python2-cairo
>  * python-psutil
>
>  Migration plan if not possible to switch to Python 3 / retire: Use
>  `%{_libdir}/python2.7/site-packages`.
>
>  === %{python2_version} ===
>
>  Affected Fedora packages:
>
>  * offlineimap
>  * pygtk2
>  * python2-setuptools
>
>  Migration plan if not possible to switch to Python 3 / retire: Use `2.7`.
>
>  === %{python2_version_nodots} ===
>
>  No affected Fedora packages.
>
>  Migration plan: Use `27`.
>
>  === %{python2_platform} ===
>
>  No affected Fedora packages.
>
>  Migration plan: Use a glob like `linux-*`.
>
>  === %{py2_shbang_opts}, %{py2_shbang_opts_nodash},
>  %{py2_shebang_flags}, %py2_shebang_fix ===
>
>  Affected Fedora packages:
>
>  * gimp-layer-via-copy-cut
>  * gimp-resynthesizer
>  * offlineimap
>
>  All use `pathfix.py -pni "%{__python2} %{py2_shbang_opts}" ...`
>
>  Migration plan: `pathfix.py -pni "%{_bindir}/python2" -kas ...`
>
>  === %py2_build, %py2_install ===
>
>  Affected Fedora packages:
>
>  * NFStest
>  * offlineimap
>  * python2-cairo
>  * python2-dns
>  * python2-setuptools
>  * python-psutil
>  * python-six
>
>  Migration plan:
>
>   %build
>   %set_build_flags
>   python2 setup.py build
>
>   %install
>   python2 setup.py install --skip-build --root %{buildroot}
>
>  Note: Add additional `sleep 1` after/before the build if you also
>  build for Python 3.
>
>  === %py2_build_egg, %py2_build_wheel, %py2_install_egg, %py2_install_wheel ===
>
>  No affected Fedora packages. There are no build dependencies in Fedora
>  to build a Python 2 wheel or install it.
>
>  Migration plan for egg: Please don't use this!
>
>   %build
>   %set_build_flags
>   python2 setup.py bdist_egg
>
>   %install
>   python2 -m easy_install -m --prefix %{buildroot}%{_prefix} -Z <path_to_egg>
>
>
>  == Benefit to Fedora ==
>  We tried to keep the macros as similar to their Python 3 counterparts
>  as possible, but we have decided that it is not worth it for 13
>  packages. We want to discourage building Python 2 packages. We want to
>  remove the package from RHEL 9 as well.
>
>  == Scope ==
>  * Proposal owners:
>  ** Drop the macros. Disable the `python2dist()`/`python2.7dist()`
>  dependency generators.
>  ** No longer require {{package|python2-rpm-macros}} and
>  {{package|python3-rpm-generators}} from {{package|python2.7}}.
>  ** Fix packages with FESCo exceptions (except already broken trac
>  plugins), leave the other affected packages broken for the maintainers
>  to fix.
>
>  * Other developers:
>  ** Fix their packages by migrating to Python 3 or expanding the macros.
>  * Release engineering: no impact on Release Engineering is anticipated
>  * Policies and guidelines: the Python 2 guidelines will be deleted
>  * Trademark approval: Not neccessary.
>  * Alignment with Objectives: Not really.
>
>
>  == Upgrade/compatibility impact ==
>  None.
>
>  == How To Test ==
>  Check the macros are not defined.
>
>  == Packager Experience ==
>  Mildly annoying (for the affected packagers), but expected to be
>  temporary as the number of Python 2 packages constantly decreases.
>
>  == Dependencies ==
>  None.
>
>  == Contingency Plan ==
>  * Contingency mechanism: If this fails, we'll introduce the macros as
>  a separate component and orphan it.
>  * Contingency deadline: Beta freeze
>  * Blocks release? No
>  * Blocks product? No
>
>  == Documentation ==
>  This page is the documentation for the affected packagers.
>  
I would really rather not have this happen until we're going to retire
Python 2 entirely. The Python 2 macros are already separate from the
Python 3 ones, so we could just leave them alone until we're ready to
just remove Python 2 entirely.

Moving the Python 2 macro files (and the python2-rpm-macros package)
from the python-rpm-macros package to the python27 package would also
simplify this eventual retirement.




--
真実はいつも一つ！/ Always, there's only one truth!

[Igor Raits]

On Fri, Jan 15, 2021 at 10:58 PM Neal Gompa <ngompa13(a)gmail.com&gt; wrote:

>  On Fri, Jan 15, 2021 at 3:57 PM Ben Cotton <bcotton(a)redhat.com&gt;
> wrote:
>  >
>  > https://fedoraproject.org/wiki/Changes/Remove_Python2_RPM_Macros
>  >
>  >
>  > == Summary ==
>  > The {{package|python2-rpm-macros}} package (containing
>  > `/usr/lib/rpm/macros.d/macros.python2`) will be removed from Fedora 34
>  > and newer. The python2 RPM macros will ceases to exist. Python 2
>  > packages are already not allowed. Around a dozen of packages use the
>  > macros in Fedora and will need to be adjusted either by expanding them
>  > or by no longer using Python 2. The `python2.7dist()` and
>  > `python2dist()` automatic provides/requires will no longer be
>  > automatically generated.
>  >
>  > == Owner ==
>  > * Name: [[User:Churchyard|Miro Hrončok]]
>  > * Email: mhroncok(a)redhat.com
>  >
>  >
>  > == Detailed Description ==
>  > Python 2 is upstream dead, usage in Fedora packages is forbidden,
>  > except for several package with an explicit FESCo approved exception.
>  > The Python Maintenance team no longer wishes to support the python2
>  > RPM macros from the {{package|python2-rpm-macros}} package (located at
>  > `/usr/lib/rpm/macros.d/macros.python2`) and hence decided to remove
>  > the package. The following macros will be undefined:
>  >
>  >  %{python2_sitelib}
>  >  %{python2_sitearch}
>  >  %{python2_version}
>  >  %{python2_version_nodots}
>  >  %{python2_platform}
>  >  %{py2_shbang_opts}
>  >  %{py2_shbang_opts_nodash}
>  >  %{py2_shebang_flags}
>  >  %py2_shebang_fix
>  >  %py2_build
>  >  %py2_build_egg
>  >  %py2_build_wheel
>  >  %py2_install
>  >  %py2_install_egg
>  >  %py2_install_wheel
>  >
>  > The following macros will remain defined for the time being to not
>  > break packages that are using Python 2 as build-time only dependency
>  > (which is also not allowed, but happens more often):
>  >
>  >  %{__python2}
>  >  %{python2}
>  >
>  > The [
>
> https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix...
>  > Python 2 section of Python packaging guidelines] will be removed.
>  >
>  > {{package|python2.7}} will no longer require
>  > {{package|python2-rpm-macros}} and {{package|python3-rpm-generators}}.
>  >
>  > Provides/requires like this will no longer be automatically generated:
>  >
>  >  python2dist(...)
>  >  python2.7dist(...)
>  >
>  > The changes will happen after the Fedora 34 mass rebuild and before
>  branching.
>  >
>  > Packages that used to use those macros in Fedora will need to be
>  > adapted to expand the macros or switch to Python 3. In January 2021,
>  > we have:
>  >
>  > * several Python 2 trac plugins, but {{package|trac}} uses Python 3 now
>  > ** only those require `python2dist(...)`
>  > * several Python 2 sugar packages, but they already don't build and/or
>  > install as {{package|sugar}} uses Python 3 now
>  > * 13 other affected packages in Fedora (some of them co-owned by Python
>  Maint):
>  > ** avahi
>  > ** gimp-layer-via-copy-cut
>  > ** gimp-resynthesizer
>  > ** chromium
>  > ** NFStest
>  > ** offlineimap
>  > ** pygobject2
>  > ** pygtk2
>  > ** python-psutil
>  > ** python-six
>  > ** python2-cairo
>  > ** python2-dns
>  > ** python2-setuptools
>  >
>  > Note: Many other packages use the macros in a disabled `%if` section.
>  > Some of the listed ones might as well (and hence are not really
>  > impacted), the list is an intersection of packages that (Build)Require
>  > Python 2 and are greppable for the macros.
>  >
>  > The removed macros and migration plan:
>  >
>  > === %{python2_sitelib} ===
>  >
>  > Affected Fedora packages:
>  >
>  > * avahi
>  > * NFStest
>  > * offlineimap
>  > * python2-dns
>  > * python2-setuptools
>  > * python-six
>  >
>  > Migration plan if not possible to switch to Python 3 / retire: Use
>  > `%{_prefix}/lib/python2.7/site-packages`.
>  >
>  > === %{python2_sitearch} ===
>  >
>  > Affected Fedora packages:
>  >
>  > * chromium
>  > * offlineimap
>  > * pygobject2
>  > * pygtk2
>  > * python2-cairo
>  > * python-psutil
>  >
>  > Migration plan if not possible to switch to Python 3 / retire: Use
>  > `%{_libdir}/python2.7/site-packages`.
>  >
>  > === %{python2_version} ===
>  >
>  > Affected Fedora packages:
>  >
>  > * offlineimap
>  > * pygtk2
>  > * python2-setuptools
>  >
>  > Migration plan if not possible to switch to Python 3 / retire: Use `2.7`.
>  >
>  > === %{python2_version_nodots} ===
>  >
>  > No affected Fedora packages.
>  >
>  > Migration plan: Use `27`.
>  >
>  > === %{python2_platform} ===
>  >
>  > No affected Fedora packages.
>  >
>  > Migration plan: Use a glob like `linux-*`.
>  >
>  > === %{py2_shbang_opts}, %{py2_shbang_opts_nodash},
>  > %{py2_shebang_flags}, %py2_shebang_fix ===
>  >
>  > Affected Fedora packages:
>  >
>  > * gimp-layer-via-copy-cut
>  > * gimp-resynthesizer
>  > * offlineimap
>  >
>  > All use `pathfix.py -pni "%{__python2} %{py2_shbang_opts}" ...`
>  >
>  > Migration plan: `pathfix.py -pni "%{_bindir}/python2" -kas ...`
>  >
>  > === %py2_build, %py2_install ===
>  >
>  > Affected Fedora packages:
>  >
>  > * NFStest
>  > * offlineimap
>  > * python2-cairo
>  > * python2-dns
>  > * python2-setuptools
>  > * python-psutil
>  > * python-six
>  >
>  > Migration plan:
>  >
>  >  %build
>  >  %set_build_flags
>  >  python2 setup.py build
>  >
>  >  %install
>  >  python2 setup.py install --skip-build --root %{buildroot}
>  >
>  > Note: Add additional `sleep 1` after/before the build if you also
>  > build for Python 3.
>  >
>  > === %py2_build_egg, %py2_build_wheel, %py2_install_egg,
>  %py2_install_wheel ===
>  >
>  > No affected Fedora packages. There are no build dependencies in Fedora
>  > to build a Python 2 wheel or install it.
>  >
>  > Migration plan for egg: Please don't use this!
>  >
>  >  %build
>  >  %set_build_flags
>  >  python2 setup.py bdist_egg
>  >
>  >  %install
>  >  python2 -m easy_install -m --prefix %{buildroot}%{_prefix} -Z
>  <path_to_egg>
>  >
>  >
>  > == Benefit to Fedora ==
>  > We tried to keep the macros as similar to their Python 3 counterparts
>  > as possible, but we have decided that it is not worth it for 13
>  > packages. We want to discourage building Python 2 packages. We want to
>  > remove the package from RHEL 9 as well.
>  >
>  > == Scope ==
>  > * Proposal owners:
>  > ** Drop the macros. Disable the `python2dist()`/`python2.7dist()`
>  > dependency generators.
>  > ** No longer require {{package|python2-rpm-macros}} and
>  > {{package|python3-rpm-generators}} from {{package|python2.7}}.
>  > ** Fix packages with FESCo exceptions (except already broken trac
>  > plugins), leave the other affected packages broken for the maintainers
>  > to fix.
>  >
>  > * Other developers:
>  > ** Fix their packages by migrating to Python 3 or expanding the macros.
>  > * Release engineering: no impact on Release Engineering is anticipated
>  > * Policies and guidelines: the Python 2 guidelines will be deleted
>  > * Trademark approval: Not neccessary.
>  > * Alignment with Objectives: Not really.
>  >
>  >
>  > == Upgrade/compatibility impact ==
>  > None.
>  >
>  > == How To Test ==
>  > Check the macros are not defined.
>  >
>  > == Packager Experience ==
>  > Mildly annoying (for the affected packagers), but expected to be
>  > temporary as the number of Python 2 packages constantly decreases.
>  >
>  > == Dependencies ==
>  > None.
>  >
>  > == Contingency Plan ==
>  > * Contingency mechanism: If this fails, we'll introduce the macros as
>  > a separate component and orphan it.
>  > * Contingency deadline: Beta freeze
>  > * Blocks release? No
>  > * Blocks product? No
>  >
>  > == Documentation ==
>  > This page is the documentation for the affected packagers.
>  >
>
>  I would really rather not have this happen until we're going to retire
>  Python 2 entirely. The Python 2 macros are already separate from the
>  Python 3 ones, so we could just leave them alone until we're ready to
>  just remove Python 2 entirely.
>
>  Moving the Python 2 macro files (and the python2-rpm-macros package)
>  from the python-rpm-macros package to the python27 package would also
>  simplify this eventual retirement.
>  
Fully agree here, unless we are really going to drop python2 stuff
completely from distribution.


>  --
>  真実はいつも一つ！/ Always, there's only one truth!
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>

[Miro Hrončok]

On 15. 01. 21 22:49, Neal Gompa wrote:
>  I would really rather not have this happen until we're going to
> retire
>  Python 2 entirely. The Python 2 macros are already separate from the
>  Python 3 ones, so we could just leave them alone until we're ready to
>  just remove Python 2 entirely. 
My point here is that I want to keep Python 2 much longer (for developers who 
unfortunately still need to support it, e.g. for RHEL 7) than I'd like to keep 
allowing packages to buiid with it.

Similarly there are no macros for Python 3.5 or 3.7 in Fedora, but the Pythons 
are available.

I'd also like to stop worrying about compatibility of the Python RPM generators 
with Python 2 packages (it gets extremely hard to test, as the real word 
scenarios in Fedora are currently almost non-existent).

>  Moving the Python 2 macro files (and the python2-rpm-macros package)
>  from the python-rpm-macros package to the python27 package would also
>  simplify this eventual retirement. 
That might be the way if the proposal is rejected (or reduced to generators 
only), I'll keep that in mind, thanks.

On 15. 01. 21 22:59, Igor Raits wrote:
>  Fully agree here, unless we are really going to drop python2 stuff
> completely
>  from distribution. 
We are really removing the Python 2 "stuff" (except for the interpreter itself)

from the distribution, slowly but steadily. This change does not in general 
impact packages that happen to need Python 2 to build only, it only impacts a 
dozen of "Python 2 packages".

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Neal Gompa]

On Sat, Jan 16, 2021 at 2:50 AM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:
>  On 15. 01. 21 22:49, Neal Gompa wrote:
>  > I would really rather not have this happen until we're going to retire
>  > Python 2 entirely. The Python 2 macros are already separate from the
>  > Python 3 ones, so we could just leave them alone until we're ready to
>  > just remove Python 2 entirely.
>
>  My point here is that I want to keep Python 2 much longer (for developers who
>  unfortunately still need to support it, e.g. for RHEL 7) than I'd like to keep
>  allowing packages to buiid with it.
>
>  Similarly there are no macros for Python 3.5 or 3.7 in Fedora, but the Pythons
>  are available.
>
>  I'd also like to stop worrying about compatibility of the Python RPM generators
>  with Python 2 packages (it gets extremely hard to test, as the real word
>  scenarios in Fedora are currently almost non-existent).
>
>  > Moving the Python 2 macro files (and the python2-rpm-macros package)
>  > from the python-rpm-macros package to the python27 package would also
>  > simplify this eventual retirement.
>
>  That might be the way if the proposal is rejected (or reduced to generators
>  only), I'll keep that in mind, thanks.
>  
I am okay with disabling the generator for Python 2, since that just
reverts us to the state before we integrated my generator into Fedora
by default (which really wasn't that long ago). But removing the
macros is a step too far for my taste.

The only reason we don't have macros for building against alternate
Python 3 versions is because our macros make that difficult to automate.
One day, maybe we'll be able to fix that...

That said, running the generator on Python 3 doesn't mean it can't
read Python 2 module metadata. The format hasn't actually *changed* to
break that...




--
真実はいつも一つ！/ Always, there's only one truth!

[Nico Kadel-Garcia]

On Sat, Jan 16, 2021 at 2:50 AM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:
>  On 15. 01. 21 22:49, Neal Gompa wrote:
>  > I would really rather not have this happen until we're going to retire
>  > Python 2 entirely. The Python 2 macros are already separate from the
>  > Python 3 ones, so we could just leave them alone until we're ready to
>  > just remove Python 2 entirely.
>
>  My point here is that I want to keep Python 2 much longer (for developers who
>  unfortunately still need to support it, e.g. for RHEL 7) than I'd like to keep
>  allowing packages to buiid with it. 
May I suggest "don't bother". The ongoing requirement of python2 for
RPM in RHEL 7 should not hinder any other work. Tools published using
python 2 should, in almost all cases, be upgraded to to python 3 for
stability and supportability. I'm particularly staring at tools in
EPEL, such as awscli and ansible.

>  Similarly there are no macros for Python 3.5 or 3.7 in Fedora, but
> the Pythons
>  are available. 
And it's a problem for people who use EPEL and who use Amazon Linux 2,
which chose to jump to python 3.7 as the base for python 3 rather than
python 3.6 as RHEL 7 and RHEL 8 did.

>  I'd also like to stop worrying about compatibility of the Python
> RPM generators
>  with Python 2 packages (it gets extremely hard to test, as the real word
>  scenarios in Fedora are currently almost non-existent). 
Yeah, especially when you start trying to chase down the dependency
chains. Don't get me started on the circular dependencies for subtly
distinct versions of "sphinx" expanding those dependency chains.Or get
me going on the unwelcome and unnecessary "bundling of three distinct
AWS published modules into the "s3transfer" package.

> > Moving the Python 2 macro files (and the python2-rpm-macros package)
> > from the python-rpm-macros package to the python27 package would also
> > simplify this eventual retirement.
>
> That might be the way if the proposal is rejected (or reduced to generators
> only), I'll keep that in mind, thanks.
>
> On 15. 01. 21 22:59, Igor Raits wrote:
>  > Fully agree here, unless we are really going to drop python2 stuff completely
>  > from distribution.
>
> We are really removing the Python 2 "stuff" (except for the interpreter
itself)
> from the distribution, slowly but steadily. This change does not in general
> impact packages that happen to need Python 2 to build only, it only impacts a
> dozen of "Python 2 packages".
>
> --
> Miro Hrončok
> --
> Phone: [phone]
> IRC: mhroncok
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]

[Miroslav Suchý]

Dne 16. 01. 21 v 8:50 Miro Hrončok napsal(a):
>  My point here is that I want to keep Python 2 much longer (for
> developers who unfortunately still need to support it, 
>  e.g. for RHEL 7) than I'd like to keep allowing packages to buiid with it. 
"Developers who unfortunately still need to support it" - they will need to
build their packages. If you remove those 
macros, you will make their life harder.
This is not about Fedora developers, but about 3rd party developers who built their
projects on top of Fedora.
-- 
Miroslav Suchy, RHCA
Red Hat, Associate Manager, Community Packaging Tools, #brno, #fedora-buildsys

[Miro Hrončok]

On 18. 01. 21 10:01, Miroslav Suchý wrote:
>  Dne 16. 01. 21 v 8:50 Miro Hrončok napsal(a):
> > My point here is that I want to keep Python 2 much longer (for developers who 
> > unfortunately still need to support it, e.g. for RHEL 7) than I'd like to keep 
> > allowing packages to buiid with it.
>  
>  "Developers who unfortunately still need to support it" - they will need to 
>  build their packages. If you remove those macros, you will make their life harder.
>  This is not about Fedora developers, but about 3rd party developers who built 
>  their projects on top of Fedora. 
Note that we don't want to support building Python 2 RPM packages any more, same 
as we don't support building Python 3.6 or 3.7 packages. We support the 
alternate Python versions for developers to be able to develop on and test on 
them, we do not support actually running applications or deploying stuff.

If there are others who'd like to support Python 2 in Fedora, speak up.

(I am actually leaning towards keeping the macros when I see the feedback, just 
trying to explain our motivation.)

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Neal Gompa]

On Mon, Jan 18, 2021 at 10:13 AM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:
>  On 18. 01. 21 10:01, Miroslav Suchý wrote:
>  > Dne 16. 01. 21 v 8:50 Miro Hrončok napsal(a):
>  >> My point here is that I want to keep Python 2 much longer (for developers who
>  >> unfortunately still need to support it, e.g. for RHEL 7) than I'd like to
> keep
>  >> allowing packages to buiid with it.
>  >
>  > "Developers who unfortunately still need to support it" - they will need
> to
>  > build their packages. If you remove those macros, you will make their life harder.
>  > This is not about Fedora developers, but about 3rd party developers who built
>  > their projects on top of Fedora.
>
>  Note that we don't want to support building Python 2 RPM packages any more, same
>  as we don't support building Python 3.6 or 3.7 packages. We support the
>  alternate Python versions for developers to be able to develop on and test on
>  them, we do not support actually running applications or deploying stuff.
>  
This is something I'd like to fix in Fedora's Python 3 stack at some
point, but there's some prep work to do before this can be done.
Supporting using alternate Python 3 stacks as a third-party packager
would be very useful...

>  If there are others who'd like to support Python 2 in Fedora,
> speak up.
>
>  (I am actually leaning towards keeping the macros when I see the feedback, just
>  trying to explain our motivation.)
>  
Your motivation makes complete sense. But as a third-party packager
who occasionally has to deal with stuff that's still in Python 2, I'd
rather at least have the macros so that building stuff still works
while porting to Python 3.

If the Python 2 interpreter is completely gone, then that's a
different story, but since it's still there for a few more years...



-- 
真実はいつも一つ！/ Always, there's only one truth!

[Richard W.M. Jones]

I wasn't clear if the packages you listed in this email are all
packages in Fedora that use the Python 2 macros (seems like a
surprisingly small number if so?)

Anyway would it be possible to get a list of all packages in Fedora
still using these macros, grouped or arranged by maintainer?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW

[Miro Hrončok]

On 19. 01. 21 10:29, Richard W.M. Jones wrote:
>  I wasn't clear if the packages you listed in this email are all
>  packages in Fedora that use the Python 2 macros (seems like a
>  surprisingly small number if so?) 
Yes, except trac plugins and sugar packages, both already broken.

>  Anyway would it be possible to get a list of all packages in Fedora
>  still using these macros, grouped or arranged by maintainer? 
I have just amended the proposal not to remove the macros yet, but here you go:

Maintainers by package:
NFStest              ajmitchell steved
avahi                lennart msekleta
chromium             spot tpopela
gimp-layer-via-copy-cut design-sw luya
gimp-resynthesizer   luya
offlineimap          cicku dodji sergesanspaille teuf
pygobject2           alexl caillon caolanm limb rhughes rstrode ssp walters
pygtk2               alexl caillon caolanm limb rhughes rstrode ssp
python-psutil        salimma
python-six           apevec bkabrda churchyard mrunge pviktori
python2-cairo        kalev
python2-dns          lbalhar pghmcfc tibbs
python2-setuptools   churchyard tibbs

Packages by maintainer:
ajmitchell NFStest
alexl      pygobject2 pygtk2
apevec     python-six
bkabrda    python-six
caillon    pygobject2 pygtk2
caolanm    pygobject2 pygtk2
churchyard python-six python2-setuptools
cicku      offlineimap
design-sw  gimp-layer-via-copy-cut
dodji      offlineimap
kalev      python2-cairo
lbalhar    python2-dns
lennart    avahi
limb       pygobject2 pygtk2
luya       gimp-layer-via-copy-cut gimp-resynthesizer
mrunge     python-six
msekleta   avahi
pghmcfc    python2-dns
pviktori   python-six
rhughes    pygobject2 pygtk2
rstrode    pygobject2 pygtk2
salimma    python-psutil
sergesanspaille offlineimap
spot       chromium
ssp        pygobject2 pygtk2
steved     NFStest
teuf       offlineimap
tibbs      python2-dns python2-setuptools
tpopela    chromium
walters    pygobject2


-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

Fedora 34 Change: Disable Python 2 Dist RPM Generators and Freeze Python 2 Macros (was: Remove Python2 RPM Macros) (Self-Contained Change proposal)

[Ben Cotton]

This proposal has been heavily modified:
https://fedoraproject.org/w/index.php?title=Changes%2FDisable_Python_2_Di...

It has therefore been resubmitted.

https://fedoraproject.org/wiki/Changes/Disable_Python_2_Dist_RPM_Generato...

= Disable Python 2 Dist RPM Generators and Freeze Python 2 Macros =

== Summary ==
The `python2.7dist()` and `python2dist()` automatic provides/requires
from {{package|python-rpm-generators}} will no longer be automatically
generated, no functional packages in Fedora use them. The `python(abi)
= 2.7` automatic provides/requires will be kept.

The {{package|python2-rpm-macros}} package will be removed from Fedora
34 and newer. The python2 RPM macros
(`/usr/lib/rpm/macros.d/macros.python2`) will be moved to
{{package|python2.7}} and frozen (no new development, except critical
bugfixes that affect packages outside the remaining Python 2
ecosystem).

Note that Python 2 packages are already not allowed in Fedora, except
explicitly granted exceptions.

== Owner ==
* Name: [[User:Churchyard|Miro Hrončok]]
* Email: mhroncok(a)redhat.com


== Detailed Description ==
Python 2 is upstream dead, usage in Fedora packages is forbidden,
except for several package with an explicit FESCo approved exception.
The Python Maintenance team no longer wishes to support the python2
dist RPM generators from the {{package|python-rpm-generators}} package
nor the python2 RPM macros from the {{package|python2-rpm-macros}}
package (located at `/usr/lib/rpm/macros.d/macros.python2`) and hence
decided to disable the generators for Python 2 and freeze the macros.

Provides/requires like this will no longer be automatically generated in Fedora:

 python2dist(...)
 python2.7dist(...)

Currently, no functional Fedora 34 packages need such provides. The
`python(abi) = 2.7` automatic provides/requires will be kept.

The following macro definitions will be moved from
{{package|python2-rpm-macros}} to {{package|python2.7}}:

 %{python2_sitelib}
 %{python2_sitearch}
 %{python2_version}
 %{python2_version_nodots}
 %{python2_platform}
 %{py2_shbang_opts}
 %{py2_shbang_opts_nodash}
 %{py2_shebang_flags}
 %py2_shebang_fix
 %py2_build
 %py2_build_egg
 %py2_build_wheel
 %py2_install
 %py2_install_egg
 %py2_install_wheel

And their definition will remain frozen unless a bug is found that
affects Fedora packages outside the remaining Python 2 ecosystem.
Packagers of Python 2 packages affected by bugs in the macros are
encouraged to apply workarounds and/or submit fixes themselves.

Only the following packages use some of the macros:

* several Python 2 trac plugins, but {{package|trac}} uses Python 3 now
** only those require `python2dist(...)` / `python2.7dist(...)`
* several Python 2 sugar packages, but they already don't build and/or
install as {{package|sugar}} uses Python 3 now
* 13 other affected packages in Fedora (some of them co-owned by Python Maint):
** avahi
** gimp-layer-via-copy-cut
** gimp-resynthesizer
** chromium
** NFStest
** offlineimap
** pygobject2
** pygtk2
** python-psutil
** python-six
** python2-cairo
** python2-dns
** python2-setuptools

Note: Many other packages use the macros in a disabled `%if` section.
Some of the listed ones might as well (and hence are not really
impacted), the list is an intersection of packages that (Build)Require
Python 2 and are greppable for the macros.

{{package|python2.7}} will provide and obsolete
{{package|python2-rpm-macros}} at least for 2 more releases (or until
it is removed entirely from Fedora).

The following macros will remain defined in
{{package|python-srpm-macros}} for the time being to not break
packages that are using Python 2 as build-time only dependency (which
is also not allowed, but happens more often):

 %{__python2}
 %{python2}

The
[https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix...
Python 2 section of Python packaging guidelines] will be amended to
say that the Python 2 macros may not completely mirror the Python 3
macros.

The changes will happen after the Fedora 34 mass rebuild and before branching.

== Feedback ==
At first, we have
[https://lists.fedoraproject.org/archives/list/[e-mail].o...
proposed to remove the macros entirely]. That proposal has received
negative feedback and we have reworked it to only freeze the macros
and disable the dist generators.


== Benefit to Fedora ==
With each change of the RPM Python dist generators we had to create
artificial packages to test the impact on Python 2, because the real
set of Python 2 package in Fedora is basically non-existent. By
disabling them, we no longer have to worry about breaking things for
our downstreams or projects building atop Fedora.

In the past, we tried to keep the macros as similar to their Python 3
counterparts as possible, but we have decided that it is not worth it
for 13 packages. This way, historical packages can still be built atop
Fedora with the macros, but we don't need to worry about keeping them
up-to-date. Once {{package|python2.7}} is retired, the macros will go
away as well. This is also true for RHEL 9.


== Scope ==
* Proposal owners:
** Disable the `python2dist()`/`python2.7dist()` dependency generators.
** Move the python2 macros from {{package|python2-rpm-macros}} to
{{package|python2.7}}.
** Remove {{package|python2-rpm-macros}} and obsolete/provide it from
{{package|python2.7}}.

* Other developers: No action required.
* Release engineering: no impact on Release Engineering is anticipated
* Policies and guidelines: a note will be added to the Python 2 guidelines
* Trademark approval: Not necessary.
* Alignment with Objectives: Not really.

== Upgrade/compatibility impact ==
The {{package|python2-rpm-macros}} package will be obsoleted by
{{package|python2.7}}. Users that happened to have the macros
installed but not the interpreter will suddenly pull the interpreter
on upgrades, but we think that's a reasonable compromise.

== How To Test ==
Build a Python 2 package that used to provide and/or require
`python2.7dist()` or `python2dist()` automatically. Verify it does not
do that. Verify it still requires `python(abi) = 2.7`. Verify the
macros are still defined. Verify {{package|python2-rpm-macros}} is no
more. Verify {{package|python2.7}} does not depend on RPM.

== Packager Experience ==
Mildly worse for packagers building Python 2 RPMs atop Fedora. If such
packagers need the dist generators to do so, they are encouraged to
build their own version. Feel free to talk to the change owner about
how to do that.

Fedora packages are not affected.

== Dependencies ==
None.

== Contingency Plan ==
* Contingency mechanism: Revert the changes.
* Contingency deadline: Beta freeze.
* Blocks release? No.
* Blocks product? No.

== Documentation ==
This page is the documentation for the affected 3rd party packagers.


-- 
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis