Re: Mass package change (python2- binary package renaming)
by Zbigniew Jędrzejewski-Szmek
On Wed, Aug 09, 2017 at 11:04:06AM -0400, R P Herrold wrote:
> On Tue, 8 Aug 2017, Zbigniew Jędrzejewski-Szmek wrote:
>
> > Hello Fedora Python package maintainers!
> >
> > This is an announcement of a mass package renaming:
> > Python 2 binary packages will be renamed to python2-*.
>
> > List 1. for those packages which will be taken care of by the mass remaining
> > Patches: https://in.waw.pl/~zbyszek/fedora/pyrename/
>
> Looking at a sample patch file, it apperas that more than a
> simple
> s/python-/python2-/g
> replacement is being done [1]
Right. A simple replacement like that wouldn't be backwards compatible:
at the very least it would break dependent packages and violate packaging
guidelines for Python.
> The patch in question also contains 'tampering' with the
> %description stanza:
>
> -%description
> -ATpy is a high-level Python package providing a way to manipulate tables of
> -astronomical data in a uniform way. It provides built-in support for NumPy
> -recarrays and common astronomical file/database formats (FITS, VO, HDF5,
> +%global _description\
> +ATpy is a high-level Python package providing a way to manipulate tables of\
> +astronomical data in a uniform way. It provides built-in support for NumPy\
> +recarrays and common astronomical file/database formats (FITS, VO, HDF5,\
> and ASCII tables) with a very simple API.
>
> +%description %_description
> +
>
> ... this does not seem 'low risk', as substitutions happen
> form many fields. It is also not mentioned in your post
When a subpackage is added, a new %description is always required.
So this is implied by "a new subpackage is added".
A lot of effort was put into making the patches minimalistic, so they
should resemble what a human would do when updating the spec file.
> Who shall be expected to repair FTB's after the patch?
No FTBs are expected. I'll rebuild all packages locally before
pushing and I'll fix any breakage caused by my changes.
Zbyszek
6 years, 3 months
- 1
- 0
[DRAFT] Mass package change proposal
by Zbigniew Jędrzejewski-Szmek
Hi!
This is a continuation of the "Finalizing Fedora's Switch to Python 3"
thread on fedora-devel, using the procedure from
https://fedoraproject.org/wiki/Mass_package_changes.
I'm proposing an automated change to ~723 packages, and manual fixes
or follow-up bug filing for the remaining ~114 packages.
### Proposed changes ###
The change is to ensure that as many as possible python packages which
provide a version for python2 have a python2- subpackage as required by
the guidelines
[https://fedoraproject.org/wiki/Packaging:Naming#Python2_binary_package_na...].
For source packages which do *not* have a python- prefix, but already
have a python-foo subpackage, that subpackage will be renamed to python2-foo.
For packages which are called python-foo and just build a main binary package
python-foo, a new python2-foo subpackage will be created.
Provides/Requires/Conflicts/Obsoletes are moved from the main package to
the new subpackage.
In all cases, the new python2- subpackage will use lower-case naming.
This may lead to a situation where an existing python3- subpackage has
a differently-cased name than the python2- subpackage.
[Q: maybe the python3- subpackage should be renamed automatically in this case?]
An effort is made (thanks Miro!) to preserve the style which is used
in the spec file, so that if e.g. python-%{real_name} was used originally,
this is changed to python2-%{real_name}. If this macro cannot be guessed or
if it resolves to a non-lowercase name, a non-macro lower-case string will
be used.
%{python_provide python2-foobar} are added as required by the guidelines
[https://fedoraproject.org/wiki/Packaging:Python#The_.25python_provide_macro].
One or two: %{python_provide python2-foobar} is always added, and if the package
has a non-lower-case name, %{python_provide python2-FooBar} is also added.
This provides backward compatible names to the package (python-foobar, python-FooBar),
as long as %{python_provide} is defined as it is now. Once %python_provide macro
is switched to prefer python3 subpackages, those compat names will be gone.
In case the source package name does not have a python- prefix, a Provides
for the old name is also added, with a comment that it should be removed later.
Example:
+%package -n python2-atpy
+Summary: %summary
+Requires: numpy python-astropy
+%{?python_provide:%python_provide python2-atpy}
+# Remove before F30
+Provides: ATpy = %{version}-%{release}
Changes are implemented in an automated way using
https://pagure.io/pyrenamer.
rpmdev-bumpspec is used to add changelog entries will be added that look like:
* Sat Aug 05 2017 Zbigniew Jędrzejewski-Szmek <zbyszek(a)in.waw.pl> - 3.3-4
- Python 2 subpackage renamed to python2-foobar
See https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3
Current diff: https://in.waw.pl/~zbyszek/fedora/python2-renaming.diff
[https://in.waw.pl/~zbyszek/fedora/python2-renaming.color.diff is a version
will color escapes, to be downloaded and viewed with less:
curl https://in.waw.pl/~zbyszek/fedora/python2-renaming.color.diff | less -RF]
### Which packages need to change ###
http://fedora.portingdb.xyz/namingpolicy/ "Misnamed subpackage" lists 877 subpackages.
It seems that the real number is a bit lower, because there are some
dead packages and some that have been fixed in the meanwhile on that
list. I'll generate a canonical maintainer: package, package: maintainer list
for the final version.
The automated rename currently works for 723 packages, 39 don't need work.
The remaining 141 packages will require manual fixes.
Some patches will be generated manually. They will be applied at the same
time.
I do *not* want maintainers to fix packages which are covered by the
automated changes by hand, since that provides no benefit. Maintainers are
of course encouraged to fix packages which are not covered, or to take the
automated patch and use it as a basis of their own fix if the automated change
is deficient for some reason.
I'm looking for volunteers to help with the remaining ~141 packages which
cannot be done automatically. If you can participate (either as a proven
packager or not), let me know. It is possible that automated cleanup can
be applied to some more packages (e.g. I just noticed that there's a
bunch of packages which have python%{python3_pkgversion}- subpackage,
which could done automatically...), so please coordinate with me to avoid
duplicated work.
[I'll be generating all the diffs as separate files and uploading them
somewhere later this weekend.]
Currently unhandled packages:
CONDITIONALS: ahkab mod_wsgi pystache python-alembic python-ansi2html
python-backports-ssl_match_hostname python-beaker python-bugzilla
python-cairocffi python-chameleon python-cliff python-cssmin
python-datanommer-models python-deltasigma python-django-dynamite
python-django-filter python-django-post_office
python-django-socialregistration python-docker-py python-flask-images
python-flask-wtf python-fmn-lib python-fmn-web python-messaging
python-nine python-postman python-pyramid-chameleon python-qt5
python-repoze-who python-requestbuilder python-rhsm python-simplevisor
python-sparklines python-sphinxcontrib-cheeseshop
python-sqlalchemy-utils python-straight-plugin python-summershum
python-tahrir python-tahrir-api python-taskw python-trollius
python-tw2-core python-tw2-jit python-tw2-sqla python-websocket-client
vertica-python zanata-python-client
MANY-SUBPACKAGES: ansible audit brltty bro catfish ceph claws-mail
dbus-python edk2 file firewalld fts-rest ganglia gfal2-python gofed
gpaw gstreamer-python kernel kross-interpreters lcgdm libappindicator
libcaca libchewing libgpod liblouis libpeas libpwquality librapi
librra libsmbios lvm2 mirrorbrain nautilus-python nemo-extensions
openvdb pam_wrapper pjproject pki-core pulp-python pygame pygobject3
pykde4 pyke PyQt4 python2-docs python-blessed python-dirq python-drat
python-narcissus-app python-networking-bigswitch python-productmd
python-pytest-cov python-qpid python-retask python-rtslib
python-taboot pywbem qpid-cpp redland-bindings root samba sip sympy
uwsgi waf
BLACKLISTED: pcp vim-syntastic
### Opting out and independent fixes ###
The script skips all packages which already have a python2- subpackage
and %python_provide, so all packages which are fixed in the meantime will be
skipped. If packages need to be skipped even without that, let me know and
I'll add the package to the blacklist.
### Timeline ###
around 2017-08-08: finalized form of this draft is sent to devel-announce and
individual maintainers.
soon after 2017-08-15 (F27 branching): automatically and manually
generated patches are applied, and packages are rebuilt.
### Contingency plan ###
For each package, a Provides for the existing name is added, so dependent
packages should be affected. It is also not a problem if only a subset of
packages is converted.
But if things go wrong, and a package cannot be fixed manually, the last
resort is to revert the change, bump release, rebuild affected package.
### Future outlook ###
Once all or most packages have been updated to provide python2- names,
I plan to start changing dependent packages to use Requires: python2-*.
But not yet ;)
Zbyszek
PS. If you have some time, please look for errors in the diff at
https://in.waw.pl/~zbyszek/fedora/python2-renaming.color.diff.
If you have a bit more time, please join the effort and help with the
remaining packages. If your are not a provenpackager, I'll be happy to
apply patches and do the rebuilds myself.
PPS. For this kind work, Python3.6 f-strings are fan-tas-tic!
6 years, 3 months
- 1
- 0
Finalizing Fedora's Switch to Python 3
by Miro Hrončok
Hello fellow Fedora contributors,
We've created a page on Fedora wiki [1] that's something like a Change
proposal*.
The document is called "Finalizing Fedora's Switch to Python 3" and it
describes steps needed in order to:
* Switch /usr/bin/python to Python 3 in cooperation with Python upstream.
* Switch `dnf install python-foo` to install python3-foo.
* Hopefully eventually get rid of Python 2.
This is all planned to line with the fact that Python 2 will be dead
upstream by 2020.
The first phase of the changes is planned for Fedora 30, which is pretty
far away today, but the proposal involves changing thousands of
packages. Hence we'd like to have something we can use to support our
efforts.
I'd like to gather feedback for the document: whether the changes are
understandable and make sense, whether you think we should do things
differently, etc.
[1] https://fedoraproject.org/wiki/FinalizingFedoraSwitchtoPython3
* It involves steps in multiple releases, so we were advised not to
create a multi-release Change proposal, but instead split it into more.
This page summarizes it all an individual Change proposals will follow.
Thank you,
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
6 years, 4 months
- 7
- 18