Hello Pythonistas.
After some recent improvements in the Python RPM dependency generators, a regression was discovered [0].
Turns out the error happened when the upstream metadata contained a requirement with a PEP 440 [1] incompatible version. A fix is pending and only one package in Fedora was affected.
However, this brings me to a question: What shall we do with such incompatible versions in the future? Implementation-wise, the generators use the pypa/packaging library to parse the requirements. When a PEP 440 incompatible version is found, it's parsed as a LegacyVersion [3]. A LegacyVersion has been deprecated for 1 year now and when found it might indicate a packaging problem.
I've experimented with a patched version of the generators that would error out right away when a LegacyVersion is parsed. I found the following categories of problems:
1) A requirement uses invalid/legacy version (2 packages):
python-celery: pytz>dev
This is a pip-compatible shortcut for 0.dev.0, which is only required as a hack for PEP 440 incompatible versions of pytz from before 2013 (2012a, 2012b, 2012c...).
python-pvc: pyvmomi>=5.5.0-2014.1.1
This is an existing PEP 440 incompatible version of pyvmomi that has been since replaced with PEP 440 compatible version on PyPI.
2) The package itself has an invalid/legacy version (3 packages):
python-haversion: main python-ipmi: unknown python-lacrosse: unknown
This is most likely a packaging problem introduced by the Fedora packager by using a GitHub tarball whereas the version information is only stored in sdist. I believe the generators should error out immediately when this happens to prevent packaging errors, but there might be a need to allow it to proceed if we want to package e.g. pyvmomi 5.5.0-2014.1.1 or pytz 2012d.
tl;dr:
- there might be cases where legacy versions are needed - but in most cases, they should be avoided
Hence, I propose we do the following in Rawhide:
We turn LegacyVersions to failures, but we provide a stop-gap measure, such as (%global python_dependency_allow_legacy_version_provides 1 / %global python_dependency_allow_legacy_version_requires 1 ) for packages that need to override this. When pypa/packaging actually drops LegacyVersion, this stop-gap measure will no longer work.
[0] https://bugzilla.redhat.com/show_bug.cgi?id=2019954 [1] https://www.python.org/dev/peps/pep-0440/ [2] https://packaging.pypa.io/ [3] https://packaging.pypa.io/en/latest/version.html#packaging.version.LegacyVer...
On Thu, Nov 4, 2021 at 7:50 AM Miro Hrončok mhroncok@redhat.com wrote:
Hello Pythonistas.
After some recent improvements in the Python RPM dependency generators, a regression was discovered [0].
Turns out the error happened when the upstream metadata contained a requirement with a PEP 440 [1] incompatible version. A fix is pending and only one package in Fedora was affected.
However, this brings me to a question: What shall we do with such incompatible versions in the future? Implementation-wise, the generators use the pypa/packaging library to parse the requirements. When a PEP 440 incompatible version is found, it's parsed as a LegacyVersion [3]. A LegacyVersion has been deprecated for 1 year now and when found it might indicate a packaging problem.
I've experimented with a patched version of the generators that would error out right away when a LegacyVersion is parsed. I found the following categories of problems:
- A requirement uses invalid/legacy version (2 packages):
python-celery: pytz>dev
This is a pip-compatible shortcut for 0.dev.0, which is only required as a hack for PEP 440 incompatible versions of pytz from before 2013 (2012a, 2012b, 2012c...).
python-pvc: pyvmomi>=5.5.0-2014.1.1
This is an existing PEP 440 incompatible version of pyvmomi that has been since replaced with PEP 440 compatible version on PyPI.
- The package itself has an invalid/legacy version (3 packages):
python-haversion: main python-ipmi: unknown python-lacrosse: unknown
This is most likely a packaging problem introduced by the Fedora packager by using a GitHub tarball whereas the version information is only stored in sdist. I believe the generators should error out immediately when this happens to prevent packaging errors, but there might be a need to allow it to proceed if we want to package e.g. pyvmomi 5.5.0-2014.1.1 or pytz 2012d.
tl;dr:
- there might be cases where legacy versions are needed
- but in most cases, they should be avoided
Hence, I propose we do the following in Rawhide:
We turn LegacyVersions to failures, but we provide a stop-gap measure, such as (%global python_dependency_allow_legacy_version_provides 1 / %global python_dependency_allow_legacy_version_requires 1 ) for packages that need to override this. When pypa/packaging actually drops LegacyVersion, this stop-gap measure will no longer work.
[0] https://bugzilla.redhat.com/show_bug.cgi?id=2019954 [1] https://www.python.org/dev/peps/pep-0440/ [2] https://packaging.pypa.io/ [3] https://packaging.pypa.io/en/latest/version.html#packaging.version.LegacyVer...
This makes sense to me, but would it also make sense to have some documentation about this case and how to resolve it to PEP 440 compatible versions, too?
On 04. 11. 21 13:42, Neal Gompa wrote:
On Thu, Nov 4, 2021 at 7:50 AM Miro Hrončok mhroncok@redhat.com wrote:
... Hence, I propose we do the following in Rawhide:
We turn LegacyVersions to failures, but we provide a stop-gap measure, such as (%global python_dependency_allow_legacy_version_provides 1 / %global python_dependency_allow_legacy_version_requires 1 ) for packages that need to override this. When pypa/packaging actually drops LegacyVersion, this stop-gap measure will no longer work.
This makes sense to me, but would it also make sense to have some documentation about this case and how to resolve it to PEP 440 compatible versions, too?
I would definitively create a description of this in the guidelines and we could link the error to it. If we want some distro-agnostic place, we could create a document in https://github.com/rpm-software-management/python-rpm-packaging
The description should definitively contain migration advice and some actionable recommendations. However, there is no deterministic way to resolve a PEP 440 incompatible version to a PEP 440 compatible version, otherwise we would just do that.
Consider that PEP 440 incompatible version can be practically anything within a reasonable character set.
On 04. 11. 21 12:49, Miro Hrončok wrote:
Hello Pythonistas.
After some recent improvements in the Python RPM dependency generators, a regression was discovered [0].
Turns out the error happened when the upstream metadata contained a requirement with a PEP 440 [1] incompatible version. A fix is pending and only one package in Fedora was affected.
[...]
tl;dr:
- there might be cases where legacy versions are needed - but in most cases, they should be avoided
Hence, I propose we do the following in Rawhide:
We turn LegacyVersions to failures, but we provide a stop-gap measure, such as (%global python_dependency_allow_legacy_version_provides 1 / %global python_dependency_allow_legacy_version_requires 1 ) for packages that need to override this. When pypa/packaging actually drops LegacyVersion, this stop-gap measure will no longer work.
I suggest removing LegacyVersion support without replacement. I think neither creating nor using "python_dependency_allow_legacy_version_requires" would be worth the extra work. All the uses of it that we've seen were mistakes, and since upstream tools are removing support for it, the stop-gap is just delaying the inevitable. If we find an actual use case for it, it can always be added.
python-devel@lists.fedoraproject.org
-
Miro Hrončok -
Neal Gompa -
Petr Viktorin