package request: PyWaffle
by Matthew Miller
Hi! I'm using this https://github.com/gyli/PyWaffle for some visualizations for Fedora Project stats.
I'm kind of out of the loop on the state of the art of python packaging, and wondered if some kind Python SIG person would like to take it on for me.
1 year, 9 months
Automatically generated Obsoletes tags?
by Tomáš Orsava
Hi,
I'm working on a way to automatically generate Obsoletes tags for Python
packages. I.e. for each `python3-foo` package, the automatic generator
would create a tag `Obsoletes: python3.10-foo < X-Y` (assuming python3
version is 3.10).
Currently we automatically generate only Provides tags, but not
Obsoletes, and when updating to a newer "main" Python version, a user
might get file conflicts during the dnf transaction (e.g. files from
python3.10-requests would conflict with files from python3-requests). If
we have the automatic Obsoletes, `python3-foo` packages would always
take precedence over any older `python3.X-foo`, as they should.
To be frank, we don't truly need this in Fedora right now: We only ship
alternative Python interpreters (e.g. python3.8, python3.9), but we do
not ship additional packages for these alternative Pythons (e.g. we have
no python3.9-requests).
However, this will be useful for downstream distributors (e.g. CentOS,
RHEL, ...), or in cases where somebody builds custom packages for
alternative stacks in Fedora.
The downside is that we will have a few thousand (est. 3624 [1])
additional Obsoletes tags in the Fedora repos that are mostly useless.
Does anyone see a problem with this? Given the amount of tags already
present (e.g. 336 thousand provides [2], 80 thousand requires [3] and
7.5 thousand obsoletes [4]), I think it won't negatively affect
anything, but I might be mistaken.
I think it would be useful to have the generator enabled by default, but
of course to account for corner cases, I'd implement an opt-out macro
(similar to `%python_disable_dependency_generator`). And to make the
automatic generator consistent with the manual macros, I would modify
the `%py_provides` macro to also generate Obsoletes tags.
What do you think?
[1] repoquery --repo=rawhide --provides -a | grep '^python3\.10-' | wc -l
[2] repoquery --repo=rawhide --provides -a | wc -l
[3] repoquery --repo=rawhide --requires -a | wc -l
[4] repoquery --repo=rawhide --obsoletes -a | wc -l
All the best,
Tomas Orsava
1 year, 9 months
python3-augeas and python3-ansible-runner file conflict?
by Matthew Miller
Got a report on Ask Fedora(*) about an upgrade conflict, and, yep, these two have a problem:
file /usr/lib/python3.10/site-packages/test/__pycache__/__init__.cpython-310.opt-1.pyc from install of python3-ansible-runner-2.0.0a1-3.fc35.noarch conflicts with file from package python3-augeas-1.1.0-2.fc35.noarch
file /usr/lib/python3.10/site-packages/test/__pycache__/__init__.cpython-310.pyc from install of python3-ansible-runner-2.0.0a1-3.fc35.noarch conflicts with file from package python3-augeas-1.1.0-2.fc35.noarch
... but, also, the particular thing being installed under "test" and particularly "test/__pycache" seems dodgy to me. Are these things even supposed to be in either package?
* https://ask.fedoraproject.org/t/dnf-upgrade-to-f35-transaction-error/18570
1 year, 10 months
Python RPM dependency generators and LegacyVersion, time to error?
by Miro Hrončok
Hello Pythonistas.
After some recent improvements in the Python RPM dependency generators, a
regression was discovered [0].
Turns out the error happened when the upstream metadata contained a requirement
with a PEP 440 [1] incompatible version. A fix is pending and only one package
in Fedora was affected.
However, this brings me to a question: What shall we do with such incompatible
versions in the future? Implementation-wise, the generators use the
pypa/packaging library to parse the requirements. When a PEP 440 incompatible
version is found, it's parsed as a LegacyVersion [3]. A LegacyVersion has been
deprecated for 1 year now and when found it might indicate a packaging problem.
I've experimented with a patched version of the generators that would error out
right away when a LegacyVersion is parsed. I found the following categories of
problems:
1) A requirement uses invalid/legacy version (2 packages):
python-celery: pytz>dev
This is a pip-compatible shortcut for 0.dev.0, which is only required as a hack
for PEP 440 incompatible versions of pytz from before 2013 (2012a, 2012b,
2012c...).
python-pvc: pyvmomi>=5.5.0-2014.1.1
This is an existing PEP 440 incompatible version of pyvmomi that has been since
replaced with PEP 440 compatible version on PyPI.
2) The package itself has an invalid/legacy version (3 packages):
python-haversion: main
python-ipmi: unknown
python-lacrosse: unknown
This is most likely a packaging problem introduced by the Fedora packager by
using a GitHub tarball whereas the version information is only stored in sdist.
I believe the generators should error out immediately when this happens to
prevent packaging errors, but there might be a need to allow it to proceed if
we want to package e.g. pyvmomi 5.5.0-2014.1.1 or pytz 2012d.
tl;dr:
- there might be cases where legacy versions are needed
- but in most cases, they should be avoided
Hence, I propose we do the following in Rawhide:
We turn LegacyVersions to failures, but we provide a stop-gap measure, such as
(%global python_dependency_allow_legacy_version_provides 1 / %global
python_dependency_allow_legacy_version_requires 1 ) for packages that need to
override this. When pypa/packaging actually drops LegacyVersion, this stop-gap
measure will no longer work.
[0] https://bugzilla.redhat.com/show_bug.cgi?id=2019954
[1] https://www.python.org/dev/peps/pep-0440/
[2] https://packaging.pypa.io/
[3]
https://packaging.pypa.io/en/latest/version.html#packaging.version.Legacy...
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
1 year, 10 months
HEADS UP: Slightly more robust Python Requires generated by RPM
by Miro Hrončok
Hello Pythonistas,
just a heads up that in rawhide, the automatically generated requirements on
Python packages, e.g. python3.Xdist(DISTNAME), ar enow generated in more strict
form.
If a package required "PyYAML < 6", it was previously generated as:
python3.10dist(pyyaml) < 6
Which was satisfied by PyYAML 6.0b1 (python3.10dist(pyyaml) 6~b1).
From now on, it will be generated as:
python3.10dist(pyyaml) < 6~
Which will not be satisfied by any alpha/beta/rc release of PyYAML 6.
There are other related differences as well neither of which should release
break anything except for when dealing with pre-releases, it is now more
strict. Details: https://bugzilla.redhat.com/show_bug.cgi?id=2014616
Any new requires will start happening as the packages are rebuilt, so as
always, please check your packages for installability when you build them and
let me know if something behaves in a weird way.
Identical changes to %pyproject_buildrequires will land soon (on all releases,
as maintaining bit-by-bit requires compatibility is not necessary there).
Thanks to Gordon for improving the generators.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
1 year, 11 months
