python-devel December 2021

python-devel@lists.fedoraproject.org

16 participants
15 discussions

The great Mailman 3 / Hyperkitty upgrade: bumping flufl-lock and mistune?
by Michel Alexandre Salim 15 Dec '21

15 Dec '21

Hi all, Neal Gompa and I have been reviving the effort to get our mailing list server infrastructure (currently running on RHEL 7 with missing packages provided in an unofficial repo) hostable on RHEL 9 + EPEL. Pagure issue: https://pagure.io/fedora-infrastructure/issue/8455 Bugzilla tracker: https://bugzilla.redhat.com/show_bug.cgi?id=2030061 Status: https://hackmd.io/Pb9otlVGQHe1r9BIC5bi7w (not fully updated yet) We're currently stuck on the following: https://bugzilla.redhat.com/show_bug.cgi?id=2032607 Your package (python-hyperkitty) Fails To Install in Fedora 36: can't install hyperkitty: - nothing provides python3.10dist(flufl-lock) >= 4 needed by hyperkitty-1.3.5-1.fc36.noarch - nothing provides python3.10dist(mistune) >= 2~rc1 needed by hyperkitty-1.3.5-1.fc36.noarch I have PRs attached to the upgrade requests for mistune: https://bugzilla.redhat.com/show_bug.cgi?id=1782288 and flufl-lock: https://bugzilla.redhat.com/show_bug.cgi?id=1852603 but both have breaking changes I detailed in the above Bz entries; if you're a maintainer cc:ed on this email please check the relevant bz: ❯ sudo dnf repoquery --enablerepo=fedora-source,rawhide,rawhide-source --whatrequires python3-flufl-lock mailman3-0:3.3.4-5.fc35.noarch mailman3-0:3.3.4-5.fc35.src mailman3-0:3.3.4-5.fc36.noarch mailman3-0:3.3.4-5.fc36.src odcs-0:0.3.4-6.fc35.noarch odcs-0:0.3.4-6.fc35.src odcs-0:0.3.4-6.fc36.noarch odcs-0:0.3.4-6.fc36.src python-cartopy-0:0.20.0-1.fc35.src python-cartopy-0:0.20.1-2.fc36.src ❯ sudo dnf repoquery --enablerepo=fedora-source,rawhide,rawhide-source --whatrequires python3-mistune python-m2r-0:0.2.1-5.20190604git66f4a5a.fc35.src python-nbconvert-0:6.1.0-2.fc35.src python-nbconvert-0:6.1.0-3.fc36.src python3-m2r-0:0.2.1-5.20190604git66f4a5a.fc35.noarch python3-nbconvert-0:6.1.0-2.fc35.noarch python3-nbconvert-0:6.1.0-3.fc36.noarch In particular, python-nbconvert specifically requires mistune < 2, and upstream doesn't seem to have a newer release yet. python-cartopy oddly only requires flufl-lock in its SRPM, not the built RPM. PRs: https://src.fedoraproject.org/rpms/python-flufl-lock/pull-request/1 https://src.fedoraproject.org/rpms/python-mistune/pull-request/5 (the packages are not in side tags yet because the PRs are not merged yet, but if it helps I can build them in a COPR for F35) We should probably bump the packages in Rawhide anyway, but also to note: - both of these packages are not co-maintained by the Python SIG - most of the recent updates have been done by non-maintainers Would it make sense to get the following groups officially added to the package ACLs? - infra-sig (admin), to ease maintaining the dependencies for Mailman and Hyperkitty - python-sig (commit or admin), for fixing issues e.g. with newer Python versions - epel-packagers-sig (collaborator, epel* branches) for helping to bootstrap on new EL releases Thanks, -- Michel Alexandre Salim profile: https://keyoxide.org/michel@michel-slm.name

5 12

Provisional %{pyproject_build_lib} macro
by Miro Hrončok 15 Dec '21

15 Dec '21

Hello Pythonistas, the pyproject-rpm-macros-0-51 update (available in Rawhide+ELN and updates ready for 33 and 34) introduces a new provisional %{pyproject_build_lib} macro. The intended use case will remain the same, but it might yet change its behavior slightly. Here is the snippet from pyproject-rpm-macros README, better viewed at https://src.fedoraproject.org/rpms/pyproject-rpm-macros/blob/rawhide/f/READ… PROVISIONAL: Importing just-built (extension) modules in %build --------------------------------------------------------------- Sometimes, it is desired to be able to import the *just-built* extension modules in the `%build` section, e.g. to build the documentation with Sphinx. %build %pyproject_wheel ... build the docs here ... With pure Python packages, it might be possible to set `PYTHONPATH=${PWD}` or `PYTHONPATH=${PWD}/src`. However, it is a bit more complicated with extension modules. The location of just-built modules might differ depending on Python version, architecture, pip version. Hence, the macro `%{pyproject_build_lib}` exists to be used like this: %build %pyproject_wheel PYTHONPATH=%{pyproject_build_lib} ... build the docs here ... This macro is currently **provisional** and the behavior might change. The `%{pyproject_build_lib}` macro expands to an Shell `$(...)` expression and does not work when put into single quotes (`'`). Depending on the pip version, the expanded value will differ: ### New pip 21.3+ with in-tree-build (Fedora 36+) Always use the macro from the same directory where you called `%pyproject_wheel` from. The value will expand to something like: * `/builddir/build/BUILD/%{name}-%{version}/build/lib.linux-x86_64-3.10` for wheels with extension modules * `/builddir/build/BUILD/%{name}-%{version}/build/lib` for pure Python wheels If multiple wheels were built from the same directory, some pure Python and some with extension modules, the expanded value will be combined with `:`: * `/builddir/build/BUILD/%{name}-%{version}/build/lib.linux-x86_64-3.10:/builddir/build/BUILD/%{name}-%{version}/build/lib` If multiple wheels were built from different directories, the value will differ depending on the current directory. ### Older pip with out-of-tree-build (Fedora 34, 35, and EL 9) The value will expand to something like: * `/builddir/build/BUILD/%{name}-%{version}/.pyproject-builddir/pip-req-build-xxxxxxxx/build/lib.linux-x86_64-3.10` for wheels with extension modules * `/builddir/build/BUILD/%{name}-%{version}/.pyproject-builddir/pip-req-build-xxxxxxxx/build/lib` for pure Python wheels Note that the exact value is **not stable** between builds (the `xxxxxxxx` part is randomly generated, neither you should consider the `.pyproject-builddir` directory to remain stable). If multiple wheels are built, the expanded value will always be combined with `:` regardless of the current directory, e.g.: * `/builddir/build/BUILD/%{name}-%{version}/.pyproject-builddir/pip-req-build-xxxxxxxx/build/lib.linux-x86_64-3.10:/builddir/build/BUILD/%{name}-%{version}/.pyproject-builddir/pip-req-build-yyyyyyyy/build/lib.linux-x86_64-3.10:/builddir/build/BUILD/%{name}-%{version}/.pyproject-builddir/pip-req-build-zzzzzzzz/build/lib` **Note:** If you manage to build some wheels with in-tree-build and some with out-of-tree-build option, the expanded value will contain all relevant directories. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

2 1

Automatically generated Obsoletes tags?
by Tomáš Orsava 15 Dec '21

15 Dec '21

Hi, I'm working on a way to automatically generate Obsoletes tags for Python packages. I.e. for each `python3-foo` package, the automatic generator would create a tag `Obsoletes: python3.10-foo < X-Y` (assuming python3 version is 3.10). Currently we automatically generate only Provides tags, but not Obsoletes, and when updating to a newer "main" Python version, a user might get file conflicts during the dnf transaction (e.g. files from python3.10-requests would conflict with files from python3-requests). If we have the automatic Obsoletes, `python3-foo` packages would always take precedence over any older `python3.X-foo`, as they should. To be frank, we don't truly need this in Fedora right now: We only ship alternative Python interpreters (e.g. python3.8, python3.9), but we do not ship additional packages for these alternative Pythons (e.g. we have no python3.9-requests). However, this will be useful for downstream distributors (e.g. CentOS, RHEL, ...), or in cases where somebody builds custom packages for alternative stacks in Fedora. The downside is that we will have a few thousand (est. 3624 [1]) additional Obsoletes tags in the Fedora repos that are mostly useless. Does anyone see a problem with this? Given the amount of tags already present (e.g. 336 thousand provides [2], 80 thousand requires [3] and 7.5 thousand obsoletes [4]), I think it won't negatively affect anything, but I might be mistaken. I think it would be useful to have the generator enabled by default, but of course to account for corner cases, I'd implement an opt-out macro (similar to `%python_disable_dependency_generator`). And to make the automatic generator consistent with the manual macros, I would modify the `%py_provides` macro to also generate Obsoletes tags. What do you think? [1] repoquery --repo=rawhide --provides -a | grep '^python3\.10-' | wc -l [2] repoquery --repo=rawhide --provides -a | wc -l [3] repoquery --repo=rawhide --requires -a | wc -l [4] repoquery --repo=rawhide --obsoletes -a | wc -l All the best, Tomas Orsava

3 3

pyproject_buildrequires choked on mistune
by Michel Alexandre Salim 11 Dec '21

11 Dec '21

I'm packaging python-mistune as a dependency for hyperkitty (so we can finally pull off a Mailman3/Hyperkitty migration from the current RHEL7+custom repo setup - tracked in https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=2030061) python-mistune review: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=2031262 One weird thing is, pyproject_buildrequires fails: ``` Executing(%generate_buildrequires): /bin/sh -e /var/tmp/rpm-tmp.KZ4hKk + umask 022 + cd /builddir/build/BUILD + cd mistune-2.0.0 + echo python3-devel + echo 'python3dist(pip) >= 19' + echo 'python3dist(packaging)' + '[' -f pyproject.toml ']' + echo 'python3dist(toml)' + rm -rfv '*.dist-info/' + '[' -f /usr/bin/python3 ']' + RPM_TOXENV=py310 + HOSTNAME=rpmbuild + /usr/bin/python3 -s /usr/lib/rpm/redhat/pyproject_buildrequires.py --generate-extras --python3_pkgver sion 3 Handling setuptools from build-system.requires Requirement satisfied: setuptools (installed: setuptools 58.5.3) Handling wheel from build-system.requires Requirement not satisfied: wheel Traceback (most recent call last): File "/usr/lib/rpm/redhat/pyproject_buildrequires.py", line 421, in main generate_requires( File "/usr/lib/rpm/redhat/pyproject_buildrequires.py", line 354, in generate_requires backend = get_backend(requirements) File "/usr/lib/rpm/redhat/pyproject_buildrequires.py", line 219, in get_backend raise FileNotFoundError('File "setup.py" not found for legacy project.') FileNotFoundError: File "setup.py" not found for legacy project. ``` so I had to manually add the BRs in https://salimma.fedorapeople.org/specs/python/python-mistune.spec the `pyproject.toml` is super simple: ``` $ cat python/mistune-2.0.0/pyproject.toml [build-system] requires = [ "setuptools", "wheel" ] ``` because the project really didn't have any dependency (beyond pytest for running tests). Is this a problem with our tooling or with upstream's pyproject.toml? Happy to follow up with them if they need a more fleshed-out project definition. Thanks, -- Michel Alexandre Salim profile: https://keyoxide.org/michel@michel-slm.name

2 1

Adding python3.10dist(pp) to python-ppft
by Ankur Sinha 10 Dec '21

10 Dec '21

Hi folks, I was looking to unretire the unmaintained python-pp because some neuro packages still use it but then was reminded by @music that we already provide python-ppft which is a maintained fork and drop-in replacement for pp. https://github.com/uqfoundation/ppft Currently python3-ppft provides these: $ rpm -q --provides python3-ppft python-ppft = 1.6.6.4-2.fc35 python3-ppft = 1.6.6.4-2.fc35 python3.10-ppft = 1.6.6.4-2.fc35 python3.10dist(ppft) = 1.6.6.4 python3dist(ppft) = 1.6.6.4 Can we also provide all these for `pp` so that packages in Fedora that require pp automatically use ppft? I tried adding `%py_provides pp` to python3-ppft, and while that does add a few of these, it does not add the python3.10dist(pp) or python3dist(pp) ones (which are autogenerated from the metadata?): $ rpm -q --provides -p ./results_python-ppft/1.6.6.4/3.fc36/python3-ppft-1.6.6.4-3.fc36.noarch.rpm python-pp = 1.6.6.4-3.fc36 python-ppft = 1.6.6.4-3.fc36 python3-pp = 1.6.6.4-3.fc36 python3-ppft = 1.6.6.4-3.fc36 python3.10-pp = 1.6.6.4-3.fc36 python3.10-ppft = 1.6.6.4-3.fc36 python3.10dist(ppft) = 1.6.6.4 python3dist(ppft) = 1.6.6.4 Would there be a way to include these machine readable provides too since these forms are used by other packages in the automatically generated requires etc. (if this is the right thing to do)? I guess the alternative would be that every package that requires pp be patched to require ppft, but tweaking a single python-ppft package would be less work. :) -- Thanks, Regards, Ankur Sinha "FranciscoD" (He / Him / His) | https://fedoraproject.org/wiki/User:Ankursinha Time zone: Europe/London

2 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

python-devel December 2021