python-devel October 2022

python-devel@lists.fedoraproject.org

13 participants
15 discussions

Help needed with running tests in mock chroot
by Sandro 15 Oct '22

15 Oct '22

Hi, I'm stuck at trying to get tests to run for a Python package I submitted for review [1]. During build the package generates skmisc/__config__.py, which is needed by pytest. I copied it over from %pyproject_build_lib/skmisc only to hit another ImportError: E ModuleNotFoundError: No module named 'skmisc.loess._loess' The only thing that comes close is the library: skmisc/loess/_loess.cpython-311-x86_64-linux-gnu.so However, there's a _loess.pyx in skmisc/loess/src/. So, before taking this back upstream [2], I thought I check here if I may have missed some (post) build steps or if the approach taken by scikit-misc is simply not feasible in the RPM build environment. For the spec file in review I'm using %pyproject_check_import -t, which succeeds. But I'd like to run the tests if at all possible. [1] https://bugzilla.redhat.com/show_bug.cgi?id=2133438 [2] https://github.com/has2k1/scikit-misc/issues/19 -- Sandro FAS: gui1ty IRC: Penguinpee Elsewhere: [Pp]enguinpee

2 3

Python 3.12 pathfix.py removal
by Miro Hrončok 12 Oct '22

12 Oct '22

Hello Pythonistas, The pathfix.py script has been removed from future Python 3.12: https://discuss.python.org/t/remove-outdated-tools-scripts-scripts/19571 https://github.com/python/cpython/pull/98167 In Fedora, we package it to python3-devel as /usr/bin/pathfix.py and we use it in: %py3_shebang_fix %pyproject_install We also use it in python3.X %install section (directly from upstream source). ------------------------- We have to decide what to do when we package python3.12 later this month. How to ship the script: 1) We can take the script (and its tests) and add them as additional sources to python3.12, carry on as we did before. The script was effectively only changed upstream by us, so this does not really make a big difference. Later on, when we have N identical scripts in python3.12, 3.13 ... and we need to change things, we will need to do it in multiple places, but that has been the case until now as well. It also allows us to gradually add improvements for newer Pythons only. This is the easiest solution in the short term. 2) We can add the script to the python-rpm-macros component instead and use one script across all Python 3 versions. It seems easier to maintain that way, but we would need to relocate the script outside of /usr/bin/ and invoke it from the macros by `%{pytohn3} <script>`, instead of the shebenag, to avoid a direct dependency on /usr/bin/python3 (to make it usable with different Python versions and to build Python itself). This is technically backwards incompatible and we would need to do this on Rawhide+ only I guess. 35 packages BuildRequire /usr/bin/pathfix.py explicitly (presumably for %py3_shebang_fix). I believe this is the most proper solution long term. 3) We might try to package this as python3-pathfix into %{python3_sitelib}, but I think it would be a really bad idea because we want to use it when building Pythons and this would create an even more complex bootstrap loop. We also allow using %py3_shebang_fix with various Python versions. I'm not saying this way is not possible, but it feels wrong. ------------------------- How to maintain the script "upstream": A) We can put it in dist-git only. When we need changes in Fedora, we do them directly where we need them. This is the easiest solution in the short term. B) We can create a project @ github.com/fedora-python and add CI etc. When we need changes, we land them "upstream" first and let them propagate down or backport them. This requires some initial work and makes changes more tedious to land, but theoretically allows easier 3rd party involvement and makes CI feedback-loop shorter (no need to e.g. scratch build Python to test the package). I believe this is the most proper solution long term. C) Like B) but we also make it pip-installable adn upload to PyPI. Not sure if worth it, but if we decide to go with 3) we will need to do this. ------------------------- Alternatively, we might want to drop the script entirely and implement the functionality from scratch with Lua/grep+sed/Rust/C instead to avoid the Python-needs-Python problem. WDYT? -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

3 3

setuptools_scm build requirement
by Sandro 04 Oct '22

04 Oct '22

Hi, I've recently submitted the python-PyMunin3 package, which has now been approved and is available in f37 and rawhide. When porting PyMunin to Python3, I also cleaned up the install process moving everything into pyproject.toml. Little did I know back then, that this would limit the availability of the package in Fedora. As f36 is currently on setuptools 59.6.0, the package can't be build there, since a pyproject.toml with the project table in it, requires setuptools >=61 as Miro pointed out to me[1]. I've since received a request to make the package available for el9. I myself already wanted it in f36, so I set out to work on lowering the setuptools requirement[2] by moving the project table back into setup.cfg just to hit another roadblock: setuptools_scm (see ticket). That package is currently not available for el9 and the version in f35 is too old. With f35 going end of life soon, that's not a big deal, but el9 is what was requested in the ticket[2]. Yet, thinking about the whole thing, all setuptools_scm does is calculate the version of the release from information in the SCM. In the RPM build environments that information is taken from the spec file. I don't know much about the internal workings of rpmbuild, but to me any BR on python-setuptools-scm (or the toml extra) seems redundant. So, can it be dropped or in some other way omitted as a build requirement? I think that would make more Python packages available to a broader audience. Although, I haven't done any research on that. [1] https://github.com/penguinpee/PyMunin3/pull/3 [2] https://github.com/penguinpee/PyMunin3/issues/4 Happy Hacktober everyone, -- Sandro

3 9

pyproject-rpm-macros 1.4.0: support for hatchling %license files and less confusing %pyproject_check_import
by Miro Hrončok 03 Oct '22

03 Oct '22

Hello Pythonistas. We have released a new version of pyproject-rpm-macros 1.4.0. The version is available in Rawhide and ELN, and updates are ready for all older Fedora releases. https://bodhi.fedoraproject.org/updates/?packages=pyproject-rpm-macros CentOS 9 Stream update is planned as well. There are some minor documentation and CI changes, but the important 2 new features are: ## Packages using the hatchling backend now correctly mark %license files The support for %license files in %pyproject_save_files/%{pyproject_files} is based on a *draft PEP 639*: https://peps.python.org/pep-0639/ For a time, the only known backend that supported the PEP was setuptools. Packages using setuptools correctly marked %license files in %{pyproject_files}. However, since the PEP is not yet finished, is still occasionally changes. The PEP was updated recently in a way that made pyproject-rpm-macros non-complaint. It introduced a *Root License Directory*, a subdirectory of the dist-info meatdata directory. In the meantime a new build backend (hatchling 1.9.0+) implemented support for the license files following the updated PEP. As a result, packages using the hatchling backend did not mark %license files in %{pyproject_files} correctly. pyproject-rpm-macros 1.4.0 added support for the current version of the PEP while maintaining support for the old way (for compatibility with setuptools). Packages using the hatchling backend will correctly mark %license files in %{pyproject_files}. Note that only hatchling 1.9.0+ (available on Fedora 36+) supports this. (Other build backends that support the current version of PEP 639 might also exist, but we are not aware of them.) For details, see https://bugzilla.redhat.com/2127946 ## %pyproject_check_import now only imports what matches %pyproject_save_files If a packager wants to use %pyproject_check_import they need to use %pyproject_save_files first to generate a list of known modules to check-import. Previously, all non-underscroed modules were always check-imported with %pyproject_check_import. That may have lead to confusing scenarios when mutliple top-level modules were present: %install %pyproject_install # <- installs modules "one" and "two" %pyproject_save_files one # <- module called "two" is skipped %check %pyproject_check_import # <-- this was also check-importing "two" pyproject-rpm-macros 1.4.0 filters the list of modules to check-import to only try importing modules that match: %install %pyproject_install # <- installs modules "one" and "two" %pyproject_save_files one # <- module called "two" is skipped %check %pyproject_check_import # <-- this will no longer check-import "two" Note that we do not recommend having packages with more than one non-underscored top-level module, but some upstreams unfortunately do that. For details, see http://bugzilla.redhat.com/2127958 In case of trouble, file a bug at https://bugzilla.redhat.com/enter_bug.cgi?product=Fedora&component=pyprojec… or email this mailing list <python-devel(a)lists.fedoraproject.org>. Happy packaging, -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 0

Selenium 4 tarball woes
by Michel Alexandre Salim 02 Oct '22

02 Oct '22

When going over my packager dashboard, I noticed that Selenium has not been updated for a while. I cleaned up and redid https://src.fedoraproject.org/rpms/python-selenium/pull-request/3, and then converted the package to rpmautospec to make future PRs easier to merge, and also requested the epel 9 branch that's been requested several times. But when looking at bumping the version for Rawhide, it turned out that... selenium authors no longer upload their source tarballs to PyPI, and on GitHub they now only provide giant multi-lingual blobs: https://github.com/SeleniumHQ/selenium/releases Looks like Debian is also stuck, their Selenium is the first alpha of Selenium 4: https://packages.debian.org/sid/python3-selenium There's an upstream issue, and apparently it's now intermittently working, so I'll build the last 3.x series for all active releases (the last version finally has a LICENSE file) and then build the last 4.x tarball I can find on PyPI for Rawhide. If it works for rebuilding Django then I will also build that for EPEL 9. Best regards, -- Michel Alexandre Salim identities: https://keyoxide.org/5dce2e7e9c3b1cffd335c1d78b229d2f7ccc04f2

1 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

python-devel October 2022