Hello Pythonistas,
The pathfix.py script has been removed from future Python 3.12:
https://discuss.python.org/t/remove-outdated-tools-scripts-scripts/19571 https://github.com/python/cpython/pull/98167
In Fedora, we package it to python3-devel as /usr/bin/pathfix.py and we use it in:
%py3_shebang_fix %pyproject_install
We also use it in python3.X %install section (directly from upstream source).
-------------------------
We have to decide what to do when we package python3.12 later this month.
How to ship the script:
1) We can take the script (and its tests) and add them as additional sources to python3.12, carry on as we did before. The script was effectively only changed upstream by us, so this does not really make a big difference.
Later on, when we have N identical scripts in python3.12, 3.13 ... and we need to change things, we will need to do it in multiple places, but that has been the case until now as well. It also allows us to gradually add improvements for newer Pythons only.
This is the easiest solution in the short term.
2) We can add the script to the python-rpm-macros component instead and use one script across all Python 3 versions. It seems easier to maintain that way, but we would need to relocate the script outside of /usr/bin/ and invoke it from the macros by `%{pytohn3} <script>`, instead of the shebenag, to avoid a direct dependency on /usr/bin/python3 (to make it usable with different Python versions and to build Python itself).
This is technically backwards incompatible and we would need to do this on Rawhide+ only I guess. 35 packages BuildRequire /usr/bin/pathfix.py explicitly (presumably for %py3_shebang_fix).
I believe this is the most proper solution long term.
3) We might try to package this as python3-pathfix into %{python3_sitelib}, but I think it would be a really bad idea because we want to use it when building Pythons and this would create an even more complex bootstrap loop. We also allow using %py3_shebang_fix with various Python versions.
I'm not saying this way is not possible, but it feels wrong.
-------------------------
How to maintain the script "upstream":
A) We can put it in dist-git only. When we need changes in Fedora, we do them directly where we need them.
This is the easiest solution in the short term.
B) We can create a project @ github.com/fedora-python and add CI etc. When we need changes, we land them "upstream" first and let them propagate down or backport them.
This requires some initial work and makes changes more tedious to land, but theoretically allows easier 3rd party involvement and makes CI feedback-loop shorter (no need to e.g. scratch build Python to test the package).
I believe this is the most proper solution long term.
C) Like B) but we also make it pip-installable adn upload to PyPI. Not sure if worth it, but if we decide to go with 3) we will need to do this.
-------------------------
Alternatively, we might want to drop the script entirely and implement the functionality from scratch with Lua/grep+sed/Rust/C instead to avoid the Python-needs-Python problem.
WDYT?
On 11. 10. 22 11:47, Miro Hrončok wrote:
Hello Pythonistas,
The pathfix.py script has been removed from future Python 3.12:
https://discuss.python.org/t/remove-outdated-tools-scripts-scripts/19571 https://github.com/python/cpython/pull/98167
In Fedora, we package it to python3-devel as /usr/bin/pathfix.py and we use it in:
%py3_shebang_fix %pyproject_install
We also use it in python3.X %install section (directly from upstream source).
Somebody asked me how we use this.
1) %pyproject_install pretty much delegates to %py3_shebang_fix
2) %py3_shebang_fix uses /usr/bin/pathfix3.X.py to fix shebangs as described in https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_shebangs It can be used by packagers directly or via %pyproject_install.
3) Python %install uses it to change shebangs of the standard library to #!/usr/bin/python3.X:
https://src.fedoraproject.org/rpms/python3.11/blob/f35/f/python3.11.spec#_92...
Hi,
Some background on pathfix.py. On Fedora, pathfix.py is available as /usr/bin/pathfix.py but also /usr/bin/pathfixX.Y.py where X.Y is the Python version (ex: pathfix3.10.py). So it's an executable program (implemented in Python).
Yep, I just removed pathfix.py from Python 3.12, as part of the larger change removing most old "example scripts" of Tools/scripts/ sub-directory. Miro gave links to discussions and the issue.
pathfix.py maintenance looks mostly quiet over 28 years (1994-2022). The script was created at Aug 19, 1994. Main changes (ignoring coding style and typo changes):
* 2006: Changed logic so it now replaces anything that has #! and python in the first line, replacing the entire line. * 2009: Port to Python 3 * 2010: Add -p option to preserve timestamp * 2018: Add -n option to disable backup * 2019:
- Add -k option to preserve flags - Add -a option to add flags - Fix for Python scripts whose name contain a dash ("-")
In 2018 and 2019, 3 new flags were added by Fedora: Red Hat employees of my team :-) The latest commit was in October 2019: 3 years ago.
---
I like option (2): add the script to the python-rpm-macros component.
Since it's mostly used by Python RPM macros, IMO it makes sense to ship it with python-rpm-macros which already provides 2 other Python scripts (compileall2.py and import_all_modules.py).
I don't think that anyone need /urs/bin/pathfix.py, outside people working on packaging (who can use Python RPM macros), so it can be removed (moved to a different directory).
If someone really wants pathfix.py, a copy a be found at: https://github.com/gvanrossum/old-demos/blob/main/scripts/pathfix.py
Victor
I vote for 2B combination. Someone else might need that script too so it's a good idea to have it somewhere publicly available even it might make the maintenance a little bit harder.
Have a nice day.
Lumír
On 10/11/22 11:47, Miro Hrončok wrote:
Hello Pythonistas,
The pathfix.py script has been removed from future Python 3.12:
https://discuss.python.org/t/remove-outdated-tools-scripts-scripts/19571 https://github.com/python/cpython/pull/98167
In Fedora, we package it to python3-devel as /usr/bin/pathfix.py and we use it in:
%py3_shebang_fix %pyproject_install
We also use it in python3.X %install section (directly from upstream source).
We have to decide what to do when we package python3.12 later this month.
How to ship the script:
- We can take the script (and its tests) and add them as additional
sources to python3.12, carry on as we did before. The script was effectively only changed upstream by us, so this does not really make a big difference.
Later on, when we have N identical scripts in python3.12, 3.13 ... and we need to change things, we will need to do it in multiple places, but that has been the case until now as well. It also allows us to gradually add improvements for newer Pythons only.
This is the easiest solution in the short term.
- We can add the script to the python-rpm-macros component instead
and use one script across all Python 3 versions. It seems easier to maintain that way, but we would need to relocate the script outside of /usr/bin/ and invoke it from the macros by `%{pytohn3} <script>`, instead of the shebenag, to avoid a direct dependency on /usr/bin/python3 (to make it usable with different Python versions and to build Python itself).
This is technically backwards incompatible and we would need to do this on Rawhide+ only I guess. 35 packages BuildRequire /usr/bin/pathfix.py explicitly (presumably for %py3_shebang_fix).
I believe this is the most proper solution long term.
- We might try to package this as python3-pathfix into
%{python3_sitelib}, but I think it would be a really bad idea because we want to use it when building Pythons and this would create an even more complex bootstrap loop. We also allow using %py3_shebang_fix with various Python versions.
I'm not saying this way is not possible, but it feels wrong.
How to maintain the script "upstream":
A) We can put it in dist-git only. When we need changes in Fedora, we do them directly where we need them.
This is the easiest solution in the short term.
B) We can create a project @ github.com/fedora-python and add CI etc. When we need changes, we land them "upstream" first and let them propagate down or backport them.
This requires some initial work and makes changes more tedious to land, but theoretically allows easier 3rd party involvement and makes CI feedback-loop shorter (no need to e.g. scratch build Python to test the package).
I believe this is the most proper solution long term.
C) Like B) but we also make it pip-installable adn upload to PyPI. Not sure if worth it, but if we decide to go with 3) we will need to do this.
Alternatively, we might want to drop the script entirely and implement the functionality from scratch with Lua/grep+sed/Rust/C instead to avoid the Python-needs-Python problem.
WDYT?
python-devel@lists.fedoraproject.org
-
Lumír Balhar -
Miro Hrončok -
Victor Stinner