Hello.
I am working on the RPM pyproject declarative buildsystem.
t;dr it turns this:
BuildSystem: pyproject
into this:
%prep %autosetup -p1 -C
%generate_buildrequires %pyproject_buildrequires
%build %pyproject_wheel
%install %pyproject_install %pyproject_save_files ...
%check %pyproject_check_import
It allows to override options to sections with BuildOption.
For detailed documentation, see https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/455 (it has README changes in it).
---------------------------------
What should be the default for BuildOption(install)? That is, what should be passed to %pyproject_save_files if the packager does not override it?
1. The safe+hard default: nothing =================================
The %pyproject_save_files macro currently has no default. It requires at least one argument. This is to avoid accidentally packaging something we don't want. Explicit is better than implicit.
For this reason, users of the pyproject declarative buildsystem would be required to always set BuildOption(install) explicitly.
Why I don't like this option: The declarative buildsystem was invented to make spec files simpler. Making BuildOption(install) mandatory defeats the purpose.
2. The easy+dangerous default: '*' ==================================
When we pass '*' to %pyproject_save_files by default, we ensure all packages work without BuildOption(install). This is the easy way. Everything is automagic, no boilerplate necessary.
Passing '*' to %pyproject_save_files is forbidden by the Fedora packaging guidelines because it is dangerous. If upstream removes a Python module or (perhaps accidentally) adds a new one that conflicts with other packages, the Fedora packager should notice.
Why I don't like this option: The default is forbidden in Fedora and hence it is only useful elsewhere (e.g. Copr). Defaults should encourage the desired pattern, defaults should not be forbidden. Packagers will use the default and the rule will be ignored as not practical.
3. Guess the module name: python-foo -> foo ===========================================
When the package is named somehow, we like the Python modules to be named identical-ish. We can create a transformation of %{name} and make that the default. Something like this:
1. Strip python- prefix if found. 2. Strip everything after a dot (including the dot) if found. 3. Convert dashes, dots etc. to underscores.
Some nice examples:
python-requests -> requests python-zope.interface -> zope python-flit-core -> flit_core tomcli -> tomcli
Examples that would need manual BuildOption(install):
python-zope-interface -> zope_interface (we need zope) python-djangorestframework -> djangorestframework (we need rest_framework) pytest -> pytest (we need pytest _pytest py)
This would produce a reasonable default that is easy and safe for sane packages but it is not dangerous for the weird ones.
Option 3a) is to do this in the pyproject declarative buildsytsem macros implementation. Explicit callers of %pyproject_save_files would not be able to use this, which only has downsides.
Option 3b) is to make this the behavior of %pyproject_save_files when it has no arguments. If additional module globs are necessary, all module globs need to be passed explicitly.
Option 3c) is to make this the behavior of `%pyproject_save_files -n` and make -n the default BuildOption(install). Users of %pyproject_save_files could combine -n with explicit module glob list (e.g. `%pyproject_save_files -n _pytest py` for pytest).
Why I don't like this: Its' heuristics. It smells like guesswork. I prefer things explicit.
4. Default to %{pypi_name} ==========================
Similarly to option 3, we prefer the PyPI name and the module name to match.
Hence, we could default to %{pypi_name} or %{srcname} like %{pypi_source} does.
There would still need to be some transformations applied, because %{pypi_name} can contain dashes and dots, etc...
Why I don't like this: %{pypi_name} is not a cardinal part of a package. It's not part of the Python packaging guidelines and does not have a clearly defined semantics (is it the project name or the name part of the source tarball?). It encourages packagers to define the macro and use it everywhere in the spec, making it harder to read. Moreover, we need to apply text transformations to it anyway, hence we might as well do them on %{name} directly (which does not have clear semantics either, but is always there).
---------------------------------
If I had to pick, I'd go with 3c. IMHO it is the most sane option.
What are your thoughts? Do you see other options?
On 7/3/24 12:07, Miro Hrončok wrote:
Hello.
I am working on the RPM pyproject declarative buildsystem.
t;dr it turns this:
BuildSystem: pyproject
into this:
%prep %autosetup -p1 -C
%generate_buildrequires %pyproject_buildrequires
%build %pyproject_wheel
%install %pyproject_install %pyproject_save_files ...
%check %pyproject_check_import
It allows to override options to sections with BuildOption.
For detailed documentation, see https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/455 (it has README changes in it).
What should be the default for BuildOption(install)? That is, what should be passed to %pyproject_save_files if the packager does not override it?
- The safe+hard default: nothing
=================================
The %pyproject_save_files macro currently has no default. It requires at least one argument. This is to avoid accidentally packaging something we don't want. Explicit is better than implicit.
For this reason, users of the pyproject declarative buildsystem would be required to always set BuildOption(install) explicitly.
Why I don't like this option: The declarative buildsystem was invented to make spec files simpler. Making BuildOption(install) mandatory defeats the purpose.
I prefer option 1. This is something the users *should* actively check and verify, so having 1 extra line for it seems reasonable to me. We're still saving dozen+ lines of boilerplate and replacing it with one line of meaningful input. That's much easier to use in view.
Tomas
- The easy+dangerous default: '*'
==================================
When we pass '*' to %pyproject_save_files by default, we ensure all packages work without BuildOption(install). This is the easy way. Everything is automagic, no boilerplate necessary.
Passing '*' to %pyproject_save_files is forbidden by the Fedora packaging guidelines because it is dangerous. If upstream removes a Python module or (perhaps accidentally) adds a new one that conflicts with other packages, the Fedora packager should notice.
Why I don't like this option: The default is forbidden in Fedora and hence it is only useful elsewhere (e.g. Copr). Defaults should encourage the desired pattern, defaults should not be forbidden. Packagers will use the default and the rule will be ignored as not practical.
- Guess the module name: python-foo -> foo
===========================================
When the package is named somehow, we like the Python modules to be named identical-ish. We can create a transformation of %{name} and make that the default. Something like this:
1. Strip python- prefix if found. 2. Strip everything after a dot (including the dot) if found. 3. Convert dashes, dots etc. to underscores.
Some nice examples:
python-requests -> requests python-zope.interface -> zope python-flit-core -> flit_core tomcli -> tomcli
Examples that would need manual BuildOption(install):
python-zope-interface -> zope_interface (we need zope) python-djangorestframework -> djangorestframework (we need rest_framework) pytest -> pytest (we need pytest _pytest py)
This would produce a reasonable default that is easy and safe for sane packages but it is not dangerous for the weird ones.
Option 3a) is to do this in the pyproject declarative buildsytsem macros implementation. Explicit callers of %pyproject_save_files would not be able to use this, which only has downsides.
Option 3b) is to make this the behavior of %pyproject_save_files when it has no arguments. If additional module globs are necessary, all module globs need to be passed explicitly.
Option 3c) is to make this the behavior of `%pyproject_save_files -n` and make -n the default BuildOption(install). Users of %pyproject_save_files could combine -n with explicit module glob list (e.g. `%pyproject_save_files -n _pytest py` for pytest).
Why I don't like this: Its' heuristics. It smells like guesswork. I prefer things explicit.
- Default to %{pypi_name}
==========================
Similarly to option 3, we prefer the PyPI name and the module name to match.
Hence, we could default to %{pypi_name} or %{srcname} like %{pypi_source} does.
There would still need to be some transformations applied, because %{pypi_name} can contain dashes and dots, etc...
Why I don't like this: %{pypi_name} is not a cardinal part of a package. It's not part of the Python packaging guidelines and does not have a clearly defined semantics (is it the project name or the name part of the source tarball?). It encourages packagers to define the macro and use it everywhere in the spec, making it harder to read. Moreover, we need to apply text transformations to it anyway, hence we might as well do them on %{name} directly (which does not have clear semantics either, but is always there).
If I had to pick, I'd go with 3c. IMHO it is the most sane option.
What are your thoughts? Do you see other options?
On 03. 07. 24 15:00, Tomáš Orsava wrote:
On 7/3/24 12:07, Miro Hrončok wrote:
Hello.
I am working on the RPM pyproject declarative buildsystem.
t;dr it turns this:
BuildSystem: pyproject
into this:
%prep %autosetup -p1 -C
%generate_buildrequires %pyproject_buildrequires
%build %pyproject_wheel
%install %pyproject_install %pyproject_save_files ...
%check %pyproject_check_import
It allows to override options to sections with BuildOption.
For detailed documentation, see https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/455 (it has README changes in it).
What should be the default for BuildOption(install)? That is, what should be passed to %pyproject_save_files if the packager does not override it?
- The safe+hard default: nothing
=================================
The %pyproject_save_files macro currently has no default. It requires at least one argument. This is to avoid accidentally packaging something we don't want. Explicit is better than implicit.
For this reason, users of the pyproject declarative buildsystem would be required to always set BuildOption(install) explicitly.
Why I don't like this option: The declarative buildsystem was invented to make spec files simpler. Making BuildOption(install) mandatory defeats the purpose.
I prefer option 1. This is something the users *should* actively check and verify, so having 1 extra line for it seems reasonable to me. We're still saving dozen+ lines of boilerplate and replacing it with one line of meaningful input. That's much easier to use in view.
On 03. 07. 24 15:14, Petr Viktorin wrote:
Hello, I recommend starting out with the explicit option. Requiring BuildOption(install) sounds good, and can be changed to a better default after people get some experience with the system.
This is now implemented in the original PR (as a fixup commit):
https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/455
On Mon, Jul 22, 2024 at 1:01 PM Miro Hrončok mhroncok@redhat.com wrote:
On 03. 07. 24 15:00, Tomáš Orsava wrote:
On 7/3/24 12:07, Miro Hrončok wrote:
Hello.
I am working on the RPM pyproject declarative buildsystem.
t;dr it turns this:
BuildSystem: pyproject
into this:
%prep %autosetup -p1 -C %generate_buildrequires %pyproject_buildrequires %build %pyproject_wheel %install %pyproject_install %pyproject_save_files ... %check %pyproject_check_import
It allows to override options to sections with BuildOption.
For detailed documentation, see https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/455 (it has README changes in it).
What should be the default for BuildOption(install)? That is, what should be passed to %pyproject_save_files if the packager does not override it?
- The safe+hard default: nothing
=================================
The %pyproject_save_files macro currently has no default. It requires at least one argument. This is to avoid accidentally packaging something we don't want. Explicit is better than implicit.
For this reason, users of the pyproject declarative buildsystem would be required to always set BuildOption(install) explicitly.
Why I don't like this option: The declarative buildsystem was invented to make spec files simpler. Making BuildOption(install) mandatory defeats the purpose.
I prefer option 1. This is something the users *should* actively check and verify, so having 1 extra line for it seems reasonable to me. We're still saving dozen+ lines of boilerplate and replacing it with one line of meaningful input. That's much easier to use in view.
On 03. 07. 24 15:14, Petr Viktorin wrote:
Hello, I recommend starting out with the explicit option. Requiring BuildOption(install) sounds good, and can be changed to a better default after people get some experience with the system.
This is now implemented in the original PR (as a fixup commit):
https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/455
Could we expand on this with dynamic subpackages to support building against multiple Python versions from a single source package and producing flavor subpackages against each Python version enabled?
On 24. 07. 24 6:01, Neal Gompa wrote:
On Mon, Jul 22, 2024 at 1:01 PM Miro Hrončok mhroncok@redhat.com wrote:
On 03. 07. 24 15:00, Tomáš Orsava wrote:
On 7/3/24 12:07, Miro Hrončok wrote:
Hello.
I am working on the RPM pyproject declarative buildsystem.
t;dr it turns this:
BuildSystem: pyproject
into this:
%prep %autosetup -p1 -C %generate_buildrequires %pyproject_buildrequires %build %pyproject_wheel %install %pyproject_install %pyproject_save_files ... %check %pyproject_check_import
It allows to override options to sections with BuildOption.
For detailed documentation, see https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/455 (it has README changes in it).
What should be the default for BuildOption(install)? That is, what should be passed to %pyproject_save_files if the packager does not override it?
- The safe+hard default: nothing
=================================
The %pyproject_save_files macro currently has no default. It requires at least one argument. This is to avoid accidentally packaging something we don't want. Explicit is better than implicit.
For this reason, users of the pyproject declarative buildsystem would be required to always set BuildOption(install) explicitly.
Why I don't like this option: The declarative buildsystem was invented to make spec files simpler. Making BuildOption(install) mandatory defeats the purpose.
I prefer option 1. This is something the users *should* actively check and verify, so having 1 extra line for it seems reasonable to me. We're still saving dozen+ lines of boilerplate and replacing it with one line of meaningful input. That's much easier to use in view.
On 03. 07. 24 15:14, Petr Viktorin wrote:
Hello, I recommend starting out with the explicit option. Requiring BuildOption(install) sounds good, and can be changed to a better default after people get some experience with the system.
This is now implemented in the original PR (as a fixup commit):
https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/455
Could we expand on this with dynamic subpackages to support building against multiple Python versions from a single source package and producing flavor subpackages against each Python version enabled?
I don't have the proper energy:motivation ratio to do that at this moment. It seems possible with nowadays RPM, but there are definitively challenges.
If I did this, I would not try to expand the declarative buildsystem, I'd try to design this first without it and only later hook it up into that.
---
Why do you think it should be the declarative buildsystem specifically that would benefit form this?
At the current state, it merely replaces the %prep, %generate_buildrequires, %build, %install, and %check sections.
We could explore the possibility to use the RPM 4.19 Dynamic Spec Generation to generate the %package and %files section from within %install or %build and enable this in the pyproject declarative buildsystem by default. The main issue with this approach is that the more options we need to pass to the Declarative Buildsystem, the more incomprehensible the invocation is.
Compare:
%pyproject_save_files -l pello ... %files -n python3-pello -f %{pyproject_files} %doc README.md %{_bindir}/pello_greeting
With something like this:
BuildOption(install): -l pello --doc README.md --bindir pello_greeting
Sure, the latter is shorter but also quite cryptic.
---
Anyway, as a demo, I think it should be fairly simple to generate this automatically from package %{name}:
%package -n python3-pello Summary: %{summary}
%description -n python3-pello ...
The biggest challenge is the value of the %description.
On 22. 07. 24 19:00, Miro Hrončok wrote:
On 03. 07. 24 15:00, Tomáš Orsava wrote:
On 7/3/24 12:07, Miro Hrončok wrote:
Hello.
I am working on the RPM pyproject declarative buildsystem.
t;dr it turns this:
BuildSystem: pyproject
into this:
%prep %autosetup -p1 -C
%generate_buildrequires %pyproject_buildrequires
%build %pyproject_wheel
%install %pyproject_install %pyproject_save_files ...
%check %pyproject_check_import
It allows to override options to sections with BuildOption.
For detailed documentation, see https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/455 (it has README changes in it).
What should be the default for BuildOption(install)? That is, what should be passed to %pyproject_save_files if the packager does not override it?
- The safe+hard default: nothing
=================================
The %pyproject_save_files macro currently has no default. It requires at least one argument. This is to avoid accidentally packaging something we don't want. Explicit is better than implicit.
For this reason, users of the pyproject declarative buildsystem would be required to always set BuildOption(install) explicitly.
Why I don't like this option: The declarative buildsystem was invented to make spec files simpler. Making BuildOption(install) mandatory defeats the purpose.
I prefer option 1. This is something the users *should* actively check and verify, so having 1 extra line for it seems reasonable to me. We're still saving dozen+ lines of boilerplate and replacing it with one line of meaningful input. That's much easier to use in view.
On 03. 07. 24 15:14, Petr Viktorin wrote:
Hello, I recommend starting out with the explicit option. Requiring BuildOption(install) sounds good, and can be changed to a better default after people get some experience with the system.
This is now implemented in the original PR (as a fixup commit):
https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/455
As a followup, I got another idea:
No BuildOption(install) -> no %pyproject_save_files
That would also work but it would allow folks who don't want to use %pyproject_save_files to also use the declarative build system without overriding %install.
Hello, I recommend starting out with the explicit option. Requiring BuildOption(install) sounds good, and can be changed to a better default after people get some experience with the system.
Hello, I recommend starting out with the explicit option. Requiring BuildOption(install) sounds good, and can be changed to a better default after people get some experience with the system.
python-devel@lists.fedoraproject.org