F38 proposal: Prevent from building RPM packages providing python3dist(...) = 0 (Self-Contained Change proposal)
9:35 a.m.
https://fedoraproject.org/wiki/Changes/Prevent-Providing-python3dist(pkg)...
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
It sometimes happens that Python packages succeed to build as RPM with
incorrect version metadata.
They generate a wrong provide in format `python3dist(...) = 0` and
`python3.Xdist(...) = 0`.
While version `0` (or equal versions like `0.0` or `0.0.0`) is
probably technically valid, in most cases this indicates a packaging
error.
We propose to prevent this error from happening by explicitly failing
the RPM build instead of generating such provides.
== Owner ==
* Name: [[User:Ksurma|Karolina Surma]]
* Email: ksurma(a)redhat.com
== Detailed Description ==
This change is about automatic RPM provides in the following form:
* `python3dist(distname) = 0`
* `python3.Xdist(distname) = 0`
Where X is the Python minor version (eg. 10, 11...).
It does not affect any other provides.
More about these provides:
https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Machine...
The provides generated during the RPM build come from the upstream
Python package metadata.
Some Python building backends, eg. setuptools, explicitly allow
creating package with version `0.0.0` when the version used by a
project is not known. This was
[https://github.com/pypa/setuptools/issues/2329 discussed upstream]
with conclusion that it's an intended behavior.
In other cases, for example when building package from a particular
git commit, the incorrect provide can be generated due to a packaging
error.
We've never encountered a situation when packaging the version `0` was
the package maintainers intention.
After a discussion on python-devel mailing list we agreed we'd like to
prevent such situations from happening in Fedora.
An example of the incorrect provides:
$ rpm -qpP python3-ssh-python-0.10.0-5.fc38.x86_64.rpm
python-ssh-python = 0.10.0-5.fc38
python3-ssh-python = 0.10.0-5.fc38
python3-ssh-python(x86-64) = 0.10.0-5.fc38
python3.11-ssh-python = 0.10.0-5.fc38
'''python3.11dist(ssh-python) = 0'''
'''python3dist(ssh-python) = 0'''
Why is it bad?
If any package requires `python3-ssh-python > 0.9` (correctly assuming
there's `0.10.0` in Fedora's repositories), the automatic dependency
generators will not discover the example package, making the other
package non-installable.
In January 2022 the
[https://bugzilla.redhat.com/showdependencytree.cgi?id=python3dist0
umbrella Bugzilla ticket] was created for Python packages providing
version `0`. In all cases the automatic provide wasn't intended.
Affected packages (as of Nov 11 2022):
b43-tools
dionaea
marisa
python-podman-api
rpkg
spectrographic
python-ssh-python
python-streamlink
The change doesn't affect a big part of the Python ecosystem.
We aim to prevent such situation from happening by increasing the
robustness of the python-rpm-generators (namely
[https://src.fedoraproject.org/rpms/python-rpm-generators/blob/rawhide/f/p...
pythondistdeps.py]).
The generator will error and fail the build if `python3dist(...) = 0`
was to be generated.
Based on discussion on python-devel mailing list there will be no way
to opt out from this change. There will be no possibility to package a
Python package with version `0`.
Packagers who encounter such need are encouraged to work with upstream
to set the version to at least `0.0.1`.
== Feedback ==
The idea was posted on
[https://lists.fedoraproject.org/archives/list/python-devel@lists.fedorapr...
python-devel mailing list] and received a positive feedback. No
alternatives to this approach were proposed.
== Benefit to Fedora ==
The correct metadata is essential for the whole package ecosystem.
More deterministic behavior of the generators will bring those
benefits:
* The packages will stop lying about the version they provide.
* The requirements generators (eg. `%pyproject_buildrequires`) will
correctly evaluate the Build- and Runtime Requirements based on the
correct Provides.
* The package maintainers who BuildRequire `%{py3dist pkgname} >= 0.2`
in their specfiles will always require the correctly evaluated
version.
== Scope ==
* Proposal owners:
# implement & test the change in python-rpm-generators
([https://src.fedoraproject.org/rpms/python-rpm-generators/blob/rawhide/f/p...
pythondistdeps.py])
* Other developers:
** Fix the packaging error to generate correct metadata and successful
build. There's no common way to deal with such packages. In most of
the cases the issue originates from packaging errors that need to be
fixed. Contact the change owners if you need help with the necessary
changes to your package.
* Release engineering: not needed for this Change
* Policies and guidelines:
[https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_source...
Python Packaging Guidelines] cover the topic of creating the version
string. This will be valid even when we change the Python RPM
generators behavior
* Trademark approval: not needed for this Change
* Alignment with Objectives: No
== Upgrade/compatibility impact ==
None.
== How To Test ==
* Prepare a Python package, set the version to `0`
or
* Use one of the known packages that provide version `0`.
* Try to build an RPM package in a regular way (eg. mockbuild)
* The build should fail.
== User Experience ==
The actual users should notice no difference.
== Dependencies ==
None
== Contingency Plan ==
* Contingency mechanism: Revert
* Contingency deadline: Beta freeze
* Blocks release? No
== Documentation ==
This page is the documentation of this change.
== Release Notes ==
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
5:03 p.m.
New subject: F38 proposal: Prevent from building RPM packages providing python3dist(...) = 0 (Self-Contained Change proposal)
So let me sum up:
Some Python building backends, eg. setuptools, explicitly allow
creating package with version `0.0.0` when the version used by a
project is not known. This was
[https://github.com/pypa/setuptools/issues/2329 discussed upstream]
with conclusion that it's an intended behavior.
Upstream says that it is intended that packages are able to set their
version to 0 or 0.0.0, but…
Based on discussion on python-devel mailing list there will be no
way
to opt out from this change. There will be no possibility to package a
Python package with version `0`.
… your proposed Change will fail those packages' build with no opt-out!? You
cannot be serious!
(Though actually, would %global __provides_exclude_from … together with a
manual Provides: python3dist(…) = 0 not work?)
A clear -1 to this Change as proposed.
We've never encountered a situation when packaging the version
`0` was
the package maintainers intention.
What if it is the *upstream* maintainer's intention? Are we now dictating
versioning schemes on upstream projects, disallowing version numbers that
upstream setuptools explicitly considers valid?
Kevin Kofler
5:23 p.m.
On Mon, Nov 14, 2022 at 6:03 PM Kevin Kofler via devel
<devel(a)lists.fedoraproject.org> wrote:
So let me sum up:
> Some Python building backends, eg. setuptools, explicitly allow
> creating package with version `0.0.0` when the version used by a
> project is not known. This was
> [https://github.com/pypa/setuptools/issues/2329 discussed upstream]
> with conclusion that it's an intended behavior.
Upstream says that it is intended that packages are able to set their
version to 0 or 0.0.0, but…
> Based on discussion on python-devel mailing list there will be no way
> to opt out from this change. There will be no possibility to package a
> Python package with version `0`.
… your proposed Change will fail those packages' build with no opt-out!? You
cannot be serious!
(Though actually, would %global __provides_exclude_from … together with a
manual Provides: python3dist(…) = 0 not work?)
A clear -1 to this Change as proposed.
> We've never encountered a situation when packaging the version `0` was
> the package maintainers intention.
What if it is the *upstream* maintainer's intention? Are we now dictating
versioning schemes on upstream projects, disallowing version numbers that
upstream setuptools explicitly considers valid?
Unfortunately, I have to agree here. Nobody said we should be
dictating the versions for people. PEP-440 does not even make 0
version illegal, so this is unnecessarily punishing.
--
真実はいつも一つ！/ Always, there's only one truth!
1:37 a.m.
New subject: F38 proposal: Prevent from building RPM packages providing python3dist(...) = 0 (Self-Contained Change proposal)
Il 15/11/22 00:23, Neal Gompa ha scritto:
On Mon, Nov 14, 2022 at 6:03 PM Kevin Kofler via devel
<devel(a)lists.fedoraproject.org> wrote:
> So let me sum up:
>
>> Some Python building backends, eg. setuptools, explicitly allow
>> creating package with version `0.0.0` when the version used by a
>> project is not known. This was
>> [https://github.com/pypa/setuptools/issues/2329 discussed upstream]
>> with conclusion that it's an intended behavior.
> Upstream says that it is intended that packages are able to set their
> version to 0 or 0.0.0, but…
>
>> Based on discussion on python-devel mailing list there will be no way
>> to opt out from this change. There will be no possibility to package a
>> Python package with version `0`.
> … your proposed Change will fail those packages' build with no opt-out!? You
> cannot be serious!
>
> (Though actually, would %global __provides_exclude_from … together with a
> manual Provides: python3dist(…) = 0 not work?)
>
> A clear -1 to this Change as proposed.
>
>> We've never encountered a situation when packaging the version `0` was
>> the package maintainers intention.
> What if it is the *upstream* maintainer's intention? Are we now dictating
> versioning schemes on upstream projects, disallowing version numbers that
> upstream setuptools explicitly considers valid?
>
Unfortunately, I have to agree here. Nobody said we should be
dictating the versions for people. PEP-440 does not even make 0
version illegal, so this is unnecessarily punishing.
IMO, the policy is right, it just have to allow to opt out, so that the
maintainer is informed that **there could be** something wrong with the
package metadata / build process. Maybe an opt-out + file a ticket in BZ
to have track of the opt out, just like the ExcludeArch is the right
approach.
Mattia
5:35 a.m.
On 11/15/22 08:37, Mattia Verga via devel wrote:
Il 15/11/22 00:23, Neal Gompa ha scritto:
> On Mon, Nov 14, 2022 at 6:03 PM Kevin Kofler via devel
> <devel(a)lists.fedoraproject.org> wrote:
>> So let me sum up:
>>
>>> Some Python building backends, eg. setuptools, explicitly allow
>>> creating package with version `0.0.0` when the version used by a
>>> project is not known. This was
>>> [https://github.com/pypa/setuptools/issues/2329 discussed upstream]
>>> with conclusion that it's an intended behavior.
>> Upstream says that it is intended that packages are able to set their
>> version to 0 or 0.0.0, but…
>>
>>> Based on discussion on python-devel mailing list there will be no way
>>> to opt out from this change. There will be no possibility to package a
>>> Python package with version `0`.
>> … your proposed Change will fail those packages' build with no opt-out!? You
>> cannot be serious!
>>
>> (Though actually, would %global __provides_exclude_from … together with a
>> manual Provides: python3dist(…) = 0 not work?)
>>
>> A clear -1 to this Change as proposed.
>>
>>> We've never encountered a situation when packaging the version `0` was
>>> the package maintainers intention.
>> What if it is the *upstream* maintainer's intention? Are we now dictating
>> versioning schemes on upstream projects, disallowing version numbers that
>> upstream setuptools explicitly considers valid?
>>
> Unfortunately, I have to agree here. Nobody said we should be
> dictating the versions for people. PEP-440 does not even make 0
> version illegal, so this is unnecessarily punishing.
>
IMO, the policy is right, it just have to allow to opt out, so that the
maintainer is informed that **there could be** something wrong with the
package metadata / build process. Maybe an opt-out + file a ticket in BZ
to have track of the opt out, just like the ExcludeArch is the right
approach.
Mattia
Hello,
The standard invocation of the Python dependency generator will be
changed to always run with option like `--fail-if-version-zero`. (This
is the subject of the current proposal.)
Based on your concerns, an opt out mechanism would be added:
If you wanted to bypass the option, you could define a macro in the
specfile, eg.
`%{!?__python_dist_allow_version_zero:--fail-if-version-zero}`
This would allow to build RPMs without bothering with versions. Also, if
it's upstream's decision to use version 0, that would be the way to go.
In Fedora this would give us an insight into how often this is needed.
What do you think of enhancing the proposal this way?
Karolina Surma
5:42 a.m.
On Tue, Nov 15, 2022 at 12:35:32PM +0100, Karolina Surma wrote:
On 11/15/22 08:37, Mattia Verga via devel wrote:
> Il 15/11/22 00:23, Neal Gompa ha scritto:
>> On Mon, Nov 14, 2022 at 6:03 PM Kevin Kofler via devel
>> <devel(a)lists.fedoraproject.org> wrote:
>>> So let me sum up:
>>>
>>>> Some Python building backends, eg. setuptools, explicitly allow
>>>> creating package with version `0.0.0` when the version used by a
>>>> project is not known. This was
>>>> [https://github.com/pypa/setuptools/issues/2329 discussed upstream]
>>>> with conclusion that it's an intended behavior.
>>> Upstream says that it is intended that packages are able to set their
>>> version to 0 or 0.0.0, but…
>>>
>>>> Based on discussion on python-devel mailing list there will be no way
>>>> to opt out from this change. There will be no possibility to package a
>>>> Python package with version `0`.
>>> … your proposed Change will fail those packages' build with no opt-out!?
You
>>> cannot be serious!
>>>
>>> (Though actually, would %global __provides_exclude_from … together with a
>>> manual Provides: python3dist(…) = 0 not work?)
>>>
>>> A clear -1 to this Change as proposed.
>>>
>>>> We've never encountered a situation when packaging the version `0`
was
>>>> the package maintainers intention.
>>> What if it is the *upstream* maintainer's intention? Are we now
dictating
>>> versioning schemes on upstream projects, disallowing version numbers that
>>> upstream setuptools explicitly considers valid?
>>>
>> Unfortunately, I have to agree here. Nobody said we should be
>> dictating the versions for people. PEP-440 does not even make 0
>> version illegal, so this is unnecessarily punishing.
>>
> IMO, the policy is right, it just have to allow to opt out, so that the
> maintainer is informed that **there could be** something wrong with the
> package metadata / build process. Maybe an opt-out + file a ticket in BZ
> to have track of the opt out, just like the ExcludeArch is the right
> approach.
>
> Mattia
>
Hello,
The standard invocation of the Python dependency generator will be
changed to always run with option like `--fail-if-version-zero`. (This
is the subject of the current proposal.)
Based on your concerns, an opt out mechanism would be added:
If you wanted to bypass the option, you could define a macro in the
specfile, eg.
`%{!?__python_dist_allow_version_zero:--fail-if-version-zero}`
This would allow to build RPMs without bothering with versions. Also, if
it's upstream's decision to use version 0, that would be the way to go.
In Fedora this would give us an insight into how often this is needed.
What do you think of enhancing the proposal this way?
I wonder how many Python packages have ever been packaged with version
0. Are we trying to solve a problem that doesn't exist?
Semver recommends[1] starting at 0.1.0 and without proof I think today
most packages start by following semver. Any old packages that predate
semver have a version > 0 by now.
[1]:
https://semver.org/#how-should-i-deal-with-revisions-in-the-0yz-initial-d...
5:10 a.m.
On 11/15/22 12:42, Ewoud Kohl van Wijngaarden wrote:
On Tue, Nov 15, 2022 at 12:35:32PM +0100, Karolina Surma wrote:
> On 11/15/22 08:37, Mattia Verga via devel wrote:
>> Il 15/11/22 00:23, Neal Gompa ha scritto:
>>> On Mon, Nov 14, 2022 at 6:03 PM Kevin Kofler via devel
>>> <devel(a)lists.fedoraproject.org> wrote:
>>>> So let me sum up:
>>>>
>>>>> Some Python building backends, eg. setuptools, explicitly allow
>>>>> creating package with version `0.0.0` when the version used by a
>>>>> project is not known. This was
>>>>> [https://github.com/pypa/setuptools/issues/2329 discussed upstream]
>>>>> with conclusion that it's an intended behavior.
>>>> Upstream says that it is intended that packages are able to set their
>>>> version to 0 or 0.0.0, but…
>>>>
>>>>> Based on discussion on python-devel mailing list there will be no
way
>>>>> to opt out from this change. There will be no possibility to
>>>>> package a
>>>>> Python package with version `0`.
>>>> … your proposed Change will fail those packages' build with no
>>>> opt-out!? You
>>>> cannot be serious!
>>>>
>>>> (Though actually, would %global __provides_exclude_from … together
>>>> with a
>>>> manual Provides: python3dist(…) = 0 not work?)
>>>>
>>>> A clear -1 to this Change as proposed.
>>>>
>>>>> We've never encountered a situation when packaging the version
`0`
>>>>> was
>>>>> the package maintainers intention.
>>>> What if it is the *upstream* maintainer's intention? Are we now
>>>> dictating
>>>> versioning schemes on upstream projects, disallowing version
>>>> numbers that
>>>> upstream setuptools explicitly considers valid?
>>>>
>>> Unfortunately, I have to agree here. Nobody said we should be
>>> dictating the versions for people. PEP-440 does not even make 0
>>> version illegal, so this is unnecessarily punishing.
>>>
>> IMO, the policy is right, it just have to allow to opt out, so that the
>> maintainer is informed that **there could be** something wrong with the
>> package metadata / build process. Maybe an opt-out + file a ticket in BZ
>> to have track of the opt out, just like the ExcludeArch is the right
>> approach.
>>
>> Mattia
>>
>
> Hello,
>
> The standard invocation of the Python dependency generator will be
> changed to always run with option like `--fail-if-version-zero`. (This
> is the subject of the current proposal.)
>
> Based on your concerns, an opt out mechanism would be added:
> If you wanted to bypass the option, you could define a macro in the
> specfile, eg.
> `%{!?__python_dist_allow_version_zero:--fail-if-version-zero}`
>
> This would allow to build RPMs without bothering with versions. Also, if
> it's upstream's decision to use version 0, that would be the way to go.
> In Fedora this would give us an insight into how often this is needed.
>
> What do you think of enhancing the proposal this way?
I wonder how many Python packages have ever been packaged with version
0. Are we trying to solve a problem that doesn't exist?
Semver recommends[1] starting at 0.1.0 and without proof I think
today
most packages start by following semver. Any old packages that predate
semver have a version > 0 by now.
Hi Ewoud,
Let me answer to two possible interpretations of your question.
The problem that exists is that you can incorrectly package a Python RPM
with Provides = 0 (without even knowing it unless inspecting the
metadata). I feel we could benefit from the proposed Change, if not for
anything else, then by getting rid of unnecessary steps of discovery
that a package is wrong (typically when it prevents other packages from
building/running), opening a Bugzilla and working to fix it. That's
costly and avoidable, so let's avoid it.
Does it happen that Python packages have a legitimate version 0 that
should make it to Fedora? We don't believe so, hence the proposed
Change, but as the others mentioned, it's not forbidden by PEP 440.
setuptools and flit will let you create package with 0.0.0. Poetry
automatically starts versioning the projects at 0.1.0.
Providing an opt-out is trivial and if this is what makes the Change
acceptable for the community, let's do it. It's definitely better than
forcing people to opt out from the generators altogether.
Cheers,
Karolina
[1]:
https://semver.org/#how-should-i-deal-with-revisions-in-the-0yz-initial-d...
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue
5:52 a.m.
On Tue, Nov 22, 2022 at 12:10:39PM +0100, Karolina Surma wrote:
On 11/15/22 12:42, Ewoud Kohl van Wijngaarden wrote:
>On Tue, Nov 15, 2022 at 12:35:32PM +0100, Karolina Surma wrote:
>>On 11/15/22 08:37, Mattia Verga via devel wrote:
>>>Il 15/11/22 00:23, Neal Gompa ha scritto:
>>>>On Mon, Nov 14, 2022 at 6:03 PM Kevin Kofler via devel
>>>><devel(a)lists.fedoraproject.org> wrote:
>>>>>So let me sum up:
>>>>>
>>>>>>Some Python building backends, eg. setuptools, explicitly allow
>>>>>>creating package with version `0.0.0` when the version used by a
>>>>>>project is not known. This was
>>>>>>[https://github.com/pypa/setuptools/issues/2329 discussed
upstream]
>>>>>>with conclusion that it's an intended behavior.
>>>>>Upstream says that it is intended that packages are able to set their
>>>>>version to 0 or 0.0.0, but…
>>>>>
>>>>>>Based on discussion on python-devel mailing list there will be no
way
>>>>>>to opt out from this change. There will be no possibility
>>>>>>to package a
>>>>>>Python package with version `0`.
>>>>>… your proposed Change will fail those packages' build with
>>>>>no opt-out!? You
>>>>>cannot be serious!
>>>>>
>>>>>(Though actually, would %global __provides_exclude_from …
>>>>>together with a
>>>>>manual Provides: python3dist(…) = 0 not work?)
>>>>>
>>>>>A clear -1 to this Change as proposed.
>>>>>
>>>>>>We've never encountered a situation when packaging the
>>>>>>version `0` was
>>>>>>the package maintainers intention.
>>>>>What if it is the *upstream* maintainer's intention? Are we
>>>>>now dictating
>>>>>versioning schemes on upstream projects, disallowing version
>>>>>numbers that
>>>>>upstream setuptools explicitly considers valid?
>>>>>
>>>>Unfortunately, I have to agree here. Nobody said we should be
>>>>dictating the versions for people. PEP-440 does not even make 0
>>>>version illegal, so this is unnecessarily punishing.
>>>>
>>>IMO, the policy is right, it just have to allow to opt out, so that the
>>>maintainer is informed that **there could be** something wrong with the
>>>package metadata / build process. Maybe an opt-out + file a ticket in BZ
>>>to have track of the opt out, just like the ExcludeArch is the right
>>>approach.
>>>
>>>Mattia
>>>
>>
>>Hello,
>>
>>The standard invocation of the Python dependency generator will be
>>changed to always run with option like `--fail-if-version-zero`. (This
>>is the subject of the current proposal.)
>>
>>Based on your concerns, an opt out mechanism would be added:
>>If you wanted to bypass the option, you could define a macro in the
>>specfile, eg.
>>`%{!?__python_dist_allow_version_zero:--fail-if-version-zero}`
>>
>>This would allow to build RPMs without bothering with versions. Also, if
>>it's upstream's decision to use version 0, that would be the way to go.
>>In Fedora this would give us an insight into how often this is needed.
>>
>>What do you think of enhancing the proposal this way?
>
>I wonder how many Python packages have ever been packaged with
>version 0. Are we trying to solve a problem that doesn't exist?
>Semver recommends[1] starting at 0.1.0 and without proof I think
>today most packages start by following semver. Any old packages that
>predate semver have a version > 0 by now.
>
Hi Ewoud,
Let me answer to two possible interpretations of your question.
The problem that exists is that you can incorrectly package a Python
RPM with Provides = 0 (without even knowing it unless inspecting the
metadata). I feel we could benefit from the proposed Change, if not
for anything else, then by getting rid of unnecessary steps of
discovery that a package is wrong (typically when it prevents other
packages from building/running), opening a Bugzilla and working to fix
it. That's costly and avoidable, so let's avoid it.
I'm 100% on board with that. Software should be hard to misuse and this
looks like a case where it's hardly ever (possibly never) what the
packager intended. That was not what my comment was about.
Does it happen that Python packages have a legitimate version 0 that
should make it to Fedora? We don't believe so, hence the proposed
Change, but as the others mentioned, it's not forbidden by PEP 440.
setuptools and flit will let you create package with 0.0.0. Poetry
automatically starts versioning the projects at 0.1.0.
That was indeed what my question was about. Yes, in theory a version 0
or 0.0.0 is possible but is there any proof that this is something that
has ever been needed?
Providing an opt-out is trivial and if this is what makes the Change
acceptable for the community, let's do it. It's definitely better than
forcing people to opt out from the generators altogether.
I'd argue that any code has a cost, even if it's trivial. Code that's
never executed is likely to be forgotten and/or have bugs.
So in short: this looks like a good proposal to me and I question the
need for an opt-out.
>>[1]:
https://semver.org/#how-should-i-deal-with-revisions-in-the-0yz-initial-d...
1:58 a.m.
On Tue, Nov 15, 2022 at 12:35:32PM +0100, Karolina Surma wrote:
On 11/15/22 08:37, Mattia Verga via devel wrote:
> Il 15/11/22 00:23, Neal Gompa ha scritto:
>> On Mon, Nov 14, 2022 at 6:03 PM Kevin Kofler via devel
>> <devel(a)lists.fedoraproject.org> wrote:
>>> So let me sum up:
>>>
>>>> Some Python building backends, eg. setuptools, explicitly allow
>>>> creating package with version `0.0.0` when the version used by a
>>>> project is not known. This was
>>>> [https://github.com/pypa/setuptools/issues/2329 discussed upstream]
>>>> with conclusion that it's an intended behavior.
>>> Upstream says that it is intended that packages are able to set their
>>> version to 0 or 0.0.0, but…
>>>
>>>> Based on discussion on python-devel mailing list there will be no way
>>>> to opt out from this change. There will be no possibility to package a
>>>> Python package with version `0`.
>>> … your proposed Change will fail those packages' build with no opt-out!?
You
>>> cannot be serious!
>>>
>>> (Though actually, would %global __provides_exclude_from … together with a
>>> manual Provides: python3dist(…) = 0 not work?)
>>>
>>> A clear -1 to this Change as proposed.
>>>
>>>> We've never encountered a situation when packaging the version `0`
was
>>>> the package maintainers intention.
>>> What if it is the *upstream* maintainer's intention? Are we now
dictating
>>> versioning schemes on upstream projects, disallowing version numbers that
>>> upstream setuptools explicitly considers valid?
>>>
>> Unfortunately, I have to agree here. Nobody said we should be
>> dictating the versions for people. PEP-440 does not even make 0
>> version illegal, so this is unnecessarily punishing.
>>
> IMO, the policy is right, it just have to allow to opt out, so that the
> maintainer is informed that **there could be** something wrong with the
> package metadata / build process. Maybe an opt-out + file a ticket in BZ
> to have track of the opt out, just like the ExcludeArch is the right
> approach.
>
> Mattia
>
Hello,
The standard invocation of the Python dependency generator will be
changed to always run with option like `--fail-if-version-zero`. (This
is the subject of the current proposal.)
Based on your concerns, an opt out mechanism would be added:
If you wanted to bypass the option, you could define a macro in the
specfile, eg.
`%{!?__python_dist_allow_version_zero:--fail-if-version-zero}`
This would allow to build RPMs without bothering with versions. Also, if
it's upstream's decision to use version 0, that would be the way to go.
In Fedora this would give us an insight into how often this is needed.
What do you think of enhancing the proposal this way?
This would provide the opt-out that was requested.
But it's also very specific to a problem that is mostly theoretical.
Maybe instead add %__python_dist_extra_flags that would contain
'--fail-if-version-zero'
by default. That would also provide an opt-out, but allow other
currently unforeseen uses.
Zbyszek
3:22 a.m.
On 11/16/22 08:58, Zbigniew Jędrzejewski-Szmek wrote:
>
> Hello,
>
> The standard invocation of the Python dependency generator will be
> changed to always run with option like `--fail-if-version-zero`. (This
> is the subject of the current proposal.)
>
> Based on your concerns, an opt out mechanism would be added:
> If you wanted to bypass the option, you could define a macro in the
> specfile, eg.
> `%{!?__python_dist_allow_version_zero:--fail-if-version-zero}`
>
> This would allow to build RPMs without bothering with versions. Also, if
> it's upstream's decision to use version 0, that would be the way to go.
> In Fedora this would give us an insight into how often this is needed.
>
> What do you think of enhancing the proposal this way?
This would provide the opt-out that was requested.
But it's also very specific to a problem that is mostly theoretical.
Maybe instead add %__python_dist_extra_flags that would contain
'--fail-if-version-zero'
by default. That would also provide an opt-out, but allow other
currently unforeseen uses.
Zbyszek
Thank you for the alternative proposal. After discussion with Miro
Hrončok we decided for the initial idea of a macro which if defined,
will disable the flag from script invocation.
The main reason being that there's already another Python macro that
behaves this way
(https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_python...),
so the new one will fit into the picture.
It'll be also easily greppable for usage statistics.
As we expect little to no need to actually use the opt-out, the trivial
(to implement and use) solution seems just enough.
Karolina
3:38 a.m.
On 15. 11. 22 0:03, Kevin Kofler via devel wrote:
(Though actually, would %global __provides_exclude_from … together
with a
manual Provides: python3dist(…) = 0 not work?)
I believe that %__provides_exclude_from would actually work.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
2:55 a.m.
On 11/14/22 16:35, Ben Cotton wrote:
https://fedoraproject.org/wiki/Changes/Prevent-Providing-python3dist(pkg)...
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
It sometimes happens that Python packages succeed to build as RPM with
incorrect version metadata.
They generate a wrong provide in format `python3dist(...) = 0` and
`python3.Xdist(...) = 0`.
While version `0` (or equal versions like `0.0` or `0.0.0`) is
probably technically valid, in most cases this indicates a packaging
error.
We propose to prevent this error from happening by explicitly failing
the RPM build instead of generating such provides.
== Owner ==
* Name: [[User:Ksurma|Karolina Surma]]
* Email: ksurma(a)redhat.com
== Detailed Description ==
This change is about automatic RPM provides in the following form:
* `python3dist(distname) = 0`
* `python3.Xdist(distname) = 0`
Where X is the Python minor version (eg. 10, 11...).
It does not affect any other provides.
More about these provides:
https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Machine...
The provides generated during the RPM build come from the upstream
Python package metadata.
Some Python building backends, eg. setuptools, explicitly allow
creating package with version `0.0.0` when the version used by a
project is not known. This was
[https://github.com/pypa/setuptools/issues/2329 discussed upstream]
with conclusion that it's an intended behavior.
In other cases, for example when building package from a particular
git commit, the incorrect provide can be generated due to a packaging
error.
We've never encountered a situation when packaging the version `0` was
the package maintainers intention.
After a discussion on python-devel mailing list we agreed we'd like to
prevent such situations from happening in Fedora.
An example of the incorrect provides:
$ rpm -qpP python3-ssh-python-0.10.0-5.fc38.x86_64.rpm
python-ssh-python = 0.10.0-5.fc38
python3-ssh-python = 0.10.0-5.fc38
python3-ssh-python(x86-64) = 0.10.0-5.fc38
python3.11-ssh-python = 0.10.0-5.fc38
'''python3.11dist(ssh-python) = 0'''
'''python3dist(ssh-python) = 0'''
Why is it bad?
If any package requires `python3-ssh-python > 0.9` (correctly assuming
there's `0.10.0` in Fedora's repositories), the automatic dependency
generators will not discover the example package, making the other
package non-installable.
In January 2022 the
[https://bugzilla.redhat.com/showdependencytree.cgi?id=python3dist0
umbrella Bugzilla ticket] was created for Python packages providing
version `0`. In all cases the automatic provide wasn't intended.
Affected packages (as of Nov 11 2022):
b43-tools
dionaea
marisa
python-podman-api
rpkg
spectrographic
python-ssh-python
python-streamlink
The change doesn't affect a big part of the Python ecosystem.
We aim to prevent such situation from happening by increasing the
robustness of the python-rpm-generators (namely
[https://src.fedoraproject.org/rpms/python-rpm-generators/blob/rawhide/f/p...
pythondistdeps.py]).
The generator will error and fail the build if `python3dist(...) = 0`
was to be generated.
Based on discussion on python-devel mailing list there will be no way
to opt out from this change. There will be no possibility to package a
Python package with version `0`.
Packagers who encounter such need are encouraged to work with upstream
to set the version to at least `0.0.1`.
== Feedback ==
The idea was posted on
[https://lists.fedoraproject.org/archives/list/python-devel@lists.fedorapr...
python-devel mailing list] and received a positive feedback. No
alternatives to this approach were proposed.
== Benefit to Fedora ==
The correct metadata is essential for the whole package ecosystem.
More deterministic behavior of the generators will bring those
benefits:
* The packages will stop lying about the version they provide.
* The requirements generators (eg. `%pyproject_buildrequires`) will
correctly evaluate the Build- and Runtime Requirements based on the
correct Provides.
* The package maintainers who BuildRequire `%{py3dist pkgname} >= 0.2`
in their specfiles will always require the correctly evaluated
version.
== Scope ==
* Proposal owners:
# implement & test the change in python-rpm-generators
([https://src.fedoraproject.org/rpms/python-rpm-generators/blob/rawhide/f/p...
pythondistdeps.py])
* Other developers:
** Fix the packaging error to generate correct metadata and successful
build. There's no common way to deal with such packages. In most of
the cases the issue originates from packaging errors that need to be
fixed. Contact the change owners if you need help with the necessary
changes to your package.
* Release engineering: not needed for this Change
* Policies and guidelines:
[https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_source...
Python Packaging Guidelines] cover the topic of creating the version
string. This will be valid even when we change the Python RPM
generators behavior
* Trademark approval: not needed for this Change
* Alignment with Objectives: No
== Upgrade/compatibility impact ==
None.
== How To Test ==
* Prepare a Python package, set the version to `0`
or
* Use one of the known packages that provide version `0`.
* Try to build an RPM package in a regular way (eg. mockbuild)
* The build should fail.
== User Experience ==
The actual users should notice no difference.
== Dependencies ==
None
== Contingency Plan ==
* Contingency mechanism: Revert
* Contingency deadline: Beta freeze
* Blocks release? No
== Documentation ==
This page is the documentation of this change.
== Release Notes ==
Hello
Based on the feedback, the proposal was updated with an opt-out:
```
Based on the discussion on fedora-devel mailing list, there will be a
mechanism to opt out from this change. A macro with a provisional name
(TBD) %__python_dist_allow_version_zero will be created. If defined in
the specfile, it will change the invocation of the Python RPM generators
not to fail if a package providing version 0 was created. It will behave
the same way as an already existing macro %_python_no_extras_required does.
```
https://fedoraproject.org/wiki/Changes/Prevent-Providing-python3dist(pkg)...
--
Karolina Surma (she/her/hers)
Software Engineer
Python Maintenance Team, Red Hat
519
days inactive
529
days old
11 comments
8 participants
participants (8)
-
Ben Cotton -
Ewoud Kohl van Wijngaarden -
Karolina Surma -
Kevin Kofler -
Mattia Verga -
Miro Hrončok -
Neal Gompa -
Zbigniew Jędrzejewski-Szmek