Re: Proposed Mass Bug Filing: Renaming "python-" binary packages to "python2-"
by Jason L Tibbitts III
First off, I really want to say that we should resist the idea that Red
Hat product decisions should hold back Fedora's progress. This
python-*/python2-*/python3-* mess has been with us for far too long, and
I applaud the Python SIG for putting in the effort to try and get this
regularized. Python packaging is still terrible, but regularizing this
one thing makes it a little bit less terrible.
>>>>> "AW" == Adam Williamson <adamwill(a)fedoraproject.org> writes:
AW> I'm not sure this list is terribly useful, because of the
AW> above. There are thousands of packages that do this, because the
AW> 'python2-' provide is not available on some older Fedora release, or
AW> on EPEL (and the package is maintained for EPEL as well as
AW> Fedora).
We can fix older releases and we can fix EPEL. We could potentially fix
RHEL by means of metapackages that live in EPEL, if we were serious
about it. Just add a "python2-foo" package that has nothing but
"Requires: python-foo".
We've talked about this for ages but until now there was never any
driving need, and while I'm sure it would work fine I don't know exactly
what would happen if the RHEL package suddenly acquires its own
Provides: python2-foo, or in any other weird dependency scenarios that
someone might invent.
If nobody can see a problem with that and we agree it would help, I will
volunteer create and maintain this metapackage thing.
AW> Sprinkling "if (some release number condition) then
AW> Requires: python2-foo else Requires: python-foo" all over your spec
AW> is a giant PITA and I for one am not very interested in doing it.
Well, here's the big thing: Read
https://pagure.io/packaging-committee/issue/686#comment-443129. In
fact, I'll quote it here. The indented text is my question, the
unindented text is ishcherb's reply:
"
First off, do we really intend for the unversioned "python-foo" to
give you the python3 version of that package?
We do, and that is what %python_provide macro was intended for. When
needed, it will be enough to only modify the macro to make Python 3
subpackages provide python- prefixed names.
"
So, basically, you simply cannot have a spec which depends on python-foo
and assumes it's going to get the python2 version. Obviously this
hasn't been done yet because it would break the world, but the point
behind posting this big list of packages is to fix them all. And even
if Fedora waits two years to do this, old EPEL releases will still be
around.
So if you really want that magical non-ifdef spec compatibility between
Fedora and EPEL, then either RHEL is going to have to add the python2-
provides soon or EPEL is going to have to introduce the metapackages I
mentioned above. And if you want compatibility between Fedora and RHEL
without EPEL present, then %if/%endif is your only real option.
The only other way I can see around this is to have some ugly macro
which hardcodes knowledge of packages which provide python-* but not
python2-* and generates your Requires: or BuildRequires: based on that.
I could write one, but I don't particularly want to. If people talk it
out and it turns out that's the only solution which people can accept,
then I will volunteer to write and maintain a macro like this.
Finally, Red Hat itself could clear up this mess a bit by officially
saying that it intends to add the python2-* provides on some schedule.
Then Fedora could coordinate with that, and we'd all live in a land of
ponies and rainbows. What a concept. But I know Red Hat never comments
on what they're doing in upcoming products and never commits to dates
for upcoming products, so....
- J<
6 years, 5 months
Re: Proposed Mass Bug Filing: Renaming "python-" binary packages to
"python2-"
by Petr Viktorin
On 06/29/2017 04:20 AM, Adam Williamson wrote:
> On Thu, 2017-06-29 at 12:11 +1000, Nick Coghlan wrote:
>> On 29 June 2017 at 11:39, Adam Williamson <adamwill(a)fedoraproject.org> wrote:
>>> On Wed, 2017-06-28 at 16:21 +0200, Iryna Shcherbina wrote:
>>>> 2) Using `python-` instead of `python2-` in the dependencies for the
>>>> Python 2 binary RPM [2].
>>>
>>> I'm not sure this list is terribly useful, because of the above. There
>>> are thousands of packages that do this, because the 'python2-' provide
>>> is not available on some older Fedora release, or on EPEL (and the
>>> package is maintained for EPEL as well as Fedora). Sprinkling "if (some
>>> release number condition) then Requires: python2-foo else Requires:
>>> python-foo" all over your spec is a giant PITA and I for one am not
>>> very interested in doing it.
>>>
>>> IMHO, if there is going to be some kind of requirement that all Python
>>> requires be explicitly versioned, there needs to be a co-ordinated
>>> effort to make sure the versioned Provides are available across at
>>> least EL6, EL7, and all supported Fedora releases *first*.
>>
>> This was the key concern I raised in response to the initial email,
>> and our conclusion at the time was:
>>
>> 1. This case does need to be addressed
>> 2. Adding an opaque dependency on buildroot configuration settings
>> wouldn't be a particularly nice way to handle it, since it forces
>> every package to switch concurrently, rather than each maintainer
>> getting to decide when to move from the Python 2 stack to the Python 3
>> stack for themselves (and that unilateral shift is already going to
>> happen for unqualified dependency declarations when the virtual
>> %python_provides macro moves from the Python 2 stack to the Python 3
>> stack)
>> 3. Ideally, the recommended approach would work for arbitrary RHEL &
>> CentOS based buildroots, not just those with the EPEL RPM macros
>> available
>>
>> The most straightforward solution we came up with was for affected
>> packages to define their own "%py_prefix" macro that selects the stack
>> they want to use based on the Python version:
>>
>> ```
>> # The block below would become the conventional
>> # "Python stack compatibility" dance for
>> # EL6, EL7, and Fedora
>> # Each package can decide for itself which version of
>> # Fedora had a sufficiently complete Py3 stack for
>> # them to be able to switch over
>>
>> # Current EL releases & older Fedora use "python-*"
>> %if 0%{?el6} || 0%{?el7} || 0%{?fedora} < 25
>> %define py_prefix python
>> %if 0%{?el6} || 0%{?el7}
>> BuildRequires: python-devel
>> %else
>> BuildRequires: python2-devel
>> %endif
>> %else
>> # Newer Fedora releases use "pythonX-*"
>> # A Py2-only project would use "python2" here
>> %define py_prefix python3
>> BuildRequires: python3-devel
>> %endif
>>
>>
>> # Dependency declarations use stack selected above
>> BuildRequires: %{py_prefix}-builddep1
>> BuildRequires: %{py_prefix}-builddep2
>> Requires: %{py_prefix}-runtimedep1
>> Requires: %{py_prefix}-runtimedep2
>> ```
>
> Erf, well, that seems kinda icky but manageable. Is it really better
> than just coming up with some kind of script to at least add the
> appropriate "Provides: python2-foo" to all packages (or at least a
> large subset of the most commonly-used packages) in EL6/EL7/F24,
> though? I mean, that doesn't seem beyond the bounds of possibility, to
> just find everything that provides 'python-foo' and make it also
> provide 'python2-foo'...
Adding python2-foo is the second half of this proposal.
We don't want to do that by *just* adding "Provides: python2-foo" to all
packages, because if everyone did that, we'd need to ask them to change
again in a few years.
Would it be reasonable to do this in phases: first ask everyone to add
python2- provides, then ask everyone to use them? Yes, and in fact, we
already did that: the python2- prefix has been recommended in the
guidelines for a quite a long time. But many packagers won't do the
change until it starts being necessary.
Rebuilding lots of packages in EL6 is practically impossible, actually.
However, some subset of commonly-used packages do have the python2-
prefixed provides even in EL6. The very example above is misleading:
EL6's packages do provide python2-devel.
--
Petr Viktorin
6 years, 5 months
Re: Proposed Mass Bug Filing: Renaming "python-" binary packages to "python2-"
by Nick Coghlan
On 29 June 2017 at 11:39, Adam Williamson <adamwill(a)fedoraproject.org> wrote:
> On Wed, 2017-06-28 at 16:21 +0200, Iryna Shcherbina wrote:
>> 2) Using `python-` instead of `python2-` in the dependencies for the
>> Python 2 binary RPM [2].
>
> I'm not sure this list is terribly useful, because of the above. There
> are thousands of packages that do this, because the 'python2-' provide
> is not available on some older Fedora release, or on EPEL (and the
> package is maintained for EPEL as well as Fedora). Sprinkling "if (some
> release number condition) then Requires: python2-foo else Requires:
> python-foo" all over your spec is a giant PITA and I for one am not
> very interested in doing it.
>
> IMHO, if there is going to be some kind of requirement that all Python
> requires be explicitly versioned, there needs to be a co-ordinated
> effort to make sure the versioned Provides are available across at
> least EL6, EL7, and all supported Fedora releases *first*.
This was the key concern I raised in response to the initial email,
and our conclusion at the time was:
1. This case does need to be addressed
2. Adding an opaque dependency on buildroot configuration settings
wouldn't be a particularly nice way to handle it, since it forces
every package to switch concurrently, rather than each maintainer
getting to decide when to move from the Python 2 stack to the Python 3
stack for themselves (and that unilateral shift is already going to
happen for unqualified dependency declarations when the virtual
%python_provides macro moves from the Python 2 stack to the Python 3
stack)
3. Ideally, the recommended approach would work for arbitrary RHEL &
CentOS based buildroots, not just those with the EPEL RPM macros
available
The most straightforward solution we came up with was for affected
packages to define their own "%py_prefix" macro that selects the stack
they want to use based on the Python version:
```
# The block below would become the conventional
# "Python stack compatibility" dance for
# EL6, EL7, and Fedora
# Each package can decide for itself which version of
# Fedora had a sufficiently complete Py3 stack for
# them to be able to switch over
# Current EL releases & older Fedora use "python-*"
%if 0%{?el6} || 0%{?el7} || 0%{?fedora} < 25
%define py_prefix python
%if 0%{?el6} || 0%{?el7}
BuildRequires: python-devel
%else
BuildRequires: python2-devel
%endif
%else
# Newer Fedora releases use "pythonX-*"
# A Py2-only project would use "python2" here
%define py_prefix python3
BuildRequires: python3-devel
%endif
# Dependency declarations use stack selected above
BuildRequires: %{py_prefix}-builddep1
BuildRequires: %{py_prefix}-builddep2
Requires: %{py_prefix}-runtimedep1
Requires: %{py_prefix}-runtimedep2
```
For dual stack libraries, the appropriate prefixes to define would be
separate ones for each stack (%py2_prefix and %py3_prefix), and either
leave the latter undefined for systems with no native Py3 stack, or
else set it to rely on EPEL, IUS, or a suitable software collection.
Cheers,
Nick.
--
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
6 years, 5 months
Proposed Mass Bug Filing: Renaming "python-" binary packages to
"python2-"
by Iryna Shcherbina
Hi all,
Currently, in Fedora package names, "python" without a version number generally means Python 2. We would like to make it mean Python 3, but to make that switch, we first need to make sure nothing uses unqualified "python". All packages should use the "python2-" or "python3-" prefix. This has already been discussed in the packaging mailing list [0], which includes the technical details.
Recently, there were two important changes to Python packaging guidelines. The first one bans using the ambiguous `python` prefix in binary RPM package names [1]. Thus you must explicitly define the Python version by using either `python2-` or `python3-`. The second change bans depending on anything using the ambiguous `python-` prefixed names in (Build)Requires [2].
The packages that violate the above-mentioned policies are being tracked in portingdb [3] and we plan to start filling bugs soon . Since th at's a lot of bugs to file (more than 1000) , we encourage all maintainers to fix the packages they maintain. The changes needed to rename the package s are documented here [4]. For your convenience, we have also implemented taskotron automated checks, which run whenever a package is built in Koji . Y ou can see the results in bodhi once you submit an update.
The templates we are going to use for bugs' desctiptions can be found here [5]. Any feedback or suggestions are welcome !
[0] https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproje...
[1] https://fedoraproject.org/wiki/Packaging:Naming?rd=Packaging:NamingGuidel...
[2] https://fedoraproject.org/wiki/Packaging:Python #Dependencies
[3] http://fedora.portingdb.xyz/namingpolicy/
[4] https://python-rpm-porting.readthedocs.io/en/latest/naming-scheme.html
[5] https://fedoraproject.org/wiki/User:Ishcherb/Package_Naming_Bug_Filing
Kind regards,
Iryna Shcherbina
Associate Software Engineer
Red Hat
6 years, 5 months
Re: Proposed Mass Bug Filing: Renaming "python-" binary packages to
"python2-"
by Iryna Shcherbina
The reason behind the second change is that there are a lot of Python 2
only packages having `python-xxx` in Requires, which will start pulling
Python 3 dependencies when we switch to Python 3 as default. This will
most possibly cause a number of unexpected issues, which is what we are
trying to avoid.
If you expect your package to work when we switch to Python 3 as
default, then you should already switch it to Python 3 or provide a
Python 3 subpackage and depend on `python3-xxx`.
Iryna
On 06/19/2017 04:11 PM, Tom Hughes wrote:
> On 19/06/17 14:59, Iryna Shcherbina wrote:
>
>> Recently, there were two important changes to Python packaging
>> guidelines. The first one bans usingtheambiguous `python` prefix in
>> binary RPM package names [1]. Thus you must explicitly
>> definethePython version by using either `python2-` or `python3-`. The
>> second change bans depending on anything using the ambiguous
>> `python-` prefixed names in (Build)Requires [2].
>
> I don't understand the logic behind the second change with regard to
> Requires, or at least I think there are cases where it makes no sense.
>
> If I have a package with a python script using #!/usr/bin/python and
> importing xxx then isn't python-xxx the correct thing to require so
> that it will continue to behave correctly when we switch to python 3
> as default?
>
> If I change it to python2-xxx then I'll just have to change it again
> when we switch the default?
>
> Tom
>
6 years, 5 months
Should Python 3 macros us UTF-8 locale?
by Miro Hrončok
Hi Pythonistas.
Regarding our Python 3 C.UTF-8 locale coercing [1], aka PEP 538 [2].
As you probably know, we build RPM packages with the C locale. So
everytime we use python3 in the spec file, the coercing message is
shown. This can be more problematic than just spamming the build logs,
see for example the related rpmlint bug [3][4].
Our macros, such as %{python3_sitelib}, %{python3_version} etc. all call
python3 and generate the warning. Should we "fix" our macros to set the
LANG to C.UTF-8?
If we change the %{__python3} macro entirely, we might get rid of all of
those warnings and we will workaround the fact that we build RPM
packages with the C locale. On the other hand the packager would not be
able to set a desired locale because it will always be overwritten:
# The crazy test suite needs Czech locale
LANG=cs_CZ.utf8 %{__python3} -m pytest
Will become:
LANG=cs_CZ.utf8 LANG=C.utf8 /usr/bin/python3 -m pytest
So I would not do that.
But we can change all other macros in
/usr/lib/rpm/macros.d/macros.python3 to set the UTF-8 locale. Would that
be wise? Desired?
Any thoughts?
[1] https://fedoraproject.org/wiki/Changes/python3_c.utf-8_locale
[2] https://www.python.org/dev/peps/pep-0538/
[3] https://bugzilla.redhat.com/show_bug.cgi?id=1457786
[4] https://bugzilla.redhat.com/show_bug.cgi?id=1436345
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
6 years, 5 months