Heads-up: rpm 4.15 alpha coming to rawhide

[Panu Matilainen]

As per https://fedoraproject.org/wiki/Changes/RPM-4.15, rpm 4.15-alpha 
will be hitting rawhide soon. A soname bump is involved but Igor kindly 
promised to handle rebuilding all dependent packages as part of the 
change, so no further action required from others on that account.

There are some other things that will require actions from others however:

A pile of language-specific macros and scripts have been eliminated from 
rpm upstream, notably %__python and %__perl and everything built around 
them, such as %python_sitelib and %perl_sitelib and their relatives. 
Python packages should be using the version specific macros already, 
provided by respective pythonN-devel packages so this shouldn't be an 
issue. On perl-side, those macros have been temporarily added to 
redhat-rpm-config instead to avoid breaking things right now, later to 
be moved to final destination in perl-macros or such.

Another major change is in Python bindings where Python 3 is now 
properly supported. In all the previous versions where Python 3 bindings 
were buildable and available, string data from header was returned as 
bytes, but the rest of the API expects strings so nothing really works 
(see https://bugzilla.redhat.com/show_bug.cgi?id=1693751). To make it 
worse, it was incompatible with every single rpm-python script ever 
written. Usually when something is unusably broken, silence means that 
it's not being used, otherwise people do complain. Not so in this case, 
the absurd API has been there for almost ten years and people have 
adopted to it with zero complaints. In short, rpm-python users still 
supporting both Python 2 and 3 are generally compatible with this 
change, but Python 3-only projects have out of necessity adopted to the 
broken API and will now need to change. The most critical distro 
components have more or less already been adopted, but here's more work 
to do on this front. This fixed interface will be made available to 
4.14.x somehow too, but the details are yet unclear. Because of that, 
we'd suggest only patching packages in rawhide to work with the change 
right now, and deal with upstreams once dust settles a bit.

Other than that, a major new thing is introduction of some parallel 
operations in rpmbuild itself. Testing hasn't shown anything odd but 
with threads and all, you never know. Keep your eyes open.

More info and details available in the preliminary release notes at 
https://rpm.org/wiki/Releases/4.15.0 and the change page linked at the 
start of this message.

	- Panu -

[Panu Matilainen]

As per https://fedoraproject.org/wiki/Changes/RPM-4.15, rpm 4.15-alpha 
will be hitting rawhide soon. A soname bump is involved but Igor kindly 
promised to handle rebuilding all dependent packages as part of the 
change, so no further action required from others on that account.

There are some other things that will require actions from others however:

A pile of language-specific macros and scripts have been eliminated from 
rpm upstream, notably %__python and %__perl and everything built around 
them, such as %python_sitelib and %perl_sitelib and their relatives. 
Python packages should be using the version specific macros already, 
provided by respective pythonN-devel packages so this shouldn't be an 
issue. On perl-side, those macros have been temporarily added to 
redhat-rpm-config instead to avoid breaking things right now, later to 
be moved to final destination in perl-macros or such.

Another major change is in Python bindings where Python 3 is now 
properly supported. In all the previous versions where Python 3 bindings 
were buildable and available, string data from header was returned as 
bytes, but the rest of the API expects strings so nothing really works 
(see https://bugzilla.redhat.com/show_bug.cgi?id=1693751). To make it 
worse, it was incompatible with every single rpm-python script ever 
written. Usually when something is unusably broken, silence means that 
it's not being used, otherwise people do complain. Not so in this case, 
the absurd API has been there for almost ten years and people have 
adopted to it with zero complaints. In short, rpm-python users still 
supporting both Python 2 and 3 are generally compatible with this 
change, but Python 3-only projects have out of necessity adopted to the 
broken API and will now need to change. The most critical distro 
components have more or less already been adopted, but here's more work 
to do on this front. This fixed interface will be made available to 
4.14.x somehow too, but the details are yet unclear. Because of that, 
we'd suggest only patching packages in rawhide to work with the change 
right now, and deal with upstreams once dust settles a bit.

Other than that, a major new thing is introduction of some parallel 
operations in rpmbuild itself. Testing hasn't shown anything odd but 
with threads and all, you never know. Keep your eyes open.

More info and details available in the preliminary release notes at 
https://rpm.org/wiki/Releases/4.15.0 and the change page linked at the 
start of this message.

	- Panu -
_______________________________________________
devel-announce mailing list -- devel-announce(a)lists.fedoraproject.org
To unsubscribe send an email to devel-announce-leave(a)lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]...

[Neal Gompa]

On Mon, Jun 10, 2019 at 8:28 AM Panu Matilainen <pmatilai(a)redhat.com&gt; wrote:
>  A pile of language-specific macros and scripts have been eliminated from
>  rpm upstream, notably %__python and %__perl and everything built around
>  them, such as %python_sitelib and %perl_sitelib and their relatives.
>  Python packages should be using the version specific macros already,
>  provided by respective pythonN-devel packages so this shouldn't be an
>  issue. On perl-side, those macros have been temporarily added to
>  redhat-rpm-config instead to avoid breaking things right now, later to
>  be moved to final destination in perl-macros or such.
>  
Unversioned Python macros need to get added to python-rpm-macros, as
there are a number of packages that do rely on their existence (as a
means of supporting only a single version of Python depending on
distro releases or legacy reasons).

CC'd python-devel, Miro, and Igor about this.


-- 
真実はいつも一つ！/ Always, there's only one truth!

[Miro Hrončok]

On 10. 06. 19 14:32, Neal Gompa wrote:
>  On Mon, Jun 10, 2019 at 8:28 AM Panu Matilainen
> <pmatilai(a)redhat.com&gt; wrote:
> >
> >
> >
> > A pile of language-specific macros and scripts have been eliminated from
> > rpm upstream, notably %__python and %__perl and everything built around
> > them, such as %python_sitelib and %perl_sitelib and their relatives.
> > Python packages should be using the version specific macros already,
> > provided by respective pythonN-devel packages so this shouldn't be an
> > issue. On perl-side, those macros have been temporarily added to
> > redhat-rpm-config instead to avoid breaking things right now, later to
> > be moved to final destination in perl-macros or such.
> >
>  
>  Unversioned Python macros need to get added to python-rpm-macros, as
>  there are a number of packages that do rely on their existence (as a
>  means of supporting only a single version of Python depending on
>  distro releases or legacy reasons). 
On it.


-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Panu Matilainen]

On 6/10/19 3:42 PM, Miro Hrončok wrote:
>  On 10. 06. 19 14:32, Neal Gompa wrote:
> > On Mon, Jun 10, 2019 at 8:28 AM Panu Matilainen <pmatilai(a)redhat.com&gt; 
> > wrote:
> >>
> >>
> >>
> >> A pile of language-specific macros and scripts have been eliminated from
> >> rpm upstream, notably %__python and %__perl and everything built around
> >> them, such as %python_sitelib and %perl_sitelib and their relatives.
> >> Python packages should be using the version specific macros already,
> >> provided by respective pythonN-devel packages so this shouldn't be an
> >> issue. On perl-side, those macros have been temporarily added to
> >> redhat-rpm-config instead to avoid breaking things right now, later to
> >> be moved to final destination in perl-macros or such.
> >>
> >
> > Unversioned Python macros need to get added to python-rpm-macros, as
> > there are a number of packages that do rely on their existence (as a
> > means of supporting only a single version of Python depending on
> > distro releases or legacy reasons).
>  
>  On it.
>   
Hmm, I somehow managed to convince myself (by looking at rawhide specs) 
that these would not be needed, but looking again now, clearly there are 
quite a few using those still. Sorry about that.

I could add them to redhat-rpm-config easily, but even better if we can 
get them directly to python-rpm-macros instead.

	- Panu -

[Miro Hrončok]

On 10. 06. 19 14:42, Miro Hrončok wrote:
>  On 10. 06. 19 14:32, Neal Gompa wrote:
> > On Mon, Jun 10, 2019 at 8:28 AM Panu Matilainen <pmatilai(a)redhat.com&gt; wrote:
> >>
> >>
> >>
> >> A pile of language-specific macros and scripts have been eliminated from
> >> rpm upstream, notably %__python and %__perl and everything built around
> >> them, such as %python_sitelib and %perl_sitelib and their relatives.
> >> Python packages should be using the version specific macros already,
> >> provided by respective pythonN-devel packages so this shouldn't be an
> >> issue. On perl-side, those macros have been temporarily added to
> >> redhat-rpm-config instead to avoid breaking things right now, later to
> >> be moved to final destination in perl-macros or such.
> >>
> >
> > Unversioned Python macros need to get added to python-rpm-macros, as
> > there are a number of packages that do rely on their existence (as a
> > means of supporting only a single version of Python depending on
> > distro releases or legacy reasons).
>  
>  On it. 
https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/22

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

[Igor Gnatenko]

So apparently new RPM segfaults on some architectures randomly, I'm
digging into it.

In worst case, I'll tag old version and try to debug it locally.

So far I can't reproduce it =(

On Mon, Jun 10, 2019 at 2:36 PM Panu Matilainen <pmatilai(a)redhat.com&gt; wrote:
>
>
> As per https://fedoraproject.org/wiki/Changes/RPM-4.15, rpm 4.15-alpha
> will be hitting rawhide soon. A soname bump is involved but Igor kindly
> promised to handle rebuilding all dependent packages as part of the
> change, so no further action required from others on that account.
>
> There are some other things that will require actions from others however:
>
> A pile of language-specific macros and scripts have been eliminated from
> rpm upstream, notably %__python and %__perl and everything built around
> them, such as %python_sitelib and %perl_sitelib and their relatives.
> Python packages should be using the version specific macros already,
> provided by respective pythonN-devel packages so this shouldn't be an
> issue. On perl-side, those macros have been temporarily added to
> redhat-rpm-config instead to avoid breaking things right now, later to
> be moved to final destination in perl-macros or such.
>
> Another major change is in Python bindings where Python 3 is now
> properly supported. In all the previous versions where Python 3 bindings
> were buildable and available, string data from header was returned as
> bytes, but the rest of the API expects strings so nothing really works
> (see https://bugzilla.redhat.com/show_bug.cgi?id=1693751). To make it
> worse, it was incompatible with every single rpm-python script ever
> written. Usually when something is unusably broken, silence means that
> it's not being used, otherwise people do complain. Not so in this case,
> the absurd API has been there for almost ten years and people have
> adopted to it with zero complaints. In short, rpm-python users still
> supporting both Python 2 and 3 are generally compatible with this
> change, but Python 3-only projects have out of necessity adopted to the
> broken API and will now need to change. The most critical distro
> components have more or less already been adopted, but here's more work
> to do on this front. This fixed interface will be made available to
> 4.14.x somehow too, but the details are yet unclear. Because of that,
> we'd suggest only patching packages in rawhide to work with the change
> right now, and deal with upstreams once dust settles a bit.
>
> Other than that, a major new thing is introduction of some parallel
> operations in rpmbuild itself. Testing hasn't shown anything odd but
> with threads and all, you never know. Keep your eyes open.
>
> More info and details available in the preliminary release notes at
> https://rpm.org/wiki/Releases/4.15.0 and the change page linked at the
> start of this message.
>
>         - Panu -
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]

[Panu Matilainen]

----- Original Message -----
>  So apparently new RPM segfaults on some architectures randomly,
> I'm
>  digging into it.
>  
>  In worst case, I'll tag old version and try to debug it locally.
>  
>  So far I can't reproduce it =( 
Too late for any real bughunting by now, just untag and I'll look at it first thing
tomorrow.

My first guess would be the multithreading, and if that's the case then building with

RPM_BUILD_NCPUS=1 in the environment (or macro %_smp_ncpus_max defined to 1) should make
it go away.

Thanks,

    - Panu -

[Petr Pisar]

On 2019-06-10, Panu Matilainen <pmatilai(a)redhat.com&gt; wrote:
>  As per https://fedoraproject.org/wiki/Changes/RPM-4.15, rpm 4.15-alpha 
>  will be hitting rawhide soon. A soname bump is involved but Igor kindly 
>  promised to handle rebuilding all dependent packages as part of the 
>  change, so no further action required from others on that account.
>
>  There are some other things that will require actions from others however:
>
>  A pile of language-specific macros and scripts have been eliminated from 
>  rpm upstream, notably %__python and %__perl and everything built around 
>  them, such as %python_sitelib and %perl_sitelib and their relatives. 
>  Python packages should be using the version specific macros already, 
>  provided by respective pythonN-devel packages so this shouldn't be an 
>  issue. On perl-side, those macros have been temporarily added to 
>  redhat-rpm-config instead to avoid breaking things right now, later to 
>  be moved to final destination in perl-macros or such.
>  
Thanks for the heads up. Perl maintainers are deliberating an optimal
resolution now. We will come up with a proposal in a few days.

-- Petr

[Jitka Plesnikova]

On 6/11/19 6:33 PM, Petr Pisar wrote:
>  On 2019-06-10, Panu Matilainen <pmatilai(a)redhat.com&gt; wrote:
> > As per https://fedoraproject.org/wiki/Changes/RPM-4.15, rpm 4.15-alpha 
> > will be hitting rawhide soon. A soname bump is involved but Igor kindly 
> > promised to handle rebuilding all dependent packages as part of the 
> > change, so no further action required from others on that account.
> >
> > There are some other things that will require actions from others however:
> >
> > A pile of language-specific macros and scripts have been eliminated from 
> > rpm upstream, notably %__python and %__perl and everything built around 
> > them, such as %python_sitelib and %perl_sitelib and their relatives. 
> > Python packages should be using the version specific macros already, 
> > provided by respective pythonN-devel packages so this shouldn't be an 
> > issue. On perl-side, those macros have been temporarily added to 
> > redhat-rpm-config instead to avoid breaking things right now, later to 
> > be moved to final destination in perl-macros or such.
> >
>  Thanks for the heads up. Perl maintainers are deliberating an optimal
>  resolution now. We will come up with a proposal in a few days. 
%{__perl} will be
moved to perl-srpm-macros. Some packages use it in
list of
BuildRequires and so it has to be defined in buildroot when source rpm
is built.
%perl_sitearch and %perl_sitelib will be removed, because they are not used
in any spec file.
%perl_vendorlib, %perl_vendorarch, %perl_archlib and %perl_privlib will be
placed in perl-macros. To prevent the build failures, perl-macros will be
added as build-require to packages, which use the macros and doesn't
build-require perl-generators (it run-requires perl-macros).

%requires_eq isn't perl macro and it stays in redhat-rpm-config. The macro
is used only in samba.spec.

Regards,
Jitka

-- 
Jitka Plesnikova
Software Engineer
Red Hat

[Panu Matilainen]

On 6/12/19 12:12 PM, Jitka Plesnikova wrote:
>  On 6/11/19 6:33 PM, Petr Pisar wrote:
> > On 2019-06-10, Panu Matilainen <pmatilai(a)redhat.com&gt; wrote:
> >> As per https://fedoraproject.org/wiki/Changes/RPM-4.15, rpm 4.15-alpha
> >> will be hitting rawhide soon. A soname bump is involved but Igor kindly
> >> promised to handle rebuilding all dependent packages as part of the
> >> change, so no further action required from others on that account.
> >>
> >> There are some other things that will require actions from others however:
> >>
> >> A pile of language-specific macros and scripts have been eliminated from
> >> rpm upstream, notably %__python and %__perl and everything built around
> >> them, such as %python_sitelib and %perl_sitelib and their relatives.
> >> Python packages should be using the version specific macros already,
> >> provided by respective pythonN-devel packages so this shouldn't be an
> >> issue. On perl-side, those macros have been temporarily added to
> >> redhat-rpm-config instead to avoid breaking things right now, later to
> >> be moved to final destination in perl-macros or such.
> >>
> > Thanks for the heads up. Perl maintainers are deliberating an optimal
> > resolution now. We will come up with a proposal in a few days.
>  %{__perl} will be moved to perl-srpm-macros. Some packages use it in
>  list of
>  BuildRequires and so it has to be defined in buildroot when source rpm
>  is built.
>  %perl_sitearch and %perl_sitelib will be removed, because they are not used
>  in any spec file.
>  %perl_vendorlib, %perl_vendorarch, %perl_archlib and %perl_privlib will be
>  placed in perl-macros. To prevent the build failures, perl-macros will be
>  added as build-require to packages, which use the macros and doesn't
>  build-require perl-generators (it run-requires perl-macros). 
Ok, sounds more or less the way I expected it to be.

>  %requires_eq isn't perl macro and it stays in redhat-rpm-config.
> The macro
>  is used only in samba.spec. 
%requires_eq was considered somehow perl-related in upstream (due to 
whatever long forgotten history) but okay, even better if its not used 
in perl either. That way we can bury it into samba.spec and remove 
elsewhere, because it violates the "dont run rpm queries from spec" 
principle and we dont want to encourage such use by providing example.

Thanks for the swift analysis.

	- Panu -

[Adam Williamson]

On Wed, 2019-06-12 at 12:40 +0300, Panu Matilainen wrote:
>  On 6/12/19 12:12 PM, Jitka Plesnikova wrote:
>  > On 6/11/19 6:33 PM, Petr Pisar wrote:
>  > > On 2019-06-10, Panu Matilainen <pmatilai(a)redhat.com&gt; wrote:
>  > > > As per https://fedoraproject.org/wiki/Changes/RPM-4.15, rpm 4.15-alpha
>  > > > will be hitting rawhide soon. A soname bump is involved but Igor kindly
>  > > > promised to handle rebuilding all dependent packages as part of the
>  > > > change, so no further action required from others on that account.
>  > > > 
>  > > > There are some other things that will require actions from others
> however:
>  > > > 
>  > > > A pile of language-specific macros and scripts have been eliminated from
>  > > > rpm upstream, notably %__python and %__perl and everything built around
>  > > > them, such as %python_sitelib and %perl_sitelib and their relatives.
>  > > > Python packages should be using the version specific macros already,
>  > > > provided by respective pythonN-devel packages so this shouldn't be an
>  > > > issue. On perl-side, those macros have been temporarily added to
>  > > > redhat-rpm-config instead to avoid breaking things right now, later to
>  > > > be moved to final destination in perl-macros or such.
>  > > > 
>  > > Thanks for the heads up. Perl maintainers are deliberating an optimal
>  > > resolution now. We will come up with a proposal in a few days.
>  > %{__perl} will be moved to perl-srpm-macros. Some packages use it in
>  > list of
>  > BuildRequires and so it has to be defined in buildroot when source rpm
>  > is built.
>  > %perl_sitearch and %perl_sitelib will be removed, because they are not used
>  > in any spec file.
>  > %perl_vendorlib, %perl_vendorarch, %perl_archlib and %perl_privlib will be
>  > placed in perl-macros. To prevent the build failures, perl-macros will be
>  > added as build-require to packages, which use the macros and doesn't
>  > build-require perl-generators (it run-requires perl-macros).
>  
>  Ok, sounds more or less the way I expected it to be.
>  
>  > %requires_eq isn't perl macro and it stays in redhat-rpm-config. The macro
>  > is used only in samba.spec.
>  
>  %requires_eq was considered somehow perl-related in upstream (due to 
>  whatever long forgotten history) 
AIUI, it is/was commonly used by perl module packages to require the
same version of perl as the module package was built with, because this
is/was a thing in perl - when perl's version got bumped, all modules
needed rebuilding too. That's probably where the 'perl-related' thing
comes in - it's not that the actual thing %requires_eq does is in any
way perl-specific, but it's a thing that was introduced in response to
this particular property of perl and its modules, and IIRC not often or
ever used by anything else.

However in Fedora, we don't use %requires_eq for perl modules, at least
not any more (perhaps we did in the past, I don't know). Instead we
have the perl(:MODULE_COMPAT_[version]) dependency that is handled by
our own perl macros (provided by the interpreter package and
automatically required by module packages). I *have* run across the use
of %requires_eq in SUSE packages, though.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

[Miroslav Suchý]

Dne 10. 06. 19 v 13:39 Panu Matilainen napsal(a):
>  More info and details available in the preliminary release notes at
> https://rpm.org/wiki/Releases/4.15.0 and the change
>  page linked at the start of this message. 
Where can I read more about this:
>  Add support for rootless chroot-operations on Linux (experimental)
?

-- 
Miroslav Suchy, RHCA
Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys

[Panu Matilainen]

On 6/12/19 1:07 PM, Miroslav Suchý wrote:
>  Dne 10. 06. 19 v 13:39 Panu Matilainen napsal(a):
> > More info and details available in the preliminary release notes at
> https://rpm.org/wiki/Releases/4.15.0 and the change
> > page linked at the start of this message.
>  
>  Where can I read more about this:
>     > Add support for rootless chroot-operations on Linux (experimental)
>  ?
>   
There's not a whole lot to write about, it just means that operations 
which require chroot() now more or less work for regular users by the 
way of user namespaces.

That more-or-less is part of the reason for the experimental status, as 
due to the way the user namespace switch is hidden inside rpm's chroot() 
helpers, it can't fork which would be required (AIUI) to properly set up 
the uid/gid mappings inside the namespace. So while you can now install 
into a chroot as a regular user, any files not owned by root (or 
yourself) will fail, so its not as useful as it seems initially:

context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[pmatilai🎩︎sopuli ~]$ rpm -Uvh --root ~/testroot mft/f28-bash.mft
warning: /mnt/Packages/b/bash-4.4.19-2.fc28.x86_64.rpm: Header V3 
RSA/SHA256 Signature, key ID 9db62fb1: NOKEY
Verifying...                          ################################# 
[100%]
Preparing...                          ################################# 
[100%]
Updating / installing...
    1:fedora-gpg-keys-28-1             ################################# 
[  6%]
    2:fedora-repos-28-1                ################################# 
[ 13%]
    3:fedora-release-28-1              ################################# 
[ 19%]
    4:setup-2.11.3-1.fc28              ################################# 
[ 25%]
    5:filesystem-3.8-2.fc28            ################################# 
[ 31%]
error: unpacking of archive failed on file /var/spool/mail: cpio: chown 
failed - No such file or directory
error: filesystem-3.8-2.fc28.x86_64: install failed
    6:basesystem-11-5.fc28             ################################# 
[ 38%]
    7:tzdata-2018d-1.fc28              ################################# 
[ 44%]
    8:ncurses-base-6.1-4.20180224.fc28 ################################# 
[ 50%]
    9:pcre2-10.31-4.fc28               ################################# 
[ 56%]
   10:libselinux-2.7-13.fc28           ################################# 
[ 63%]
   11:ncurses-libs-6.1-4.20180224.fc28 ################################# 
[ 69%]
   12:glibc-langpack-en-2.27-8.fc28    ################################# 
[ 75%]
   13:glibc-common-2.27-8.fc28         ################################# 
[ 81%]
   14:glibc-2.27-8.fc28                ################################# 
[ 88%]
   15:bash-4.4.19-2.fc28               ################################# 
[ 94%]
   16:libsepol-2.7-6.fc28              ################################# 
[100%]
[pmatilai🎩︎sopuli ~]$ rpm -Va --root ~/testroot
Unsatisfied dependencies for bash-4.4.19-2.fc28.x86_64:
	filesystem >= 3 is needed by (installed) bash-4.4.19-2.fc28.x86_64
missing     /usr/lib/systemd/system-preset/85-display-manager.preset
.M.......  g /usr/lib/variant
[pmatilai🎩︎sopuli ~]$

It remains to be seen if there's something that we can do to make this 
work, or whether it's just too much of a hack to live.

	- Panu -