Re: Please use side tags for backwards-incompatible bumps of major packages, not buildroot overrides
On 22. 03. 22 19:48, Adam Williamson wrote:
I found quite a big mess today, caused by an attempt to bump perl to
5.34.1 in Fedora 36:
https://bodhi.fedoraproject.org/updates/FEDORA-2022-cea638ebd4
Because some packages depend on the exact perl interpreter version, the
maintainer made a buildroot override for perl:
https://bodhi.fedoraproject.org/overrides/perl-5.34.1-486.fc36
and rebuilt them. Okay.
The problem with this is, we have lots of perl modules. People build
them all the time. And buildroot overrides apply to *everything*.
So, while the buildroot override has been valid, multiple *other* perl
modules have been unwittingly built against it:
perl-Scalar-List-Utils:
https://koji.fedoraproject.org/koji/buildinfo?buildID=1936732
perl-Parallel-Pipes:
https://koji.fedoraproject.org/koji/buildinfo?buildID=1937527
perl-PPIx-Regexp:
https://koji.fedoraproject.org/koji/buildinfo?buildID=1937522
perl-Crypt-SSLeay:
https://koji.fedoraproject.org/koji/buildinfo?buildID=1937521
perl-Crypt-OpenSSL-PKCS10:
https://koji.fedoraproject.org/koji/buildinfo?buildID=1937471
...and those are just the ones I found so far. Because they were built
against 5.34.1, all of those builds require
"perl(:MODULE_COMPAT_5.34.1)" , which means they require perl-5.34.1
from updates-testing. They cannot be installed with perl-5.34.0 from
stable. But the maintainers likely had no idea about this - buildroot
overrides are not at all discoverable, there is zero indication your
package is building against one when you build it - and have created
separate updates for those packages:
https://bodhi.fedoraproject.org/updates/FEDORA-2022-cb1170abf2
https://bodhi.fedoraproject.org/updates/FEDORA-2022-d83d0ba901
https://bodhi.fedoraproject.org/updates/FEDORA-2022-fac98e635f
https://bodhi.fedoraproject.org/updates/FEDORA-2022-c2203f1964
https://bodhi.fedoraproject.org/updates/FEDORA-2022-0990e3309e
Users testing those updates will likely not notice the problem, because
they'll have *all* of updates-testing enabled when they update, and
perl-5.34.1 will be pulled in. But if any of those updates is pushed
stable before the perl update is pushed stable, it will fail to install
for anyone who does not have updates-testing enabled.
This is quite a mess. I'm going to try and clean it up, but it'll be a
bit ugly (there are a couple of ways I can do it, but both will involve
doing bumps-and-rebuilds of the affected packages with no changes).
I've been worried about this sort of thing happening for a while due to
the undesirable properties of buildroot overrides. This is the biggest
real-world case I've seen, but it could certainly happen again. It
might even have happened without anyone noticing exactly what was going
on (just mysterious dependency issues that magically went away after a
bit).
So: now we have convenient self-service side tags, *please use them*.
Especially for something as major as a bump of perl that changes
dependencies of packages built against it like this. Side tags avoid
this mess entirely. Using the mechanism to produce an update from a
side tag also makes it easier to make sure all the right packages are
in the update and they don't get incorrectly submitted as separate
updates.
Thanks folks!
Thanks for this, Adam.
I will add one more reason:
When the automatic "fails to install" reports run, they use the Koji buidlroot.
Every now and then, a bugzilla is reported like this one:
https://bugzilla.redhat.com/show_bug.cgi?id=2066728
It is not a false positive, the dependency problem exists, but it only exists
because there is an override:
https://bodhi.fedoraproject.org/overrides/R-4.1.3-1.fc36
Please, use side tags, not overrides, when updating inter-dependent packages,
it avoid (even temporary) breakage:
https://docs.fedoraproject.org/en-US/fesco/Updates_Policy/#updating-inter...
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok