Hello Adam,
Adam Williamson [2022-10-18 10:37 -0700]:
that would define a critical path for Server. That would mean
updates
containing any of those packages or their dependencies would be
'critical path' updates, meaning they have higher requirements to be
pushed stable (+2 karma or 14 days waiting, as opposed to +1 karma or 7
days waiting for regular updates)
What is the reason for increasing the waiting period, what would it achieve?
- It sets the wrong motivation, as it indiscriminately penalizes all of these
updates. You can not land faster (or at all) by adding proper CI, and
instead can blame "the community" for not having spotted a regression once
the update gets in eventually.
- It will easily lead to packages not being updated at all any more. We
release every 14 days. Updates in the current stable are usually getting
karma feedback rather easily, but we also support the previous stable, or
the pending fork (like Fedora 37 a few weeks ago), and they usually don't
get *any* karma. We can work around that by giving karma ourself from our
team after gating tests get green [1], but that is just fighting the process
and gives zero new information.
- It is a step backwards. CI helps to increase velocity of the distribution,
this step decreases it. This is walking away from CI and towards more human
interaction.
- It tries to address the problem at the wrong place. What we are sorely
missing in Fedora is reverse dependency testing, i.e. making sure that a
package update does not break *other* packages. New selinux-policy, systemd,
or kernel versions break the distro all the time, but they wouldn't be
covered by the "critical path". Human manual testing of e.g. freeipa or
cockpit *could* spot this, but (1) it takes much more work than just running
the automated tests that we already have, and (2) it barks at the wrong
tree.
As a workaround, we run Cockpit's integration tests (which are kinda sorta
"Fedora/RHEL server QA") against fedora-37 + updates-testing twice a week, and
occasionally we are able to -1 a package update in bodhi that breaks things.
But 7 days are more than enough for that. Our problem has been that at least in
the past, e.g. selinux-policy updates got like +5 "WFM" karma and went into
-updates in *hours*, way too fast to catch it with that method. A human "WFM"
is simply not good enough for these low-level package updates, and it also
doesn't address the "test consumers of that package" problem.
So in summary, I don't believe this is helpful at all, sorry.
and openQA would test them automatically and they would be blocked
from
stable if the tests fail.
That is a good move, of course. Any failing gating test should block updates
(for every package), otherwise we'll never fix them. (Developers can waive, of
course, but without that you wouldn't even know)
Martin
[1] And I'd automate that, hello
https://github.com/osbuild/fedora-bot