Non-image blocker process change proposal

[Adam Williamson]

Hi, folks! It's been a recurring issue in the blocker review / release
validation process in recent times that we run across bugs that qualify
as blockers, but for which the fix does not need to be in the final
frozen media or install trees.

Common cases are bugs related to upgrading, especially since the
introduction of fedup and even more so of dnf-system-upgrade; most
upgrade-related issues can now be sufficiently fixed by package updates
to either the source or target release. Bugs to do with writing USB
media from the previous release, for instance, also often fall in this
category.

Up until now we've been sort of handwaving these as 'special blockers',
following the regular blocker process but noting in comments that they
don't block the compose. We haven't been tracking very hard if they
actually *are* being fixed with updates promptly, we've just been sort
of waving a magic wand and assuming it will happen. I just found one
which was supposed to be fixed with a 0-day update for Beta, but hadn't
been fixed yet: https://bugzilla.redhat.com/show_bug.cgi?id=1263230

So, we kinda need to do this better.

Up top I'd like to note there are really kind of two buckets of
'special blockers' for any given release. If the release being
validated is N, they are:

1) Bugs for which the fix must be in the 0-day update set for N
2) Bugs for which the fix must be stable in N-1 and N-2 by N release day

There will be a lot of process nerd detail involved in any fix, but
before any detailed drafts I'd like to suggest two broad possible
approaches and see what people think:

#1 Separate trackers
--------------------

As a sort of on-the-spot PoC for F23 Beta, I created a new tracker bug
for bucket 1: https://bugzilla.redhat.com/show_bug.cgi?id=1264167

We could formalize that approach, and have a '0-day' blocker tracker
for each milestone. We could either just have one '0-day' tracker and
throw bugs for both the pending release and previous stable releases on
the same tracker and keep track of what needs updating where in each
bug, or we could have two 0-day trackers for each milestone, one for
the pending release, and one for previous stable releases.

So we'd have something like:

F24AlphaBlocker
F24AlphaFreezeException
F24Alpha0Day
(F24AlphaStable) (? - better alias suggestions welcome)

and so on. This is a lot of bugs, but there is a script to create them,
so we're not adding a bunch of onerous work.

So far as proposing bugs goes, I think we'd probably want to extend the
current somewhat flexible approach; formally you would nominate a bug
as a particular type of blocker/FE (by marking it as blocking the
appropriate tracker), but we would move things around in blocker review
meetings sensibly, as we currently do (when something is nominated as
FE but should really be a blocker, or vice versa). In blocker review
we'd go through all bugs nominated for any of the trackers, probably
starting with 'Blocker', then '0Day' and 'Stable', then
'FreezeException'.

#2 MOAR METADATA
----------------

The alternative is to make the existing Blocker trackers do more work.
In this model we wouldn't add any new tracker bugs; we'd just add new
'magic words' in the Whiteboard field. Right now, an accepted blocker
is identified by the string 'AcceptedBlocker' appearing in the
whiteboard field. We could simply add some more magical strings like
that: 'Accepted0Day' and 'AcceptedStable', say (better suggestions
welcome).

I kind of like this idea as it's less change and involves creating
fewer new bugs. We'd have to make some changes to blockerbugs either
way - tflink can say if either approach would be more work in
blockerbugs, but I'm gonna guess they'd be fairly similar.

With either approach, the basic goal is to make it more feasible to
keep an eye on each of the different categories of 'release blocker'
bugs; right now we have solid processes in place for ensuring the
'normal' blockers are all addressed in the release media, but we don't
have any processes in place for ensuring 0Day and Stable bugs actually
get updates shipped when we say they must.

My suggestion would be that we make sure 'blockerbugs' includes lists
of each type of blocker. Ahead of and at Go/No-Go meetings, we would
want to have a formal assurance from the person responsible for fixing
the bug that the fix would be provided by a certain time - say, one day
or two days ahead of the release date - and it would be QA's
responsibility to ensure the updates are tested promptly, and releng's
responsibility to ensure they are pushed on time after being tested. I
would suggest the Program Manager ought to have overall responsibility
for keeping an eye on the 0Day and Stable blocker lists and making sure
the maintainer, QA, and releng all did their jobs on time.

It'd be great if folks could post their general thoughts on this, and
any preference for option 1 or option 2. Thanks!
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net