Non-image blocker process change proposal

[Sudhir D]

On 11/19/2015 06:06 AM, Adam Williamson wrote:
> Hi, folks! It's been a recurring issue in the blocker review / release
> validation process in recent times that we run across bugs that qualify
> as blockers, but for which the fix does not need to be in the final
> frozen media or install trees.
>
> Common cases are bugs related to upgrading, especially since the
> introduction of fedup and even more so of dnf-system-upgrade; most
> upgrade-related issues can now be sufficiently fixed by package updates
> to either the source or target release. Bugs to do with writing USB
> media from the previous release, for instance, also often fall in this
> category.
>
> Up until now we've been sort of handwaving these as 'special blockers',
> following the regular blocker process but noting in comments that they
> don't block the compose. We haven't been tracking very hard if they
> actually *are* being fixed with updates promptly, we've just been sort
> of waving a magic wand and assuming it will happen. I just found one
> which was supposed to be fixed with a 0-day update for Beta, but hadn't
> been fixed yet: https://bugzilla.redhat.com/show_bug.cgi?id=1263230
>
> So, we kinda need to do this better.
>
> Up top I'd like to note there are really kind of two buckets of
> 'special blockers' for any given release. If the release being
> validated is N, they are:
>
> 1) Bugs for which the fix must be in the 0-day update set for N
> 2) Bugs for which the fix must be stable in N-1 and N-2 by N release day
>
> There will be a lot of process nerd detail involved in any fix, but
> before any detailed drafts I'd like to suggest two broad possible
> approaches and see what people think:
>
> #1 Separate trackers
> --------------------
>
> As a sort of on-the-spot PoC for F23 Beta, I created a new tracker bug
> for bucket 1: https://bugzilla.redhat.com/show_bug.cgi?id=1264167
>
> We could formalize that approach, and have a '0-day' blocker tracker
> for each milestone. We could either just have one '0-day' tracker and
> throw bugs for both the pending release and previous stable releases on
> the same tracker and keep track of what needs updating where in each
> bug, or we could have two 0-day trackers for each milestone, one for
> the pending release, and one for previous stable releases.
>
> So we'd have something like:
>
> F24AlphaBlocker
> F24AlphaFreezeException
> F24Alpha0Day
> (F24AlphaStable) (? - better alias suggestions welcome)
>
> and so on. This is a lot of bugs, but there is a script to create them,
> so we're not adding a bunch of onerous work.
>
> So far as proposing bugs goes, I think we'd probably want to extend the
> current somewhat flexible approach; formally you would nominate a bug
> as a particular type of blocker/FE (by marking it as blocking the
> appropriate tracker), but we would move things around in blocker review
> meetings sensibly, as we currently do (when something is nominated as
> FE but should really be a blocker, or vice versa). In blocker review
> we'd go through all bugs nominated for any of the trackers, probably
> starting with 'Blocker', then '0Day' and 'Stable', then
> 'FreezeException'.
>
> #2 MOAR METADATA
> ----------------
>
> The alternative is to make the existing Blocker trackers do more work.
> In this model we wouldn't add any new tracker bugs; we'd just add new
> 'magic words' in the Whiteboard field. Right now, an accepted blocker
> is identified by the string 'AcceptedBlocker' appearing in the
> whiteboard field. We could simply add some more magical strings like
> that: 'Accepted0Day' and 'AcceptedStable', say (better suggestions
> welcome).
>
> I kind of like this idea as it's less change and involves creating
> fewer new bugs. We'd have to make some changes to blockerbugs either
> way - tflink can say if either approach would be more work in
> blockerbugs, but I'm gonna guess they'd be fairly similar.
>
> With either approach, the basic goal is to make it more feasible to
> keep an eye on each of the different categories of 'release blocker'
> bugs; right now we have solid processes in place for ensuring the
> 'normal' blockers are all addressed in the release media, but we don't
> have any processes in place for ensuring 0Day and Stable bugs actually
> get updates shipped when we say they must.
>
> My suggestion would be that we make sure 'blockerbugs' includes lists
> of each type of blocker. Ahead of and at Go/No-Go meetings, we would
> want to have a formal assurance from the person responsible for fixing
> the bug that the fix would be provided by a certain time - say, one day
> or two days ahead of the release date - and it would be QA's
> responsibility to ensure the updates are tested promptly, and releng's
> responsibility to ensure they are pushed on time after being tested. I
> would suggest the Program Manager ought to have overall responsibility
> for keeping an eye on the 0Day and Stable blocker lists and making sure
> the maintainer, QA, and releng all did their jobs on time.
>
> It'd be great if folks could post their general thoughts on this, and
> any preference for option 1 or option 2. Thanks!

I suggest we have only one ZeroDay i.e., for Final and do away with 
intermediate ones.
As I see it, ZeroDay comes with cost and we also need to have basic 
sanity testcases automated to ensure ZeroDay fixes won't 
introduce/regress blocker.

How about automatically qualifying any freeze exception in current phase 
as blocker for next phase and keep 0day only for RC?
AlphaBlocker --> AlphaFreezeException --> BetaBlocker --> 
BetaFreezeException --> FinalBlocker --> FinalBlockerException --> ZeroDay

This would mean we will be not so liberal in allowing blockers linger 
around in a phase for more time, but I think that is okay tradeoff.

 From tracking perspective, I think we may just want to have trackers 
for phaseBlocker for each milestone and FinalBlocker and 0Day for Final 
along with backPortfix tracker one for the pending release, and one for 
previous stable releases.

Regards,
Sudhir