F-14 ON_QA release blockers in need of testing

[Kamil Paral]

----- "Sandro \"red\" Mathys" <red at fedoraproject.org> wrote:

> On 10/19/2010 04:57 PM, Kamil Paral wrote:
> > ----- "Sandro \"red\" Mathys" <red at fedoraproject.org> wrote:
> > 
> >>> List of ON_QA bugs - http://bit.ly/dx4ehO
> >>
> >> [16:17:55] <red_alert> many of those ON_QA bugs have been VERIFIED
> >> before bodhi changed it back [to ON_QA]...do we need to re-test
> >> those?
> >>
> >>
> >> [16:19:47] <jlaska> red_alert: I think that's a really good
> >> discussion
> >> for the list.  kparal mentioned that earlier, let's discuss it
> there
> >> so
> >> others can benefit as well
> >>
> >> #####
> >>
> >> I've often seen people setting the status back to VERIFIED in such
> a
> >> case but I'm not exactly sure if that's the best possible
> behavior.
> >>
> >> So, does it make sense to re-test fixes because the newly pushed
> >> version
> >> might have broken things again or do we generally assume that
> fixes
> >> included in older versions are still working in newer versions?
> >>
> >> IMHO it doesn't make sense to re-test again. There'll always be
> newer
> >> versions and we can't always re-test every bug with every new
> >> version.
> >>
> >> +1 for just setting back to VERIFIED again
> > 
> > 
> > It's a little more complicated. Let's suppose we have bug #1234 and
> > foobar-1.1 claims to fix it. If you post proposed update to Bodhi 
> > and fill into details that it fixes #1234, then Bodhi will set
> > #1234 to ON_QA. Some tester will test that and will set the status
> > to VERIFIED.
> > 
> > Now, there was some serious issue with foobar-1.1, unrelated to 
> > #1234. The developer will *unpush* that proposed update, create
> > foobar-1.2 and post it to proposed updates again. Because there
> > is still no foobar update released that would fix #1234, the 
> > developer will again mark foobar-1.2 as fixing #1234. Bodhi will
> > change #1234 back from VERIFIED to ON_QA. This is correct, because
> > although foobar-1.1 is verified to fix that issue, there were some
> > additional changes that could negatively impact that fix. So the
> > tester should test foobar-1.2 and mark #1234 as VERIFIED again,
> > if everything works ok.
> > 
> > On the other hand, let's suppose that foobar-1.1 was released
> > and pushed into 'updates' repository. Now, when the developer
> > creates foobar-1.2, he *should not* mark bug #1234 as being fixed
> > by foobar-1.2. That was was already fixed by foobar-1.1 and the fix
> > was *released*. If such event happens (developer mistakenly marks
> > foobar-1.2 as fixing #1234 and Bodhi sets the bug again to ON_QA),
> > there is really no need to test it again. We can set it back to
> > VERIFIED.
> > 
> > I think I have described how it should work. But maybe I'm wrong.
> > Does it make sense? What do you think?
> 
> Right, I never thought about unpushed updates (i.e. I hardly ever met
> such cases yet). 

Not just unpushed updates, but simply all "not pushed updates" :)
foobar-1.1 may stay in updates-testing, but it never comes to stable 
updates. And foobar-1.2 is then posted into updates-testing. In this
case we still need to re-verify relevant bugs.


> So I think we need to quickly check the details in
> bodhi and then either re-test or (in most cases) set it back to
> VERIFIED.
> 
> Is that about it or is it more complicated? :)

That's about it, I guess :)