Building and submitting updates for Fedora 20

[Adam Williamson]

On Mon, 2013-10-07 at 04:03 +0200, Ralf Corsepius wrote:
> On 10/07/2013 03:49 AM, Tim Flink wrote:
> > On Sun, 06 Oct 2013 15:09:24 -0500
> > Rex Dieter <rdieter at math.unl.edu> wrote:
> >
> >> Michael Schwendt wrote:
> >>
> >>> On Sun, 06 Oct 2013 19:11:41 +0200, Ralf Corsepius wrote:
> >>>
> >>>> I now see ... the version in f19 was greater than that in
> >>>> f20+rawhide, for whatever reasons.
> >>>> Actually, I wonder why AutoQA did not complain.
> >>> There are no AutoQA comments in that bodhi ticket at all. Almost as
> >>> if AutoQA has not been run for that update. Normally it would add a
> >>> comment also for PASSED tests.
> >> Is AutoQA enabled globally yet?  Last I knew, it was an opt-in
> >> service.
> > AutoQA is run globally on all current fedora branches other than
> > rawhide but if maintainers want results emailed to them, it's opt-in.
> >
> Bummer - Why? Isn't it AutoQA's job is to catch "must-not-happen" 
> package bugs, i.e. to be *mandatory*?

In addition to Tim's response explaining why we can't make it mandatory
*yet*, a few additional notes:

* Notification and enforcement are not the same thing. Some maintainers
still might not want email notifications generated directly by AutoQA to
be sent to them even if we made the tests mandatory. For instance, they
might have a workflow of checking their updates manually, or they might
already treat the emails generated by *Bodhi* - which include comments
left by AutoQA - as high-priority mails.

* AutoQA's aim is/was ('was' as it's now being replaced by TaskBot, but
TaskBot's aim is much the same) not just what you describe. AutoQA/TB
themselves aim to be generic frameworks for running automated tests
within Fedora processes; the 'special sauce' is to allow tests to be
hooked into the Fedora dev process - Koji, Bodhi and all the rest of it.
It is perfectly possible and expected that some of the tests that run on
AutoQA/TB will not be 'critical' tests that cause an update to be
blocked or a build withdrawn if they fail, but more 'informational'
tests. rpmguard is an example of one that we run mainly to provide
information to be interpreted manually, rather than with the expectation
that we could automatically 'approve' or 'reject' a build based on the
result.

Along with writing the framework, it's certainly a high-priority target
of the automated QA effort to implement some specific tests for the
purpose of update gating - depcheck and upgradepath being the ones we've
worked on so far - with the intent that we can make them sufficiently
foolproof to actually automatically block updates if they fail. But
that's not *all* the automated QA effort is about.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net