Avoiding conditioning ignorance towards AutoQA

[Tim Niemueller]

On 27.04.2011 15:38, Kamil Paral wrote:
>> 
>> - Post only errors It is common, for example, in automated build or
>> continuous integration systems to send out emails only on errors.
>> Similar goes for Unix tools, which tend to be quiet if everything
>> is ok, and only bother you with output if something is not.
>> Therefore, I propose to have AutoQA messages posted only in case
>> that there has been an error.
> 
> How can you then distinguish an update for which the tests have
> passed from an update for which the tests haven't yet been executed?
> 
> Moreover, currently not all updates are tested. Sometimes our tests
> simply don't work properly. Not just the updates are being tested,
> the whole AutoQA is being tested (and developed) in this whole
> effort.

Please don't force testing, fixing, and maintaining AutoQA on the rest
of us. Integrate it such that stuff is pushed to testing only after
AutoQA has been run, or have a flag display "tests ran". Or post the
"PASSED" messages, but make Bodhi not sent messages in the case the
tests passed.

Keeping the current way will just make me (and possibly others) add
filters to throw away messages from AutoQA. Please be aware of how much
contributor time you waste by making them hope through useless (because
the tests have passed and no information is gained) mails. I realize you
want to improve things, but at the current stage its consuming the most
valuable resource we have, packager/developer time.


>> - Accumulate error messages An email is sent for every single
> 
> We have that in plan, believe me.
> 
>> Combined with the earlier proposal, the states for all platforms
>> should be collected by an intermediate node, and if and only if a
>> test failed on any of the platforms, one message with all status
>> messages is posted to the update.
> 
> Sending Bodhi comments is just a quick way how to inform the
> maintainers. We are working on a results database with API that other
> Fedora services (Koji, Bodhi) could query and use the results as they
> seem fit. For most tests I expect it will be similar to what you
> describe. But that's future. Until that's implemented we can only
> either send comments to Bodhi or send no comments at all.

I understand you like to have a quick and working solution for now, and
that great stuff is coming. But you cost a lot of time right now. Please
reconsider to make your development and testing time less intrusive for
others.


>> 
>> On a related note: it'd be much appreciated if Bodhi would provide
>> an option to get a daily digest with all comments of all the
>> packages I'm involved with.
> 
> Great idea, you can ask lmacken about that (or create ticket in its
> Trac). Or, you can filter your emails and check the relevant folder
> once a day :)

That still means striving through many messages with lots of non-info
text. One concise email would make things much better.

>> 
>> I hope the fine folks of the AutoQA effort take these proposals
>> into account when proceeding in the development of the system and
>> help me to stop ignorance from taking over.
> 
> It will take some time, but we see the deficiencies, same as you do. 
> We try to improve as fast as possible.

Please try to find a way to do this without costing as much time as atm.
There must be ways, for example have a list of packages to use it for
that packagers can opt-in and make AutoQA developers the first to use it.

> PS: We have a special mailing list for AutoQA: 
> https://fedorahosted.org/mailman/listinfo/autoqa-devel

Sorry, to keep me involved in Fedora I have to make it a reasonable
effort, joining yet another project is out of my possibilities atm.

	Tim

-- 
    Tim Niemueller <tim at niemueller.de>      www.niemueller.de
=================================================================
 Imagination is more important than knowledge. (Albert Einstein)