Fwd: Re: Helping to improve advertising of test days and other things

[Adam Williamson]

On 2012-08-23 2:28, Samuel Greenfeld wrote:
> I think there is a more fundamental issue.
>
> From casual observation, it seems that more people like announcing
> they are a new Bugzapper than being a new general QA volunteer for
> Fedora.
>
> Besides the cool name, what causes this?

Your thoughts are interesting, but I think the answer is in fact 
simpler than that: the instructions for becoming a proven tester or a 
Bugzapper ask you to send a self-introduction mail, but there are no 
instructions for becoming a 'general QA volunteer' and no guide that 
tells you to send a self-introduction mail. Hence we see 
self-introduction mails for proventesters and Bugzappers but not for 
'QA'. It's really that straightforward :)

Since proventesters is officially dormant and Bugzappers 
is...unofficially dormant at present, I and the others who handle those 
two 'applications processes' have been trying to point volunteers to the 
other QA tasks when we respond to them. I think this is working out 
okay, though it sure would be nice if someone wanted to step up and try 
to bring Bugzappers out of hibernation once again.

> While I am not a Bugzapper, it would seem to me that QA runs on 
> strict
> cycles.  A new build comes out, and everyone is asked to test it in a
> timely manner.  A test day for X is held on day Y and if you are not
> available on Y (give or take a day) you may feel that no one will 
> look
> at your results.

That's a reasonable description of the updates-testing process and the 
test day process, though I think it's a fairly unavoidable aspect of 
each process. Update testing and release validation do *need* to be done 
in a timely manner; neither devs nor users want to wait three months to 
ship an update, and our release cycle is extremely tight and flat 
requires us to do validation testing extremely quickly. As for test 
days, one the main points of the 'Test Day' concept is the real-time 
nature of it: the benefits of getting devs and testers together at one 
time in one 'place' (IRC channel) to work through issues quickly. There 
have been some detailed proposals for Test Day-ish 'events' over a 
longer time period, and that might certainly be something someone might 
be interested in doing, but it has clear trade-offs compared to the 
single-day, real-time concept.

Aside from that, we do have other processes that come under the 'QA' 
banner and which aren't really so time-critical: see 
https://fedoraproject.org/wiki/QA/Join . 'Reporting bugs in Fedora 
releases', 'Testing Fedora pre-releases', 'Testing Rawhide', 'Creating 
test cases' and 'Developing tools' would all come under that header, I'd 
say.

> Meanwhile a Bugzapper sounds like they do not have to install 
> anything
> if they do not want to verify a fix.  They can go through Bugzilla at
> their leisure.  Their changes appear in realtime and they can
> immediately feel they made a difference.
>
> But given I primarily work on a downstream Fedora Remix, there is 
> only
> so much time I have to do upstream QA.  It takes time to research
> each update fedora-easy-karma says needs feedback so I can determine
> if I can personally test the fix(es).  And there is no guarantee that
> what I do makes a difference; many updates get pushed because someone
> says "works for me" or the maintainer pushes on a timeout.

Bodhi karma as currently implemented really kind of sucks, and we have 
big plans to improve it with Bodhi 2.0, which we've discussed several 
times already so I won't repeat them - but they certainly address a lot 
of the issues above.

> Most times when I file ABRT reports it seems like I hit duplicates.

This is still useful, actually - it helps to know if multiple people 
are hitting a crash, and sometimes later reports have info that was 
missing from the original report. So just because it's a dupe doesn't 
make it useless.

-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net