[Fedora QA] #268: asking to join the proven testers and requesting a mentor

[Michael Schwendt]

Let me disclose a bit of stuff that could help understanding my point
of view.


* It is true that some other testers also post brief comments such as "no
regressions noted during casual use". I also do that occasionally, but I
also post neutral feedback as recommended in the guidelines.

Mentioning "casual use" could be an important detail, and a difference
e.g. from someone else who runs Squid in a corporate intranet.


* If there are too many terse "works for me" +1 comments on packages,
accountability becomes a problem. If major bugs are found after
approval of the package, it becomes all to easy for a "works for me"
tester to claim that a bug could not and cannot be reproduced or
that the crucial feature has not been tested. Therefore, terse +1
votes should not be the norm, but the exception for testers who have
shown familiarity with the same package/software before. More verbose
comments to build up trust, and subsequent terse comments depending on
how confident a tester is when signing off a package.


* Testing requirements aren't very specific yet. For major apps like Claws
Mail, for example, it would not be feasible to test every feature. Still,
regularly I try to explain a +1 with a few words and, for example, mention
that I've used Claws Mail with IMAP or specific plugins that have been
fixed. Sometimes it also helps to point out that none of the bug-fixes
have been verified, or that none of them had been reproducible before.


* I think it isn't asked too much to add a few words on what kind of
testing the tester has performed. Example "Audacity": Has arifiauo only
installed and started the new upstream release? Or has he sampled and
edited a one hour long track, too? Has he used it often or sporadically
only?


* Or "pptp", which crashed regularly, albeit only once a week, with the
bodhi ticket giving details:

https://admin.fedoraproject.org/updates/FEDORA-2012-0135
|
| bodhi - 2012-01-05 21:04:20
|   This update has been pushed to testing
| 
| arifiauo - 2012-01-06 06:05:51
|   Works for me 

Uh?


* Another example, the "Perl" base package. Watch the difference:

https://admin.fedoraproject.org/updates/FEDORA-2011-17271
|
| watzkej (proventesters) - 2012-01-04 02:23:13
|   767931 seems to be fixed per running the reproducer script.
|   I haven't noticed any regressions in my personal use of Perl.
|
| arifiauo - 2012-01-05 20:43:04
|   Works for me 

What usage patterns has Perl seen here?


* "gsm" is mostly a library in ordinary installs and deserves an
explanation of how it has been tested:

https://admin.fedoraproject.org/updates/FEDORA-2011-16629
|
| mschwendt (proventesters) - 2011-12-13 11:49:48
|   Tested .au to .gsm and .gsm to .au conversion plus playback.
|
| bodhi - 2011-12-16 22:06:25
|  This update has reached 14 days in testing and can be pushed
|  to stable now if the maintainer wishes
|
| arifiauo - 2012-01-05 20:47:26
|  Works for me 

Has he performed the same tests or different ones?
Is the "Works for me" supposed to say "Package installed, machine still
rebooted"? Is that the only goal of the entire testing procedure?


There are many more examples within yesterday's flood of +1 votes on a
multitude of different software types, not limited to stuff like tor or
nfoview. That makes me nervous.

https://admin.fedoraproject.org/updates/FEDORA-2012-0272
|
| bodhi - 2012-01-10 18:29:09
|  This update is currently being pushed to the Fedora 16 testing updates repository.
|
| arifiauo (proventesters) - 2012-01-11 00:22:59
|  Works for me
| 
| bodhi - 2012-01-11 05:59:57
|  This update has been pushed to testing 

:-/