gathering useful statistics for QA team
"Jóhann B. Guðmundsson"
johannbg at gmail.com
Tue Jul 20 16:52:57 UTC 2010
On 07/20/2010 01:13 PM, Kamil Paral wrote:
> we would like to help QA team to increase public participation in its
> activities. We believe that an important step in achieving that is in
> rewarding the most active participants with "fame" in different top
> tens, ladders and charts. Therefore we would like to extend the current
> Fedora Community website  with different statistics regarding user
> participation in different areas relevant to your team. We have called
> this project "Fedora Hall of Fame" . This information can then be used
> in different newsletters (FWN, etc) to praise the best people and motivate
> the others.
Ah the carrot game here a couple of things you need to know when playing
Wont this lead to less quality of testing/triaging as in
reporters/triagers will end up competing amongst themselves to reach the
"carrot" the "hall of fame", resulting in every reporter starts either
filing bug for every error msg they come across and provide +karma for
every package that goes through bodhi with similar behaviour in bugzilla
How do you think new reporters/triagers that see those stats are going
How far back into time are you thinking about gathering those stats?
How are you going to compare Red Hat and other company's employs that
literally are paid to work at this vs those that are freely given what
ever time they have to the project?
Do you think they will ever manage to be equally and or more productive
than those that get full time pay to do this?
Do you believe that newcomers and those that give freely what ever
limited time they have will look the hall of fame and think by
themselves hey I think I can get my name up there?
Seventhly and perhaps one of the more devastating downside when playing
the carrot game..
How do you suggest that the community handles those that have spent
countless hours trying to get their name in "Hall of Fame" when the cold
hard reality strikes them and they realize they cant and we risk loosing
them and their valuable contribution from the project forever?
> The information we hope to receive from you is:
> 1. What are the most important tools you would like to have tracked?
> For example it can be your wiki, mailing lists, Bugzilla, Koji, Bodhi,
> Transifex, packages' source code, and so on.
> 2. What are the most important characteristics you would like to see
> gathered? For example: # of wiki edits, # of new bug reports, # of
> package updates released, etc. Some of our ideas are at .
> In short, we're looking for hints which statistical data would help
> your team most in evaluating your best contributors.
Here's an Idea..
How about starting from the right end and gather those information about
components then about the maintainers then about QA and the rest.
We need to know components stats first and foremost so we can
effectively focus the project resource where they are mostly needed.
So I can get on the same page what's the end game here as in what do you
think can be achieved by this?
Was the solution of "playing the carrot game" the only solution on the
table that could achieve that goal?
More information about the test