AutoQA Testcase Stats

[Tim Flink]

On Fri, 19 Jul 2013 08:56:48 -0600
Tim Flink <tflink at redhat.com> wrote:

> Note that these numbers reflect execution in staging autotest since I
> rebuilt the clients (including the queued jobs that weren't run while
> rebuilding), not pass/fail results and not from production. I was only
> interested in numbers about execution of the tests (GOOD - executed
> cleanly, reported results. ERROR - problems in execution, probably
> didn't report results)

I've switched the clients in stg to all fc19 clients to highlight the
problems with more data. I recalculated the stats from roughly the last
24 hours

http://tflink.fedorapeople.org/autoqa/jobstats/20130724_jobstats-stg_summary.txt

The two that are still jumping out at me are:

depcheck
   163 runs ( 45 GOOD, 116 ERROR )
    -  43 fc18 runs ( 43 GOOD, 0 ERROR )
    -  118 fc19 runs ( 2 GOOD, 116 ERROR )

I think that almost all of the fc19 failures are due to #442 which we
should probably at least look at to see if it's going to be feasibly
fixable. If all else fails, we can either get rid of the fc19 clients
or keep depcheck from being scheduled on fc19.


conflicts
    154 runs ( 34 GOOD, 116 ERROR )
    -  30 fc18 runs ( 0 GOOD, 30 ERROR )
    -  120 fc19 runs ( 34 GOOD, 86 ERROR )

I'm not really sure what's going on with conflicts but it is crashing a
lot for both fc18 and fc18. I've filed #441 to track the issue here.

I've not looked into either of these bugs yet, so please take them if
you're interested - just make a comment in the bug to say that you're
working on it so that we don't have multiple people working on the same
thing :)

Tim
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: not available
URL: <http://lists.fedoraproject.org/pipermail/qa-devel/attachments/20130725/a86417f9/attachment.sig>