[Test-Announce] 2010-09 Graphics Test Week recap

Tue Oct 5 21:50:19 UTC 2010

Another Graphics Test Week has gone by, so here's the statistical recap!

A foreword - participation was substantially down this year, for which I
entirely blame myself; it kind of crept up on me and I didn't get enough
advance publicity out there. Most especially I didn't get the word out
to Phoronix, which is just dumb, because I think the majority of testers
come from there. So I'm sorry about that and I'll try and make sure it
doesn't happen for F15. If anyone else wants to take over or help out
with Graphics Test Week organization and publicity, btw, please do drop
me a line!

Here's an updated version of the numbers I came up with for F13:

f11 nouveau: 104 tests, 42 bugs - ratio 0.40
f12 nouveau: 53 tests, 34 bugs - ratio 0.64
f13 nouveau: 78 tests, 26 bugs - ratio 0.33
f14 nouveau: 39 tests, 8 bugs - ratio 0.21

f11 radeon: 55 tests, 46 bugs - ratio 0.84
f12 radeon: 61 tests, 81 bugs - ratio 1.33
f13 radeon: 48 tests, 33 bugs - ratio 0.69
f14 radeon: 32 tests, 18 bugs - ratio 0.56

f11 intel: 23 tests, 21 bugs - ratio 0.91
f12 intel: 29 tests, 31 bugs - ratio 1.07
f13 intel: 38 tests, 38 bugs - ratio 1.00
f14 intel: 33 tests, 28 bugs - ratio 0.84

Each driver posted its best ever ratio of bugs to tests, which is great
(though I imagine the Phoronix testers may be quite sophisticated and
demanding in their use cases, and this may impact the results). The
ranking stays the same, with nouveau at number 1, radeon at number 2 and
intel at number 3. The same caveats as last time of course apply -
different drivers implement different features and may have to deal with
more problematic hardware.

Here's the other chart I ran last time, tracking how we handle the bugs
that are reported - obviously, this time adding the numbers for F13 as
we can't see into the future for the F14 bugs...

f11 nouveau: 42 bugs, 4 open, 8 closeddupe, 24 closedfixed, 6 closedunfixed - 70.59%
f12 nouveau: 34 bugs, 11 open, 8 closeddupe, 14 closedfixed, 1 closedunfixed - 53.85%
f13 nouveau: 27 bugs, 17 open, 6 closeddupe, 3 closedfixed, 1 closedunfixed - 14.29%

f11 radeon: 46 bugs, 14 open, 10 closeddupe, 19 closedfixed, 3 closedunfixed - 52.78%
f12 radeon: 81 bugs, 19 open, 32 closeddupe, 28 closedfixed, 2 closedunfixed - 57.14%
f13 radeon: 36 bugs, 28 open, 3 closeddupe, 5 closedfixed, 0 closedunfixed - 15.15%

f11 intel: 21 bugs, 7 open, 1 closeddupe, 12 closedfixed, 1 closedunfixed - 60%
f12 intel: 31 bugs, 7 open, 12 closeddupe, 12 closedfixed, 0 closedunfixed - 63.16%
f13 intel: 42 bugs, 26 open, 4 closeddupe, 11 closedfixed, 1 closedunfixed - 28.95%

To refresh your memories, the percentage is 'closedfixed' / (bugs -
'closeddupe') * 100, the intention being to give an indication of what
proportion of bug reports wind up in a fix. Full details in my previous
recap -
http://lists.fedoraproject.org/pipermail/test/2010-April/090271.html .
(I know the bug counts for F13 do not match those in the previous table;
I took the numbers in the previous table from my last recap, but did
these numbers just now, and I suspect further results were added to the
F13 pages following my last recap, accounting for the discrepancy).

This one's obviously worrying and indicates our rate of closing reported
X bugs slowed markedly in F13 for all three drivers (though intel fared
noticeably better than the others). I definitely intend to look into
this. I don't think we could assume this is down to less development
effort; it could well be down to less X triage, I know I have not been
able to triage X very often for the last few months. I'll definitely
look into this and see if any follow-up action is necessary, however.

The raw lists of bugs reported from the F14 Test Days follow. Thanks
very much to all testers, and to the wonderful Fedora X.org developers
and triagers:

Adam Jackson
Dave Airlie
Jerome Glisse
Ben Skeggs
Matej Cepl
François Cami
Chris Campbell

Nouveau
-------

573096 NEW  - NV1f Chipset - Garbled Display. Needs nouveau.tv_disable=1 on boot to work
611059 NEW  - Screen artifacts in KDE with nouveau
638317 NEW  - Rendercheck is not passed completely
638329 NEW  - Resume freezes with nouveau, GeForce 6150
517917 ASSIGNED  - NV1f chipset. Screen moved one line down.
638174 CLOSED NOTABUG - F14 graphics test Xorg falls back to VESA instead of Nouveau
638103 CLOSED DUPLICATE - Second monitor not enabled on F14 live CD by default
638258 CLOSED DUPLICATE - garbled video on boot

Radeon
------

638646 NEW  - [abrt] mesa-demos-7.9-0.8.fc14: Process /usr/lib64/mesa/fbo_firecube was killed by signal 11 (SIGSEGV)
638807 NEW  - waiting for X server to shut down gnome-settings-daemon: Fatal IO error 11
638793 NEW  - suspend/resume corrupts a compiz  enabled desktop (ATI Radeon XPRESS 200M 5955)
638806 NEW  - Radeon XPRESS 200M 5955: Corrupted OpenGL images; problem goes away after running an XV program (i.e. Totem)
639660 NEW  - Missing resolutions on second monitor (ATI RV730XT [Radeon HD 4670])
638827 NEW  - Suspend/Resume of Fedora livecd fails.
628400 NEW  - Fedora 14 ATI Radeon HD 5750 graphics corruption when moving windows
630918 NEW  - Horizontal grey needles graphic glitch with kernels 2.6.34 and upwards on x86_64 with ATI HD3650
638680 NEW  - testday:mesa-demos:tunel2 hangs computer
638745 NEW  - Default resolution is wrong when connected via VGA
638754 NEW  - Radeon test day  - errors leaving X to return to console
638759 NEW  - Resume from suspend fails for radeon test day livecd
638779 NEW  - glxgears window is completely black
638799 NEW  - Testday: 2010-09-29 Testcase_radeon_xvideo totem-1:2.31.6-3.fc14 crash
638803 NEW  - Activating multiple monitors causes modeset loop
638823 NEW  - Radeon HD 5750 rotating the screen crashes X
550431 ASSIGNED  - KMS:R200:kernel command stream rejection
533567 ASSIGNED  - KMS:RS480:X200M - GPU lockup (screen goes black)

Intel
-----

582532 NEW  - DPMS - display wakes up after 15 seconds
582861 NEW  - Poor 3D performance unless CPU is under load
628145 NEW  - Displayport issues with Lenovo T510 when *just* using external display
638123 NEW  - Does not correctly detect resolution of external screen
638124 NEW  - Image on external scren unstable
638850 NEW  - Multi head tests: Possibly wrong mode for second head
638852 NEW  - Multihead, pointer goes off top/bottom on second head with lower resolution
638857 NEW  - Mirrors instead of spans on bootup
639016 NEW  - Multiple displays fails with GDM
639021 NEW  - Switching virtual terminals fails
639041 NEW  - [Intel Test Day] Delayed VT/user switch with compiz running
639057 NEW  - display switching fails with desktop effects enabled
639088 NEW  - Screen goes black between Plymouth and X
639104 NEW  - [Intel Test Day] gstreamer-properties fails
639140 NEW  - Fullscreen XV video doesn't work with metacity compositing
639144 NEW  - External display disabled on start
639146 NEW  - Blank display (backlight on) after KMS initialization
639150 NEW  - After resuming via lid-switch, multi-head displays remain turned off
639157 NEW  - System does not start with desktop spanning all displays
639195 NEW  - Doesn't resume from suspend, display remains off
639198 NEW  - Gnome panel disappears when compositing is enabled in Metacity
639229 NEW  - Closing latop lid changes dualhead configuration wildly
639243 NEW  - Plugging out and into docking station again doesn't restore previous dualhead mode
639165 NEW  - Strange result after swapping left/right display in multi-head configuration with Compiz
623824 NEW  - Won't display on VGA-connected monitor
512958 NEW  - Open/close laptop lid doesn't generate acpi signal on HP mini 2140
639173 NEW  - DMA errors cause failure of graphics test day livecd image boot
639007 CLOSED DUPLICATE - KMS no longer working with external monitor
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net

_______________________________________________
test-announce mailing list
test-announce at lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/test-announce