5:11 p.m.
The insanity that is Graphics Test Week is now over, so it's time for
the recap!
https://fedoraproject.org/wiki/Test_Day:2010-04-13_Nouveau
https://fedoraproject.org/wiki/Test_Day:2010-04-14_Radeon
https://fedoraproject.org/wiki/Test_Day:2010-04-15_Intel
We had a great turnout again; thanks to everyone for testing. Here's
some interesting numbers I just pulled out...
f11 nouveau: 104 tests, 42 bugs - ratio 0.40
f12 nouveau: 53 tests, 34 bugs - ratio 0.64
f13 nouveau: 78 tests, 26 bugs - ratio 0.33
f11 radeon: 55 tests, 46 bugs - ratio 0.84
f12 radeon: 61 tests, 81 bugs - ratio 1.33
f13 radeon: 48 tests, 33 bugs - ratio 0.69
f11 intel: 23 tests, 21 bugs - ratio 0.91
f12 intel: 29 tests, 31 bugs - ratio 1.07
f13 intel: 38 tests, 38 bugs - ratio 1.00
The 'ratio' is the number of bugs per test. Obviously there's wiggle
room here; different people report different bugs, and some of the
drivers implement features the others don't (and hence have more
'surface area' for bugs). But I think they're quite fun anyway. Nouveau
and Radeon both regressed from f11 to f12, according to the numbers, and
got better than either previous release for f13 (at Test Day time).
Intel has stayed fairly steady. According to this analysis, Nouveau wins
the 'least buggy driver' contest by a fair margin, which is interesting!
For F13, it has half as many bugs per test as Radeon, and a third as
many as Intel. (Of course, I'm sure some of our erstwhile devs would
argue it could fairly be renamed the 'least crack-addled hardware
manufacturer contest'...)
The F11 Test Days happened in March 2009, the F12 Test Days in September
2009, and the F13 last week; these are pretty comparable points in the
respective cycles.
In terms of participation, we had the largest number of testers for F11,
with the nouveau number accounting for most of that; I suspect this is
because nouveau was very new and shiny in F11 and impossible to use on
most distros, so people were very interested in trying it out for the
first time. F12 had the fewest tests run, and F13 pretty much splits the
difference.
Since I'm getting up a head of steam, let's look at fixes!
f11 nouveau: 42 bugs, 4 open, 8 closeddupe, 24 closedfixed, 6 closedunfixed - 70.59%
f12 nouveau: 34 bugs, 11 open, 8 closeddupe, 14 closedfixed, 1 closedunfixed - 53.85%
f11 radeon: 46 bugs, 14 open, 10 closeddupe, 19 closedfixed, 3 closedunfixed - 52.78%
f12 radeon: 81 bugs, 19 open, 32 closeddupe, 28 closedfixed, 2 closedunfixed - 57.14%
f11 intel: 21 bugs, 7 open, 1 closeddupe, 12 closedfixed, 1 closedunfixed - 60%
f12 intel: 31 bugs, 7 open, 12 closeddupe, 12 closedfixed, 0 closedunfixed - 63.16%
I counted CANTFIX, WONTFIX and INSUFFICIENT_DATA as 'unfixed', ERRATA,
RAWHIDE, CURRENTRELEASE and NEXTRELEASE as 'fixed'. NOTABUG I lumped in
with DUPLICATE as the 'closeddupe' number (these are reports that should
be discarded from consideration entirely). The percentage is calculated
as:
closedfixed / (bugs - closeddupe) * 100
i.e. it roughly indicates the percentage of genuine unique bugs reported
that have been fixed so far. These numbers are pretty close, both across
drivers and across releases; we've fixed just over half the bugs
reported. I think the outlying high nouveau result is probably a
consequence of 'low-hanging fruit' - the driver was in a pretty initial
state at that point, so the bugs exposed are likely to have been, on the
whole, easier to fix. Obviously I've left F13 out as the maintainers
have had only half a week to work on the bugs!
The raw lists of bugs reported from the F13 Test Days follow. Thanks
very much to all testers, and to the wonderful Fedora X.org developers
and triagers:
Adam Jackson
Dave Airlie
Jerome Glisse
Ben Skeggs
Matej Cepl
François Cami
Chris Campbell
for helping to organize the events, set up the test cases, man the IRC
channel and triage - and of course fix! - all the bugs.
Nouveau
-------
573096 NEW - F13 Beta - Garbled Display. NV18 chipset. Both DVI-D and VGA outputs
579897 NEW - Xorg uses 100% cpu with nouveau and GeForce 8400 GS
581769 NEW - nVidia Corporation GeForce 6150SE nForce 430 (rev a2) fails with compiz
581934 NEW - When booting the test day live image machine is very slow and various
services crash.
581945 NEW - Screen corruption occurs when turning on desktop effects
581956 NEW - Doesn't detect secondary display (connected via DVI cable) on GeForce
6150SE nForce 430
582070 NEW - Screen Flicker Playing Totem Video - Testcase nouveau xvideo
582168 NEW - testday case: XVideo FAILED, on 8400 GS
582271 NEW - Fails to boot and display only shows thin white lines and dots with a
blinking cursor
582272 NEW - Suspend fails on nVidia Corporation G84M [Quadro NVS 140M]
582379 NEW - compiz fails on GeForce 7050 PV
582489 NEW - Garbage when swithing from plymouth to gdm
582606 NEW - starting X server from runlevel 3 does not work - black screen
582617 NEW - [dualhead] when leaving fulscreen with changed resolution, only one screen
comes back
582664 NEW - Brightness controls do not work after resume
582866 NEW - Window side edges tear when moved horizontally
583365 NEW - Minor video corruption in totem when in fullscreen mode
583679 NEW - Rendercheck passes only 2057222 tests of 2124238 on nVidia 7600GS
580570 NEW - [abrt] crash in upower-0.9.2-1.fc13: Process /usr/libexec/upowerd was killed
by signal 11 (SIGSEGV)
517917 ASSIGNED - X11 moved one line down. console moved left one character. NV18
chipset
560444 ASSIGNED - Black screen (with backlight on) after suspend
577142 ON_QA - Glitches with GMA Mobile 945 and compiz enabled.
581887 CLOSED DUPLICATE - Graphical boot does not reach the login screen.
582064 CLOSED NOTABUG - Unable to enable desktop-effects
583540 CLOSED NOTABUG - secondary HDMI display not detected on GeForce GT220
583690 CLOSED UPSTREAM - Video problems in Totem when paused
Radeon
------
582166 NEW - rv730: shadowtex demo has corrupt shadow texture
582413 NEW - rv730: glxgears in fedora twice as slow as on Debian
582209 NEW - rv730: html5 video in firefox hogs CPU, makes Xorg unusable with KMS!
(firefox shouldn't use REPEAT_NONE)
540743 NEW - KMS:RV280:9250:AGP:MESA Black windows/no menus while rotating compiz cube
572480 NEW - KMS fails to set mode (radeon / RV670)
583065 NEW - If suspending two times in a row, the second suspension hangs the system
581692 NEW - Graphic corruption when first entering X
582151 NEW - rv730: rendercheck fails (1957932 tests passed of 2124238 total)
582158 NEW - rv730: transition from graphical boot to login screen has a short flicker
582161 NEW - rv730:extremetuxracer reflections artifacts
582207 NEW - KMS:R6XX html5 video in firefox hogs CPU, makes Xorg unusable
582264 NEW - blank screen for laptop w/ Radeon HD 3400 / [1002:95c4] test day
582400 NEW - glxgears drawing problems when desktop effects enabled
582793 NEW - KMS:RV610:HD2400 fails to resume
582952 NEW - content of partially covered window during scrolling is messy
583382 NEW - Minor Video distortion in gstreamer-properties test window
583521 NEW - gstreamer get black on the left when video paused
583845 NEW - Washed-out output (wrong gamma) on RV100 with KMS
583910 NEW - X server crash when rotating a screen with dual-head
584044 NEW - Most of desktop area garbled with compiz and dual-head on RV100
584095 NEW - Screen flickers a couple of times after entering X (and also when running
xrandr) on X1600Pro
584105 NEW - 3D very slow and even halts mouse cursor on X1600Pro
524057 ASSIGNED - KMS:ES1000 installing F13 Beta: black screen in graphical install mode
484692 ASSIGNED - Regression. Display flickers on M76 [Radeon Mobility HD 2600 Series]
560048 ASSIGNED - X crashes with segfault on startup w/ kernel 2.6.32 Radeon Mobility M6
LY
497055 ASSIGNED - KMS:RV250 svideo broken (distorted picture)
522176 ASSIGNED - KMS: set mode with null fb leads to video junk
531825 ASSIGNED - KMS:RV250|M9:9000:AGP Suspend/Resume fails (ThinkPad T41)
560463 ASSIGNED - KMS:RV250:M9 unable to use external monitor on laptop
571874 MODIFIED - KMS:RV630:HD2600 vga-out wavey unstable picture (regression)
583690 CLOSED UPSTREAM - Video problems in Totem when paused
582442 CLOSED DUPLICATE - during cold boot, after boot progress and before login, display
looks corrupted but mouse pointer OK
582881 CLOSED NEXTRELEASE - Missing 3D Support for Radeon HD 5670
Intel
-----
572771 NEW - Thaw doesn't work after hibernate with F-13 on Dell Inspiron 6400
513736 NEW - Compiz crash if dual monitor & compiz on i915 driver
583471 NEW - dual display, enabling compiz / desktop effects causes bars and stripes on
screen
581424 NEW - Problems with Compiz desktop effects on Dell D400
582415 NEW - XV: tearing on windows moves!
582419 NEW - intel: VT switching slow
582420 NEW - fast user switching: black screen
582447 NEW - Transition to GDM fails with nomodeset
582480 NEW - [abrt] crash in compiz-0.8.6-1.fc13: Process /usr/bin/compiz was killed by
signal 11 (SIGSEGV)
582525 NEW - Powered off display is detected as powered on
582532 NEW - DPMS - display wakes up after 15 seconds
582541 NEW - intel: tearing in openarena
582654 NEW - Tooltip shadows appear randomly.
582712 NEW - Failure of the basic KMS test on Fedora test day
582734 NEW - Failure of the multihead test
582741 NEW - Sony TV became confused after playing with monitor properties
582742 NEW - Mouse cursor becomes jumpy for a few seconds after changing resolutions
582746 NEW - Trouble when switching monitor resolutions while running glxgears under
compiz
582751 NEW - Freeze due to fast user switching
582759 NEW - Just crash with login to GNOME, intel 855GM
582861 NEW - Poor 3D performance unless CPU is under load
583709 NEW - Compiz crashes on i915 with dual-head
583764 NEW - Compiz crashes on i865G
572799 NEW - Install never appears on Lenovo x201
582699 NEW - xset dpms force off doesn't stick.
546721 NEW - gdm/X doesn't start
537494 ASSIGNED - With compiz and KMS enabled, hibernate/thaw doesn't work on Dell
Inspiron 6400 (Intel Corporation Mobile 945GM/GMS/GME, 943/940GML Express Integrated
Graphics Controller)
501435 ASSIGNED - Xvideo failure on i855GM
497069 ASSIGNED - XVideo fails if multihead monitor size exceeds 2k in any direction
582843 MODIFIED - kernel oops on suspend/resume
560628 MODIFIED - KMS: Disturbed display with 2.6.32.x / 2.6.33.x
570517 ON_QA - intel crash with KMS on resume/startup ... "Failed to submit batch
buffer, expect rendering corruption or even a frozen display: Input/output error."
577142 ON_QA - Glitches with GMA Mobile 945 and compiz enabled.
572772 CLOSED DUPLICATE - Intel graphics driver leaves artefacts and doesn't refresh
with F-13 on Dell Inspiron 6400
582375 CLOSED DUPLICATE - Graphic artifacts when desktop effects are enabled
582821 CLOSED ERRATA - VT switching makes X disappear on a 915
582856 CLOSED DUPLICATE - Under Compiz windows don't update consistently
583690 CLOSED UPSTREAM - Video problems in Totem when paused
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net