Metrics: RFC

[Christopher Blizzard]

Bill Nottingham wrote:
> Christopher Blizzard (blizzard at redhat.com) said: 
>> Keeping in mind what we want to measure, I would guess that we have a 
>> few things here:
>>
>> 1. Something that's run every time you change the kernel that can ask 
>> users to run through a set of simple tests?  Annoying, but maybe we can 
>> only run it for 1/20 people and since we're just interested in 
>> statistical information, that's enough.  Think high level (external 
>> monitor support, sound, suspend/resume, etc.)
> 
> How often is this needed? I would think that when hardware
> *regresses*, we tend to hear about it fairly quickly.
> 

If I could point at something that said "between kernel version X and 
version Y we broke sound on all T42 laptops" that would get people 
hopping.  And doing so in a way that doesn't require us to filter 
through a few hundred bug reports to be able to figure out that it's T42 
users as opposed to X31 users.  I'm talking about a way that makes our 
jobs faster and easier and lets us prove the impact (negative or 
otherwise!) on our user community.

So, yeah, we have these mechanisms today.  But they are manual and we're 
relatively slow to respond to them.  (Not that we don't work hard - I'm 
just saying we could be _much_ better about getting information into the 
right people's hands.)

--Chris