On Mon, 2014-08-11 at 16:53 -0400, Jon Stanley wrote:
So I have to admit that I saw that Fedora badges were now being
awarded for kernel testing, so I went ahead and submitted my results.
In the process, I made a few observations (both on the process and on
the tests themselves):
First from the process standpoint, has it been thought about to make
the process easier for testers than going to a webpage and uploading
results manually? Could we make it even easier by somehow asking for a
FAS identity and then uploading it?
Yes, the code was just put into python-fedora last week to allow us to
do CLI authentication, and now I am rewriting submit to make that work.
It should be available at some point this week.
As for the test content itself, the test header has no information
as
to what hardware it is running on, which for performance tests
matters, and possibly for some functionality tests as well (vendors
with buggy firmware, etc). I'm not sure what format patches are
preferred in, but there's a branch hw-info of
git://githuib.com/jds2001/kernel-tests that has this. A log header now
looks like:
Thanks for the patch, I will take a look at it. To be honest when we
first thought of it, we were worried about the balance of privacy. If we
collect too much information, we might end up with fewer testers. I will
mull it over a bit, but I don't think the information here would be too
much. As for performance, user submitted performance data will be of
less use anyway, as performance really has to be done in a controlled
environment. Any time userspace changes, we have to rebaseline on a
known kernel, etc.
Another test that would be worthwhile that I don't think I'm
smart
enough to write on my own is whether AutoNUMA is working as intended.
This would of course require NUMA hardware (or booting a system with
fake NUMA).
We can easily check for NUMA and simply skip if the test is run without
it. I will add it to my list of tests to write.
Thanks,
Justin