[Fedora QA] #175: Improve transfer of previous test results in the installation matrix

[Fedora QA]

#175: Improve transfer of previous test results in the installation matrix
-------------------------+--------------------------------------------------
 Reporter:  kparal       |       Owner:  rhe      
     Type:  enhancement  |      Status:  new      
 Priority:  major        |   Milestone:  Fedora 15
Component:  Wiki         |     Version:           
 Keywords:               |  
-------------------------+--------------------------------------------------
 Currently in our installation matrix there are some test results which are
 from "anonymous" source (no name filled in). Example:

 https://fedoraproject.org/wiki/Test_Results:Fedora_15_Alpha_RC2_Install

 I was little confused by that, but rhe explained me that those are the
 test results that were transfered from the previous test result (i.e. from
 F15 Alpha RC1 in this case). We already agreed if would be better if we
 added <ref> with explanation to each such result, so other testers
 understand the meaning of it (I suppose many of them might be confused
 same as I was). Still, I think there are some improvements that could be
 made. I'd like to discuss them in this ticket.

 My proposal:
  1. Transferring old results to the new matrices is a great idea, I
 strongly support that.
  2. In order to avoid confusion about "anonymous" results, we should
 clearly separate new results from transfered results. Using a <ref>
 comment is possible, but I'd like to do something more visible. We could
 do something like {{result|fail|rc1|12345}} and then link rc1 to correct
 wiki page (if {{result}} template is not suitable for it, we could create
 {{oldresult}} template or similar). But even though it could be scripted,
 it seems like too much work. Easier solution is to do
 {{result|fail|previous run|12345}}, and if "previous run" is linked to
 [[User:previous run]], we could use that wiki page for short explanation.
  3. We should transfer only fails and warnings, not passes. The rationale
 is that the pass result makes you (me, many people) think: "Hey, there's a
 pass, let's work on something else". But pass from previous run doesn't
 mean pass in current run (as I experienced today, when I changed two
 previous passes to fails). Therefore warnings and fails are good to know
 (we can check whether they are fixed or not, and we won't forget about
 them), but passes are counter-productive - let's leave empty fields
 instead.
  4. All transfered results must have bugzilla number assigned. Otherwise
 there's no information value. If I don't know what was broken, I can't
 check whether it's been fixed or not.
  5. We should document all of this in the "Test Results Format" table and
 also advise people how to interact with these transfered results. My idea:
 If there was a transfered warn/fail, and you checked that the problem is
 fixed, remove it from the table (and put your result in there instead). If
 you checked that the problem is still present, again remove it from the
 table and put your result in there instead (referencing the same bug
 number). If you didn't check the problem, put your result in the table and
 leave the transfered result intact.

 What do you think?

-- 
Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/175>
Fedora QA <http://fedorahosted.org/fedora-qa>
Fedora Quality Assurance