[Fedora QA] #154: Tracker: critical path test case creation

[Fedora QA]

#154: Tracker: critical path test case creation
--------------------------+-------------------------------------------------
  Reporter:  adamwill     |       Owner:  adamwill 
      Type:  enhancement  |      Status:  new      
  Priority:  major        |   Milestone:  Fedora 15
 Component:  Wiki         |     Version:           
Resolution:               |    Keywords:           
--------------------------+-------------------------------------------------
Comment (by jlaska):

 Awesome, I like how you've focused on writing the test case *once*, and
 distinguishing the content from how it's grouped/organized.  That
 distinction will help us with a possible future TCMS migration.

 We discussed on IRC, but following up here for a larger audience....
 I wasn't sure if the 3 groups you listed (critpath_test_cases,
 mdadm_test_cases, mdadm_critpath_test_cases) were all needed.  Certainly
 they help for convenience, but it sounds like the consumers of this
 hierarchy would only need to find
  1. all mdadm test cases (''Category:mdadm_test_cases'')
      ... OR ...
  2. All critical path mdadm test cases (intersection between
 ''Category:mdadm_test_cases'' and ''Category:Critpath_test_cases'').

 I didn't initially think the intersection would be challenging to do.  But
 thinking ahead to some remote API that consumers would use to find
 applicable tests, we may want to use the three categories as you suggest
 to avoid having each consumer defining logic for how it finds applicable
 tests.  So in your example, we would treat
 ''Category:Critpath_test_cases'' as a way to denote test priority (where
 high == critpath, media, low).  ''Category:mdadm_test_cases'' would be
 used to indicate membership to a test plan (the mdadm test plan).  And
 finally, ''Category:Critpath_mdadm_test_cases'' would be our way to
 provide all *high* priority tests in the mdadm test plan.  Does that make
 sense?

 I know I'm getting too far ahead, but I'm trying to articulate our wiki
 organization in TCMS terms, and to reduce the requests consumers would
 have down to a simple API.

 Using the 3 category suggestion, my ASCII-art attempt at a top-down view
 from ''Category:Test_cases'' would be ...
 {{{
 Category:Test_Cases
   |
   +-> Category:mdadm_test_cases
   |   |
   |   +-> Category:Critpath_mdadm_test_cases (e.g. high priority tests)
   |   |   |
   |   |   +-> QA:Testcase_mdadm_important_use_case_1
   |   |   +-> QA:Testcase_mdadm_important_use_case_2
   |   |   +-> QA:Testcase_mdadm_important_use_case_3
   |   |
   |   +-> QA:Testcase_mdadm_obscure_use_case_4
   |   +-> QA:Testcase_mdadm_obscure_use_case_5
   |   +-> QA:Testcase_mdadm_obscure_use_case_6
   |
   +-> Category:kernel_test_cases
   |   |
   |   +-> Category:Critpath_kernel_test_cases (e.g. high priority tests)
   |       |
   |       +-> ...
   .
   .
   .
 }}}

 And then the same view, but for just ''Category:Critpath_test_cases''.
 This is probably the easiest thing for consumers of this data to manage.

 {{{
 Category:Critpath_Test_Cases
   |
   +-> Category:Critpath_mdadm_test_cases (e.g. high priority tests)
   |   |
   |   +-> QA:Testcase_mdadm_important_use_case_1
   |   +-> QA:Testcase_mdadm_important_use_case_2
   |   +-> QA:Testcase_mdadm_important_use_case_3
   +-> Category:Critpath_kernel_test_cases (e.g. high priority tests)
   |   |
   |   +-> QA:Testcase_kernel_important_use_case_1
   |   +-> QA:Testcase_kernel_important_use_case_2
   |   +-> ...
   .
   .
   .
 }}}

 While it's an extra category for each src.rpm, this seems like a really
 sane approach.

-- 
Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/154#comment:12>
Fedora QA <http://fedorahosted.org/fedora-qa>
Fedora Quality Assurance