[Fedora QA] #35: Sync up install test priorities with the release criteria

[Fedora QA]

#35: Sync up install test priorities with the release criteria
--------------------------+-------------------------------------------------
  Reporter:  jlaska       |       Owner:  jlaska   
      Type:  task         |      Status:  assigned 
  Priority:  major        |   Milestone:  Fedora 13
 Component:  Test Review  |     Version:           
Resolution:               |    Keywords:           
--------------------------+-------------------------------------------------
Comment (by rhe):

 Replying to [comment:9 jlaska]:
 > <skip>
 >
 > You make a good point.  I keep going back and forth on ... if the use
 cases aren't release critical, should we continue testing them?  Anyone
 have a suggested phrasing for tests we want to run, but they aren't
 release critical?
 >
 Wow, I keep thinking this for a looong time but still can't find a good
 solution. I think it depends on the goal for installation tests. We test
 it whether to meet the release criteria or to fully check the
 installation. In my view so far we design it for a full check, or we won't
 create the repo cases. Then I think we can have extra label(s) for our
 own. But it's really tough to think out one. ：）

 > > Yes, PUD should be Alpha, I think.
 >
 > Okay, thanks.  Updated wiki.
 >
 > > traceback debug mode is Alpha, isn't it? From Alpha requirements: "The
 installer must be able to report failures to Bugzilla, with appropriate
 information included"
 >
 > Maybe.  As I read it, the criteria captures the test
 [https://fedoraproject.org/wiki/QA:Testcase_Anaconda_save_traceback_to_bugzilla
 QA:Testcase_Anaconda_save_traceback_to_bugzilla].  I feel that might need
 to be expanded or a new criteria is added.
 >
 > How do folks feel about adding a criteria to the
 [https://fedoraproject.org/wiki/Fedora_13_Beta_Release_Criteria
 Fedora_13_Beta_Release_Criteria] ... something like ... ''The installer
 must be able to copy failures logs to a remote system and offer a debug
 shell for investigating failures, and support copying failure information
 to a remote system.''
 >
 Agree.

 > > updates.img working is probably Beta or Final, I'd think.
 >
 > Agreed.  We currently have
 [https://fedoraproject.org/wiki/QA:Testcase_Anaconda_updates.img_via_URL
 QA:Testcase_Anaconda_updates.img_via_URL] listed as an Alpha priority
 case.  I think this is correct since the use case offers the only
 mechanism to work around issues once the installer is released as Alpha.
 Unfortunately, I can't see a Alpha (or Beta) release criteria to link this
 back to.  Any suggested updates to the criteria to accommodate this?
 >
 > With regards to
 [https://fedoraproject.org/wiki/QA/TestCases/PackageSetsMinimalPackageInstall
 QA/TestCases/PackageSetsMinimalPackageInstall].  Any concerns around
 obsoleting this test?

 Sometimes when sth wrong happens during default package installation,
 minimal install is another choice. So I think we can just keep it.

-- 
Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/35#comment:11>
Fedora QA <http://fedorahosted.org/fedora-qa>
Fedora Quality Assurance