[Fedora QA] #151: Tests consistent with Criterion

[Fedora QA]

#151: Tests consistent with Criterion
--------------------------+-------------------------------------------------
  Reporter:  rhe          |       Owner:  rhe          
      Type:  enhancement  |      Status:  new          
  Priority:  major        |   Milestone:  Fedora 16    
 Component:  Wiki         |     Version:               
Resolution:               |    Keywords:  retrospective
--------------------------+-------------------------------------------------
Comment (by adamwill):

 I have done a survey of the alpha criteria, checking for associated tests.
 Here are the results. 'partial' indicates that relevant tests are present
 but do not entirely cover the criterion's requirements. 'not-in-matrix'
 indicates a test case is written but not included in the current test
 matrices. 'NO TEST' indicates that, as far as I could find, there is no
 test case written for the criterion. Following these keywords, I have
 listed the relevant test cases for each criterion (where applicable).

 We can open subsidiary tickets to this bug for each specific issue
 identified here.

 There must be no file conflicts (cases where the files in some packages
 conflict but the packages have explicit Conflicts: tags are acceptable) or
 unresolved package dependencies during a media-based (CD/DVD) install

 https://fedoraproject.org/wiki/QA:Testcase_Mediakit_FileConflicts

 All dedicated installer images must boot to the graphical boot menu and
 allow the user to select install options. If no option is selected, the
 installer should load after a reasonable timeout

         partial
 https://fedoraproject.org/wiki/QA/TestCases/BootMethodsBootIso
 https://fedoraproject.org/wiki/QA/TestCases/BootMethodsDvd
 https://fedoraproject.org/wiki/QA:Testcase_Boot_Methods_efidisk.img

 The installer must boot (if appropriate) and run on all primary
 architectures from default live image, DVD, and boot.iso install media

         not-in-matrix
 https://fedoraproject.org/wiki/QA:Testcase_Live_Image_Boot
 https://fedoraproject.org/wiki/QA/TestCases/BootMethodsBootIso
 https://fedoraproject.org/wiki/QA/TestCases/BootMethodsDvd

 The graphical boot menu for all installation images should include an
 entry which causes both installation and the installed system to use a
 generic, highly compatible video driver (such as 'vesa'). This mechanism
 should work correctly, launching the installer and attempting to use the
 generic driver

         NO TEST

 The installer must be able to use at least one of the HTTP or FTP remote
 package source options

 https://fedoraproject.org/wiki/QA:Testcase_Http_Repository (redundant with
 ftp_repository?)
 https://fedoraproject.org/wiki/QA:Testcase_Additional_Http_Repository
 https://fedoraproject.org/wiki/QA:Testcase_Additional_Ftp_Repository
 https://fedoraproject.org/wiki/QA:Testcase_Ftp_Repository

 The installer must be able to use the DVD local package source options

 https://fedoraproject.org/wiki/QA/TestCases/InstallSourceDvd

 The installer must be able to complete an installation using the text,
 graphical and VNC installation interfaces

 https://fedoraproject.org/wiki/QA/TestCases/InstallSourceDvd
 https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_Text
 https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_VNC

 The installer must be able to complete package installation with the
 default package set for each supported installation method

 https://fedoraproject.org/wiki/QA/TestCases/InstallSourceDvd
 https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_Text
 https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_VNC

 The installer must be able to complete an installation using IDE, SATA and
 SCSI storage devices, with the default file system and LVM

 https://fedoraproject.org/wiki/QA/TestCases/PartitioningExt4OnNativeDevice
 https://fedoraproject.org/wiki/QA:Testcase_install_to_SATA_device
 https://fedoraproject.org/wiki/QA:Testcase_install_to_SCSI_device
 https://fedoraproject.org/wiki/QA:Testcase_install_to_PATA_device

 The installer must be able to complete an installation using the entire
 disk, existing free space, or existing Linux partitions methods, with or
 without encryption enabled

         partial
 https://fedoraproject.org/wiki/QA:Testcase_Anaconda_autopart_install
 https://fedoraproject.org/wiki/QA:Testcase_Anaconda_autopart_%28use_free_space%29_install
 https://fedoraproject.org/wiki/QA:Testcase_Anaconda_autopart_%28encrypted%29_install
         (existing partitions method does not seem to be tested)

 The rescue mode of the installer must start successfully and be able to
 detect and mount an existing default installation

 https://fedoraproject.org/wiki/QA:Testcase_Anaconda_rescue_mode

 The installer must be able to report failures to Bugzilla, with
 appropriate information included

 https://fedoraproject.org/wiki/QA:Testcase_Anaconda_save_traceback_to_bugzilla

 In most cases (see Blocker_Bug_FAQ), a system installed according to any
 of the above criteria (or the appropriate Beta or Final criteria, when
 applying this criterion to those releases) must boot to the 'firstboot'
 utility on the first boot after installation, without unintended user
 intervention. This includes correctly accessing any encrypted partitions
 when the correct passphrase is supplied. The firstboot utility must be
 able to create a working user account

         NO TEST

 Following on from the previous criterion, after firstboot is completed and
 on subsequent boots, a system installed according to any of the above
 criteria (or the appropriate Beta or Final criteria, when applying this
 criterion to those releases) must boot to a working graphical environment
 without unintended user intervention. This includes correctly accessing
 any encrypted partitions when the correct passphrase is supplied

         NO TEST (partly implied by more sophisticated desktop tests)

 When booting a system installed without a graphical environment, or when
 using a correct configuration setting to cause an installed system to boot
 in non-graphical mode, the system should boot to a state where it is
 possible to log in through at least one of the default virtual consoles

         partial
 https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_Text

 It must be possible to run the default web browser and a terminal
 application from all release-blocking desktop environments. The web
 browser must be able to download files, load extensions, and log into FAS

 https://fedoraproject.org/wiki/QA:Testcase_desktop_browser

 The installed system must be able to download and install updates with yum
 and the default graphical package manager in all release-blocking desktops

 https://fedoraproject.org/wiki/QA:Testcase_desktop_updates

 The default Fedora artwork must either refer to the current Fedora release
 under development (Fedora 16), or reference an interim release milestone
 (e.g. Alpha or Beta). If a release version number is used, it must match
 the current Fedora release under development. This includes artwork used
 in the installer, firstboot, graphical boot, graphical login and desktop
 background.

         NO TEST

 A system logging infrastructure must be available and enabled by default.
 It must provide at least basic local file-based logging of kernel
 messages, and allow other components to write log messages. This must be
 done in accordance with relevant standards accepted by the Project

         NO TEST

-- 
Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/151#comment:3>
Fedora QA <http://fedorahosted.org/fedora-qa>
Fedora Quality Assurance