Draft Test Cases for Fedora Server Final Criteria

[Adam Williamson]

On Tue, 2014-11-04 at 16:58 -0500, Stephen Gallagher wrote:
> So, at today's Server WG meeting, I was asked to come up with a
> high-level draft of some additional requirements to validate for Final
> criteria. Most of these probably should be Beta criteria in F22, but we
> didn't have them as such this time around. I'll differentiate them as
> such below. It should be understood that Final validation is a superset
> of Beta validation, so anything we add in that category applies to both.
> 
> == Beta Criteria ==
> 
> === Domain Controller ===
> * The Domain Controller must be capable of serving LDAP requests on port
> 389. This should be validated by the use of the ldapsearch tool.
> 
> * The Domain Controller must be capable of serving TLS-encrypted LDAP
> requests on port 389. This should be validated by the use of the
> ldapsearch tool.
> 
> * The Domain Controller must be capable of serving LDAPS (LDAP encrypted
> with SSL) over port 636. This should be validated by the use of the
> ldapsearch tool.
> 
> * The Domain Controller must be capable of returning LDAP and LDAPS
> search results using simple auth (the -x option to ldapsearch) or
> SASL/GSSAPI auth (the -Y GSSAPI option). This should be validated by the
> use of the ldapsearch tool.
> 
> * The Domain Controller must be capable of serving DNS host records on
> port 53. This should be validated by the use of the 'dig' tool.
> 
> === FreeIPA Domain Client ===
> * Enrolled clients must be capable of authenticating against a valid
> user account using SSSD.
> * Enrolled clients must honor FreeIPA HBAC rules for access-control.
> * Enrolled clients must be able to change their passwords according to
> the password policy specified by the FreeIPA server
> * Users must be capable of performing password-less single-sign-on
> between two enrolled clients using GSSAPI.
> 
> 
> 
> == Final Criteria ==
> 
> === Domain Controller ===
> * The Domain Controller must be capable of serving DNS SRV records for
> ldap and kerberos on port 53. This is used for auto-discovery.
> 
> === FreeIPA Domain Client ===
> * When configured to use the Domain Controller for DNS services, the
> domain client must be able to use DNS to discover the Domain Controller
> address using SRV records.
> 
> * When configured to use FreeIPA for host-key validation, initial SSH
> between domain clients should not prompt the user to accept the SSH
> public key.

I have added these criteria to the Fedora 21 pages now.

I revised the Beta page slightly:

https://fedoraproject.org/w/index.php?title=Fedora_21_Beta_Release_Criteria&diff=395310&oldid=392111

to move the 'clients must be able to authenticate' criterion out of the
domain controller role requirements section (as it really is a
requirement for clients, and thus applies more widely) and into a new
section I created for domain client requirements, and explain the
circumstances in which those apply.

On the Final page, I added all the new domain controller role
requirements, following the format we established at Beta for role
requirements:

https://fedoraproject.org/w/index.php?title=Fedora_21_Final_Release_Criteria&diff=395309&oldid=390615

and then created the same new 'domain client requirements' section as I
created for Beta, and added all the new client requirements to it:

https://fedoraproject.org/w/index.php?title=Fedora_21_Final_Release_Criteria&diff=395311&oldid=395309

I added all the new requirements to Final, not Beta, because it would be
misleading to list them in the 21 Beta criteria when we did not in fact
test the 21 Beta against those criteria. We can move the Beta ones to
the Beta section for F22 (for the 'client requirements'; as already
noted, we should move *all* the role requirements out of the criteria
into some more sustainable framework for F22).

I tweaked the wording a bit to fit into the overall flow of the
criteria, but nothing major. 

I dropped the bits about 'this should be validated with blah' as they
are not properly part of the criteria. They're really extremely
minimalist test cases. What actually needs to happen is that we write
proper test cases for each criterion and reference them in the
References expander - you'll note that each criterion's References
section currently has FIXME as its 'Test Case'. You can see the existing
Server test cases at
https://fedoraproject.org/wiki/Template:Server_test_matrix ; we need to
write new ones following the normal layout (use, say,
https://fedoraproject.org/wiki/QA:Testcase_realmd_join_cockpit or
https://fedoraproject.org/wiki/QA:Testcase_Server_cockpit_basic as a
model), add them to that matrix, and reference them from the release
criteria. I can do some work on that tomorrow, any help anyone else can
extend in that work would be greatly appreciated.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net