Proposed Server release criteria for F21 Beta and Final

Adam Williamson adamwill at fedoraproject.org
Tue Sep 30 00:19:25 UTC 2014


Hi, folks. So, I drew up a rough draft of the Server release criteria
for Beta and Final as I suggest they might be. We could kick it around
at tomorrow's meeting if desired. Here we go:

BETA

=== Remote logging ===

It must be possible to forward system logs to a remote system using
Server packages.

=== Firewall configuration ===

Release-blocking roles must be able to report their status in regard to
the system firewall as described in the
[[Server/Technical_Specification#Firewall|technical specification]].

=== Roles ===

Release-blocking roles and the supported role configuration interfaces
must meet the core functional
[[Server/Technical_Specification#Role_Definition_Requirements|Role
Definition Requirements]] to the extent that supported roles can be
successfully started, stopped, and queried.

=== Cockpit ===

??????

FINAL

=== Roles ===
Release-blocking roles and the supported role configuration interfaces
must fully meet the
[[Server/Technical_Specification#Role_Definition_Requirements|Role
Definition Requirements]].

=== Cockpit ===

??????

You will note first of all the big ?????? for Cockpit. Cockpit's role
really isn't terribly well defined in the tech spec or PRD, so it is
hard to draft requirements. Is it primarily supposed to be an interface
for controlling roles? If so, can we treat it that way in the criteria,
and require things from it as regards its job in configuring roles?

I think the Role criteria described above would be okay, and at least
unambitious enough not to cause huge problems, for F21, but I'm really
not that happy with them for the long term.

It's obviously not practical to start hardcoding specific requirements
for specific roles into the criteria. What I would like instead is to
have a little more process on the Product side for Roles. I'd suggest
that Roles themselves should have definitions of their expected
functionality baked in at the point of creation / approval.

These could be split into sections to be required at Alpha, Beta and
Final. You could, for instance, expect roles to basically start up and
not explode at Alpha, expect them to be broadly 'feature complete' and
testable at Beta, and expect them to fully meet their listed functional
requirements at Final, just as a quick cut. Here's a very rough mock-up
for the "Domain Controller" role as a guide:

Alpha
-----

The Role must start and stop successfully. When the role is running, a
client system must be able to enrol into the FreeIPA domain.

Beta
----

When the Role is running, it must be able to enrol and unenrol multiple
clients. Client systems must be able to authenticate user accounts via
Kerberos. The FreeIPA configuration web UI must be available and allow
at least basic configuration of user accounts and permissions.

Final
-----

Oh, I don't know, I'm running out of ideas, I don't do that much more
with my FreeIPA system than is described in Beta. You know better than
me! That's why you should be writing this! :)

Well, I sort of ran out of steam at the end there, but you kinda get the
idea, right?

We can set things up appropriately so the release criteria can sensibly
call out to them. I'd expect the criteria would just say something like
'Release-blocking roles must meet the functional requirements listed in
their definition pages for the Beta milestone' or whatever, with a link
to a sensible target from which you could easily find the Role
definition pages.

I'd be very grateful for feedback on both the proposed criteria for F21,
and the idea above. We do need to get the Beta criteria in place ASAP,
Beta TC1 is due tomorrow (yes, I know, no rest for the wicked :/) I'll
start drafting up test cases ASAP.

-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net



More information about the server mailing list