Initial set of proposed release criteria for Server product

[Adam Williamson]

On Mon, 2014-06-09 at 17:05 -0700, Adam Williamson wrote:
> On Mon, 2014-06-09 at 11:33 -0400, Stephen Gallagher wrote:
> > On 06/09/2014 08:26 AM, Miloslav Trmač wrote:
> > > Hello, 2014-06-07 0:55 GMT+02:00 Adam Williamson
> > > <awilliam at redhat.com <mailto:awilliam at redhat.com>>:
> > > 
> > > The other is remote system management. The spec says:
> > > 
> > > "Software updates on the Fedora Server must be possible to perform 
> > > either locally using command-line tools (e.g. yum/dnf)..."
> > > 
> > > (okay, we already cover that in current criteria)
> > > 
> > > "...or centrally by common management systems (e.g. Puppet, Chef, 
> > > Satellite, Spacewalk, OpenLMI)."
> > > 
> > > well, that's extremely broad. Do we really want to have the
> > > criteria say "it must be possible to update a Fedora Server system
> > > via Puppet, Chef, Satellite, Spacewalk or OpenLMI", write a test
> > > case for each, and block releases unless all of those mechanisms
> > > work? Or do we want to focus down a bit?
> > > 
> > > 
> > > I don’t recall precisely; AFAICT this is mainly a “we shall not
> > > break what works” criterion, e.g. that Server should not add a
> > > mandatory interactivity requirement for updates that would make it
> > > impossible to use some of these tools.
> > 
> > Yes, I'm pretty sure that was the intent of that portion: mainly that
> > we commit to not actively hampering the efforts of those (and similar)
> > projects from working with Fedora Server (even if we eventually
> > support and provide a "better" option).
> 
> OK. I can write a criterion based on that interpretation fine; would it
> be worth rewording the tech spec a bit, perhaps something like this:
> 
> "Fedora Server will provide a command line utility for performing system
> updates and will ensure that updates can always be run
> non-interactively. Server will aim to maintain compatibility with
> management systems such as Puppet, Chef, Satellite, Spacewalk and
> OpenLMI for remote update deployment."
> 
> (it occurs to me that we don't specify anything about repository
> configuration or package signing there, I guess they're stuff we kind of
> assume is baked into Fedora...)

Having thought about this a bit more, I think it possibly doesn't need a
release criterion. It seems quite unlikely that some kind of *bug* is
going to introduce interactivity into the update process; if it
happened, it'd be a design/policy issue, not something to handle via
what's a fairly bug-centred process. OK, maybe that's a bit squishy, but
it kinda makes sense in my head; it just doesn't seem to fit into my
release criteria mental box, every time I draft a criterion for it it
comes out sounding...odd.

Can anyone see something specific here which should be called out as a
release criterion and have a particular validation test added? Something
specific that we can (and need to) test at each release milestone?
Beyond the existing generic criteria/tests? I can't, quite. So unless
someone else can, I'll leave it out.

We could, I guess, promote one or a few particular remote management
system(s) to 'formally supported' status, and have a criterion / test
case for "it must be possible to perform an automated remote update of
the system from Puppet [or Chef or Satellite or whichever one(s) we
pick]". That's the thing that feels like a criterion / test case to me,
if anything. Thoughts?
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net