[F24 Criteria Change] FreeIPA Upgrades

[Adam Williamson]

On Mon, 2015-11-02 at 10:02 -0500, Stephen Gallagher wrote:
> Up to now, all of our release criteria for FreeIPA have been in terms
> of fresh installations on a newly-installed system. However, there
> are
> numerous individuals out there who are running FreeIPA on Fedora
> systems and who would suffer greatly if an upgrade from the previous
> version did not work.
> 
> I propose that the following release criterion be added for Fedora 24
> and onwards:
> 
> Beta Criterion:
> * For any system with a configured and enabled FreeIPA server (or
> Domain Controller Role powered by the same), it must be possible to
> upgrade from Fedora N to Fedora N+1 and have the same set of domain
> controller services available after upgrade.
> 
> 
> As a secondary criterion, I'd also like to recommend that we add:
> 
> Final Criterion:
> * For any system with a configured and enabled FreeIPA server (or
> Domain Controller Role powered by the same), it must be possible to
> upgrade from Fedora N to Fedora N+1 and have the same set of domain
> controller services available after upgrade without manual
> intervention.

I have three issues with this:

1. The addition of the Final criterion rather explicitly implies that
the Beta criterion allows for 'manual intervention', and you can fix
just about anything with 'manual intervention' - by that standard the
current situation wouldn't fail the test, arguably, as you *can* make
FreeIPA work on upgrade, you just have to hand-edit the LDAP server
config and run the Tomcat migration script and tweak an SELinux
boolean, simples!

I know we fuzz this a bit in other cases, but I'm worried about the
implications of making it super explicit like that. I'd rather qualify
the level of 'manual intervention' that's acceptable.

2. I'm not sure this approach of specifying individual bits of
functionality that must work is gonna scale. It would make more sense
to me to require instead that upgrade must work for the 'featured'
roles.

3. More fundamentally, I don't think the release criteria is the place
to be doing this *at all*. One of the things on my list for post-F23
process tasks is to propose an alternative system for handling stuff
we've been treating as 'special blockers' lately, and this kind of
upgrade payload stuff is likely to fall into that category. I don't
like the 'special blocker' (lack of a) process at all, really, as it
has no teeth: we just say 'oh, this must happen' and then assume it
magically will. Which is not a great way to go about stuff.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net