Release criterion proposal: "Package sets" (Alpha and Beta)

[Adam Williamson]

On Tue, 2015-01-20 at 12:46 -0800, Adam Williamson wrote:
> On Tue, 2014-12-23 at 10:21 -0800, Adam Williamson wrote:
> > The "Package sets" criterion for Alpha currently reads:
> > 
> > "When doing a graphical install using the dedicated installer 
> > images, the installer must be able to install each of the release 
> > blocking desktops, as well as the minimal package set."
> > 
> > This was drafted prior to Product-ization. It has a bug - you 
> > can't do that from the Server DVD, and that's intended - and two 
> > problems - it's too focused on desktops for the new Product-y 
> > world, and the 'graphical' restriction seems arbitrary (TUI should 
> > work regarding package sets too). It also is missing something: 
> > there's no
> > requirement about what the *default* package set should be.
> > 
> > I propose we re-word the Alpha criterion to:
> > 
> > "When installing with a release-blocking dedicated installer 
> > image, the installer must be able to install the default package 
> > set."
> > 
> > and add a Beta criterion:
> > 
> > "When installing with a release-blocking dedicated installer 
> > image, the default package set must be correct."
> > 
> > with an explanatory note that 'correct' means the package set 
> > intended by the group responsible for the image - Product WG, 
> > FESCo or whoever.
> > 
> > I'm not sure whether we need a requirement for non-default package 
> > sets. Note that the case for offline media is already covered by 
> > Alpha criterion "No broken packages":
> > 
> > "There must be no errors in any package on the release-blocking 
> > images which cause the package to fail to install."
> > 
> > network installs using updates media don't really need to block on 
> > package set issues, as they can be fixed. That leaves the question 
> > of whether we'd want to block the release if, say, there was a bug 
> > which meant that if you tried to netinst KDE without the updates 
> > repos enabled, it failed. What do folks think about that?
> 
> Here's a ping on this (as I only got feedback from Mike before - 
> anyone else?) and a modification: I'd like to extend the Beta 
> criterion to read:
> 
> "When installing with a release-blocking dedicated installer image, 
> the default package set must be correct, and choosing a different 
> package set must work."
> 
> with a footnote something like:
> 
> "'work' means that the package set selection mechanism itself must 
> work; when used, the packages that form the chosen set must actually 
> be the ones marked for installation. Package issues that render one 
> or more selectable package sets un-installable do not constitute a 
> violation of this criterion, though they may be violations of other 
> criteria."
> 
> this is to cover things like 
> https://bugzilla.redhat.com/show_bug.cgi?id=1179362 , which I 
> noticed when filing it is a bit of a loophole in the proposed 
> criteria.
> 
> Any more thoughts, folks? Thanks!

Hum, so thinking about it a bit further, I'd like to add one more 
thing to this proposal.


On reflection I think we *should* require install of release blocking 
desktops and minimal package set to work with the frozen release 
package set, because I can see cases where it would be important to be 
able to fire off an install and be 100% confident you're not going to 
have package set issues; people may want to use the frozen repo for 
that purpose, so they're sure updates won't introduce any new problems.

We also seem to be pretty much committed to still having the universal 
network install image, now - at least for F22.

So, to cover those goals, propose we basically move the existing Alpha 
criterion to Final, with a tweak to refer to the network install image 
and the frozen repos specifically:

When installing with the network install image with no update 
repositories enabled, the installer must be able to install each of 
the release blocking desktops, as well as the minimal package set.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net