Release criterion proposal: upgrade methods

Tue Sep 25 01:15:43 UTC 2012

On Mon, 2012-09-24 at 20:21 -0400, Richard Ryniker wrote:
> >I think we could link to
> >https://fedoraproject.org/wiki/Upgrading to define
> >'supported/recommendation upgrade mechanism'
> 
> OK, but to illustrate the problem with indirect references:  the
> "Upgrading" page you cite above says to read
>   http://docs.fedoraproject.org/en-US/Fedora/17/html/Installation_Guide/index.html
> for details.  This document says (Chapter 20):
> 
>   Before upgrading to Fedora 17 you should first bring your current version
>   up to date. However, it is not then necessary to upgrade to intermediate
>   versions. For example, you can upgrade from Fedora 14 to Fedora 17
>   directly.
> 
> Therefore, it seems your recent post to this list:
>   http://lists.fedoraproject.org/pipermail/test/2012-September/110331.html
> may be incorrect.  I think you are right, but the quotation above
> contradicts your statement:
> 
>   Only 17-18: using the definite article 'the' rather than the indefinite
>   article 'a' implies this. It says 'the previous stable Fedora release' -
>   which strictly means "only the single preceding release"
> 
> My point here is that details in the QA criteria that specify explicitly
> what operations must work are valuable.  Indirect references to dynamic
> documents (which you properly describe as not owned by QA) mean somebody
> else, who may have different interests and objectives than QA and does
> not intend to write a QA criterion, defines what your criteria are.

Well, to an extent, yeah. To me that's not really a problem as much as
it is an opportunity, though. ;) It's a left hand, right hand issue; the
one should know what the other is doing. As you explain above, obviously
that's not quite the case there. I don't think that's an inherent
weakness of the idea as much as it is a case where we should correct
something, though. Either we should start testing upgrades from ancient
releases and taking it seriously when they fail, or we should stop
advising people to do so in the installation guide.

Another way to put it...I'd say that interlinking the criteria and the
installation guide, as you outline it above, has provided us with a
*benefit*: we have identified an inconsistency between what the
installation guide reckons is reliable and what QA and devel are making
any kind of effort to ensure actually is reliable. If we just wrote the
criteria in some kind of silo where we had our own definitions of
everything, it wouldn't make that problem go away, would it? The
installation guide would still be out there and would still be advising
people to do something that maybe it shouldn't be advising people to do.
Given that Fedora's a collaborative effort, I'd say we should be
consciously trying to interlink between teams as much as possible as a
way to ensure we're all on the same page, not silo'ing off our efforts
from each other because we're scared of what we might find out from
working together...

> >'official install media' is not quite as obvious; maybe it
> >should be a link to the Deliverables SOP when I or someone else finally
> >gets that done (my last draft is still at
> >https://fedoraproject.org/wiki/User:Adamwill/Draft_releng_SOP_deliverables
> 
> Your draft makes no mention of LiveUSB Creator or livecd-tools.  It only
> mentions "'dd' or similar tools".  Does this mean persistent storage,
> encrypted /home, and other features, are no longer supported on live
> media (or perhaps are simply not of concern to QA)?  

Before this goes viral, let me provide the all-important context. This
is the text Richard's referring to:

"All image files for PC architectures should be prepared for writing to
USB directly with 'dd' or similar tools, and should be prepared for both
EFI and BIOS booting whether written to an optical disc or a USB disk."

The answer to your question is no, it doesn't mean I don't care about
litd or livecd-creator. It just means there isn't any 'preparation' work
that has to be done to an image to make it writeable via
livecd-iso-to-disk. Our images are litd-able as they pop out of the
creator. That's not the case for dd; releng has to run mkisohybrid on
the images at some point to make them dd'able. Once or twice this hasn't
got done, hence I decided to specify it in the SOP.

> I gather there has
> been a lot of "back and forth" in this area of late - perhaps another
> case where explicit language in QA criteria may be valuable, even if it
> has to track evolving decisions about what sophisticated live system
> features will be "supported".

We do in fact have this. At Alpha:

"The installer must boot (if appropriate) and run on all primary
architectures, with all system firmware types that are common on those
architectures, from default live image, DVD, and boot.iso install media
when written to an optical disc and when written to a USB stick with at
least one of the officially supported methods"

('at least one' is in boldface, and 'officially supported methods' links
to https://fedoraproject.org/wiki/How_to_create_and_use_Live_USB .)

At Beta we have a criterion with the precise same text, except that 'at
least one of the officially supported methods' is replaced by 'any of
the officially supported methods', with 'any' in boldface.

What this means is simply that at Alpha we require at least one of the
supported USB writing methods to work, and at Beta we require all of
them to work. That's what we decided in the discussions on this list and
as part of the Alpha validation process - the 'back and forth' you
referred to.

> It is unlikely there will ever be perfect QA criteria, but these criteria
> are important: the value of the QA effort depends in significant ways on
> their quality.  Whatever their eventual form, I hope my comments help to
> make them a little better.

Agreed! Thanks for the feedback.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net