Proposed release criteria and test case adjustments relating to artwork and self-identification

Adam Williamson awilliam at redhat.com
Tue Aug 28 21:29:13 UTC 2012


Hey, folks. So discussion around 
https://bugzilla.redhat.com/show_bug.cgi?id=849982 rather indicates that 
our existing release criteria relating to artwork are problematic. I 
talked to the design team on IRC this morning, looking at the criteria 
in their initial context (see 
http://meetbot.fedoraproject.org/fedora-bugzappers/2010-09-03/fedora-bugzappers.2010-09-03-16.00.log.html 
around 16:25 and 
https://lists.fedoraproject.org/pipermail/test/2010-September/093476.html 
if you're interested), and how artwork requirements have changed since 
then.

The current criteria are:

Alpha - "The default Fedora artwork must either refer to the current 
Fedora release under development (Fedora 18), or reference an interim 
release milestone (e.g. Alpha or Beta). If a release version number is 
used, it must match the current Fedora release under development. This 
includes artwork used in the installer, graphical bootloader menu, 
firstboot, graphical boot, graphical login and desktop background."

Final - "The proposed final Fedora artwork must be included and enabled 
by default for the installer, graphical boot, firstboot, graphical login 
and desktop background. All Fedora artwork must be consistent with the 
proposed final theme, and if any artwork contains a graphical version 
number, the version number used must match the Fedora release number. 
Generic release artwork (e.g. Alpha, Beta, Development) must not be used 
for the final release"

There are several problems with these, now:

1) The use of 'refer to' is somewhat ambiguous and creates confusion - 
I'd been reading it as meaning 'explicitly specify', but the artwork 
team have been reading it as meaning 'specify or reference in some way' 
(so they consider the desktop backgrounds as 'referring to' specific 
releases by association with their codenames, even though the 
backgrounds don't state a version number).

2) They're probably over-strict for their real intent. The intent of 
the Alpha criterion, especially, is really only that no-one should 
mistake an Alpha release for the previous stable release or anything 
like that. It's about avoiding confusion.

3) The space for artwork in current Fedora is much more limited than it 
was when we wrote those criteria. Lots of the things that were 
previously explicitly themed per release now have either generic artwork 
or no 'artwork' to speak of at all. anaconda doesn't really have 
'artwork' any more. The bootsplash has been the Fedora logo on a blue 
blackground for several releases. firstboot is not themed any more, it's 
just solid grey and blue with the Fedora logo. From Fedora 18 onwards, 
even gdm will not be themed any more, it will not feature a background 
image. We're not sure of the status of kdm, but design team considers 
KDE SIG responsible for that, it's not part of the 'official' artwork 
stuff. So basically, we're down to bootloaders and the desktop 
background, when it comes to 'artwork'.

With all those factors in mind, here's what we boiled it down to. I 
propose we replace the Alpha criterion with this one:

* The default desktop background must be different from that of the two 
previous stable releases

and add a new Alpha criterion:

* Any component which prominently identifies a Fedora release version 
number or phase (Alpha, Beta, Final) must do so correctly

The first of these might be a bit over-clever - we could just say "must 
be different from any previous release", but, perhaps over-thinking 
things, I wondered if maybe at some point in the future we'll want to 
use a 'retro' wallpaper for some reason, revive the Fedora Core 1 
artwork or something. Which, with enough of a time gap, obviously 
wouldn't create any confusion. So I tried to allow for the possibility.

The second one, I think, frames the confusion problem better: it's not 
only artwork that can create confusion. The new anaconda UI identifies 
the Fedora version and phase with a text string - that's not 'artwork', 
but if it's wrong, it could certainly confuse things. plymouth in text 
mode also specifies the Fedora release. There are probably others I'm 
not thinking of right now. I don't see why the 'confusion' criterion 
shouldn't cover _anything_ that could plausibly cause confusion, there's 
no reason to restrict it to artwork. The 'prominently' weasel-word gives 
us an out to avoid slipping the release just because it's wrong in some 
obscure text file somewhere that no-one ever reads.

I propose we modify the Final criterion a bit, just to update it for 
the more restricted modern artwork era:

* The proposed final Fedora artwork must be included and enabled by 
default for all graphical bootloaders and the desktop background. All 
Fedora artwork must be consistent with the proposed final theme

'all graphical bootloaders' is phrased that way because we use multiple 
bootloaders - grub2 for installed systems and EFI boots of installer 
discs, syslinux for BIOS boots of installer discs - and sometimes these 
aren't graphical, we have to be allowed to not include *any* artwork in 
a bootloader if that seems necessary/unavoidable. We can drop the 
'version number must match' sentence entirely, if we adopt the proposed 
new Alpha criterion which would subsume it.

We would also have to modify some test cases slightly, the changes just 
follow from the above.

In https://fedoraproject.org/wiki/QA:Testcase_base_startup , the result 
"If the artwork used at the bootloader screen, during graphical boot, on 
the login manager, or the desktop background references a release number 
and/or pre-release phase (Alpha, Beta etc), it must be the current 
number and/or phase for the release or pre-release under test" would be 
changed to "If the bootloader screen, graphical bootsplash, login 
manager, desktop background or any other prominent part of the start 
process references a release number and/or pre-release phase (Alpha, 
Beta etc), it must be the current number and/or phase for the release or 
pre-release under test". Similarly, the phrase "the artwork used in the 
login manager and on the default desktop (particularly the background) 
must be the correct and current artwork" proposed for the release in 
question by the design team" would be changed to "the artwork used on 
the default desktop (particularly the background) must be the correct 
and current artwork".

In https://fedoraproject.org/wiki/QA:Testcase_base_firstboot, the 
expected result relating to artwork would simply be removed.

In 
https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_Graphical, 
"used in the boot menu and installer" would be changed to just "used in 
the boot menu". "If a release version number is used" could be clarified 
to "If the boot menu and/or installer specify a release version number".

Sorry for the essay, hope it all makes sense! Comments, suggestions, 
improvements, complaints welcome.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net


More information about the test mailing list