Review of Fedora 18 Release Criteria

[Adam Williamson]

On Tue, 2012-10-09 at 22:43 -0400, Akira TAGOH wrote:
> ----- 元のメッセージ -----
> | As we're getting closer to the scheduled time for beta freeze, we'd
> | like to find out now if any of the current criteria or proposed
> | criteria
> | changes are unreasonable to expect for beta. There may be more
> | changes
> | for final as we get closer to that but I think that we're pretty
> | close
> | to being done with the release requirements for beta.
> | 
> | The current (as of this writing) release criteria are available at:
> |  - http://fedoraproject.org/wiki/Fedora_18_Alpha_Release_Criteria
> |  - http://fedoraproject.org/wiki/Fedora_18_Beta_Release_Criteria
> |  - http://fedoraproject.org/wiki/Fedora_18_Final_Release_Criteria
> | 
> | There have been several changes to the release criterion lately
> | (mostly
> | due to changes in anaconda's functionality) and there are still a
> | couple of modifications pending (identified at the end of this email
> | with links to the most recent versions of the proposal)
> | 
> | The questions we're interested in are:
> |  - Is there anything in the current criteria or proposed revisions
> |  that
> |    seem unreasonable?
> |  - Are there any areas where the criteria are lacking?
> 
> I was expecting not to see the sort of locale issues anymore since
> it's essential technology to support i18n/l10n and didn't doubt we
> haven't ever had any criteria for that and in fact it wasn't broken in
> a long time. however it happened and affected a lot of applications
> and testings. the sort of this issue should be caught up by alpha
> because of that. in fact it messed up a lot. so I'd like to propose
> adding something for locales/langauges in alpha criteria.
> 
> Though it may be not happened forever. but that would be good to make
> sure if we won't break it every releases.
> 
> Any thoughts?

So this is an area we've been over a few times...

Last time we went over it, what we did was add an explicit translation
criterion for Final - "All critical path actions on release-blocking
desktop environments should correctly display all sufficiently complete
translations available for use" - and agree that other i18n/l10n issues
should be considered conditional infringements of the other criteria and
handled that way. e.g. if you can't pick a keyboard map properly, that's
a conditional infringement of "In most cases (see Blocker_Bug_FAQ), a
system installed according to any of the above criteria (or the
appropriate Beta or Final criteria, when applying this criterion to
those releases) must boot to the 'firstboot' utility on the first boot
after installation, without unintended user intervention, unless the
user explicitly chooses to boot in non-graphical mode. This includes
correctly accessing any encrypted partitions when the correct passphrase
is supplied. The firstboot utility must be able to create a working user
account", because the passphrase you picked for an encrypted partition
at install time would not be typed the same way when you came to unlock
it.

This more or less _works_, but on reflection (it came up in the 18
blocker review process), it seems pretty baroque and non-raptor-proof -
I don't think someone just reading the process documentation is
necessarily going to figure out that dodge. So it fails that test. (As
neatly shown by the fact that you posted this mail - obviously you
didn't figure it out :>) So I'm now rather of the opinion we should have
explicit criteria for the major i18n/l10n stuff, indeed.

We do actually cover this in the test cases, but again, we did it in an
'indirect' way: we added 'test it with a non-English keymap' to
https://fedoraproject.org/wiki/QA:Testcase_desktop_login  and
https://fedoraproject.org/wiki/QA:Testcase_Anaconda_autopart_%
28encrypted%29_install . But again this isn't particularly obvious if
you just browse the test list. And now I look at it, it covers only
keymap, not locale, so we might need to add that somehow.

We have/had a trac ticket which was closed after the last go-round; I
re-opened it after having the thoughts described above.
https://fedorahosted.org/fedora-qa/ticket/81 - let's follow up on it
there.

I _think_ this actually broke between Alpha and Beta, didn't it? I might
be wrong, but I thought it worked at Alpha but then had got broken for
Beta.

Thanks for the thoughts!
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net