Criterion proposal - keyboard layouts

Tue Feb 21 01:49:08 UTC 2012

On Tue, 2012-02-21 at 00:11 +0000, "Jóhann B. Guðmundsson" wrote:
> On 02/20/2012 11:42 PM, Adam Williamson wrote:
> > I wish I had better background on it, but I'm pretty sure we did discuss
> > having an explicit criterion before going with the 'vague' paragraph
> > instead. All that mail (by me) says is that I didn't think a specific
> > criterion 'worked well', which is pretty useless looking back - sigh.
> >
> > One problem I do remember is that, in practice, we don't really want to
> > block the release if a single extremely obscure keyboard layout turns
> > out to be broken at the last minute; that's one factor in favor of the
> > vague hand-wavey judgment call paragraph. We want to block only on
> > reasonably popularly-used layouts, your Frenches and Germans and
> > whatevers, but it's a bit hard to write a criterion that properly
> > restricts the list without being too restrictive.
> >
> > Still, I'm not really against the criterion, just trying to remember the
> > thought process from before.
> 
> If I remember correctly the vague part was put there as an escape clause 
> to go to if enough number of community members voiced up for their 
> layout not working to block the release.  ( which basically is what you 
> say there with "popular" )

Yes, indeed, I kinda 'assumed' that knowledge in my post but should have
made it explicit: the function of that paragraph is to give us an
explicit thing to point to to justify making judgment calls about bugs
which break the criteria _only in specific configurations_.

The question now is whether it's appropriate to always simply handle
keymap breakage by that mechanism - consider it a conditional breakage
of the encrypted partition criterion and judge based on how serious the
breakage is and how many keymaps it affects - or whether we should
actually add a specific criterion for keymaps.

> The fact is that we cant implement any kind of formal test procedure to 
> test all available layouts as in we cant really cover any other layout 
> then our current community members use ( I for one cant use Chinese, or 
> Japanese layouts as an example ).

It is actually possible to test a keyboard map you don't really know how
to use, though a bit painful. The only things you need to type in a
normal install are passwords. All you need to do to test a keymap is to
look at a diagram of the keymap - GNOME can help with that - and
identify keys which are different from en-US, then use those keys to
make up your passwords. Then you can tell whether the keymap is properly
used because if it isn't, the password entry will fail.

You can select any keyboard layout you like, even one which doesn't
match your actual physical keyboard at all, and you'll still usually be
able to complete an install well enough to test that the correct layout
is used at all points.

> Due to the fact the we cant cover them all, implementing such criteria 
> in any shape or form would force us to start chosing which layouts to 
> test which is not fair to others  + community members on different 
> layout are already reporting any issues that they find which begs the 
> question what benefits would it bring implementing such criteria.

Well, we do have a precedent for having criteria which are not fully
testable. One of the final criteria is "The installer must be able to
create and install to any workable partition layout using any file
system offered in a default installer configuration, LVM, software,
hardware or BIOS RAID, or combination of the above" - that's impossible
to test fully. We've always simply interpreted this that we will test as
many layouts as possible, and any bug roughly of the form 'I tried to
create <some valid partition layout> and anaconda crashed' that comes to
our attention will be accepted as a blocker.

I'm not sure 'fairness' in this way is really worth applying to QA.
After all, it's a truism that we can _never_ test everything. It doesn't
really mean we're being 'unfair' to the things we don't have time to
test. The way I see it, testing one layout is better than testing none,
testing two is better than testing one, testing three is better than
testing two...it's never really 'unfair' to the ones we don't manage to
test.

Say in Scenario A we test only English, and in Scenario B we test
English, French, German and Japanese. We don't test Austrian in either
case. It's not like Austrian's any *worse* off in Scenario B than it was
in Scenario A...

> I for one aggree with Adam that broken keyboard layout should not block 
> the release ( as I was uhum when this decition was made in the first 
> place If I can recall correctly me and Dennis vote hard -1 on the 
> relevant go/no go meeting but to no prevail since we got outnumbered by 
> +1 by community members with broken keyboard layout which lead to the 
> vague thingy ) however if such criteria such be implemented I ack the 
> proposed proposal...

Thanks for your thoughts!
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net