On Fri, 2014-07-11 at 13:23 -0400, Matthias Clasen wrote:
Hey Adam,
I've looked over the workstation testcases quickly.
I just found this mail a half year later, so some very belated
replies...
Some comments:
https://fedoraproject.org/wiki/QA:Testcase_desktop_updates talks
about yum and PackageKit - should it include dnf ?
As Jaro said at the time, yum is still the default/supported package
manager for F21. For F22 we're currently slated to change to dnf, and
if that *happens* we'll change the test case, but there's still some
discussion about it at present.
https://fedoraproject.org/wiki/QA:Testcase_desktop_panel_basic could
benefit from some outline of what desktop 'panel' elements we expect
to see. Sticking with the most common laptop case, I would mention
network, sound and battery status as expected
So yeah, a thing to bear in mind is that most of the cases were
written around F13 time (IIRC), with the GNOME 2 and KDE 3 of those
days in mind, when everything looked more or less like Windows 98 and
everyone's 'panel' actually had some bits running in it that they
didn't write (like nm-applet was a pretty separate thing from gnome-
panel, for e.g.)
When I'm running this test (and the 'advanced' version) against Shell I
usually check all the bits of the Shell top bar, user menu, and
overview. Re-wording it to be too specific about the expected elements
runs the risk of no longer being applicable to the other desktops, but
of course if it's necessary we can have two (or more) cases, or some
conditional language.
I think this just about scrapes by as 'OK' at present, but if I can
find the time I'll see if I can improve it for the Modern World.
https://fedoraproject.org/wiki/QA:Testcase_desktop_login seems to be
a bit of a misnomer; the test is really about keyboard layouts.
Hum, not really, it really *does* test login/DM stuff - it just also
tests that there aren't screwups with the keyboard layout not being
consistent between installer, DM, and desktop. IIRC, I initially
started writing a separate 'keyboard layouts' test case at some point,
then realized that what it was actually doing was duplicating a bunch
of existing test cases but with 'oh, and do it with a different
keyboard layout' - so instead of doing that, I just wrote 'check it
with a different keyboard layout' into each of the existing tests.
There are similar bits in the disk encryption test case, for e.g., to
make sure the right keyboard layout is used for entering the
passphrase on boot.
https://fedoraproject.org/wiki/QA:Testcase_audio_basic talks about
selection hw profiles. I really think that the expectation for
'common hw' (ie your typical laptop) should be that sound works out
of the box.
Reasonable point, I'll have to check on the current actual status of
this with different hardware; I *think* at least on modern systems the
drivers/PA should be able to sense what's connected and choose the
appropriate output profile, but I'm not 100% sure.
I think the note for multiple *devices* is still valid, though. If you
have multiple output devices, PA can't magically tell which one you
want sound to come out of, you do have to tell it.
https://fedoraproject.org/wiki/QA:Testcase_desktop_updates same
comment as above - should this use dnf instead of yum ? I think it
would be good to make this testcase more specific wrt to the way
offline updates are integrated in the desktop now. Test that we
notify (once per day) about available updates. Test that the
logout/poweroff dialog offers to install pending updates. Test that
we notify about successful and unsuccessful offline updates, and
show details.
There's definitely some ancient text in there ("observing whether the
'checking for updates' and then 'updates available' icons appear in
the notification area", no, that's not going to happen). The same
problem as I mentioned above applies to making this *too* specific to
Workstation - the same test case is applied to KDE and other lives too
- but I think the same approach will work, first try and 'modernize'
it as a generic test case, and then evaluate whether it's
necessary/worthwhile to split out a Workstation-specific case from it.
https://fedoraproject.org/wiki/QA:Testcase_workstation_theming would
benefit from adding some mention of accessibility - we expect the
HighContrast theme to work and be of similar quality to the default
theme.
Thanks, I'm not sure whether that'd work best as an addon to this test
case or a new one, but either way, I'll try and add something to cover
it.
Things that would be good to capture in new testcases:
- Screen locking. Explicit locking via keyboard shortcut or menu
should
work, as well as lock on idle. Screen blanking after lock is
something that frequently has issues. Can also test that inhibiting
works (when watching video in totem, or in the browser.
Noted, thanks!
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net