For the Fedora Magazine: Suggested QA activities during the pre-F21 lull

Wed Jan 22 14:43:34 UTC 2014

On 01/22/2014 08:21 AM, Ankur Sinha wrote:
> Hi,
> 
> Adam sent this out to the QA list. I'm wondering if either a summary, or
> the entire text would make a good post for the magazine?

A summary that points to the mailing list is probably the best way to
go. For stuff like this, I think we can safely emulate LWN's approach -
quick summary, quote from the mail, and link to the mailing list.

Best,

jzb

> -------- Forwarded Message --------
>> From: Adam Williamson <awilliam at redhat.com>
>> Reply-to: For testing and quality assurance of Fedora releases
>> <test at lists.fedoraproject.org>
>> To: test at lists.fedoraproject.org
>> Subject: Suggested activities during the pre-F21 lull
>> Date: Mon, 20 Jan 2014 17:49:43 -0800
>>
>> Hi folks! As a throwaway at the end of the meeting today I mentioned a
>> few tasks we could be doing during this 'quiet time' between F20 and
>> F21. People seemed interested, and so I promised to send out a more
>> detailed version of the list with links and references and stuff. This
>> (obviously!) isn't a set of orders, and it's not really even a 'top
>> priority' list, just my little list of ideas for things you could work
>> on if you're looking to spend some time on Fedora but aren't sure what
>> to do when we don't have Test Days or validation testing running.
>>
>> * It's not glamorous or exciting, but it always needs doing: test
>> updates-testing on the stable releases (F19 and F20) and file feedback.
>> You can always use a virtual machine to test a release you don't have
>> installed - you can't check everything in a VM, but you can do a lot of
>> things, and sometimes you can test stuff in a VM that you wouldn't want
>> to mess with on your main system.
>> https://fedoraproject.org/wiki/QA:Updates_Testing has all the info you
>> need on doing updates-testing work: if you don't use fedora-easy-karma -
>> https://fedoraproject.org/wiki/Fedora_Easy_Karma - or gooey karma -
>> stuck in package review ATM, but there's a Copr at
>> http://copr.fedoraproject.org/coprs/blaskovic/fedora-gooey-karma/builds/
>> - consider doing so, it'll really help!
>>
>> * You can always help test Rawhide - the development tree of Fedora, so
>> right now, the current state of Fedora 21. All the details on Rawhide
>> are at https://fedoraproject.org/wiki/Releases/Rawhide . Depending on
>> how brave/experienced you are (and how many forms of backup you have!),
>> you can even run it in production (the machine I'm typing this on has
>> been running Rawhide for several weeks now...but I have backups of
>> everything, webmail to use when Evolution breaks, Xfce to use when GNOME
>> breaks, a Fedora 20 laptop to use if Rawhide breaks and a Fedora 19
>> laptop to use if Fedora 20 breaks ;>). If that's a bit too much for you,
>> you can test the nightly live images from
>> http://alt.fedoraproject.org/pub/alt/nightly-composes/ , or run it in a
>> VM. Once you're actually running Rawhide, file any significant bugs you
>> encounter just as usual, and if you run into any real showstoppers, you
>> can propose them as Fedora 21 blockers following the usual process
>> ( https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process ) - the
>> blocker bug app has not yet been updated for F21 so you can't use that,
>> but all you have to do is mark the bug as blocking "AlphaBlocker",
>> "BetaBlocker" or "FinalBlocker". If you see any unusual or unexpected
>> changes but you're not sure if they're bugs, post about them here (on
>> test@), it's one of the purposes of the list!
>>
>> * One thing we never seem to quite do enough of is creating
>> package-specific test cases: again this isn't highly visible work, but
>> it is useful to do it. The details on doing it are at
>> https://fedoraproject.org/wiki/QA:SOP_package_test_plan_creation (and
>> https://fedoraproject.org/wiki/QA:SOP_test_case_creation explains how to
>> write a test case). If you write some test cases and 'associate' them
>> with a package, as the SOPs explain, they will be attached to every
>> Bodhi update for that package, and people doing updates-testing testing
>> will see your test cases as something they can use to check the package
>> is working properly. If Bodhi 2.0 ever shows up, we'll be able to use
>> these test cases in more concrete ways too - you're building up our
>> resources for the future.
>>
>> * Of course, we have work to do on improving the release criteria and
>> validation test cases. Johann is right that Fedora.next introduces some
>> uncertainty here, but the majority of our validation process covers
>> stuff that's more than likely going to be in the 'shared base' of the
>> new Fedora.next 'products', so I don't expect there'll be *too* much
>> change - we'll probably still be validating the shared base as usual.
>> I'm trying to do some of this, but it always helps to have more folks.
>> We came across several pain points in the criteria and test cases during
>> F20 validation, and it'd be good to resolve some of those. You can
>> always poke through blocker bug meeting logs to remind yourself of the
>> issues we came up against - what I've been doing is browsing through
>> http://meetbot.fedoraproject.org/fedora-blocker-review - the F20
>> meetings start from 2013-08-21 - and just skim-reading the HTML full
>> logs of each meeting; you can easily see where the discussion of each
>> bug starts from the bolded lines. When the discussion lasts only a few
>> lines and finishes quick, we probably didn't have any trouble deciding
>> the status of that bug, so you don't need to worry about it. If the
>> discussion of a bug goes on for pages and pages, it's a good hint that
>> that bug was a 'pain point': we might be able to improve our criteria
>> and/or test cases to make the situation clearer. We welcome submissions
>> of proposed improvements to the validation process from anyone, there's
>> no secret club you need to be in to submit ideas! Proposing a new
>> release criterion or test case is really as simple as writing it down
>> and mailing it to the list with an explanation - you can do your test
>> case just as plain text if you don't want to deal with all the wiki
>> stuff, or you can create it in your personal space on the wiki and link
>> to it from your proposal email. It's probably a good idea to include the
>> word 'proposal' and/or 'criteria' and/or 'validation' in the subject (I
>> tend to search for those 'magic words' when looking through the list
>> archives for such proposals).
>>
>> Thanks for all your continued work, everyone!
>> -- 
>> Adam Williamson
>> Fedora QA Community Monkey
>> IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
>> http://www.happyassassin.net
>>
>> -- 
>> test mailing list
>> test at lists.fedoraproject.org
>> To unsubscribe:
>> https://admin.fedoraproject.org/mailman/listinfo/test
> 
> 
> 

-- 
Joe Brockmeier | Principal Cloud & Storage Analyst
jzb at redhat.com | http://community.redhat.com/
Twitter: @jzb  | http://dissociatedpress.net/