For the Fedora Magazine: Suggested QA activities during the pre-F21 lull

Wed Jan 22 14:21:34 UTC 2014

Hi,

Adam sent this out to the QA list. I'm wondering if either a summary, or
the entire text would make a good post for the magazine?

-------- Forwarded Message --------
> From: Adam Williamson <awilliam at redhat.com>
> Reply-to: For testing and quality assurance of Fedora releases
> <test at lists.fedoraproject.org>
> To: test at lists.fedoraproject.org
> Subject: Suggested activities during the pre-F21 lull
> Date: Mon, 20 Jan 2014 17:49:43 -0800
> 
> Hi folks! As a throwaway at the end of the meeting today I mentioned a
> few tasks we could be doing during this 'quiet time' between F20 and
> F21. People seemed interested, and so I promised to send out a more
> detailed version of the list with links and references and stuff. This
> (obviously!) isn't a set of orders, and it's not really even a 'top
> priority' list, just my little list of ideas for things you could work
> on if you're looking to spend some time on Fedora but aren't sure what
> to do when we don't have Test Days or validation testing running.
> 
> * It's not glamorous or exciting, but it always needs doing: test
> updates-testing on the stable releases (F19 and F20) and file feedback.
> You can always use a virtual machine to test a release you don't have
> installed - you can't check everything in a VM, but you can do a lot of
> things, and sometimes you can test stuff in a VM that you wouldn't want
> to mess with on your main system.
> https://fedoraproject.org/wiki/QA:Updates_Testing has all the info you
> need on doing updates-testing work: if you don't use fedora-easy-karma -
> https://fedoraproject.org/wiki/Fedora_Easy_Karma - or gooey karma -
> stuck in package review ATM, but there's a Copr at
> http://copr.fedoraproject.org/coprs/blaskovic/fedora-gooey-karma/builds/
> - consider doing so, it'll really help!
> 
> * You can always help test Rawhide - the development tree of Fedora, so
> right now, the current state of Fedora 21. All the details on Rawhide
> are at https://fedoraproject.org/wiki/Releases/Rawhide . Depending on
> how brave/experienced you are (and how many forms of backup you have!),
> you can even run it in production (the machine I'm typing this on has
> been running Rawhide for several weeks now...but I have backups of
> everything, webmail to use when Evolution breaks, Xfce to use when GNOME
> breaks, a Fedora 20 laptop to use if Rawhide breaks and a Fedora 19
> laptop to use if Fedora 20 breaks ;>). If that's a bit too much for you,
> you can test the nightly live images from
> http://alt.fedoraproject.org/pub/alt/nightly-composes/ , or run it in a
> VM. Once you're actually running Rawhide, file any significant bugs you
> encounter just as usual, and if you run into any real showstoppers, you
> can propose them as Fedora 21 blockers following the usual process
> ( https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process ) - the
> blocker bug app has not yet been updated for F21 so you can't use that,
> but all you have to do is mark the bug as blocking "AlphaBlocker",
> "BetaBlocker" or "FinalBlocker". If you see any unusual or unexpected
> changes but you're not sure if they're bugs, post about them here (on
> test@), it's one of the purposes of the list!
> 
> * One thing we never seem to quite do enough of is creating
> package-specific test cases: again this isn't highly visible work, but
> it is useful to do it. The details on doing it are at
> https://fedoraproject.org/wiki/QA:SOP_package_test_plan_creation (and
> https://fedoraproject.org/wiki/QA:SOP_test_case_creation explains how to
> write a test case). If you write some test cases and 'associate' them
> with a package, as the SOPs explain, they will be attached to every
> Bodhi update for that package, and people doing updates-testing testing
> will see your test cases as something they can use to check the package
> is working properly. If Bodhi 2.0 ever shows up, we'll be able to use
> these test cases in more concrete ways too - you're building up our
> resources for the future.
> 
> * Of course, we have work to do on improving the release criteria and
> validation test cases. Johann is right that Fedora.next introduces some
> uncertainty here, but the majority of our validation process covers
> stuff that's more than likely going to be in the 'shared base' of the
> new Fedora.next 'products', so I don't expect there'll be *too* much
> change - we'll probably still be validating the shared base as usual.
> I'm trying to do some of this, but it always helps to have more folks.
> We came across several pain points in the criteria and test cases during
> F20 validation, and it'd be good to resolve some of those. You can
> always poke through blocker bug meeting logs to remind yourself of the
> issues we came up against - what I've been doing is browsing through
> http://meetbot.fedoraproject.org/fedora-blocker-review - the F20
> meetings start from 2013-08-21 - and just skim-reading the HTML full
> logs of each meeting; you can easily see where the discussion of each
> bug starts from the bolded lines. When the discussion lasts only a few
> lines and finishes quick, we probably didn't have any trouble deciding
> the status of that bug, so you don't need to worry about it. If the
> discussion of a bug goes on for pages and pages, it's a good hint that
> that bug was a 'pain point': we might be able to improve our criteria
> and/or test cases to make the situation clearer. We welcome submissions
> of proposed improvements to the validation process from anyone, there's
> no secret club you need to be in to submit ideas! Proposing a new
> release criterion or test case is really as simple as writing it down
> and mailing it to the list with an explanation - you can do your test
> case just as plain text if you don't want to deal with all the wiki
> stuff, or you can create it in your personal space on the wiki and link
> to it from your proposal email. It's probably a good idea to include the
> word 'proposal' and/or 'criteria' and/or 'validation' in the subject (I
> tend to search for those 'magic words' when looking through the list
> archives for such proposals).
> 
> Thanks for all your continued work, everyone!
> -- 
> Adam Williamson
> Fedora QA Community Monkey
> IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
> http://www.happyassassin.net
> 
> -- 
> test mailing list
> test at lists.fedoraproject.org
> To unsubscribe:
> https://admin.fedoraproject.org/mailman/listinfo/test

-- 
Thanks,
Warm regards,
Ankur (FranciscoD)

http://fedoraproject.org/wiki/User:Ankursinha

Join Fedora! Come talk to us!
http://fedoraproject.org/wiki/Fedora_Join_SIG

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: This is a digitally signed message part
URL: <http://lists.fedoraproject.org/pipermail/marketing/attachments/20140123/109a966a/attachment.sig>