For the Fedora Magazine: Suggested QA activities during the pre-F21 lull

[Ankur Sinha]

Hi,

Adam sent this out to the QA list. I'm wondering if either a summary, or
the entire text would make a good post for the magazine?

-------- Forwarded Message --------
>  From: Adam Williamson <awilliam(a)redhat.com&gt;
>  Reply-to: For testing and quality assurance of Fedora releases
>  <test(a)lists.fedoraproject.org&gt;
>  To: test(a)lists.fedoraproject.org
>  Subject: Suggested activities during the pre-F21 lull
>  Date: Mon, 20 Jan 2014 17:49:43 -0800
>  
>  Hi folks! As a throwaway at the end of the meeting today I mentioned a
>  few tasks we could be doing during this 'quiet time' between F20 and
>  F21. People seemed interested, and so I promised to send out a more
>  detailed version of the list with links and references and stuff. This
>  (obviously!) isn't a set of orders, and it's not really even a 'top
>  priority' list, just my little list of ideas for things you could work
>  on if you're looking to spend some time on Fedora but aren't sure what
>  to do when we don't have Test Days or validation testing running.
>  
>  * It's not glamorous or exciting, but it always needs doing: test
>  updates-testing on the stable releases (F19 and F20) and file feedback.
>  You can always use a virtual machine to test a release you don't have
>  installed - you can't check everything in a VM, but you can do a lot of
>  things, and sometimes you can test stuff in a VM that you wouldn't want
>  to mess with on your main system.
>  https://fedoraproject.org/wiki/QA:Updates_Testing has all the info you
>  need on doing updates-testing work: if you don't use fedora-easy-karma -
>  https://fedoraproject.org/wiki/Fedora_Easy_Karma - or gooey karma -
>  stuck in package review ATM, but there's a Copr at
>  http://copr.fedoraproject.org/coprs/blaskovic/fedora-gooey-karma/builds/
>  - consider doing so, it'll really help!
>  
>  * You can always help test Rawhide - the development tree of Fedora, so
>  right now, the current state of Fedora 21. All the details on Rawhide
>  are at https://fedoraproject.org/wiki/Releases/Rawhide . Depending on
>  how brave/experienced you are (and how many forms of backup you have!),
>  you can even run it in production (the machine I'm typing this on has
>  been running Rawhide for several weeks now...but I have backups of
>  everything, webmail to use when Evolution breaks, Xfce to use when GNOME
>  breaks, a Fedora 20 laptop to use if Rawhide breaks and a Fedora 19
>  laptop to use if Fedora 20 breaks ;>). If that's a bit too much for you,
>  you can test the nightly live images from
>  http://alt.fedoraproject.org/pub/alt/nightly-composes/ , or run it in a
>  VM. Once you're actually running Rawhide, file any significant bugs you
>  encounter just as usual, and if you run into any real showstoppers, you
>  can propose them as Fedora 21 blockers following the usual process
>  ( https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process ) - the
>  blocker bug app has not yet been updated for F21 so you can't use that,
>  but all you have to do is mark the bug as blocking "AlphaBlocker",
>  "BetaBlocker" or "FinalBlocker". If you see any unusual or
> unexpected
>  changes but you're not sure if they're bugs, post about them here (on
>  test@), it's one of the purposes of the list!
>  
>  * One thing we never seem to quite do enough of is creating
>  package-specific test cases: again this isn't highly visible work, but
>  it is useful to do it. The details on doing it are at
>  https://fedoraproject.org/wiki/QA:SOP_package_test_plan_creation (and
>  https://fedoraproject.org/wiki/QA:SOP_test_case_creation explains how to
>  write a test case). If you write some test cases and 'associate' them
>  with a package, as the SOPs explain, they will be attached to every
>  Bodhi update for that package, and people doing updates-testing testing
>  will see your test cases as something they can use to check the package
>  is working properly. If Bodhi 2.0 ever shows up, we'll be able to use
>  these test cases in more concrete ways too - you're building up our
>  resources for the future.
>  
>  * Of course, we have work to do on improving the release criteria and
>  validation test cases. Johann is right that Fedora.next introduces some
>  uncertainty here, but the majority of our validation process covers
>  stuff that's more than likely going to be in the 'shared base' of the
>  new Fedora.next 'products', so I don't expect there'll be *too* much
>  change - we'll probably still be validating the shared base as usual.
>  I'm trying to do some of this, but it always helps to have more folks.
>  We came across several pain points in the criteria and test cases during
>  F20 validation, and it'd be good to resolve some of those. You can
>  always poke through blocker bug meeting logs to remind yourself of the
>  issues we came up against - what I've been doing is browsing through
>  http://meetbot.fedoraproject.org/fedora-blocker-review - the F20
>  meetings start from 2013-08-21 - and just skim-reading the HTML full
>  logs of each meeting; you can easily see where the discussion of each
>  bug starts from the bolded lines. When the discussion lasts only a few
>  lines and finishes quick, we probably didn't have any trouble deciding
>  the status of that bug, so you don't need to worry about it. If the
>  discussion of a bug goes on for pages and pages, it's a good hint that
>  that bug was a 'pain point': we might be able to improve our criteria
>  and/or test cases to make the situation clearer. We welcome submissions
>  of proposed improvements to the validation process from anyone, there's
>  no secret club you need to be in to submit ideas! Proposing a new
>  release criterion or test case is really as simple as writing it down
>  and mailing it to the list with an explanation - you can do your test
>  case just as plain text if you don't want to deal with all the wiki
>  stuff, or you can create it in your personal space on the wiki and link
>  to it from your proposal email. It's probably a good idea to include the
>  word 'proposal' and/or 'criteria' and/or 'validation' in the
> subject (I
>  tend to search for those 'magic words' when looking through the list
>  archives for such proposals).
>  
>  Thanks for all your continued work, everyone!
>  -- 
>  Adam Williamson
>  Fedora QA Community Monkey
>  IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
>  http://www.happyassassin.net
>  
>  -- 
>  test mailing list
>  test(a)lists.fedoraproject.org
>  To unsubscribe:
>  https://admin.fedoraproject.org/mailman/listinfo/test 
-- 
Thanks,
Warm regards,
Ankur (FranciscoD)

http://fedoraproject.org/wiki/User:Ankursinha

Join Fedora! Come talk to us!
http://fedoraproject.org/wiki/Fedora_Join_SIG

[Joe Brockmeier]

On 01/22/2014 08:21 AM, Ankur Sinha wrote:
>  Hi,
>  
>  Adam sent this out to the QA list. I'm wondering if either a summary, or
>  the entire text would make a good post for the magazine? 
A summary that points to the mailing list is probably the best way to
go. For stuff like this, I think we can safely emulate LWN's approach -
quick summary, quote from the mail, and link to the mailing list.

Best,

jzb

>  -------- Forwarded Message --------
> > From: Adam Williamson <awilliam(a)redhat.com&gt;
> > Reply-to: For testing and quality assurance of Fedora releases
> > <test(a)lists.fedoraproject.org&gt;
> > To: test(a)lists.fedoraproject.org
> > Subject: Suggested activities during the pre-F21 lull
> > Date: Mon, 20 Jan 2014 17:49:43 -0800
> >
> > Hi folks! As a throwaway at the end of the meeting today I mentioned a
> > few tasks we could be doing during this 'quiet time' between F20 and
> > F21. People seemed interested, and so I promised to send out a more
> > detailed version of the list with links and references and stuff. This
> > (obviously!) isn't a set of orders, and it's not really even a 'top
> > priority' list, just my little list of ideas for things you could work
> > on if you're looking to spend some time on Fedora but aren't sure what
> > to do when we don't have Test Days or validation testing running.
> >
> > * It's not glamorous or exciting, but it always needs doing: test
> > updates-testing on the stable releases (F19 and F20) and file feedback.
> > You can always use a virtual machine to test a release you don't have
> > installed - you can't check everything in a VM, but you can do a lot of
> > things, and sometimes you can test stuff in a VM that you wouldn't want
> > to mess with on your main system.
> > https://fedoraproject.org/wiki/QA:Updates_Testing has all the info you
> > need on doing updates-testing work: if you don't use fedora-easy-karma -
> > https://fedoraproject.org/wiki/Fedora_Easy_Karma - or gooey karma -
> > stuck in package review ATM, but there's a Copr at
> > http://copr.fedoraproject.org/coprs/blaskovic/fedora-gooey-karma/builds/
> > - consider doing so, it'll really help!
> >
> > * You can always help test Rawhide - the development tree of Fedora, so
> > right now, the current state of Fedora 21. All the details on Rawhide
> > are at https://fedoraproject.org/wiki/Releases/Rawhide . Depending on
> > how brave/experienced you are (and how many forms of backup you have!),
> > you can even run it in production (the machine I'm typing this on has
> > been running Rawhide for several weeks now...but I have backups of
> > everything, webmail to use when Evolution breaks, Xfce to use when GNOME
> > breaks, a Fedora 20 laptop to use if Rawhide breaks and a Fedora 19
> > laptop to use if Fedora 20 breaks ;>). If that's a bit too much for you,
> > you can test the nightly live images from
> > http://alt.fedoraproject.org/pub/alt/nightly-composes/ , or run it in a
> > VM. Once you're actually running Rawhide, file any significant bugs you
> > encounter just as usual, and if you run into any real showstoppers, you
> > can propose them as Fedora 21 blockers following the usual process
> > ( https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process ) - the
> > blocker bug app has not yet been updated for F21 so you can't use that,
> > but all you have to do is mark the bug as blocking "AlphaBlocker",
> > "BetaBlocker" or "FinalBlocker". If you see any unusual or
> unexpected
> > changes but you're not sure if they're bugs, post about them here (on
> > test@), it's one of the purposes of the list!
> >
> > * One thing we never seem to quite do enough of is creating
> > package-specific test cases: again this isn't highly visible work, but
> > it is useful to do it. The details on doing it are at
> > https://fedoraproject.org/wiki/QA:SOP_package_test_plan_creation (and
> > https://fedoraproject.org/wiki/QA:SOP_test_case_creation explains how to
> > write a test case). If you write some test cases and 'associate' them
> > with a package, as the SOPs explain, they will be attached to every
> > Bodhi update for that package, and people doing updates-testing testing
> > will see your test cases as something they can use to check the package
> > is working properly. If Bodhi 2.0 ever shows up, we'll be able to use
> > these test cases in more concrete ways too - you're building up our
> > resources for the future.
> >
> > * Of course, we have work to do on improving the release criteria and
> > validation test cases. Johann is right that Fedora.next introduces some
> > uncertainty here, but the majority of our validation process covers
> > stuff that's more than likely going to be in the 'shared base' of the
> > new Fedora.next 'products', so I don't expect there'll be *too* much
> > change - we'll probably still be validating the shared base as usual.
> > I'm trying to do some of this, but it always helps to have more folks.
> > We came across several pain points in the criteria and test cases during
> > F20 validation, and it'd be good to resolve some of those. You can
> > always poke through blocker bug meeting logs to remind yourself of the
> > issues we came up against - what I've been doing is browsing through
> > http://meetbot.fedoraproject.org/fedora-blocker-review - the F20
> > meetings start from 2013-08-21 - and just skim-reading the HTML full
> > logs of each meeting; you can easily see where the discussion of each
> > bug starts from the bolded lines. When the discussion lasts only a few
> > lines and finishes quick, we probably didn't have any trouble deciding
> > the status of that bug, so you don't need to worry about it. If the
> > discussion of a bug goes on for pages and pages, it's a good hint that
> > that bug was a 'pain point': we might be able to improve our criteria
> > and/or test cases to make the situation clearer. We welcome submissions
> > of proposed improvements to the validation process from anyone, there's
> > no secret club you need to be in to submit ideas! Proposing a new
> > release criterion or test case is really as simple as writing it down
> > and mailing it to the list with an explanation - you can do your test
> > case just as plain text if you don't want to deal with all the wiki
> > stuff, or you can create it in your personal space on the wiki and link
> > to it from your proposal email. It's probably a good idea to include the
> > word 'proposal' and/or 'criteria' and/or 'validation' in the
> subject (I
> > tend to search for those 'magic words' when looking through the list
> > archives for such proposals).
> >
> > Thanks for all your continued work, everyone!
> > -- 
> > Adam Williamson
> > Fedora QA Community Monkey
> > IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
> > http://www.happyassassin.net
> >
> > -- 
> > test mailing list
> > test(a)lists.fedoraproject.org
> > To unsubscribe:
> > https://admin.fedoraproject.org/mailman/listinfo/test
>  
>  
>   

-- 
Joe Brockmeier | Principal Cloud & Storage Analyst
jzb(a)redhat.com | http://community.redhat.com/
Twitter: @jzb  | http://dissociatedpress.net/

[Ankur Sinha]

On Wed, 2014-01-22 at 08:43 -0600, Joe Brockmeier wrote:
>  A summary that points to the mailing list is probably the best way
> to
>  go. For stuff like this, I think we can safely emulate LWN's approach
>  -
>  quick summary, quote from the mail, and link to the mailing list.
>  
>  Best,
>  
>  jzb 
Aye. I'll try to write up the summary sometime before the weekend. 

Do we have a proper procedure to submit articles for consideration to
the magazine? Do we just send them here to the list, or do we file trac
tickets? If we want the entire community to send stuff to the magazine,
maybe the trac will be a good place? We could add a template and a
component to make it easier?
-- 
Thanks,
Warm regards,
Ankur (FranciscoD)

http://fedoraproject.org/wiki/User:Ankursinha

Join Fedora! Come talk to us!
http://fedoraproject.org/wiki/Fedora_Join_SIG

[Joe Brockmeier]

On 01/23/2014 07:10 AM, Ankur Sinha wrote:
>  On Wed, 2014-01-22 at 08:43 -0600, Joe Brockmeier wrote:
> > A summary that points to the mailing list is probably the best way to
> > go. For stuff like this, I think we can safely emulate LWN's approach
> > -
> > quick summary, quote from the mail, and link to the mailing list.
> >
> > Best,
> >
> > jzb
>  
>  Aye. I'll try to write up the summary sometime before the weekend. 
>  
>  Do we have a proper procedure to submit articles for consideration to
>  the magazine? Do we just send them here to the list, or do we file trac
>  tickets? If we want the entire community to send stuff to the magazine,
>  maybe the trac will be a good place? We could add a template and a
>  component to make it easier? 
Let's keep this simple. We have a howto for posting to WP here:

http://fedoraproject.org/wiki/Magazine/Howtopost

If you're going to be contributing regularly, ask on the list.

If you are sending a one-off or don't think you'll need to contribute
regularly, you can just send properly formatted HTML, Markdown, or plain
text to the list in an email with the subject:

Fedora Magazine Submission: [$subject]

Someone with proper credentials should have it up quickly. If it's not
up or at least acknowledged w/in 24 hours, feel free to ping the list.
(Maybe more on weekends...)

Sound reasonable? I can put this on the wiki if we're agreed.

Best,

jzb
-- 
Joe Brockmeier | Principal Cloud & Storage Analyst
jzb(a)redhat.com | http://community.redhat.com/
Twitter: @jzb  | http://dissociatedpress.net/