----- Original Message -----
From: "Adam Williamson" adamwill@fedoraproject.org To: "For testing and quality assurance of Fedora releases" test@lists.fedoraproject.org Sent: Thursday, April 12, 2018 3:57:33 AM Subject: Re: [Proposal] Test Day Guidelines
On Wed, 2018-04-11 at 03:23 -0400, Sumantro Mukherjee wrote:
Hey All,
I would like to make a proposal for updating and fleshing out some of test day guideline.
To start with the wording on the Top of Test Day page: If you come to this page before or after the test day is completed, your testing is still valuable, and you can use the information on this page to test, file any bugs you find at Bugzilla, and add your results to the results section. If this page is more than a month old when you arrive here, please check the current schedule and see if a similar but more recent Test Day is planned or has already happened.
# Proposal 1: We would also like people to *not* do testing if we are testing something which are compose sensitive and post-test day chasing and finding the same bugs in the compose and filing dups won't make much sense. So, it will be best to reduce the time frame from a month to a week. Also, check the current schedule for something fresher and ready requiring testing love.
I picked 'a month' more or less arbitrarily, 'a week' is certainly an OK choice too. Note that the template really is just a *template*, and can (and should!) be adapted to each event as appropriate. So if *some specific event* is very time sensitive, this text could be changed as appropriate - to say 'a day' or whatever.
True. I will keep this in mind from next time :)
# Proposal 2: We would love to see more coverage on the test cases, let's take Modularity and as an example, Contributors found bugs in Rpi 3 (ARM) which weren't the case with x86_64, Profile field in the test day results page becomes very important and crucial. Maybe, we should avoid putting "Fedora-Server-dvd-xxxxxyyyyyyz.n.0.iso" which doesn't give much info about the testing environment or how it was executed. We might want to flesh out the importance of that field and it will be very helpful in coming future.
I'm actually not quite sure what you're referring to here, or where the "Fedora-Server-dvd-xxxxxyyyyyyz.n.0.iso" example comes from. Could you provide a bit more context?
The context where it stems from was, if you have a look at http://testdays.fedorainfracloud.org/events/38 we have mostly all testers, mentioning Fedora 28 Server on (KVM,ARM,x86_64,VMware,ppc64le) which gives us a bit more idea if there is a failure. Just writing Fedora server dvd gives us absolutely no idea as to how the test were executed. In one certain case during Modularity test day, a certain issue just came up which was specific to ARM ... the profile also lets us know about the coverage across archs and various virtualizations. I felt, we should, flesh out a bit more about the importance of the same.
In general I'd certainly say that, again, the way in which result reporting is set up should be carefully considered for each individual test day; the template and SOP are only guides. You can set up a result table in many different ways to capture results from different combinations of environment factors...
yes, thats something I will look at :)
# Proposal 3: Test Days have a specific team which takes on the task of curating the #fedora-test-day channel and these are mostly the people who are working on the feature being tested or developed for a significant amount of time. This is done to ensure that all the contributors/participants can pin point folks who are playing the role they are designed for.It's in the best interest of contributors that they shouldn't edit this section as they misleading if done without proper knowledge.
Are you saying there should be a specifically-identified 'team' for each individual test day, or an ongoing permanent 'team' for test days as a whole? Either could potentially work, but I think the proposal could do with a few more details...
No, I am good with either, I am strictly against people editing the organizing list. The team is defined and kept there to make sure the contributors know where to come and whom to grab hold of , when something is breaking. Randomly editing the section renders to pupose moot. Context: [http://fedoraproject.org/wiki/Test_Day:2018-02-22_Kernel_4.15_Test_Day] [http://fedoraproject.org/wiki/Test_Day:2018-04-10_Add-On_Modularity_Test_Day]
-- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net _______________________________________________ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-leave@lists.fedoraproject.org