6:04 p.m.
Hi folks! So the F35 retrospective has been up for a while:
https://fedoraproject.org/wiki/Fedora_35_QA_Retrospective
I thought I'd send out my planned set of actions in response to the
points raised there. Please suggest any improvements you can think of.
I'll also put this on the agenda for the meeting on Monday, then I'll
go ahead and implement these (with any improvements that are suggested)
next week.
1. "adamwill - It took a long time to work through all the iterations
of the fedora-third-party/SELinux bug. Seems like we could have
tightened the loop on that somehow." - I don't think there's any
specific action to take immediately here, but if a similar situation
occurs in future I intend to be more hands-on about asking the relevant
component developers and the SELinux developers to work faster to get
to the bottom of the problem pile sooner.
2. "adamwill - Movement on the KDE blockers was initially slow, if we
could have resolved them earlier we would have had more time to find
and fix the bugs that eventually delayed Final." - I think we kinda
addressed this during the cycle, by pulling in some folks who were not
really involved in the blocker fix cycle previously. For F36 and later
we now have more KDE folks we can tap to get problems solved promptly.
3. "Ahmedalmeleh - Bugzilla is challenging to me and still getting used
to it. I wish I was able to learn how to operate OpenQA's automated
tests, in the given timeframe and was given guidance sooner.", also
Ahmed's 'wishlist' item and Matt's item - I'll plan to file a ticket
for improved guidance on using Bugzilla, and ask Ahmed for specific
input on what might help. I wonder if it might be a good idea to drag
ourselves into the 21st century and record some videos covering common
QA activities too, that might be something some folks would appreciate
more than text instructions? For openQA, I'll ask Ahmed for more detail
on specifically what he'd like to do with it here, as "operating" the
tests isn't something we envisage most contributors typically needing
to do.
4. "kparal - After F34, we talked in our team about doing a selected
compose testing in a complete full run (all test cases), way ahead of
the milestones (Beta and Final), and as a group effort to encourage
people to really fill out all test cases. Yet we never got to it in
F35. Some of the bugs might've been discovered earlier this way." - I
wonder if we could do this as a "test day" or "test week", or two:
schedule one event around Beta freeze and one event around Final
freeze, and have the goal of the event be to fill out the test
matrices. What do people think of this idea? If it seems like a good
one, I'll file tickets for creating those events.
5. "Mattdm - The timing of KDE Plasma releases is not well-aligned with
our schedule..." - this honestly doesn't seem like something QA can
really take action on, maybe it's more suited to the wider
retrospective Ben has been talking about?
6. "Mattdm - Lots of people (as non-scientifically measured by my
personal impressions of help requests I'm seeing) are hitting the
wireplumber-not-enabled problem. Maybe we should have blocked on that,
actually." - I've already updated the commonbugs note on this, but
another action we could take is to test upgrades from an older release
and see if we can spot the "pulseaudio still enabled" case happening
and if so what triggers it. Aside from that, I think we'll just have to
keep this one on record as an example to keep in mind when making these
kinds of subjective blocker calls in future.
7. "sumantrom - Sending some testing event updates to our "users" list.
I see surge in contributors when there is an article posted in Fedora
Magazine which means Fedora's user base wants to contribute/ try out
things. Posting things in the user-list will help." - this seems like a
good idea for sure. If we still work from an SOP for organizing these
kinds of events, a concrete action would be to add this step to the
SOP, I can file a ticket for that.
8. "sumantrom - Onboarding session on Bugzilla and test case writing."
- again this seems like a good idea, and we can file a ticket to have
such an event. Perhaps we could also link this up with point 3) and
record the session to use, perhaps after editing it down, as a "getting
started with Bugzilla" video?
9. "frantisekz - Some testing could have been conducted earlier in the
development, especially KDE/Discover blockers and bugs have been
present for a long time before the release." - I think the practical
step we can take here is already in progress: add more specific package
manager criteria and test cases. See the "new criterion proposal:
Graphical package managers" thread. Once the new criteria are decided,
we can create new test cases or extend existing ones, and once that's
done, the testing should get done more often and earlier. We can even
automate it. I can file tickets for those actions.
How does this sound to everyone? Please add any suggestions you have!
And there's still time to add new items to the retrospective itself, if
any new ones show up, I'll update this list of proposed actions.
Thanks everyone!
--
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net