On 8/21/20 8:06 AM, Kamil Paral wrote:
Hello folks,
Hey Kamil!
wanted to thank you all for supporting the FCOS Test Day which turned
out to be a very successful event! As we looked back, we thought there might be room for
more community participation in FCOS QA. On those lines, we were wondering if there are
good ways for other Fedora teams and for the community in large to get a good picture of
how quality validation and decisions are done for the FCOS stable stream, and how people
who are interested can participate in this process.
In particular, I tried to find some documentation on:
* The pool of bugs which are currently reported against the testing and next stream
* The subset of aforementioned bugs which were decided to be "blockers" and
must be fixed before the stream can be promoted to stable
* Where this blocker decision-making takes place and how people can get involved
* What are the procedures and rules for this blocker decision-making process
Unfortunately, I was not able to find any. I might be looking at a wrong location. Can
you please point me to the correct place, if such documentation exists?
Our high level bugs mostly exist in the issue tracker. You are right we could use more
transparency in our process.
https://github.com/coreos/fedora-coreos-tracker/issues
Since FCOS might soon be an official Fedora edition, it would be great to have insight
into the FCOS community release process. But this is far from just a QA team thing. This
process transparency is important for recruiting new passionate community members who can
help you run and improve things. Another use case might be a party affected by a bug who
might feel motivated to visit such a blocker discussion and argue their case about why
they think such a bug should not hit the stable stream.
+1. That's a good point. Obviously our release process happens more often and we
don't want to add a lot of
manual "process" around it because then it gets hard to manage, but I think we
should improve.
We can put together all the relevant documents in one place that is accessible to the
public so that the community is aware of important decisions made during the release
process and feel included. I imagine that there could be a "Participate!"
section in the FCOS docs page [1] which would point people to areas where they can help
FCOS, and one of those areas could be quality control. What do you think?
I think the most organic way people can get this information into the community today is
just open a tracker issue
and come to our meeting. For example some people raised issues with the new version of
podman and we're currently
holding it back in FCOS. See the following two issues:
-
https://github.com/coreos/fedora-coreos-tracker/issues/560
-
https://github.com/coreos/fedora-coreos-tracker/issues/575
As an example, the current quality validation workflow in Fedora includes a regular
"blocker review" meeting in a dedicated IRC channel, where bugs proposed as
"blockers" get discussed [2][3]. Anyone can propose a bug as a blocker and we
regularly receive bug nominations from the community. The blocker review meeting is open
to anybody and we try to base our votes on "release criteria" [4] after
assessing that particular case and its circumstances. Release criteria help us bring a bit
more consistency into this blocker decision-making and it saves us some time from looking
up historical evidence ("how did we vote last time on a similar bug?"). All
currently proposed and accepted blockers are visible from our custom page [5]. This whole
process is a collaborative effort between Release Engineering, Quality Assurance,
Development, and Project Management. Please note that while I'm describing our current
process as an example, I'm not implying you need to do it in the same way.
I'm just showing how we do it in general. Also, our documentation is quite complex
and lengthy, but essentially it can be just a couple of paragraphs with links, to steer
people into the right direction.
Looking forward to hearing your thoughts. Thanks!
The existing process for fedora is well thought out but maybe a little overwhelming for us
for right now. Clement, who
recently joined our team, is going to try to help us with this in the future.
In the mean time, I think the best way to go about making change here would be for some
concrete proposals for how we
could implement some baby steps to increase transparency. Kamil, do you have any
suggestions there?
Thanks!
Dusty