#171: [VOTE] To mark recent failure on Atomic images as non blocking --------------------+-------------------- Reporter: kushal | Owner: Type: task | Status: new Priority: normal | Milestone: Future Component: --- | Keywords: --------------------+-------------------- You can view the error at https://apps.fedoraproject.org/autocloud/jobs/606/output#227
Related bugzilla bus
* https://bugzilla.redhat.com/show_bug.cgi?id=1265295 * https://bugzilla.redhat.com/show_bug.cgi?id=1353688
We want to mark these issues as non-blocking.
Can the WG members please vote in? (Aye, Nay, Abstain).
#171: [VOTE] To mark recent failure on Atomic images as non blocking --------------------+--------------------- Reporter: kushal | Owner: Type: task | Status: new Priority: normal | Milestone: Future Component: --- | Resolution: Keywords: | --------------------+---------------------
Comment (by kushal):
First I should vote: Aye (to make them non-blocking).
#171: [VOTE] To mark recent failure on Atomic images as non blocking --------------------+--------------------- Reporter: kushal | Owner: Type: task | Status: new Priority: normal | Milestone: Future Component: --- | Resolution: Keywords: | --------------------+---------------------
Comment (by trishnag):
What is the point of having the testcase if we want to release images even though they bugs?
This testcase is also **not a non-gating test** AFAIK. https://github.com/kushaldas/tunirtests/blob/master/cloudtests.py#L61-L98
#171: [VOTE] To mark recent failure on Atomic images as non blocking --------------------+--------------------- Reporter: kushal | Owner: Type: task | Status: new Priority: normal | Milestone: Future Component: --- | Resolution: Keywords: | --------------------+---------------------
Comment (by dustymabe):
Replying to [comment:2 trishnag]:
What is the point of having the testcase if we want to release images
even though they have bugs?
So that we can find and fix bugs faster when they do happen. If we choose to release with the bug anyway then we know it exists and can guide users accordingly until it is fixed.
This testcase is also **not a non-gating test** AFAIK.
https://github.com/kushaldas/tunirtests/blob/master/cloudtests.py#L61-L98
Does that link prove it is a non-gating test? Where is the list of gating vs non-gating?
#171: [VOTE] To mark recent failure on Atomic images as non blocking --------------------+--------------------- Reporter: kushal | Owner: Type: task | Status: new Priority: normal | Milestone: Future Component: --- | Resolution: Keywords: | --------------------+---------------------
Comment (by dustymabe):
I vote aye (+1) to release with this known bug.
#171: [VOTE] To mark recent failure on Atomic images as non blocking --------------------+--------------------- Reporter: kushal | Owner: Type: task | Status: new Priority: normal | Milestone: Future Component: --- | Resolution: Keywords: | --------------------+---------------------
Comment (by trishnag):
Replying to [comment:3 dustymabe]:
Replying to [comment:2 trishnag]:
What is the point of having the testcase if we want to release images
even though they have bugs?
So that we can find and fix bugs faster when they do happen. If we
choose to release with the bug anyway then we know it exists and can guide users accordingly until it is fixed.
Okay, if we choose to release with the bug anyway does that mean that the bug is not going to affect users much?
This testcase is also **not a non-gating test** AFAIK.
https://github.com/kushaldas/tunirtests/blob/master/cloudtests.py#L61-L98
Does that link prove it is a non-gating test? Where is the list of
gating vs non-gating? We put non-gating tests here https://github.com/kushaldas/tunirtests/blob/master/nongatingtests.py Rest of the files have gating tests. Journal logging test is a gating test. So we have it in a separate file.
#171: [VOTE] To mark recent failure on Atomic images as non blocking --------------------+--------------------- Reporter: kushal | Owner: Type: task | Status: new Priority: normal | Milestone: Future Component: --- | Resolution: Keywords: | --------------------+---------------------
Comment (by dustymabe):
Replying to [comment:5 trishnag]:
Replying to [comment:3 dustymabe]:
Replying to [comment:2 trishnag]:
What is the point of having the testcase if we want to release
images even though they have bugs?
So that we can find and fix bugs faster when they do happen. If we
choose to release with the bug anyway then we know it exists and can guide users accordingly until it is fixed.
Okay, if we choose to release with the bug anyway does it mean that the
bug is not going to affect users much?
Yeah. The idea is that we put items to a vote and determine the severity. This bug is not that severe as it is just a logging issue. While important, it shouldn't block release in this case IMHO. My opinion could change in the future, but since this has been going out in our images for some time now we aren't making the currently released image any worse by releasing the next image.
This testcase is also **not a non-gating test** AFAIK.
https://github.com/kushaldas/tunirtests/blob/master/cloudtests.py#L61-L98
Does that link prove it is a non-gating test? Where is the list of
gating vs non-gating?
We put non-gating tests here
https://github.com/kushaldas/tunirtests/blob/master/nongatingtests.py
Rest of the files have gating tests. Journal logging test is a gating
test. So we have it in a separate file.
Thanks
#171: [VOTE] To mark recent failure on Atomic images as non blocking --------------------+--------------------- Reporter: kushal | Owner: Type: task | Status: new Priority: normal | Milestone: Future Component: --- | Resolution: Keywords: | --------------------+---------------------
Comment (by jberkus):
So:
My reason for unblocking these tests is simple:
The version we are *currently distributing* also fails these tests.
If we had a version on getfedora which passed the tests, and it was only the pending version which was failing, I'd keep them blocking. But I see no point in continuing to block on tests which the "stable" version also doesn't pass. Clearly we've decided it's acceptable to ship a version of Fedora Atomic which fails the log tests.
#171: [VOTE] To mark recent failure on Atomic images as non blocking --------------------+--------------------- Reporter: kushal | Owner: Type: task | Status: new Priority: normal | Milestone: Future Component: --- | Resolution: Keywords: | --------------------+---------------------
Comment (by trishnag):
Replying to [comment:6 dustymabe]:
Replying to [comment:5 trishnag]:
Replying to [comment:3 dustymabe]:
Replying to [comment:2 trishnag]:
What is the point of having the testcase if we want to release
images even though they have bugs?
So that we can find and fix bugs faster when they do happen. If we
choose to release with the bug anyway then we know it exists and can guide users accordingly until it is fixed.
Okay, if we choose to release with the bug anyway does it mean that
the bug is not going to affect users much?
Yeah. The idea is that we put items to a vote and determine the
severity. This bug is not that severe as it is just a logging issue. While important, it shouldn't block release in this case IMHO. My opinion could change in the future, but since this has been going out in our images for some time now we aren't making the currently released image any worse by releasing the next image. Got you. Thanks dustymabe.
This testcase is also **not a non-gating test** AFAIK.
https://github.com/kushaldas/tunirtests/blob/master/cloudtests.py#L61-L98
Does that link prove it is a non-gating test? Where is the list of
gating vs non-gating?
We put non-gating tests here
https://github.com/kushaldas/tunirtests/blob/master/nongatingtests.py
Rest of the files have gating tests. Journal logging test is a gating
test. So we have it in a separate file.
Thanks
#171: [VOTE] To mark recent failure on Atomic images as non blocking --------------------+--------------------- Reporter: kushal | Owner: Type: task | Status: new Priority: normal | Milestone: Future Component: --- | Resolution: Keywords: | --------------------+---------------------
Comment (by mattdm):
Replying to [comment:7 jberkus]:
If we had a version on getfedora which passed the tests, and it was only
the pending version which was failing, I'd keep them blocking. But I see no point in continuing to block on tests which the "stable" version also doesn't pass. Clearly we've decided it's acceptable to ship a version of Fedora Atomic which fails the log tests.
This implies a second decision to be made: once there ''is'' a working version, should the non-blocking test become a blocking one?
#171: [VOTE] To mark recent failure on Atomic images as non blocking --------------------+--------------------- Reporter: kushal | Owner: Type: task | Status: new Priority: normal | Milestone: Future Component: --- | Resolution: Keywords: | --------------------+---------------------
Comment (by jberkus):
My inclination is to say "non-blocking" because I do a lot of testing on Atomic Host and never noticed the bug, personally. I generally figure that gating bugs should be ones which either (a) carry significant security risk or (b) make the system unusable, or significantly less usable, than the prior release.
However, surely the Fedora project has some standards around what's considered a gating bug?
#171: [VOTE] To mark recent failure on Atomic images as non blocking --------------------+--------------------- Reporter: kushal | Owner: Type: task | Status: new Priority: normal | Milestone: Future Component: --- | Resolution: Keywords: | --------------------+---------------------
Comment (by mattdm):
Replying to [comment:10 jberkus]:
However, surely the Fedora project has some standards around what's
considered a gating bug?
We do for the traditional 6-month releases:
https://fedoraproject.org/wiki/Fedora_Release_Criteria
For example, functional system logging is considered to block an alpha release:
https://fedoraproject.org/wiki/Fedora_25_Alpha_Release_Criteria#System_loggi...
The Atomic release doesn't have to follow the exact same rules, of course, but I'd highly suggest keeping in the ballpark and coordinating with QA for expertise.
#171: [VOTE] To mark recent failure on Atomic images as non blocking --------------------+--------------------- Reporter: kushal | Owner: Type: task | Status: new Priority: normal | Milestone: Future Component: --- | Resolution: Keywords: | --------------------+---------------------
Comment (by jberkus):
Yeah, so then the recommendation would be to unblock until this bug is fixed, but then make it a gating issue thereafter.
#171: [VOTE] To mark recent failure on Atomic images as non blocking --------------------+--------------------- Reporter: kushal | Owner: Type: task | Status: new Priority: normal | Milestone: Future Component: --- | Resolution: Keywords: | --------------------+---------------------
Comment (by sayanchowdhury):
Replying to [comment:12 jberkus]:
Yeah, so then the recommendation would be to unblock until this bug is
fixed, but then make it a gating issue thereafter.
Agree. I wonder if we can have a concrete release criteria as emphasis on a issue can change quickly.
I don't know if my vote counts but I am +1 (Aye) to mark these as non- blocking.
#171: [VOTE] To mark recent failure on Atomic images as non blocking --------------------+--------------------- Reporter: kushal | Owner: Type: task | Status: new Priority: normal | Milestone: Future Component: --- | Resolution: Keywords: | --------------------+---------------------
Comment (by trishnag):
Replying to [comment:12 jberkus]:
Yeah, so then the recommendation would be to unblock until this bug is
fixed, but then make it a gating issue thereafter. +1 to this.
#171: [VOTE] To mark recent failure on Atomic images as non blocking --------------------+--------------------- Reporter: kushal | Owner: Type: task | Status: closed Priority: normal | Milestone: Future Component: --- | Resolution: fixed Keywords: | --------------------+--------------------- Changes (by kushal):
* status: new => closed * resolution: => fixed
Comment:
Done in commit https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=5e5354b...
#171: [VOTE] To mark recent failure on Atomic images as non blocking --------------------+--------------------- Reporter: kushal | Owner: Type: task | Status: closed Priority: normal | Milestone: Future Component: --- | Resolution: fixed Keywords: | --------------------+---------------------
Comment (by kushal):
Worked as a non gating test.
https://apps.fedoraproject.org/autocloud/jobs/634/output#235
-
Fedora Cloud Trac Tickets