Related to a recently proposed blocker [0], I propose a new release criterion which would cover that case. There is a push from the Dekstop team to have toolbox as a default app in the installer iso as well. Here it is:
~~~~~~~~~~~~ Toolbox aka Toolbx should be usable. Toolbox should be able to list, create and provide the desired containerized environment to the user without root privileges. Toolbox images should be generally functional.
Footnote - "What does this cover?": Toolbox's general functionality as covered by this test cases https://fedoraproject.org/wiki/QA:Testcase_toolbox ~~~~~~~~~~~~~
I think this criterion could target F39 Beta Release Criteria or Basic Release Criteria [2].
Please tell me your thoughts, thanks!
[0] https://bugzilla.redhat.com/show_bug.cgi?id=2183038 [1] https://fedoraproject.org/wiki/Basic_Release_Criteria
On Tue, Apr 11, 2023 at 11:56 PM Sumantro Mukherjee sumukher@redhat.com wrote:
Toolbox aka Toolbx should be usable.
As a style matter, let's just cut this sentence. "usable" is vague and the next sentence offers more clarity.
Toolbox should be able to list, create and provide the desired containerized environment to the user without root privileges.
Do we need to tighten the scope here? What sorts of failures are acceptable? If someone tries to create a toolbox that pulls in software from third-party repositories, I think we'd want to call that not our problem.
Toolbox images should be generally functional.
How does this differ from the previous sentence? Is "generally functional" more than list, create, and provide the containerized environment?
Footnote - "What does this cover?": Toolbox's general functionality as covered by this test cases https://fedoraproject.org/wiki/QA:Testcase_toolbox
The test case seems to be much narrower than the text of the criterion implies. Is the test case insufficient or is the criterion too broad?
I think this criterion could target F39 Beta Release Criteria or Basic Release Criteria [2].
I'd say this is a Beta (or possibly Final, although I'm not sure if I believe that) criterion for now. When/if ostree variants become our primary desktop deliverable, moving it to Basic would make sense to me.
On Wed, 2023-04-12 at 09:00 -0400, Ben Cotton wrote:
Footnote - "What does this cover?": Toolbox's general functionality as covered by this test cases https://fedoraproject.org/wiki/QA:Testcase_toolbox
The test case seems to be much narrower than the text of the criterion implies. Is the test case insufficient or is the criterion too broad?
Also, this isn't really how we do things. The criterion should explain itself. The test case covers the criterion. It's not really procedurally sound for the criterion definition to reference the test case.
I think this criterion could target F39 Beta Release Criteria or Basic Release Criteria [2].
I'd say this is a Beta (or possibly Final, although I'm not sure if I believe that) criterion for now. When/if ostree variants become our primary desktop deliverable, moving it to Basic would make sense to me.
So aside from the good notes Ben provided, there's another problem: how exactly does toolbox functionality intersect with the release process?
We don't produce the toolbox container image as part of the compose process, so what exactly are we covering here? Is this a criterion for the toolbox container image itself? For running toolbox from the media produced as part of the compose?
The latter seems more practicable. In that case I'd probably write the criterion to state that, i.e. something like "For any release-blocking deliverable whose default deployment includes toolbox, it must be possible to..." or something along those lines.
On Wed, Apr 12, 2023 at 5:57 AM Sumantro Mukherjee sumukher@redhat.com wrote:
There is a push from the Dekstop team to have toolbox as a default app in the installer iso as well.
The 'toolbox' package is already installed by default. Or do you mean some related GUI app?
Toolbox aka Toolbx should be usable. Toolbox should be able to list, create and provide the desired containerized environment to the user without root privileges. Toolbox images should be generally functional. Footnote - "What does this cover?": Toolbox's general functionality as covered by this test cases https://fedoraproject.org/wiki/QA:Testcase_toolbox
Already covered by Ben and Adam, so just a few additional notes: * We can't block on toolbox *images*, unless they are marked as release-blocking artifacts. If the Desktop team wants that, Ben will know the right process to take. * We can block on the toolbox cli command. We can say that it must list existing containers, enter them, create a new one, etc. But since the toolbox images are not release-blocking at the moment, they might be broken, and then we won't be able to verify toolbox cli functionality (that's probably not a showstopper to creating this criterion). It would certainly make more sense to have both parts blocking, though :-) * The criterion can't refer to the testcase for definitions. The criterion must contain definitions, and the testcase is then written to test them, that's the correct relation.
On Mon, Apr 17, 2023 at 2:44 PM Kamil Paral kparal@redhat.com wrote:
On Wed, Apr 12, 2023 at 5:57 AM Sumantro Mukherjee sumukher@redhat.com wrote:
There is a push from the Dekstop team to have toolbox as a default app in the installer iso as well.
The 'toolbox' package is already installed by default. Or do you mean some related GUI app?
Toolbox aka Toolbx should be usable. Toolbox should be able to list, create and provide the desired containerized environment to the user without root privileges. Toolbox images should be generally functional. Footnote - "What does this cover?": Toolbox's general functionality as covered by this test cases https://fedoraproject.org/wiki/QA:Testcase_toolboxAlready covered by Ben and Adam, so just a few additional notes:
- We can't block on toolbox *images*, unless they are marked as
release-blocking artifacts. If the Desktop team wants that, Ben will know the right process to take.
- We can block on the toolbox cli command. We can say that it must list
existing containers, enter them, create a new one, etc. But since the toolbox images are not release-blocking at the moment, they might be broken, and then we won't be able to verify toolbox cli functionality (that's probably not a showstopper to creating this criterion). It would certainly make more sense to have both parts blocking, though :-)
- The criterion can't refer to the testcase for definitions. The criterion
must contain definitions, and the testcase is then written to test them, that's the correct relation.
test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Thanks for all the comments and advice!!
I have rewritten this
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ For any release-blocking deliverable whose default deployment includes toolbox, it must be possible to list existing containers, enter them, and create a new one.
Footnote - "What does this cover?": Toolbox CLI should offer commands to be executed by the user which includes listing existing container images created by the toolbox, creating container images with the latest Fedora and RHEL and entering these containers. This criterion aims at blocking only the toolbox rpm functionality and not the OCI image which is supposed to be downloaded by user's choice. https://fedoraproject.org/wiki/QA:Testcase_toolbox ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For any release-blocking deliverable whose default deployment includes toolbox, it must be possible to list existing containers, enter them, and create a new one. Footnote - "What does this cover?": Toolbox CLI should offer commands to be executed by the user which includes listing existing container images created by the toolbox, creating container images with the latest Fedora and RHEL and entering these containers. This criterion aims at blocking only the toolbox rpm functionality and not the OCI image which is supposed to be downloaded by user's choice. https://fedoraproject.org/wiki/QA:Testcase_toolbox ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The criterion and the footnote seem a bit duplicated, perhaps we can shorten it. I'd also specify that entering is related just to those containers which we're blocking on creating. What about this?
~~~~ For any release-blocking deliverable whose default deployment includes toolbox, the toolbox CLI must be able to list existing containers, create a new container with the latest Fedora and RHEL image, and enter it.
Footnote - "What does this cover?": This criterion aims at blocking only on the toolbox CLI functionality. It does not require that the specified image is present in the registry or functional. ~~~~
It would be also good to have a Workstation WG member (since this is driven by them) comment here that this is indeed what they want.
Hello
This has been silent for a long time now but here's the update. Debarshi and I decided to re-write the whole thing as a changeset[0]. We DO care about not just the rpm but also the OCI image that we ship with Toolbx. As a result, we filed a releng ticket[1] where we justified the following: We need the image ready for branch point, beta and final and that image be blocked implying we resort to "no-go" if the expected image is not there at all.
As the basic functionality of toolbx continues to work as intended, we would also like to block on any upcoming changeset, if that has the potential to break toolbx or toolbx OCI image deliverables as well. Read [2] for more exact things that might break toolbx.
Finally, GNOME WS, Silverblue, FCOS include Toolbx as part of their deliverable. RHEL does the same. It's crucial that we block on this. However, as the changeset process rolls, we will go through FESCO as well. This is just a good time for us to ensure we all are on the same page and have the plumbing done beforehand!
The revised criterion I have now stands as:
~~~~ For any release-blocking deliverable whose default deployment includes toolbx, the toolbox CLI must be able to list existing containers, create a new container with the latest Fedora and RHEL image, and enter it. OCI images should get updates in every stage of the release cycle (Branched, Beta and Final).
Footnote - "What does this cover?": This criterion aims at blocking Toolbx OCI image and Toolbx rpms. In cases of a breaking changeset or regression which affects toolbx, we will need to test well ahead of time to ensure things are fine. The images must be present in registry.fedoraproject.org and must keep being updated like for branchpoints, beta and final. Missing images or broken images, will be blocking the release. Changes in Toolbx stack will warrant for a test day in that particular release cycle and regular validation to ensure there are no regressions.
~~~~
[0] https://fedoraproject.org/w/index.php?title=Changes/ToolbxReleaseBlocker [1] https://pagure.io/releng/issue/11399 [2] https://fedoraproject.org/w/index.php?title=Changes/ToolbxReleaseBlocker#Det...
On Mon, May 1, 2023 at 10:48 PM Sumantro Mukherjee sumukher@redhat.com wrote:
Hello
This has been silent for a long time now but here's the update. Debarshi and I decided to re-write the whole thing as a changeset[0]. We DO care about not just the rpm but also the OCI image that we ship with Toolbx. As a result, we filed a releng ticket[1] where we justified the following: We need the image ready for branch point, beta and final and that image be blocked implying we resort to "no-go" if the expected image is not there at all.
As the basic functionality of toolbx continues to work as intended, we would also like to block on any upcoming changeset, if that has the potential to break toolbx or toolbx OCI image deliverables as well. Read [2] for more exact things that might break toolbx.
Finally, GNOME WS, Silverblue, FCOS include Toolbx as part of their deliverable. RHEL does the same. It's crucial that we block on this. However, as the changeset process rolls, we will go through FESCO as well. This is just a good time for us to ensure we all are on the same page and have the plumbing done beforehand!
I'd also like to preload toolbx on Fedora KDE: https://pagure.io/fedora-kde/SIG/issue/346
We already ship it for Fedora Kinoite.
The revised criterion I have now stands as:
For any release-blocking deliverable whose default deployment includes toolbx, the toolbox CLI must be able to list existing containers, create a new container with the latest Fedora and RHEL image, and enter it. OCI images should get updates in every stage of the release cycle (Branched, Beta and Final). Footnote - "What does this cover?": This criterion aims at blocking Toolbx OCI image and Toolbx rpms. In cases of a breaking changeset or regression which affects toolbx, we will need to test well ahead of time to ensure things are fine. The images must be present in registry.fedoraproject.org and must keep being updated like for branchpoints, beta and final. Missing images or broken images, will be blocking the release. Changes in Toolbx stack will warrant for a test day in that particular release cycle and regular validation to ensure there are no regressions.
"will warrant for a" -> "will warrant a"
otherwise lgtm.
On Tue, May 2, 2023 at 9:04 AM Neal Gompa ngompa13@gmail.com wrote:
On Mon, May 1, 2023 at 10:48 PM Sumantro Mukherjee sumukher@redhat.com wrote:
Hello
This has been silent for a long time now but here's the update. Debarshi and I decided to re-write the whole thing as a changeset[0]. We DO care about not just the rpm but also the OCI image that we ship with Toolbx. As a result, we filed a releng ticket[1] where we justified the following: We need the image ready for branch point, beta and final and that image be blocked implying we resort to "no-go" if the expected image is not there at all.
As the basic functionality of toolbx continues to work as intended, we would also like to block on any upcoming changeset, if that has the potential to break toolbx or toolbx OCI image deliverables as well. Read [2] for more exact things that might break toolbx.
Finally, GNOME WS, Silverblue, FCOS include Toolbx as part of their deliverable. RHEL does the same. It's crucial that we block on this. However, as the changeset process rolls, we will go through FESCO as well. This is just a good time for us to ensure we all are on the same page and have the plumbing done beforehand!
I'd also like to preload toolbx on Fedora KDE: https://pagure.io/fedora-kde/SIG/issue/346
We already ship it for Fedora Kinoite.
yaaaayyyy!!!!
The revised criterion I have now stands as:
For any release-blocking deliverable whose default deployment includes toolbx, the toolbox CLI must be able to list existing containers, create a new container with the latest Fedora and RHEL image, and enter it. OCI images should get updates in every stage of the release cycle (Branched, Beta and Final). Footnote - "What does this cover?": This criterion aims at blocking Toolbx OCI image and Toolbx rpms. In cases of a breaking changeset or regression which affects toolbx, we will need to test well ahead of time to ensure things are fine. The images must be present in registry.fedoraproject.org and must keep being updated like for branchpoints, beta and final. Missing images or broken images, will be blocking the release. Changes in Toolbx stack will warrant for a test day in that particular release cycle and regular validation to ensure there are no regressions."will warrant for a" -> "will warrant a"
otherwise lgtm.
Thanks for the review!
On Tue, May 2, 2023 at 12:00 AM Sumantro Mukherjee sumukher@redhat.com wrote:
On Tue, May 2, 2023 at 9:04 AM Neal Gompa ngompa13@gmail.com wrote:
On Mon, May 1, 2023 at 10:48 PM Sumantro Mukherjee sumukher@redhat.com wrote:
Hello
This has been silent for a long time now but here's the update. Debarshi and I decided to re-write the whole thing as a changeset[0]. We DO care about not just the rpm but also the OCI image that we ship with Toolbx. As a result, we filed a releng ticket[1] where we justified the following: We need the image ready for branch point, beta and final and that image be blocked implying we resort to "no-go" if the expected image is not there at all.
As the basic functionality of toolbx continues to work as intended, we would also like to block on any upcoming changeset, if that has the potential to break toolbx or toolbx OCI image deliverables as well. Read [2] for more exact things that might break toolbx.
Finally, GNOME WS, Silverblue, FCOS include Toolbx as part of their deliverable. RHEL does the same. It's crucial that we block on this. However, as the changeset process rolls, we will go through FESCO as well. This is just a good time for us to ensure we all are on the same page and have the plumbing done beforehand!
I'd also like to preload toolbx on Fedora KDE: https://pagure.io/fedora-kde/SIG/issue/346
We already ship it for Fedora Kinoite.
yaaaayyyy!!!!
Note that my desire to preload it depends on a guarantee that it isn't broken and works on GA.
-- 真実はいつも一つ！/ Always, there's only one truth!
On Tue, May 2, 2023 at 4:48 AM Sumantro Mukherjee sumukher@redhat.com wrote:
Debarshi and I decided to re-write the whole thing as a changeset[0].
Great, I think this is the best approach.
The revised criterion I have now stands as:
Does it make sense to wait for the FESCo approval before we finalize the exact criterion wording? I don't want to spend too much time on it in advance, when we're not sure about the outcome :-)
On Tue, May 2, 2023, 16:46 Kamil Paral kparal@redhat.com wrote:
On Tue, May 2, 2023 at 4:48 AM Sumantro Mukherjee sumukher@redhat.com wrote:
Debarshi and I decided to re-write the whole thing as a changeset[0].
Great, I think this is the best approach.
The revised criterion I have now stands as:
Does it make sense to wait for the FESCo approval before we finalize the exact criterion wording? I don't want to spend too much time on it in advance, when we're not sure about the outcome :-)
Every changeset goes through FESCO any which way. Since, toolbox is anyway going to be default out of the box , things breaking apart before release isn't a great user experience. I think we should keep polishing the criteria as this is going to be asked for by FESCO and I might sight this thread for further reference.
test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On Tue, May 2, 2023 at 4:46 PM Kamil Paral kparal@redhat.com wrote:
On Tue, May 2, 2023 at 4:48 AM Sumantro Mukherjee sumukher@redhat.com wrote:
Debarshi and I decided to re-write the whole thing as a changeset[0].
Great, I think this is the best approach.
The revised criterion I have now stands as:
Does it make sense to wait for the FESCo approval before we finalize the exact criterion wording? I don't want to spend too much time on it in advance, when we're not sure about the outcome :-)
FESCo has approved[0] this changeset and now the criterion needs to be in place. I will gather all the knowledge and the corrections that have been talked about in this thread and start a fresh one for voting?
[0] https://pagure.io/fesco/issue/3002
On Thu, Jun 8, 2023 at 7:07 AM Sumantro Mukherjee sumukher@redhat.com wrote:
FESCo has approved[0] this changeset and now the criterion needs to be in place. I will gather all the knowledge and the corrections that have been talked about in this thread and start a fresh one for voting?
Either continue this one or start a fresh one, as you wish. If you start a fresh one, please include a link to this conversation as a reference. Thanks.
The new proposed criteria taking Kamil's and Adam's suggestions goes as follows :
Once people are okay with it, I can go ahead and add this to the criterion?
~~~~~~~~~~~~~~~~~~~~~ For any release-blocking deliverable whose default deployment includes Toolbx, the <code>toolbox</code> CLI must be able to list existing containers, create a new container with the latest Fedora and RHEL image, and enter it.
Footnote - "What does this cover?": The images must be present in registry.fedoraproject.org ~~~~~~~~~~~~~~~~~~~~~~~~~
On Fri, 2023-06-09 at 09:24 +0530, Sumantro Mukherjee wrote:
The new proposed criteria taking Kamil's and Adam's suggestions goes as follows :
Once people are okay with it, I can go ahead and add this to the criterion?
For any release-blocking deliverable whose default deployment includes Toolbx, the <code>toolbox</code> CLI must be able to list existing containers, create a new container with the latest Fedora and RHEL image, and enter it. Footnote - "What does this cover?": The images must be present in registry.fedoraproject.org
Thanks Sumantro! Sorry for the late reply. Kamil and I just talked about this. We think it still needs some tweaking. Here's my suggestion:
~~~~~~~~~~~~~~~~~~~~~ For any release-blocking deliverable whose default deployment includes Toolbx, the {{command|toolbox}} CLI must be able to list existing containers. For each of the following, it must be able to create and enter a new container: * The image from the compose under test * The most recent stable image for the current stable Fedora release * The most recent stable image for the current stable RHEL release
Footnote - "What does this cover?": Bugs in either the {{command|toolbox}} CLI or the container image from the compose under test may constitute a violation of the criteria for that compose. Any bugs found in the current stable Fedora and RHEL images should be reported appropriately, but don't constitute a violation of the criteria for the compose under test - the requirement is only that the CLI be capable of correctly creating and entering them. If a stable Fedora or RHEL container needs updates in order for the CLI from the compose under test to be able to run it correctly, that may constitute a PreviousRelease blocker (see [[QA:SOP_blocker_bug_process#Normal,_0- Day_and_Previous_Release_blockers|the blocker SOP]]). ~~~~~~~~~~~~~~~~~~~~~~~~~
This is working out the tricky details around the difference between the CLI and the images and what requirements we have for both as part of the release process. I've noted on both the FESCo and releng tickets that the details about producing and publishing the container images do not seem to be worked out yet:
https://pagure.io/releng/issue/11399 https://pagure.io/fesco/issue/3002
On Tue, Jun 20, 2023 at 12:07 PM Adam Williamson adamwill@fedoraproject.org wrote:
For any release-blocking deliverable whose default deployment includes Toolbx, the {{command|toolbox}} CLI must be able to list existing containers.
One nitpick, it's not clear whether "list" means listing local containers or remote containers (all in the registry). So perhaps say "list existing local containers", or something similar?
On Tue, 2023-06-20 at 12:30 +0200, Kamil Paral wrote:
On Tue, Jun 20, 2023 at 12:07 PM Adam Williamson adamwill@fedoraproject.org wrote:
For any release-blocking deliverable whose default deployment includes Toolbx, the {{command|toolbox}} CLI must be able to list existing containers.One nitpick, it's not clear whether "list" means listing local containers or remote containers (all in the registry). So perhaps say "list existing local containers", or something similar?
Sure, makes sense. Sorry, I forgot to change that. Thanks.
On Tue, Jun 20, 2023 at 4:14 PM Adam Williamson adamwill@fedoraproject.org wrote:
On Tue, 2023-06-20 at 12:30 +0200, Kamil Paral wrote:
On Tue, Jun 20, 2023 at 12:07 PM Adam Williamson <
adamwill@fedoraproject.org>
wrote:
For any release-blocking deliverable whose default deployment includes Toolbx, the {{command|toolbox}} CLI must be able to list existing containers.One nitpick, it's not clear whether "list" means listing local containers or remote containers (all in the registry). So perhaps say "list existing local containers", or something similar?
Sure, makes sense. Sorry, I forgot to change that. Thanks.
Adam Williamson (he/him/his) Fedora QA Fedora Chat: @adamwill:fedora.im | Mastodon: @adamw@fosstodon.org https://www.happyassassin.net
test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Thanks a lot Kamil and Adamw!! Can I go and publish this under the Beta criteria? :) @Kamil Paral kparal@redhat.com it will be awesome, if you can point me to the correct criteria template , please?
On Mon, Jun 26, 2023 at 10:21 AM Sumantro Mukherjee sumukher@redhat.com wrote:
Thanks a lot Kamil and Adamw!! Can I go and publish this under the Beta criteria? :) @Kamil Paral kparal@redhat.com it will be awesome, if you can point me to the correct criteria template , please?
Thumbs up from me. Here's the criterion page [1]. It will probably fit under "Post-install requirements". Please make sure to add a References footnote with a hyperlink to this email thread. When saving the wiki page, provide a reasonable summary so that the page history is easy to read. Please link to the page diff here afterwards.
The next action would be to figure out how to test the nightly/RC toolbox image (it will require uploading to some testing branch/tag [2]), write a test case, add it to one of the release validation matrices, and link it from the release criterion ;-)
Thanks, Kamil
[1] https://fedoraproject.org/wiki/Fedora_39_Beta_Release_Criteria [2] https://pagure.io/fesco/issue/3002#comment-862116
On Tue, May 02, 2023 at 08:18:07AM +0530, Sumantro Mukherjee wrote:
Hello
This has been silent for a long time now but here's the update. Debarshi and I decided to re-write the whole thing as a changeset[0]. We DO care about not just the rpm but also the OCI image that we ship with Toolbx. As a result, we filed a releng ticket[1] where we justified the following: We need the image ready for branch point, beta and final and that image be blocked implying we resort to "no-go" if the expected image is not there at all.
If we start building the oci image as part of composes, it will be not failable, that is, if it doesn't compose, the compose fails and nothing is produced. :) Of course, it might compose and still have some kind of breakage and will need testing for that.
Note that as far as I know there's not any critera/testing for branch point. We just get a successfull compose there. Beta and Final are actually releases with testing, etc.
kevin
On Tue, 2023-05-02 at 14:20 -0700, Kevin Fenzi wrote:
On Tue, May 02, 2023 at 08:18:07AM +0530, Sumantro Mukherjee wrote:
Hello
This has been silent for a long time now but here's the update. Debarshi and I decided to re-write the whole thing as a changeset[0]. We DO care about not just the rpm but also the OCI image that we ship with Toolbx. As a result, we filed a releng ticket[1] where we justified the following: We need the image ready for branch point, beta and final and that image be blocked implying we resort to "no-go" if the expected image is not there at all.
If we start building the oci image as part of composes, it will be not failable, that is, if it doesn't compose, the compose fails and nothing is produced. :) Of course, it might compose and still have some kind of breakage and will need testing for that.
Note that as far as I know there's not any critera/testing for branch point. We just get a successfull compose there. Beta and Final are actually releases with testing, etc.
Right. I think just making it a non-failable image achieves everything desired.
We can automate testing of the image, I guess, so long as there's an explanation somewhere of how we can test the specific image built as part of the compose when running the tests for e.g. the Workstation or Silverblue image from the same compose...
On Tue, May 2, 2023 at 4:48 AM Sumantro Mukherjee sumukher@redhat.com wrote:
The revised criterion I have now stands as:
For any release-blocking deliverable whose default deployment includes toolbx, the toolbox CLI must be able to list existing containers,
toolbx -> Toolbx to denote you're talking about the project/software
and we can also do toolbox CLI -> <code>toolbox</code> CLI to distinguish the command name
OCI images should get updates in every stage of the release cycle (Branched, Beta and Final).
"get updates" is probably a bit confusing. You want to say that the Toolbx OCI image must be composed from the same software versions as other compose artifacts, right? This is actually a bit tricky, because sometimes some artifacts might have slightly different versions (IoT, respinning some Spins when they fail to compose, a hotfix in just one Spin, etc, it has happened before). Perhaps we can neglect that. But I wonder if this sentence is actually needed. Once Toolbox OCI is release-blocking, it will have to be built with every compose, just as Kevin said. So this will have to be true each time. And we have the same assumption basically for each release-blocking artifact.
Footnote - "What does this cover?": This criterion aims at blocking Toolbx OCI image and Toolbx rpms.
I think this is already covered in the criterion above.
In cases of a breaking changeset or regression which affects toolbx, we will need to test well ahead of time to ensure things are fine.
This sentence doesn't fit into the release criteria. Just leave it out.
The images must be present in registry.fedoraproject.org
This is fine. It adds extra detail to the main requirement specified above (it could be merged there or kept inside the footnote).
and must keep being updated like for branchpoints, beta and final. Missing images or broken images, will be blocking the release.
I think this is just assumed, as described above, for each release-blocking artifact. Honestly, I thought we have some criterion saying "all release-blocking artifacts must exist" or something along those lines, but I haven't found it :-)
Adam, am I looking wrong?
Changes in Toolbx stack will warrant for a test day in that particular release cycle and regular validation to ensure there are no regressions.
This sentence doesn't fit into the release criteria. Just leave it out.
On Wed, May 3, 2023 at 6:38 PM Kamil Paral kparal@redhat.com wrote:
On Tue, May 2, 2023 at 4:48 AM Sumantro Mukherjee sumukher@redhat.com wrote:
The revised criterion I have now stands as:
For any release-blocking deliverable whose default deployment includes toolbx, the toolbox CLI must be able to list existing containers,toolbx -> Toolbx to denote you're talking about the project/software
and we can also do toolbox CLI -> <code>toolbox</code> CLI to distinguish the command name
OCI images should get updates in every stage of the release cycle (Branched, Beta and Final).
"get updates" is probably a bit confusing. You want to say that the Toolbx OCI image must be composed from the same software versions as other compose artifacts, right? This is actually a bit tricky, because sometimes some artifacts might have slightly different versions (IoT, respinning some Spins when they fail to compose, a hotfix in just one Spin, etc, it has happened before). Perhaps we can neglect that. But I wonder if this sentence is actually needed. Once Toolbox OCI is release-blocking, it will have to be built with every compose, just as Kevin said. So this will have to be true each time. And we have the same assumption basically for each release-blocking artifact.
Footnote - "What does this cover?": This criterion aims at blocking Toolbx OCI image and Toolbx rpms.
I think this is already covered in the criterion above.
In cases of a breaking changeset or regression which affects toolbx, we will need to test well ahead of time to ensure things are fine.
This sentence doesn't fit into the release criteria. Just leave it out.
The images must be present in registry.fedoraproject.org
This is fine. It adds extra detail to the main requirement specified above (it could be merged there or kept inside the footnote).
and must keep being updated like for branchpoints, beta and final. Missing images or broken images, will be blocking the release.
I think this is just assumed, as described above, for each release-blocking artifact. Honestly, I thought we have some criterion saying "all release-blocking artifacts must exist" or something along those lines, but I haven't found it :-)
Adam, am I looking wrong?
Adam & Kamil, since my criteria starts with"for each release-blocking artifact", do you folks think its a good idea to site the criterion? or at least have it? I am not able to find it as well.
Changes in Toolbx stack will warrant for a test day in that particular release cycle and regular validation to ensure there are no regressions.
This sentence doesn't fit into the release criteria. Just leave it out.
test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On Fri, May 5, 2023 at 6:39 AM Sumantro Mukherjee sumukher@redhat.com wrote:
Adam & Kamil, since my criteria starts with"for each release-blocking artifact", do you folks think its a good idea to site the criterion? or at least have it? I am not able to find it as well.
You mean "cite", right? :-) Cite which one? Sorry I don't understand. Are you talking about the following sentence?
Honestly, I thought we have some criterion saying "all release-blocking
artifacts must exist" or something along those lines, but I haven't found it :-)
Let's wait for Adam to reply. (Ping him if he doesn't, thanks).
On Wed, 2023-05-03 at 15:08 +0200, Kamil Paral wrote:
and must keep being updated like for branchpoints, beta and final. Missing images or broken images, will be blocking the release.
I think this is just assumed, as described above, for each release-blocking artifact. Honestly, I thought we have some criterion saying "all release-blocking artifacts must exist" or something along those lines, but I haven't found it :-)
Adam, am I looking wrong?
I pretty much consider it to be implied: an image that doesn't exist doesn't meet *any* of the criteria that apply to it, so we can cite just about any criterion. It *is* more explicitly mentioned in the automatic blockers bit, too:
"Bugs which entirely prevent the composition of one or more of the release-blocking images required to be built for a currently-pending (pre-)release"
https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process#Automatic_blockers
So I would say it's not necessary to specify this explicitly in the new criteria. We may want to adjust the "Release-blocking images must boot" and "Expected image boot behavior" blocks at https://fedoraproject.org/wiki/Basic_Release_Criteria#Release-blocking_image... , though.
-
Adam Williamson -
Ben Cotton -
Kamil Paral -
Kevin Fenzi -
Neal Gompa -
Sumantro Mukherjee