Two-Week Atomic actual deliverables

[Adam Miller]

On Thu, Sep 10, 2015 at 12:02 PM, Matthew Miller
<mattdm at fedoraproject.org> wrote:
> Okay, so, we are getting down to the wire with
> https://fedoraproject.org/wiki/Changes/Two_Week_Atomic, and
> particularly with the Flock/post-Flock decision to make Atomic the main
> thing, there are a few specifics which should be nailed down.
>
> Particularly, I want to call attention to this part of the Change:
>
>   In addition to basic "What is this all about?" information, the
>   website will need text properly setting expectations for development
>   status, lack of non-automated testing, support level ("you can help
>   us fix it!"), and so on.
>
> The web team will help us update
> <https://getfedora.org/en/cloud/download/>, but they aren't the ones to
> write the actual text. I'm thinking a box like this:
>
>
>    Fedora Atomic is meant to represent the leading edge of Project
>    Atomic development. These releases showcase that work. They've
>    passed various levels of automated testing, but have not been
>    hand-vetted.
>
>    Please test before using new versions in production. If you do
>    discover a problem, the Atomic tools make it easy to flip back to an
>    earlier release — and if that happens, please also help us by filing
>    bugs or submitting fixes.

I think it would be good to have a click-able URL or button here to
take people to the location to report issues.

>
>    Note that different Fedora Atomic media are subject to different
>    levels of automatic testing. We are making continuous improvements
>    to these tests; [click here] to learn about the current test status.
>
>
> "Click here" would go to a page which would describe (and this is the
> accurate *current* state as I understand it):
>
> * qcow2/raw.xz downloadable images:
>   - tunir-based test suite run in a VM
>    - link to actual tests
>   - not currently testing in live openstack environment
>
> * Vagrant boxes:
>   - same tunir-based test suite in VM environemnt
>
> * EC2 AMIs:
>   - bitwise identical to qcow2/raw.xz images
>   - basic "does it run?" test in live EC2 environment
>
> * installer ISOs:
>   - based on Fedora 22 [23, etc] installer (which got full Fedora QA including
>     manual testing) plus updates (which haven't)
>   - used to generate all images above, so basic function is verified
>   - however, not actually tested to boot on non-VM hardware, for UI
>     problems, etc.
>
>
> In talking with some of the people doing the actual implementation work
> (hi Adam Miller!), there's quite a lot of concern over whether it's
> appropriate to include at all the items lower down on the list with
> such minimal testing. Particularly, the installer ISO images are not
> really like the others.

Big shout out to Kushal and dgilmore for doing the bulk of the work,
I've been face down in Layered Image Build tooling land for a while.
:)

Also, thanks to threebean for helping us with fedmsg and web bits.

>
> I'm not sure that the F22 installer ISO got any testing at all for the
> F22 release -- it was non blocking and there aren't test cases for it.
> But it is on https://getfedora.org/en/cloud/download/atomic.html under
> "Other Downloads" on the right. For F23, I suggest keeping it in the
> sidebar like that, but with more text explaining that this is available
> to test but not supported.
>

I'm aiming to wrap up something I'm working on right now and will
*hopefully* be able to find time tomorrow or Monday to get an Atomic
nightly ISO build (nightly now, aiming for more rapid
compose/iteration later) installer test in OpenQA[0] so that the ISO
installer will also have automated testing.

>
> Jumping back up to earlier in this message, an alternative to the
> [click here] approach would be to outline the testing level for each
> image type in the paragraph right above the download (or launch) link.
> But that seems like it might be too much text.
>
> What do you think?

+1

As long as people know that this is cutting-edge and not quite as
tested as the normal Fedora deliverables then I think that's fine.
Hopefully we can continue to iterate on this and make the test cases
cover more ground so we're more confident in the artifacts the
automated pipeline is churning out in the Fedora 24 time frame but for
now this is probably best.

-AdamM

[0] - https://openqa.happyassassin.net/

>
> --
> Matthew Miller
> <mattdm at fedoraproject.org>
> Fedora Project Leader
> _______________________________________________
> cloud mailing list
> cloud at lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/cloud
> Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct