Two-Week Atomic actual deliverables

[Matthew Miller]

Okay, so, we are getting down to the wire with
https://fedoraproject.org/wiki/Changes/Two_Week_Atomic, and
particularly with the Flock/post-Flock decision to make Atomic the main
thing, there are a few specifics which should be nailed down.

Particularly, I want to call attention to this part of the Change:

  In addition to basic "What is this all about?" information, the
  website will need text properly setting expectations for development
  status, lack of non-automated testing, support level ("you can help
  us fix it!"), and so on.

The web team will help us update
<https://getfedora.org/en/cloud/download/>, but they aren't the ones to
write the actual text. I'm thinking a box like this:

  
   Fedora Atomic is meant to represent the leading edge of Project
   Atomic development. These releases showcase that work. They've
   passed various levels of automated testing, but have not been
   hand-vetted. 

   Please test before using new versions in production. If you do
   discover a problem, the Atomic tools make it easy to flip back to an
   earlier release — and if that happens, please also help us by filing
   bugs or submitting fixes.

   Note that different Fedora Atomic media are subject to different
   levels of automatic testing. We are making continuous improvements
   to these tests; [click here] to learn about the current test status.


"Click here" would go to a page which would describe (and this is the
accurate *current* state as I understand it):

* qcow2/raw.xz downloadable images:
  - tunir-based test suite run in a VM
   - link to actual tests
  - not currently testing in live openstack environment

* Vagrant boxes:
  - same tunir-based test suite in VM environemnt

* EC2 AMIs:
  - bitwise identical to qcow2/raw.xz images
  - basic "does it run?" test in live EC2 environment

* installer ISOs:
  - based on Fedora 22 [23, etc] installer (which got full Fedora QA including
    manual testing) plus updates (which haven't)
  - used to generate all images above, so basic function is verified
  - however, not actually tested to boot on non-VM hardware, for UI
    problems, etc.


In talking with some of the people doing the actual implementation work
(hi Adam Miller!), there's quite a lot of concern over whether it's
appropriate to include at all the items lower down on the list with
such minimal testing. Particularly, the installer ISO images are not
really like the others.

I'm not sure that the F22 installer ISO got any testing at all for the
F22 release -- it was non blocking and there aren't test cases for it.
But it is on https://getfedora.org/en/cloud/download/atomic.html under
"Other Downloads" on the right. For F23, I suggest keeping it in the
sidebar like that, but with more text explaining that this is available
to test but not supported.


Jumping back up to earlier in this message, an alternative to the
[click here] approach would be to outline the testing level for each
image type in the paragraph right above the download (or launch) link.
But that seems like it might be too much text.

What do you think?

-- 
Matthew Miller
<mattdm at fedoraproject.org>
Fedora Project Leader