basic plan for two-week atomic images

[Matthew Miller]

On Thu, May 21, 2015 at 04:14:51PM -0500, Dennis Gilmore wrote:
> > 1. F22 trees built nightly (already happening)
> there is new atomic trees built as part of the package updates
> process managed by bodhi.

Cool.

> > 2. Images built from those nightly (I think right now only rawhide?)
> once f22 is final only rawhide.

What would it take to make those happen post-release?

> > 3. Images go through automated tests, automatically (tunir; not fully
> >    automatic yet, right?)
> there is zero automated testing available, but we really need it, afaik it is 
> all manual

See <https://fedoraproject.org/wiki/Changes/tunir>. Plus glorious
Taskotron future, I'm sure. :)

> > 4. Every two weeks, latest image to pass those tests gets linked
> >    on atomic.fpo, again all automatically (NEEDS DESIGN AND CODE)
> >     - probably need a manual override
> >     - page will present these images with appropriate setting of
> >       expectations ("fedora qa not to blame for anything here").
> >     - need something to happen if there aren't any that pass for
> >       two weeks, which should not happen I hope, but, y'know)
> >         * automatic message about image being out of date
> >         * next successful image automatically posted? Or do we
> >           skip two weeks?
> >     - also needs: commitment to follow and fix if things break
> We need people to test, and a process to move things through stages.
> build all the images, etc. a location to put things, i,e lots of
> things

Let's work at building the full list of lots of things. If we want to
have people testing, I'd say that'd go something like:

  - every two weeks, latest image to pass is flagged as needing human
    testing, a la a package update in bodhi
  - some sort of karma system — maybe it _is_ bodhi, but don't want to
    block on that
  - enough +1s, it goes to the atomic.fpo page
  - maybe expectation that new image is flagged on Monday, passed by...
    Wednesday? Something like that.

Alternately, we could skip that, or slot it into a later phase, and
just say "these images are auto-generated".

What process is missing for building the images?

For location, I guess
<https://dl.fedoraproject.org/pub/alt/fedora-atomic/images/>? Maybe
with separate subdirs for "all", "passed autotest", "passed human
tests" — and/or a subdir for the two-week "releases" separate from the
nightlies.

> 
> > Phase II
> > 
> > 5. Ability to include packages from a side tag, repo, something.
> >    All full, legit Fedora packages destined for main repo but maybe at
> >    a different speed. (Implementation needs work)
> >    - possibly initially just selected packages from updates-testing?
> really we just need to push things stable. if we do updates-testing composes. 
> we should be able to get the testing done and pushed through. there there is 
> no need to go off and do special things

Atomic devs have expressed a strong worry about the lack of something
like this, unless I'm misunderstanding. Possibly an alternate image
including updates-testing would suffice (or, maybe, is actually
separately necessary, since atomic doesn't let individual testers just
pull in individual RPMs to test).

> > Phase III
> > 6. Something to only expose Atomic users to _update sets_ that pass the
> >    tests (because, hey, we have tests!)
> it would be most ideal to have things signed off as good. we kinda
> have it for updates with bodhi, we likely need something new or to
> extend something existing.

It's kind of different from the Bodhi, in that these will be
integration tests rather than per-package tests.

I'm also worried about sending updates through a human-intervention
gate _twice_ will be a big bottleneck.

> > 7. Presumably, there will be a switch to F23 as base at F23 release;
> >    will there be overlap (beta images) to make that smooth?
> we probably should do all supported releases and rawhide

I know Atomic devs have previously suggested wanting to encourage
people to stay on $current, at least while the thing is under such
rapid development. Possibly the right thing here is to generate
nightlies for all but point the phase I two-week-selection process at
only $current.


> > 8. What else?
> > 
> > Phase IV
> > 
> > 9. Expand tests beyond tunir — test installs on hardware, etc.
> test everything :) including other parts of fedora. automated tests for cloud 
> images would be good also. I fear we currently do not do anywhere near the 
> testing that we should.

Absolutely. 

> 
> > 10. Maybe create a new, special Fedora repository for packages
> >     with version skew mainline Fedora — e.g. newer systemd
> >     or hold older Docker version for some reasons. (Up for
> >     discussion — that's why this is in phase 3!)
> We could do so, I would want to see a really good reason to do so. I
> would hope that what is done for one part of fedora does not break or
> have negative impacts on other parts of fedora, so just pushing the
> updates in should be good. making rawhide more of a place to be, for
> me should be our goal. get the new awesome things in there and used
> by people, leave the stable releases stable

The concern I'm hearing is that rawhide is hard to develop on, because
parts other than what you're working on are may change from underneath
you too quicky. So, there's a desire to develop on top of the stable
release — but since Atomic isn't really an "on top of" thing entirely,
it might need new systemd or something that the mainline isn't ready
for. This area _definitely_ needs clear details, I agree.


Thanks Dennis!

-- 
Matthew Miller
<mattdm at fedoraproject.org>
Fedora Project Leader