In A World Where...TCs don't exist?

Friday, 29 January 2016

Hi, folks! I thought this might be about the appropriate time to throw
this out there.

There hasn't been a big news press on this, but some of you may know
that releng is fairly close to switching over to Pungi 4 for composes.
For those of you who don't know:

releng is fairly close to switching over to Pungi 4 for composes.

This will have various interesting effects on QA and the whole process
of building Fedora releases.

With the current releng process, TC / RC composes are one beast, and
nightly composes are another, very different beast. In fact nightly
composes barely really 'exist' at all - when we say 'nightly compose'
we really mean 'pungify the rawhide/branched repo, and fire off a bunch
of koji tasks'. After the fact, there is no real relationship between
any of those bits, which is why I had to write fedfind to go out and
synthesize the concept of a 'nightly compose' by finding all the Koji
tasks and treating them plus the repository boot.iso's as a single
'compose'.

With Pungi 4, all composes will look a lot more similar. 'nightly'
composes (which, in point of fact, will probably happen more than once
per day - I'm not sure if we came up with a new name yet) look a lot
more like current TC/RC composes than current nightly composes. You can
see approximately what a Pungi 4 compose currently looks like here:

https://kojipkgs.fedoraproject.org/compose/rawhide/

as of right now, the Koji built bits - lives, cloud and ARM disk
images, etc - aren't integrated with the installer images, but they
*will* be, and they'll all show up in the same location. As you can see
it has all the different variants, and a Server DVD image. (A Pungi 4
compose also has a bunch of metadata, which means we can more or less
kill off fedfind, thank God).

The implication of this I wanted to talk about in this thread is: what
does this mean for the release validation process, in terms of what
composes we cut and what release validation events we have?

So as you probably know, right now, the validation process is built
around the milestone 'TC' and 'RC' images. We build a series of Alpha
TCs and run a bunch of tests for each of these composes, reporting the
results to wiki pages named for the composes. Then we do Alpha RCs,
then Beta TCs, and so on through Final RCs.

For the last few releases we've added on some 'nightly' validation
events, where we create wiki pages named for nightly composes and run
the same set of tests on the nightly boot.iso's and Koji images, but
these have been framed as kind of an 'early warning system' for use
before Alpha TC1 arrives, and once Alpha TC1 arrives we stop doing the
nightly validation events.

With Pungi 4, I don't think this makes a lot of sense any more. Dennis
and I have been talking about this and I think we broadly agree on it. 

TCs and RCs used to be kinda the only way we *could* do validation
testing. For long periods we didn't have reliable nightly builds of
Rawhide or Branched at all, certainly not all the Koji-produced images.
The process for doing 'real' composes was quite long and painful and
required squishy human intervention.

If we have automated, more-than-nightly composes that look much like a
regular release compose would, there's no clear case for having TCs at
all. We could simply stop building them and extend the "nightly"
validation process. I think the way to do that would be to keep
'nominating' nightly composes for validation testing all the time,
*except* when we're doing RCs. So instead of going something like:

24 Rawhide 20160120
24 Rawhide 20160215
	== BRANCH POINT ==
24 Branched 20160301
24 Branched 20160315
24 Alpha TC1
24 Alpha TC2
       == ALPHA FREEZE ==
24 Alpha RC1
24 Alpha RC2
       == ALPHA RELEASE ==
24 Beta TC1
....

we'd go something like:

24 Rawhide 20160120
24 Rawhide 20160215
	== BRANCH POINT ==
24
Branched 20160301
24 Branched 20160315
24 Branched 20160401
24 Alpha RC1
24
Alpha RC2
       == ALPHA RELEASE ==
24 Branched 20160501
24 Branched
20160515
24 Beta RC1
....

note: all dates completely made up, this is just for illustration.

I think it would be plausible to do this for Fedora 24, if the Pungi 4 switchover happens
soon and goes well. There would be some details to pin down in relval and wikitcms and
stuff (we might need to tweak the validation event naming approach a bit so that it's
possible to identify the sequence of events from the names - i.e. so you know where the
RCs fit in), but nothing unsolvable.

We'll be talking about a lot of this stuff at DevConf, if anyone's going to be
there, pin down me or Dennis or someone else involved in release-y stuff and we'd be
happy to discuss it. But I wanted to throw something up on the lists for discussion as
well. What do you think? Thanks!

One point that's come up already is the way that we manually pull newer packages to
fix blocker/FE bugs into TC and RC composes via the 'bleed' repo. We're
currently envisaging something like the 'buildroot override' mechanism for the
compose process - some kind of system which would tag packages to be pulled into the
composes somehow. It would still be gated through the blocker/FE review process at least
during freezes, and probably all the time (it wouldn't be open season for any packager
to request a 'compose override' at any time). This would also allow us to do stuff
like 'tag new anaconda builds into the composes as soon as they land in
updates-testing, so we can actually test them and provide karma'.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002