On Mon, 2015-08-17 at 15:31 -0400, Matthew Miller wrote:
On Mon, Aug 17, 2015 at 09:08:31AM -0700, Adam Williamson wrote:
> Hi folks! So I was wondering: what do folks think about running
> openQA's 'all' mode nightly (on the BOS deployment) instead of the
> 'current' mode?
+1; do it!
> compose. I think it'd be nice to have an idea every day if Rawhide
> and/or Branched are broken. We could then actually write some kind
> of
> 'israwhidebroken.com' thing. Ideally we'd be doing this with fedmsg
How do we get from "it's broken today" to "and someone should do
something about it?"
Electric shock collars?
No, seriously, I'm kinda working on some stuff to do some sort of daily
compose status report, which should help. openQA doesn't do automatic
bug submission and we probably don't want to do it (because of cases
where the failures are more to do with openQA than Fedora, and avoiding
potential excessive duplication when libreport dupe detection fails),
so the first thing that needs to happen is someone (i.e. one of us)
goes and looks at the failures and submits bugs if they're actually
valid problems.
If the bugs are significant enough they'll wind up as release blockers,
which is probably an adequate short-term mechanism.
We're still a reasonably long distance from true CI in the sense of
'...and if it's broken, we block the change that broke it', because
daily is still far too coarse granularity for that. For that we'd need
Dennis' pet project that somehow knows all the things that can possibly
influence the behaviours of the images, and we'd need to run the tests
every time any of those things changed, and we'd need to do it fast
enough to gate the changes. Which is all fairly challenging stuff. This
is more 'observe and report' level stuff now and for the medium-term
foreseeable future, I'd say.
What we could (probably) do right now is hack up a dumb lil' thing with
a hardcoded list of packages which would re-spin an image every time a
build of one of those packages happened, and run maybe a subset of the
most critical tests on it (to avoid the tests taking too long). But it
would probably be a bit of a duct-tape project for now, and it'd be a
few days' work at least (and that's an estimate with my 'pretend
developer' hat on, so I'm now going to switch into my 'QA' hat and
revise that idiot's estimate to 'two weeks at least').
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net