Planning: Fedmsg Integration

[Tim Flink]

On Fri, 12 Dec 2014 11:59:30 -0500
Ralph Bean <rbean at redhat.com> wrote:

> On Thu, Dec 11, 2014 at 11:10:55PM -0700, Tim Flink wrote:
> > Almost forgot about this, even though I was talking about it earlier
> > today.
> > 
> > The current plan for bodhi2, as I understand it, is shooting for
> > production deployment in January or February. This affects us
> > because we will no longer be submitting bodhi comments as a
> > feedback mechanism for completed tasks.
> > 
> > Before this happens, we will need to enhance resultsdb-backend to
> > send out fedmsgs that can be processed by fedmsg-notifier (fmn [1])
> > 
> > [1] https://apps.fedoraproject.org/notifications/
> > 
> > Outside of the technical requirements here, we need to figure out
> > whether or not receiving notifications will be optional for
> > maintainers or not. This will require some conversation with FESCo
> > as it's more of a policy thing but will need driving from our end.
> 
> IMHO, it should be optional, but opt-out.  (So we would need to
> include it in the new defaults that go out with the next release of
> fmn).

Normally, I'd agree with you but if we're going to start gating updates
or builds based on automated check results, that changes things a bit.

Then again, it's not like that's going to happen right away so it's
probably something we can deal with later.
 
> > The more technical bits that I'm aware of at this point are:
> >  - Decide on a schema for the messages we emit
> >    * org.fedoraproject.taskotron.result.package.foo-1.2-3.fc99 etc.
> >      (not even sure that's a valid message)
> 
> I'd rather not have a variable in the package topic -- that will give
> us an enormous number of total unique topic in the datagrepper
> database.
> 
> How about something like:
> * org.fedoraproject.taskotron.result.new
> 
> ..and embed the update, build, and/or package name in the body of the
> message.

Yeah, that would work for me. I've not spent much time looking into
fedmsg structure yet and the one I listed was just the first thing I
thought of.

If we did embed the task name and other data in the message itself,
would fmn be able to differentiate between results for different
affected packages? I don't think that anyone wants emails for every
result from taskotron - I don't even have that set up.

Another thing we're going to have to figure out is how to send
notifications for depcheck. Update title isn't always consistent, so it
makes for a poor method to create notification rules by. We could start
posting results per-package but that is going to require some new code
to figure out what all would be affected if a build was witheld.

> >  - Figure out what information should be in the emitted messages
> > 
> >  - Figure out where the messages should be emitted from and
> > add/modify code to get that happening (resultsdb-backend makes the
> > most sense to me right now but am open to other ideas)
> > 
> >  - Get certs for the emitting machines
> > 
> >  - Start emitting messages in staging
> > 
> > 
> > As a reward when it's done:
> > 
> >  - rip out the bodhi comment code from litaskotron and burn it over
> > a bonfire while we toast marshmallows for s'mores
> > 
> > I'd like to have this ready for deployment in staging by early
> > January, so that it can run for a while before we move it into
> > production but I'm not sure if that's compatible with vacation
> > schedules.
> 
> For what its worth, bodhi2 (unreleased) currently just queries the
> resultsdb JSON API to figure out how it should mark or block
> updates... all without fedmsg.  fedmsg just gives us a nice
> notification layer for end users.

Yeah, I'm looking forward to that. The code we're currently using to
make bodhi comments is ... not well liked or a great piece of code but
it was the best reasonable solution we had at the time.

> There is another layer of integration that we've not yet begun to stub
> out in bodhi2, which is to publish a message when its backend begins a
> mash.  We have been hoping that taskotron could listen for that new
> message and begin a depcheck run against all the packages in the koji
> tag that bodhi is about to mash -- and then later, publish a message
> when that check is done.  Bodhi would block, waiting for that
> response, and then only proceed with the mash if that depcheck run
> passes.

Yeah, we've talked about doing that in the past. Most of the stuff to
do that is in place for Taskotron but it would probably require some
tweaking to return depcheck results in a useful format for mash -
"which builds in this set are OK to push?". Has there been any farther
discussion on when we'd want to start doing that and how we'd handle
the nitty-gritty of actually doing it (overriding bad results, exact
policy on what gets pushed and what doesn't etc.)?

Tim
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://lists.fedoraproject.org/pipermail/qa-devel/attachments/20141212/5bd726ac/attachment.sig>