[fedmsg] Proposal on a replay mechanism

[Simon Chopin]

Quoting Ralph Bean (2013-07-09 14:57:00)
> On Tue, Jul 09, 2013 at 01:35:59PM +0200, Simon Chopin wrote:
> > Quoting Ralph Bean (2013-07-09 03:08:24)
> > > On Mon, Jul 08, 2013 at 03:39:56PM +0200, Simon Chopin wrote:
> 
> .. <snip> ..
> 
> > > One problem I see is in the implementation details.  How long is an
> > > endpoint expected to hold on to its old messages before discarding
> > > them?  Whereas currently, an application that gets a fedmsg hook added
> > > doesn't retain much extra state as a result, this replay-request
> > > proposal would require a book keeping mechanism added to every
> > > endpoint (in our case, every mod_wsgi/httpd process, others).
> > 
> > Not every endpoint. For instance, I don't expect your wiki endpoints to
> > provide such mechanism, as the systems depending on it are not critical
> > (unless I'm missing something). For those that are critical, well, I'd
> > say it is the price to pay.
> 
> Cool.  If we can keep it heterogeneous like this, then Fedora
> Infrastructure can keep our endpoints as they are.  What do you think
> about designing this as a persistence plugin system, so the Debian
> crew could implement a schema that meets your needs, while Fedora can
> do something else or nothing in this respect.

Yes, that's completely what I had in mind. That would be rude to force
you to change your setup for our only benefit :-).

The way I see the API would be an object implementing a set of methods
to access the persistent store. I should have something for you by
tonight on my clone :-)

[...]
> Have you thought much yet about how the API should be changed if
> anywhere?
> 
> fedmsg.publish will need to record outgoing messages.  That seems
> pretty simple; no API changes needed.
> 
> Should fedmsg.tail_messages automatically detect outages and handle
> sending the replay request for the user?  Or should the user be
> responsible for detecting the outage and making the request
> themselves?  Same question goes for hub daemons and their consumers.

I have given it some thoughts, but not to the point of being able to
pinpoint where there will need some changes. The only thing I am almost
certain we will have to break is the way endpoints are handled, so that
the listeners can take into account the special cases.

For the persistent thing, ISTR moksha has already some way to store
messages, we might be able to leverage that in the hubs.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: signature
URL: <http://lists.fedoraproject.org/pipermail/messaging-sig/attachments/20130709/4f0cbd15/attachment.sig>