Attempting to allow for a source of truth for Fedora release cycle information: releasestream
Hi folks!
Before the RH holiday shutdown, I started a new project designed to address a long-standing pain point in Fedora: we don't really have a good source of truth for release cycle information. I'm thinking of situations where something:
* Needs to know what the "current stable" Fedora release(s) are * Needs to know if Fedora X is Branched * Needs to know whether Fedora X Beta happened yet * Needs to know whether Fedora X is frozen
...etc etc etc. Some of this information is available...sort of...from various different systems, with various awkward caveats that I won't dive into unless someone asks. But there isn't really a single source of truth for it, and some of it just isn't really available in any easily machine-consumable way.
So I wrote releasestream: https://pagure.io/fedora-qa/releasestream
releasestream is intended to be a system that will let us do this. It is a very very simple webapp with three capabilities:
* It can read a simple record ("stream") of "release events" for a "release" and produce several static JSON representations of the information * It can write an entry to one of these streams in response to a properly-formatted POST request * It can publish a message when a new entry is received
that is all it does. The "releases" and "events" are entirely arbitrary; a "release" can be any string, and so can the "state" for a given "event". An "event" is defined as a state being either reached or left; any number of events for the same state can be present for a release.
The JSON outputs are basically "states by release" and "releases by state", as you might want either depending on what you're doing. You can conveniently look up "what releases are currently in state X?" or "what states is release X currently in?".
This all works right now; the main thing that isn't implemented yet is any form of authentication. Right now if you can talk to the server you can submit events. I wanted to check if there is interest in moving forward with this, and discuss various options, before working on that. There is a ticket where I sketched out various possible approaches: https://pagure.io/fedora-qa/releasestream/issue/2
My idea for using this is basically that we deploy an 'official' releasestream instance in infra, and then find things that do the actual work of moving Fedora releases into and out of given "states" and tack on a bit at the end to tell releasestream about it. So when e.g. Mohan is putting Fedora 34 into the Beta freeze, we would add a "submit 'enter freeze' event to releasestream" to that process somehow - ideally something that has to be run as part of the process anyway would send the POST, but in a pinch a human could do it too. When Fedora 34 is being 'released', we tweak that process to include sending a releasestream event POST. And so on.
The thing I like about this design is that there's a low barrier to entry, and can easily be adopted piecemeal but still be immediately useful for some things - as long as the event your code needs to watch for has been "onboarded", you're good. It also allows for the implementation details of a state change to change radically - it can go from being done by a human to being done by system X to being done by system Y, and all that needs to happen is to ensure the same dead simple POST request is sent to the same server.
So, what do folks think? Does this seem like a good idea? Should I go ahead with trying to get it deployed and onboard things to it? Any comments, ideas, potential problems? Thanks!
I'm definitely in favor of this effort. ;)
A few questions inline...
On Wed, Jan 13, 2021 at 09:53:36AM -0800, Adam Williamson wrote: ...snip...
that is all it does. The "releases" and "events" are entirely arbitrary; a "release" can be any string, and so can the "state" for a given "event". An "event" is defined as a state being either reached or left; any number of events for the same state can be present for a release.
Might it be good to standardize these? I mean we don't want to have:
state: "Fedora 34 Beta Released to mirrors" and then in f35 cycle: state: "f35b released/mirrors"
Or at least record somewhere in a doc what we want to use after the first cycle in use?
...snip...
My idea for using this is basically that we deploy an 'official' releasestream instance in infra, and then find things that do the actual work of moving Fedora releases into and out of given "states" and tack on a bit at the end to tell releasestream about it. So when e.g. Mohan is putting Fedora 34 into the Beta freeze, we would add a "submit 'enter freeze' event to releasestream" to that process somehow
- ideally something that has to be run as part of the process anyway
would send the POST, but in a pinch a human could do it too. When Fedora 34 is being 'released', we tweak that process to include sending a releasestream event POST. And so on.
I see right now this keeps the state/messages in a file store? I think this is a great app to just run in openshift, but if we do that I wouldn't want to loose state, so perhaps we store things in a database? I know thats overkill for this simple stuff, but it would mean we have everything easily stored/backed up, etc. I guess we could also just make a persistent volume and make sure to back it up.
...snip...
So, what do folks think? Does this seem like a good idea? Should I go ahead with trying to get it deployed and onboard things to it? Any comments, ideas, potential problems? Thanks!
Yeah, I like it and definitely think it's worth a try!
Thanks for implementing it.
kevin
On Wed, 2021-01-13 at 13:42 -0800, Kevin Fenzi wrote:
I'm definitely in favor of this effort. ;)
A few questions inline...
On Wed, Jan 13, 2021 at 09:53:36AM -0800, Adam Williamson wrote: ...snip...
that is all it does. The "releases" and "events" are entirely arbitrary; a "release" can be any string, and so can the "state" for a given "event". An "event" is defined as a state being either reached or left; any number of events for the same state can be present for a release.
Might it be good to standardize these? I mean we don't want to have:
state: "Fedora 34 Beta Released to mirrors" and then in f35 cycle: state: "f35b released/mirrors"
Or at least record somewhere in a doc what we want to use after the first cycle in use?
I was following the lessons we learned with taskotron: make the tool simple and flexible, standardize the usage by convention. (This also means the tool is flexible: if it works out for Fedora, it could be used for...pretty much any other software project that has this issue).
Typically submissions should be done by code or at least scripted, and there shouldn't be *too* many systems submitting events, so it should be relatively easy to make sure they follow the same names. But yes, we definitely want everything that submits to call the same release the same thing :)
We *could* write a "fedora helper" for submitting events that enforces a house style, but for something this simple I'm not sure it's worthwhile.
We could use one of the name formats Bodhi has, or we could use the 'dist' / 'short' names from productmd compose metadata, I guess.
...snip...
My idea for using this is basically that we deploy an 'official' releasestream instance in infra, and then find things that do the actual work of moving Fedora releases into and out of given "states" and tack on a bit at the end to tell releasestream about it. So when e.g. Mohan is putting Fedora 34 into the Beta freeze, we would add a "submit 'enter freeze' event to releasestream" to that process somehow
- ideally something that has to be run as part of the process anyway
would send the POST, but in a pinch a human could do it too. When Fedora 34 is being 'released', we tweak that process to include sending a releasestream event POST. And so on.
I see right now this keeps the state/messages in a file store? I think this is a great app to just run in openshift, but if we do that I wouldn't want to loose state, so perhaps we store things in a database? I know thats overkill for this simple stuff, but it would mean we have everything easily stored/backed up, etc. I guess we could also just make a persistent volume and make sure to back it up.
Yeah, it could be a database too, I guess, I just used a text file because it was easy, honestly, and it also means you can hand edit stuff quite easily (this would allow fixups for incorrect names, and it'd also make it easy to 'reconstruct' historical schedules, which I might do at some point). If you file a ticket for adding db support I'll look at it.
...snip...
So, what do folks think? Does this seem like a good idea? Should I go ahead with trying to get it deployed and onboard things to it? Any comments, ideas, potential problems? Thanks!
Yeah, I like it and definitely think it's worth a try!
Thanks for implementing it.
Thanks for the feedback!
On Wed, 13 Jan 2021 at 18:53, Adam Williamson adamwill@fedoraproject.org wrote:
Hi folks!
Before the RH holiday shutdown, I started a new project designed to address a long-standing pain point in Fedora: we don't really have a good source of truth for release cycle information. I'm thinking of situations where something:
- Needs to know what the "current stable" Fedora release(s) are
- Needs to know if Fedora X is Branched
- Needs to know whether Fedora X Beta happened yet
- Needs to know whether Fedora X is frozen
...etc etc etc. Some of this information is available...sort of...from various different systems, with various awkward caveats that I won't dive into unless someone asks. But there isn't really a single source of truth for it, and some of it just isn't really available in any easily machine-consumable way.
So I wrote releasestream: https://pagure.io/fedora-qa/releasestream
releasestream is intended to be a system that will let us do this. It is a very very simple webapp with three capabilities:
- It can read a simple record ("stream") of "release events" for a
"release" and produce several static JSON representations of the information
- It can write an entry to one of these streams in response to a
properly-formatted POST request
- It can publish a message when a new entry is received
that is all it does. The "releases" and "events" are entirely arbitrary; a "release" can be any string, and so can the "state" for a given "event". An "event" is defined as a state being either reached or left; any number of events for the same state can be present for a release.
The JSON outputs are basically "states by release" and "releases by state", as you might want either depending on what you're doing. You can conveniently look up "what releases are currently in state X?" or "what states is release X currently in?".
This all works right now; the main thing that isn't implemented yet is any form of authentication. Right now if you can talk to the server you can submit events. I wanted to check if there is interest in moving forward with this, and discuss various options, before working on that. There is a ticket where I sketched out various possible approaches: https://pagure.io/fedora-qa/releasestream/issue/2
My idea for using this is basically that we deploy an 'official' releasestream instance in infra, and then find things that do the actual work of moving Fedora releases into and out of given "states" and tack on a bit at the end to tell releasestream about it. So when e.g. Mohan is putting Fedora 34 into the Beta freeze, we would add a "submit 'enter freeze' event to releasestream" to that process somehow
- ideally something that has to be run as part of the process anyway
would send the POST, but in a pinch a human could do it too. When Fedora 34 is being 'released', we tweak that process to include sending a releasestream event POST. And so on.
The thing I like about this design is that there's a low barrier to entry, and can easily be adopted piecemeal but still be immediately useful for some things - as long as the event your code needs to watch for has been "onboarded", you're good. It also allows for the implementation details of a state change to change radically - it can go from being done by a human to being done by system X to being done by system Y, and all that needs to happen is to ensure the same dead simple POST request is sent to the same server.
So, what do folks think? Does this seem like a good idea? Should I go ahead with trying to get it deployed and onboard things to it? Any comments, ideas, potential problems? Thanks!
I am pretty sure you did :-) but have you looked at adding the missing information to Bodhi ? Now that we have rawhide in Bodhi we should be able to expose most the information needed.
-- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net
rel-eng mailing list -- rel-eng@lists.fedoraproject.org To unsubscribe send an email to rel-eng-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/rel-eng@lists.fedoraproject.or...
On Thu, 2021-02-11 at 09:55 +0100, Clement Verna wrote:
So, what do folks think? Does this seem like a good idea? Should I go ahead with trying to get it deployed and onboard things to it? Any comments, ideas, potential problems? Thanks!
I am pretty sure you did :-) but have you looked at adding the missing information to Bodhi ? Now that we have rawhide in Bodhi we should be able to expose most the information needed.
I didn't look at that in any technical detail but conceptually it seems kinda wrong to me. The problem with using any existing system is that all existing systems are *for* something. Bodhi is for dealing with updates. There's no reason why Bodhi would track, for instance, whether 34 is under a string freeze, right? Because Bodhi doesn't have anything to do with translations. There are a zillion other 'states' that it would be useful to have info on which aren't relevant to any *one* existing system, and that's why to me it makes sense to have a separate thing for that.
On Thu, Feb 11, 2021 at 09:04:12AM -0800, Adam Williamson wrote:
On Thu, 2021-02-11 at 09:55 +0100, Clement Verna wrote:
So, what do folks think? Does this seem like a good idea? Should I go ahead with trying to get it deployed and onboard things to it? Any comments, ideas, potential problems? Thanks!
I am pretty sure you did :-) but have you looked at adding the missing information to Bodhi ? Now that we have rawhide in Bodhi we should be able to expose most the information needed.
I didn't look at that in any technical detail but conceptually it seems kinda wrong to me. The problem with using any existing system is that all existing systems are *for* something. Bodhi is for dealing with updates. There's no reason why Bodhi would track, for instance, whether 34 is under a string freeze, right? Because Bodhi doesn't have anything to do with translations. There are a zillion other 'states' that it would be useful to have info on which aren't relevant to any *one* existing system, and that's why to me it makes sense to have a separate thing for that.
The other problem that comes from this is:
When we are 'go' for a new release, we start pushing '0-day' updates. Usually this is the day after the 'go' decision and a week or two ahead of the actual release. At that point bodhi thinks the release is "released", even though it is not.
kevin
On Thu, 11 Feb 2021 at 18:04, Adam Williamson adamwill@fedoraproject.org wrote:
On Thu, 2021-02-11 at 09:55 +0100, Clement Verna wrote:
So, what do folks think? Does this seem like a good idea? Should I go ahead with trying to get it deployed and onboard things to it? Any comments, ideas, potential problems? Thanks!
I am pretty sure you did :-) but have you looked at adding the missing information to Bodhi ? Now that we have rawhide in Bodhi we should be
able
to expose most the information needed.
I didn't look at that in any technical detail but conceptually it seems kinda wrong to me. The problem with using any existing system is that all existing systems are *for* something. Bodhi is for dealing with updates. There's no reason why Bodhi would track, for instance, whether 34 is under a string freeze, right? Because Bodhi doesn't have anything to do with translations. There are a zillion other 'states' that it would be useful to have info on which aren't relevant to any *one* existing system, and that's why to me it makes sense to have a separate thing for that.
I was wondering about the 4 Needs bullet points in your original email which could be in Bodhi but yes if we want to add more and more info a seperate app makes sense :-)
rel-eng@lists.fedoraproject.org
-
Adam Williamson -
Clement Verna -
Kevin Fenzi