On Fri, Apr 17, 2020 at 8:07 AM Clement Verna <cverna@fedoraproject.org> wrote:
Hi all,

I wanted to start a discussion and possibly some work around automating the manual tasks involved in the release engineering work.

In particular I have in mind the following tasks :
 - processing the unretire package tickets.
 - processing the requests for a new package or a new branch.
 - container base image release.

Instead of looking at each of these individually I was thinking that it might be interesting to look at having an automation framework or something like a bot that could be flexible enough to add more tasks or process in the future.

To do that we have different possibilities, one could be to build a bot that has a similar architecture than the compose-tracker (https://pagure.io/releng/compose-tracker) ie a fedora-messaging consumer processing messages.
Another option is to use loopabull (https://github.com/maxamillion/loopabull) to trigger ansible playbook on fedora-messaging messages.

Both solutions are quite similar, but one (loopabull) provides already the boilerplate to trigger a script or a function based on messages received (a bit like AWS lambda or other serverless framework). We also have already a few process automated that way (https://pagure.io/Fedora-Infra/loopabull-tasks).
So I believe that using loopabull might be the best way forward, but I would be interested to hear about other ideas :-)

I would lean to use loopabull because it already works in a "reactive way" with mq messages and we don't need to write a full application since we will be using ansible (which still can be "extended" developing modules for complex tasks) - the above script could become a couple of ansible modules to be used in a playbook with loopabull.
 

Now if we look at the tasks to automate, I was thinking that we could implement that automation in different phases :

un-retiring tickets:
  • First step would be to run automatically the check-unretirement script (https://pagure.io/releng/blob/master/f/scripts/check-unretirement.py) and redirect its output into the ticket comments. That way people processing the ticket have all the information available in the ticket.
  • Second step would be to process automatically the tickets that do not require a new bz review (retired less than 8 weeks ago)
  • Finally see if we can process automatically all the tickets.

creating new dist-git repo or branches:

I think it would also need to check for missed tickets from time to time, like every 5 minutes for example, in case it missed one for whatever reason.
 
container base image release:
  • Instead of building the image every night, we could rebuild the image only when at least 1 package present in the base image has been updated.
  • Push the image weekly to the registry if a build happened during that week.

Is there any particular reason for using a weekly push schedule (assuming doing one on every build is way too much)?
 


Again here are some ideas, and I would very much appreciate feedback or other ideas :-). Also if you think about another process that could be automated that way please share it :-).

Thanks
Clément

_______________________________________________
rel-eng mailing list -- rel-eng@lists.fedoraproject.org
To unsubscribe send an email to rel-eng-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/rel-eng@lists.fedoraproject.org


--

Leonardo Rossetti

Senior Software Engineer,

Red Hat