Mirror the ansible repo from pagure in the batcave

[Pierre-Yves Chibon]

Good Morning Everyone,

Kevin and I had a discussion on IRC the other day about mirroring the ansible
repo.
The options we considered are:
- mirroring to pagure
- mirroring from pagure
  - using pagure's mirroring feature
  - using a different approach

- Mirroring to pagure
  This meant, the canonical place for ansible remains in the batcave and pagure
  only periodically pulls from there to update itself (or a hook updates pagure
  on push).
  This would work, but it basically make the pull-request workflow for reviewers
  a little more complex, as you would have to download the patch of the PR,
  apply it (potentially adding the: Merging <id> so it marks the PR as merged)
  and then push to the batcave.
  If you merged via the UI in pagure, you would then have to pull from pagure
  and then push to the batcave, with the risk of a race condition if someone or
  something updates pagure right after your merge and suddenly the commit(s) of
  the PR disapear (since the content in pagure would be overriden by what is
  coming from the batcave).

  So, not an ideal workflow.

- Mirroring from pagure
  This means, the canonical place for ansible is in pagure.io.
  However, we need to still be able to run ansible when pagure.io goes down
  (keeping in mind batcave and pagure.io are in two different data-center).
  So we want to keep a mirror of the ansible repo in the batcave for emergencies
  and that repo should be as up to date as possible.

  With this approach, all the work happens on pagure.io, the PR workflow is the
  usual, straight forward one and we have a mirror in the batcave that is mostly
  up to date.

  - Using pagure's mirroring feature
  Pagure can mirror in and out, out is done right after a push, however, batcave
  is not accessible from pagure.io, so we just can't use this.

  - Using a different approach
  We have two ways we could do this: a simple cron job, a message-based service.

  For a first step I went with a third approach: a small python service that
  runs every 3 minutes (configurable): git fetch && git fsck (to ensure the git
  is in a correct state). It actually doesn't run every 3 minutes, it calls git,
  then wait for 3 minutes then calls git again and so on. So if git was ever to
  takes 4 minutes to run, there is no risk of the program stepping on its own
  toes.


I would like to propose that install and configure this for a test.
I propose that we keep /git/ansible.git as is and just have a
/git/mirrors/ansible.git which will be the mirror from pagure.
We can migrate the hook from /git/ansible.git to /git/mirrors/ansible.git so
/srv/web/infra/ansible (ie: the exploded tree) is coming from the mirror.

I guess the lag time to get /git/mirrors updated will quickly be annoying (up to
3 minutes between when the PR is merged and when the playbook is updated), so
if we like the workflow we will likely want to switch pretty quickly to a
message-based service and then we could keep the current cron-like service to
run hourly and update /git/ansible.git which would then become some kind of
backup.


What do you think?
FBR worthy? (ie: if F32 goes out next week, I think we can wait, if it doesn't
do we want to try this sooner?)

If we like the idea, I'll start looking into the fedora-messaging based
consumer, using the approach from the current service, it should be pretty
straight forward (I'll re-use the same project for it).


Oh, and if you want to see the code of the current service:
https://pagure.io/Fedora-Infra/mirror_from_pagure


Thanks,
Pierre