About MirrorManager2

Pierre-Yves Chibon pingou at pingoured.fr
Wed Feb 11 13:10:01 UTC 2015


Hi everyone,

We currently have MirrorManager2 running on staging. It's apparently not 100%
set-up since we get emails once in a while that one of the crons failed
(iirc among other we need to finish configuring fedmsg).

Other that this, MirrorManager2 is currently in a decent shape I think.
However, we really need to make sure nothing broke in the re-write and we
want to make sure we won't break it in the future.
To try to ensure that last part, I have try to write some tests for the UI but
also for the backend part (all the different scripts).
The pull-request is opened for review:
https://github.com/fedora-infra/mirrormanager2/pull/14

I have also been trying to capitalize on the knowledge we acquired during the
FAD by starting to write down how mirrormanager works in the documentation:
https://github.com/fedora-infra/mirrormanager2/compare/tests...doc
(pull-request to be opened once the tests branch is merged)
I would appreciate if those that were at the FAD could go through this
branch/changes and adjust as needed.
I have been thinking about asking Matt to do the review so that we can adjust
and improve the documentation.

 
Before we move MirrorManager2 to prod, here is what I think would be nice to
do/have:

- Pickle validation
  - Figure a way to validate a pickle after its creation and before moving it to
    the mirrorlist boxes
- Find out if we can improve our tests some more
  (to improve our confidence that we're ready)
- Engage the mirror mailing list and try to get them to react on the coming
  changes

The pickle validation might also be an interesting idea to check if there is a
difference between the pickle generated by prod and the one generated in stg.

 
Finally, at DevConf we have been speaking quite a bit with Dennis around
updates and MirrorManager and here is some of the ideas we spoke about:
 
- Be able to run the UMDL script on only a part of the tree
  (ie: be able to say, we updated f21-updates and we only update this part)
- Crawl the mirrors for only a part of the tree
  (This goes together with updating only part of the tree via UMDL)
- Consider if we should/could drop the content of the host_category_dir table
  before running the crawler
- Mirror versioning:
  - run UMDL, detect changes, increase master mirror's version by 1
  - run the crawler, check for the changes, align that mirror's version with the
    master mirror one
  - be able to see the difference between two versions
  - be able to crawl a mirror only for the difference between the version it is
    at and the version the master mirror is at
  note: we might still want to run a full crawl once in a while (daily?
  bi-daily?)
 
This list of ideas is more a long term todo list, not something we would want to
have working for pushing MirrorManager2 to prod.


Thoughts? Agreements? Disagreements?

Thanks,
Pierre
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.fedoraproject.org/pipermail/infrastructure/attachments/20150211/b4c926c5/attachment.sig>


More information about the infrastructure mailing list