Sync github projects on fedorahosted

[Pierre-Yves Chibon]

On Tue, Feb 25, 2014 at 09:16:19AM -0500, Konstantin Ryabitsev wrote:
>    On Tue, Feb 25, 2014 at 5:53 AM, Pierre-Yves Chibon <pingou at pingoured.fr>
>    wrote:
> 
>      grok-dumb-pull seems to be the way to go in the current environment.
>      However I
>      have a couple of questions:
>      - has there been any thoughts on trying to get grokmirror included in
>      github as
>        one of the hooks they provide by default? This would reduce their
>      bandwith
>        usage for all the project that, like us, would like to mirror their
>      github
>        repo elsewhere.
> 
>    I've not approached them for anything like this, but I would imagine a
>    single manifest file would not work at all for all of github's
>    repositories. The largest collection we currently manage with grokmirror
>    is 5,500 repositories and though it does admirably well, it's getting to
>    the point where parsing/writing the manifest file is taking upwards of a
>    second. They could probably generate a manifest file per user, but not a
>    single manifest for all the repositories they host.

Per-user or per-organization might already just do the job. Tbh, I was even
thinking of a per-project manifest.

>      - One thing that isn't clear to me is how the manifest.js is made
>      available. It
>        is just provided at a specified URL or is it in fact included in the
>      git repo?
> 
>    It's made available outside git repositories as a simple http download.
>    This way we can make use of an extremely lightweight HTTP with
>    "if-newer-than" header and bail out early if the remote manifest hasn't
>    changed.

So, in theory, it should be duable for github, might as simple as
https://github.com/fedora-infra/fedocal/manifest where they have
https://github.com/fedora-infra/fedocal/branches now.

Food for thoughts I guess :)

Thanks for the feedbacks!
Pierre