up2date, mirror repositories, and performance

[seth vidal]

> To start with, something like the CPAN system that checks "freshness"
> would be good (although I'm not sure what's up with CPAN - I sent an
> update of my record and nothing has happened).  I don't know if anything
> uses that data, but it could be used.
> 
> However, I am not sure about multi-level tiered mirroring.  When I
> started mirroring CPAN, I mirrored from another mirror.  They had
> occasional problems, so I switched.  That mirror had problems and then
> went away, so I just switched to the master.  Since then, the only time
> I've touched my CPAN mirror is when I got a heads up one afternoon when
> (IIRC) 5.8.0 was released (normally I sync in the morning so I did a
> manual mid-day sync).
> 
> To have a tiered mirror system, there'd have to be some front-end tool
> for the mirror admins to catch "upstream" problems and move to another
> mirror (but you've got to handle different "freshness" - there are
> occasional sync problems between some of the Red Hat master servers;
> adding multiple tiers will not help).  Mirrors would need to be
> registered with each other (to get sync in advance of release and for
> notification of problems).
> 
> Mirror update schedules would also need to be synchronized; if I have my
> update cron job as "0-23/6" in Central time and I mirror off someone
> that has "0-23/6" but in Pacific time, I'll always be 4-12 hours behind.
> How long should I schedule "behind" the next tier to get updates (15
> minutes, 1 hour)?

> There also does need to be some automated (or at least semi-automated
> with a one-click RH review) way for mirror admins to maintain and update
> listings.

Here's the problem with comparison to CPAN- how often does CPAN have
11GB releases or complete re-rolls of rawhide which require something
like 15GB of changing files?

CPAN can work w/o tiering b/c they have a TRIVIAL amount of files in
comparison to a fedora release or beta or devel.

-sv