Fedora extras metadata
Thomas M Steenholdt
tmus at tmus.dk
Sat Mar 17 20:55:24 UTC 2007
Matt Domsch wrote:
>
> First, the problems with the current mirroring is really twofold:
>
( ... snip ... )
> 2) the global mirrors aren't being notified when content has
> changed on the master, such that they should start a new rsync
> run. mirrormanager takes a per-host email address, which the
> master sign-and-push scripts will eventually send an email to when
> the content changes. As it stands, syncing every 6 hours when
> nothing has changed doesn't make any sense.
>
> Now, mirrormanager has 2 methods by which it can know a give mirror
> host is up-to-date. First is a new report_mirrors script that uploads
> directory data back to the database from the mirror itself. Not
> everyone will want to run that, and there's always the 'trust but
> verify' model, so I've also got a (fast?) crawler that crawls each
> host using HTTP HEADs and keepalives or FTP DIR calls looking for
> content it should be carrying compared to the master list, and
> tracking presence and up-to-date-ness on a per-directory level. Those
> that aren't up2date get dropped from the appropriate per-directory
> lists (e.g. the repodata dirs) in real time.
>
Sounds good, but it still seems like we need special tools to determine
on the fly, with standard tools, if a mirror is in a sync'ed state. If I
need to create a local mirror of fedora X with updates, with standard
tools, I have to figure out when to mirror by trial and error and have
no way to check before the sync, if the mirror is in the middle of
syncing itself.
The mirrormanager you're working on will be useful for monitoring and
flagging stale mirrors. It could work well for yum too, but for the
basic mirroring, we still lack a simple mechanism to allow us to track
the state of upstream mirrors. Mirroring a few gigs of inconsistent data
makes no sense at all and this can be implemented very very easily. It's
all a matter of documenting "HOWTO setup a fedora mirror".
> That's the idea. A lot of the code is implemented, there's more to
> go. If you're good with python, turbogears, and the like, I'm sure I
> could put you to work on it. Drop me a note.
>
Never worked with turbogears, sorry ;-)
/Thomas
More information about the devel
mailing list