Mirrormanager thoughts
Matt Domsch
Matt_Domsch at dell.com
Mon Jan 8 22:06:23 UTC 2007
On Sun, Jan 07, 2007 at 11:38:31PM -0600, Matt Domsch wrote:
> https://hosted.fedoraproject.org/projects/mirrormanager/wiki/WikiStart
>
> I've redone the mirrormanager schema and initialization data. I
> haven't redone the controllers or templates at all yet, so this won't
> run except in the tg-admin shell and tg-admin toolbox (catwalk is
> nice). Please take a look. I append the schema file for review. The
> hierarchy is as follows:
More schema updates. Three new objects.
Product is a top-level content name, e.g. 'rhel' or 'fedora'.
Products can have many ProductVersions. Some versions are 'test' versions
(e.g. 6.90). This needed as the path to fedora-test-6.90 looks like
fedora/linux/core/test/6.90/ while the path to fedora-core-6 looks
like fedora/linux/core/6/ (the extra 'test' indirection there...)
Category is meant to be able to capture the standard directory
structure for 'core', 'extras', 'updates', 'updates-testing', 'test',
and 'epel' cleanly. It uses $VERSION and $ARCH in the path names to
be replaced when creating a Content.
> Content is the highest level unit of data that can be synced.
Content refers to the combination of a Category, a ProductVersion, and
an Arch, and is still the highest level unit of data that can be synced.
These have names like fedora-core-6-i386. (product, category, version, arch).
The other objects are unchanged fundamentally.
Your comments would be appreciated.
Thanks,
Matt
--
Matt Domsch
Software Architect
Dell Linux Solutions linux.dell.com & www.dell.com/linux
Linux on Dell mailing lists @ http://lists.us.dell.com
from sqlobject import *
from turbogears.database import PackageHub
hub = PackageHub("mirrors")
__connection__ = hub
from mirrors.identity_models import *
class Site(SQLObject):
name = StringCol(alternateID=True)
robot_email = StringCol(default=None)
admin_active = BoolCol(default=False)
user_active = BoolCol(default=True)
private = BoolCol(default=False)
admins = MultipleJoin('SiteAdmin')
hosts = MultipleJoin('Host')
ips = MultipleJoin('SiteIP')
class SiteAdmin(SQLObject):
username = StringCol()
site = ForeignKey('Site')
# IP addresses to go in the rsync ACLs
class SiteIP(SQLObject):
site = ForeignKey('Site')
address = StringCol()
class Host(SQLObject):
name = StringCol()
site = ForeignKey('Site')
country = StringCol(default=None)
internet2 = BoolCol(default=False)
pull_from = ForeignKey('Site')
mirrors = MultipleJoin('Mirror')
private_rsyncs = MultipleJoin('HostPrivateRsync')
ip_blocks = MultipleJoin('HostIPBlocks')
# for other mirrors to pull from in tiering
class HostPrivateRsync(SQLObject):
host = ForeignKey('Host')
url = StringCol()
user = StringCol(default=None)
password = StringCol(default=None)
# some hosts only serve some IP CIDR blocks
# such as private mirrors behind
# company firewalls. This lets them first try
# the global yum mirrorlist redirector which will
# return them their local mirror URL
class HostIPBlocks(SQLObject):
host = ForeignKey('Host')
cidr = StringCol()
class Arch(SQLObject):
name = StringCol(alternateID=True)
class Category(SQLObject):
# e.g. core, extras, updates, updates-testing, test, ...
name = StringCol(alternateID=True)
# with $VERSION and $ARCH for later substitution
path = StringCol()
canonicalhost = StringCol()
sourcepkgpath = StringCol()
sourceisopath = StringCol(default=None)
contents = MultipleJoin('Content')
class Product(SQLObject):
name = StringCol(alternateID=True)
versions = MultipleJoin('ProductVersion')
class ProductVersion(SQLObject):
name = StringCol()
product = ForeignKey('Product')
isTest = BoolCol(default=False)
class Content(SQLObject):
# e.g. fedora-core-6-i386
name = StringCol(alternateID=True)
category = ForeignKey('Category')
# 5, 6, development
version = ForeignKey('ProductVersion')
# default subdir starting below /
# e.g. 'pub/fedora/core/6/$ARCH/'
path = StringCol()
arch = ForeignKey('Arch')
# these are paths below $ARCH/
# e.g. iso/, os/, debuginfo/
# of course Extras doesn't have these ;-(
isos = StringCol(default=None)
binarypackages = StringCol(default=None)
repodata = StringCol(default=None)
debuginfo = StringCol(default='debug/')
repoview = StringCol(default=None)
mirrors = MultipleJoin('Mirror')
# one per Host per Content
# fortunately these will be created/modified
# by a crawler program rather than manually
class Mirror(SQLObject):
host = ForeignKey('Host')
content = ForeignKey('Content')
urls = MultipleJoin('MirrorURL')
# sync status
dvd_isos_synced = BoolCol(default=False)
cd_isos_synced = BoolCol(default=False)
binarypackages_synced = BoolCol(default=False)
debuginfo_synced = BoolCol(default=False)
repoview_synced = BoolCol(default=False)
# One per protocol/path one can get at this same data
# checking one MirrorURL is the same as checking them all
class MirrorURL(SQLObject):
mirror = ForeignKey('Mirror')
# The equivalent of the dir structure on the masters
# e.g. http://mirrors.kernel.org/fedora/core/6/i386/os/
path = StringCol(default=None)
More information about the infrastructure
mailing list