proposed MM patch: improve rsync_acl query

[Matt Domsch]

mirrormanager has a query that lets the mirrors, and eventually, the
master mirrors, get the Access Control List directly from the
database.  The current query is poor in several ways: it's very slow
as it uses python object walks, which results in hundreds of database
hits instead of only one.  It doesn't let the user specify they want
to limit by hosts on Internet2, or only public hosts.

Patch below fixes these.  The URL will work as:

http://admin.fedoraproject.org/mirrormanager/rsync_acl?internet2_only=1&a...

The arguments on the end are optional.

The mirrors are starting to use this query to populate their own rsync
ACLs.  It's not critical that it go in before F9 launch, but I think
it's quite low-risk.  I've tested it locally and it works as expected.

Thanks,
Matt

-- 
Matt Domsch
Linux Technology Strategist, Dell Office of the CTO
linux.dell.com & www.dell.com/linux

[Toshio Kuratomi]

Matt Domsch wrote:
>  mirrormanager has a query that lets the mirrors, and eventually, the
>  master mirrors, get the Access Control List directly from the
>  database.  The current query is poor in several ways: it's very slow
>  as it uses python object walks, which results in hundreds of database
>  hits instead of only one.  It doesn't let the user specify they want
>  to limit by hosts on Internet2, or only public hosts.
>  
>  Patch below fixes these.  The URL will work as:
>  
>
> http://admin.fedoraproject.org/mirrormanager/rsync_acl?internet2_only=1&a...
>  
>  The arguments on the end are optional.
>  
>  The mirrors are starting to use this query to populate their own rsync
>  ACLs.  It's not critical that it go in before F9 launch, but I think
>  it's quite low-risk.  I've tested it locally and it works as expected.
>   
Err... no patch was attached but the feature sounds good.  If it's 
really low risk it sounds like a win to put it in.

-Toshio

[Matt Domsch]

On Sun, Apr 13, 2008 at 11:35:55AM -0700, Toshio Kuratomi wrote:
>  Matt Domsch wrote:
>  >mirrormanager has a query that lets the mirrors, and eventually, the
>  >master mirrors, get the Access Control List directly from the
>  >database.  The current query is poor in several ways: it's very slow
>  >as it uses python object walks, which results in hundreds of database
>  >hits instead of only one.  It doesn't let the user specify they want
>  >to limit by hosts on Internet2, or only public hosts.
>  >
>  >Patch below fixes these.  The URL will work as:
>  >
>
> >http://admin.fedoraproject.org/mirrormanager/rsync_acl?internet2_only=1&public_only=1
>  >
>  >The arguments on the end are optional.
>  >
>  >The mirrors are starting to use this query to populate their own rsync
>  >ACLs.  It's not critical that it go in before F9 launch, but I think
>  >it's quite low-risk.  I've tested it locally and it works as expected.
>  >
>  Err... no patch was attached but the feature sounds good.  If it's 
>  really low risk it sounds like a win to put it in. 
Doh.  mailman ate the patch.  Here I committed it to the upstream, but
I haven't yet pushed to puppet.  Here's the patch.

http://git.fedorahosted.org/git/mirrormanager?p=mirrormanager;a=commitdif...


-- 
Matt Domsch
Linux Technology Strategist, Dell Office of the CTO
linux.dell.com & www.dell.com/linux

[Matt Domsch]

On Sun, Apr 13, 2008 at 02:09:08PM -0500, Matt Domsch wrote:
>  On Sun, Apr 13, 2008 at 11:35:55AM -0700, Toshio Kuratomi wrote:
>  > Matt Domsch wrote:
>  > >mirrormanager has a query that lets the mirrors, and eventually, the
>  > >master mirrors, get the Access Control List directly from the
>  > >database.  The current query is poor in several ways: it's very slow
>  > >as it uses python object walks, which results in hundreds of database
>  > >hits instead of only one.  It doesn't let the user specify they want
>  > >to limit by hosts on Internet2, or only public hosts.
>  > >
>  > >Patch below fixes these.  The URL will work as:
>  > >
>  >
> >http://admin.fedoraproject.org/mirrormanager/rsync_acl?internet2_only=1&public_only=1
>  > >
>  > >The arguments on the end are optional.
>  > >
>  > >The mirrors are starting to use this query to populate their own rsync
>  > >ACLs.  It's not critical that it go in before F9 launch, but I think
>  > >it's quite low-risk.  I've tested it locally and it works as expected.
>  > >
>  > Err... no patch was attached but the feature sounds good.  If it's 
>  > really low risk it sounds like a win to put it in.
>  
>  Doh.  mailman ate the patch.  Here I committed it to the upstream, but
>  I haven't yet pushed to puppet.  Here's the patch.
>  
>
> http://git.fedorahosted.org/git/mirrormanager?p=mirrormanager;a=commitdif...
With no objections and two acks, I went ahead and put this in
production.  Query time down from 60s to 1s.

-- 
Matt Domsch
Linux Technology Strategist, Dell Office of the CTO
linux.dell.com & www.dell.com/linux

[Jima]

On Mon, 14 Apr 2008, Matt Domsch wrote:
>  With no objections and two acks, I went ahead and put this in
>  production.  Query time down from 60s to 1s. 
  Awesome, nice work.

      Jima