speeding up the group/dump query in FAS

[Toshio Kuratomi]

On Mon, Sep 06, 2010 at 09:53:55PM -0500, Mike McGrath wrote:
> On Mon, 6 Sep 2010, Matt_Domsch at Dell.com wrote:
> 
> > Currently this query takes several minutes.  HAProxy (or something in the proxy series) times out the request, and returns a 500 to the client.
> >   https://admin.fedoraproject.org/accounts/group/dump/
> >
> > Any chance that could be optimized?  FWIW, I use this query in my ftbfs script to convert package owner names to email addresses, so I can send the FTBFS report to the owner's emails directly (on bcc).
> >
> 
> This must be related to the alchemy alterations we did recently.  I'll
> take a look and see what it would take.  I was under the assumption this
> particular dump was memcached so it should be very quick but I could be
> wrong on that.
> 
I took a look and we should be able to do something similar to what we did
for /groups/list => switch from using code that targets the ORM layer to
code that targets the SQL layer and do the privacy_filtering type of stuff
manually.  The tough part is that we probably should figure out how to make
that type of code more generic so that /gorups/list and groups/dump and
anything else that needs it can use the same thing.

It took roughly a day to create the changes to /groups/list -- since we
could copy and paste if we don't figure out how to genericify, probably less
time to get this up and running.

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
Url : http://lists.fedoraproject.org/pipermail/infrastructure/attachments/20100907/9e195e32/attachment.bin