speeding up the group/dump query in FAS
a.badger at gmail.com
Tue Sep 7 15:07:28 UTC 2010
On Mon, Sep 06, 2010 at 09:53:55PM -0500, Mike McGrath wrote:
> On Mon, 6 Sep 2010, Matt_Domsch at Dell.com wrote:
> > Currently this query takes several minutes. HAProxy (or something in the proxy series) times out the request, and returns a 500 to the client.
> > https://admin.fedoraproject.org/accounts/group/dump/
> > Any chance that could be optimized? FWIW, I use this query in my ftbfs script to convert package owner names to email addresses, so I can send the FTBFS report to the owner's emails directly (on bcc).
> This must be related to the alchemy alterations we did recently. I'll
> take a look and see what it would take. I was under the assumption this
> particular dump was memcached so it should be very quick but I could be
> wrong on that.
I took a look and we should be able to do something similar to what we did
for /groups/list => switch from using code that targets the ORM layer to
code that targets the SQL layer and do the privacy_filtering type of stuff
manually. The tough part is that we probably should figure out how to make
that type of code more generic so that /gorups/list and groups/dump and
anything else that needs it can use the same thing.
It took roughly a day to create the changes to /groups/list -- since we
could copy and paste if we don't figure out how to genericify, probably less
time to get this up and running.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 198 bytes
Desc: not available
Url : http://lists.fedoraproject.org/pipermail/infrastructure/attachments/20100907/9e195e32/attachment.bin
More information about the infrastructure