On 7/30/14, 2:32 PM, Mike McLean wrote:
On 07/29/2014 05:43 PM, Mike McLean wrote:
> These changes cause the hub to use generators for some potentially large
> queries internally. The point is to reduce memory usage.
>
> In the case that a large query is returned via rpc, the marshaller has
> been extended to handle generators. The marshaller will still create a
> giant pile of xml, but we at least still save the memory for the
> original data.
>
> I also have some work on an iterating marshaller that will not keep the
> entire xml return in memory, but that's a bit more complicated, so I'm
> going to leave that for another day.
>
> There are other cases (e.g. repo_init) where we have very large queries
> internally. In these cases, this patch set will result in significant
> memory savings.
A few more related changes, additional logging and a bugfix.
Also, will probably drop the chunksize patch, doesn't seem to matter
when unless the db connection has a lot of lag.
These changes are working well for me and are likely to land in git soon.
Here's a small patch to fix chain-maven, which can override a G:A:V in
the tag with a build from the chain. Tested locally and it fixes the
regression in chain-maven builds for me.