Hey Presto!

[Michael Schroeder]

On Sat, Jun 12, 2010 at 04:35:42PM +0100, Richard W.M. Jones wrote:
> On Sat, Jun 12, 2010 at 05:27:32PM +0200, drago01 wrote:
> > We don't generate deltas for packages with a size of >= 100MB ....
> > which kind of makes it useless for this case but it seems that delta
> > generation is to expensive to do for such large packages on the re-eng
> > boxes.
> 
> It's because the program that generates the delta RPMs reads the whole
> RPMs into memory, according to:
> 
> http://lwn.net/Articles/329484/

It currenlty uses three big chunks of memory
- the source rpm payload (unpacked)
- the target rpm payload (unpacked)
- a hash over the source rpm (exactly the size of the unpacked
  source rpm payload)

So it needs roughly three times the size of the unpacked payload.

> Anyone know if there's a genuine reason why it does it, or if it's
> just a Simple Matter Of Programming to fix it?  (And can point us to
> the actual bit of code that could be fixed ...)

It's not that hard to fix, there's no need to keep the "target"
rpm in memory at all. The "source" rpm can be limited to some
max size with the down side that the end of the "target" rpm
cannot match the start of the "source" rpm anymore. This shouldn't
do much harm in the real world.

I've already looked at the code, it shouldn't be much work to
implement. I'll try to do it this or next week.

Cheers,
  Michael.

-- 
Michael Schroeder                                   mls at suse.de
SUSE LINUX Products GmbH, GF Markus Rex, HRB 16746 AG Nuernberg
main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);}