A more efficient up2date service using binary diffs

[Michael Schroeder]

On Wed, Mar 23, 2005 at 01:22:20PM +0000, Joe Desbonnet wrote:
> Just a quick followup: I finally managed to cobble together a machine
> so that I could do a full test (FC3 fresh install -> current).
> Unfortunately a bug in the 0.1.0 release prevented the test from
> working -- I wasn't handlng byte range requests properly. That's fixed
> now and I'll post a new release in the next 1 or 2 days once I've
> finished testing.
> 
> Interim results are encouraging: overall bandwidth savings between 66
> - 78% (at the expense of local processing power -- those with slow
> machines on fast links may not benefit).

Hmm, compression rate should even be a bit better than that, at
least that's what we (SuSE) see with the deltarpms.

As most of you don't seem to know how our deltarpms work (and
as I'm the author ;-) ) here's a short description:

Our deltarpms are a binary diff of the rpm payload with the cpio
header information dropped, i.e. only the file contents. Files
with verify flags indicating that they might change (e.g. config
files) are excluded from the diff.

Because of this, applying a deltarpm can work with either the old
rpm or files from the filesystem, i.e. there's no need to keep
the old rpms. As one can't be 100% sure that the files are still
intact one can test if a deltarpm would apply, for this check only
the so called signature is necessary. So the update process can
first get the signature, test if the delta would work, and fall
back to the full rpm if the test fails.

The code is available from
    ftp://ftp.suse.com/pub/projects/deltarpm/deltarpm-2.2.tar.bz2

Creating and applying deltarpms is very easy:

    makedeltarpm oldrpm newrpm deltarpm

creates deltarpm,

    applydeltarpm deltarpm newrpm2

re-creates newrpm. You can also use

    applydeltarpm -r oldrpm deltarpm newrpm2

if you don't want to use files from the filesystem. Check out the 
manpage for more options. Makedeltarpm uses a modified bsdiff which
uses a hash instead of a suffix array to speed things up.

Btw, Seth Vidal claimed that the Zen/Red Carpet people hate deltarpms.
I don't know why he thinks so, they don't even know about deltarpms,
how could they hate them?

Cheers,
  Michael.

-- 
Michael Schroeder                                   mls at suse.de
main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);}