ostree/fedora atomic and impact on the mirror network

[Colin Walters]

On Tue, Mar 11, 2014 at 9:34 AM, Martin Langhoff 
<martin.langhoff at gmail.com> wrote:
> 
> Ouch -- so updates fetch EVERY file regardless of whether it has
> changed between the snapshot installed and the target snapshot? That
> is kind of bad.
> 

No.  Clients only fetch objects they don't have.

> Are you aware of the work done on OLPC's os-builder, which used rsync
> "informed" by a per-snapshot metatada file?
> 

I'd definitely looked at older versions of olpc-update, but not at the 
latest.

Bottom line though is that while rsync is a very good general purpose 
tool (for ad-hoc sync between two machines in particular), it spends 
CPU and I/O time on both client and server in order to attempt to 
minimize network bandwidth.  This is not good for the "pool of servers 
replicating to many clients" case.

Now rsync's "batch mode" allows precomputing on the server once, which 
helps a lot - and it starts to get closer to where I'm going with 
OSTree static deltas.  The main difference then is that OSTree static 
deltas are much more flexible - the rsync style "rolling checksum" is 
only one of several possible algorithms.

The OSTree static delta design was inspired by the network protocol 
part of:
http://www.chromium.org/chromium-os/chromiumos-design-docs/autoupdate-details

There are other details too, like the fact that static deltas are 
chunked to say 16MB, so an interrupted client won't have to re-download 
more than one chunk.  They also reuse the integrated GPG verification 
in OSTree, etc.

> In our experience, the problem we had was network latency. On a local
> network, a small OS image, and with rsync client sending requests
> speculatively a bit ahead of receiving the response to its prev
> request... the lag for clients updating was horrible.
> 

The main latency with OSTree comes from metadata fetches.

> How long does an update of a full Fedora 
> 

You'd have to define "full Fedora" =)  And furthermore what you're 
updating from and to.

> (Is there a good overview I should be reading?)
> 

There's not a good overview of this particular level.  But once you 
understand that OSTree is like git, with commits, trees, and content 
objects, and that in the "archive" mode they're just stored as 
individual zlib-compressed files, the rest of the design kind of 
follows from there.

Two recent blog posts are:
http://samthursfield.wordpress.com/2014/01/08/os-level-version-control/
http://samthursfield.wordpress.com/2014/01/16/the-fundamentals-of-ostree/

I also have several here:
http://blog.verbum.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20140311/24b09883/attachment.html>