Re: Yum, Proxy Cache Safety, Storage Backend

[Nicolas Mailhot]

Le jeudi 24 janvier 2008 à 12:49 -0600, Les Mikesell a écrit :

>  I'm missing something here.  A cache is only going to resend what
> it got 
>  the first time.  If the first user got the right thing, so will the 
>  second. 
No it won't. Common failures are:

- T0: user A updates foo and in the course of doing it populates the
proxy cache with yum T0 metadata and foo rpm only.
- T1: upstream updates repo changing bar rpm version. Metadata is
regenerated with the same filenames as before. For the proxy its T0 copy
is still valid till the expiration delay  it got in http headers the
first time.
- T2: user B wants to update bar. The proxy happily serves yum the
cached T0 metadata (since yum didn't request any special treatment for
it). It tells yum the T0 bar is available. Unfortunately it's not - user
A didn't download it so it's not in the cache, and upstream moved to a
new version

- T0: user A updates foo and in the course of doing it populates the
proxy cache with yum T0 metadata and foo rpm 
- T1: upstream signs foo rpm and regenerates the repo. Metadata is
regenerated with the same filenames as before. For the proxy its T0 copy
is still valid till the expiration delay  it got in http headers the
first time.
- T2: proxy cache is contended, it evicts foo rpm since it takes a lot
of room. T0 metadata is small so it's retained for now
- T3: user B wants to update foo. The proxy happily serves yum the
cached T0 metadata (since yum didn't request any special treatment for
it). It tells yum foo is available with T0 checksum. Unfortunately it's
not - T0 foo rpm was evicted from the cache, when user B requests the
same URL as user A it gets T1 signed foo with a new checksum.

(you can invert this case with a proxy which strategy is to keep big
files which are expensive to download, and drop small metadata)

In all those cases the proxy worked as designed and was fooled by
yum/createrepo using the same URLs for different objects.

A proxy will always serve you a set of files as they existed on the
original location at some time. But there is zero insurance they were
taken from this location at the same time, so you *must* take care new
content is created with new filenames, or your web server tells the
proxy when old data will be replaced by new one (via expires), or your
system is able to cope with a mix of files from different times.

-- 
Nicolas Mailhot