Yum, Proxy Cache Safety, Storage Backend
Les Mikesell
lesmikesell at gmail.com
Thu Jan 24 19:49:30 UTC 2008
Nicolas Mailhot wrote:
>
>> I'm missing something here. A cache is only going to resend what it got
>> the first time. If the first user got the right thing, so will the
>> second.
>
> No it won't. Common failures are:
>
> - T0: user A updates foo and in the course of doing it populates the
> proxy cache with yum T0 metadata and foo rpm only.
> - T1: upstream updates repo changing bar rpm version. Metadata is
> regenerated with the same filenames as before. For the proxy its T0 copy
> is still valid till the expiration delay it got in http headers the
> first time.
> - T2: user B wants to update bar. The proxy happily serves yum the
> cached T0 metadata (since yum didn't request any special treatment for
> it). It tells yum the T0 bar is available. Unfortunately it's not - user
> A didn't download it so it's not in the cache, and upstream moved to a
> new version
Can't the T1->T2 transition happen while user A is in mid-update? I'm
fairly sure I've seen something like that without any proxies involved.
> - T0: user A updates foo and in the course of doing it populates the
> proxy cache with yum T0 metadata and foo rpm
> - T1: upstream signs foo rpm and regenerates the repo. Metadata is
> regenerated with the same filenames as before. For the proxy its T0 copy
> is still valid till the expiration delay it got in http headers the
> first time.
> - T2: proxy cache is contended, it evicts foo rpm since it takes a lot
> of room. T0 metadata is small so it's retained for now
> - T3: user B wants to update foo. The proxy happily serves yum the
> cached T0 metadata (since yum didn't request any special treatment for
> it). It tells yum foo is available with T0 checksum. Unfortunately it's
> not - T0 foo rpm was evicted from the cache, when user B requests the
> same URL as user A it gets T1 signed foo with a new checksum.
>
> (you can invert this case with a proxy which strategy is to keep big
> files which are expensive to download, and drop small metadata)
>
> In all those cases the proxy worked as designed and was fooled by
> yum/createrepo using the same URLs for different objects.
Again, I think you'll still have the same issues if user A's update
spans any of these arbitrary Tx designations - at least the ones where
you've got metadata and the rpms are changed before you get them.
> A proxy will always serve you a set of files as they existed on the
> original location at some time. But there is zero insurance they were
> taken from this location at the same time, so you *must* take care new
> content is created with new filenames, or your web server tells the
> proxy when old data will be replaced by new one (via expires), or your
> system is able to cope with a mix of files from different times.
But you are doing things that aren't guaranteed to work in the first
place. All the cache does is expand the window for breakage a bit.
--
Les Mikesell
lesmikesell at gmal.com
More information about the devel
mailing list