RFC: Old packages remain on the mirrors for one week

[Richard Hughes]

Hi all,

I'd like to ask for comments on a feature I need for the Fedora
Application Installer. The current yum backend in PackageKit does
something like this:

* yum install foo
* depsolve transaction using cached metadata
* download foo-0.1.noarch.rpm
* error! foo-0.1.noarch.rpm doesn't exist
* download latest repomd, primary
* re-depsolve
* download latest filelists
* continue to re-depsolve
* download foo-0.2.noarch.rpm
* install foo using librpm

Now, we do this as the metadata is cached on the client side for up to
a week as we don't want to unconditionally update the metadata for
every transaction, but we don't know if we can download the package
without downloading all the metadata beforehand. This is incompatible
with the swish UX in the application installer where we can search for
things straight away without having "Downloading..." in the UI
appearing at odd times. So my proposal is thus:

1. We retain old packages on the mirrors for a minimum of 7 days.
2. We regenerate the metadata on every compose like before
3. We only include the latest package version in the metadata
4. If the user is installing an "old" package we check if the new
package is a security or important update and re-download all metadata
if so

Point 3 means that the metadata size does not explode, and CLI tools
like yum don't spend minutes depsolving a much larger set of packages.
Although this increases the amount of space required on the mirrors
(by about 15% for fedora-19 by my approximation), the amount of
bandwidth saved is huge. By my calculations, over the last 7 weeks
[with ~10 offline updates, and hundreds of 'yum' commands] over 60% of
my traffic from the mirrors is metadata!

FWIW; 1,2,3 is what Debian and Ubuntu do. Comments welcome, thanks.

Richard