Keeping old versions of packages

[Bill Nottingham]

Richard Hughes (hughsient at gmail.com) said: 
> Using PackageKit and yum on the command line is often painful as we
> have to always download metadata unless it's less than a few hours
> old. Being able to update the metadata once a week would be awesome
> (with the possible exception of security updates) so that we could
> schedule the 20Mb+ metadata update when the user is idle rather than
> waiting for updates.
> 
> I'm wondering what the interest would be in keeping packages
> referenced in metadata on the mirrors for say a month? We'd probably
> need a separate fedora-security repo too that's designed to be kept
> small enough so that metadata checks every day would be not costly in
> terms of bandwidth and time.
> 
> If anyone is interested in doing this, you'd be awesome. Thanks.

The thing that creates the repos is 'mash', so that's where such a change
needs to be. It currently talks to koji to get the builds for a particular
tag, and has the following config settings:

 latest = True

Grab only the latest package

 latest = False

Grab *all* the versions of the packages.

It only supports these two variations, because that's what koji supports.

If you wanted to keep more versions on the mirrors, you have the following
options:

1) Have mash create everything, and then run a script that prunes versions
older than X, and re-runs createrepo.

We used to do this in the old days. It's an utter hack, and not very
efficient.

2) Have mash try and implement a date-based expiry itself from what it
requests from koji.

The problem here is that mash pulls from koji, and only has two times
that it could use as a key:

- build time (not really what you want, as packages can build and sit
  for a while)

- time it was tagged into the repo (which is affected by rel-eng operations
  if packages need to be retagged or moved)

3) Have mash sort through the packages retrieved by koji, and pull some
subset less than 'all' (the last 2/3/4 versions).

This is probably doable - it requires requesting all tagged packages from
koji and then doing some postprocessing on the list before you download
them. It's merely a matter of code.

So, needless to say, I'd suggest anyone interested in this to look at option
#3. Note that enabling something like that on rawhide would have a large
effect on the repository creation time - there's only so many ways to speed
up scanning across 50000 packages & 100GB on a SAN.

Bill (former mash maintainer)