RFC: Old packages remain on the mirrors for one week

[Dennis Gilmore]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mon, 12 Aug 2013 14:47:03 +0100
Richard Hughes <hughsient at gmail.com> wrote:

> Hi all,
> 
> I'd like to ask for comments on a feature I need for the Fedora
> Application Installer. The current yum backend in PackageKit does
> something like this:
> 
> * yum install foo
> * depsolve transaction using cached metadata
> * download foo-0.1.noarch.rpm
> * error! foo-0.1.noarch.rpm doesn't exist
> * download latest repomd, primary
> * re-depsolve
> * download latest filelists
> * continue to re-depsolve
> * download foo-0.2.noarch.rpm
> * install foo using librpm
> 
> Now, we do this as the metadata is cached on the client side for up to
> a week as we don't want to unconditionally update the metadata for
> every transaction, but we don't know if we can download the package
> without downloading all the metadata beforehand. This is incompatible
> with the swish UX in the application installer where we can search for
> things straight away without having "Downloading..." in the UI
> appearing at odd times. So my proposal is thus:
> 
> 1. We retain old packages on the mirrors for a minimum of 7 days.

without completely rewriting how we compose the trees this is not a
possibility.

> 2. We regenerate the metadata on every compose like before
> 3. We only include the latest package version in the metadata

this would need the tools to be completely rewritten also.

> 4. If the user is installing an "old" package we check if the new
> package is a security or important update and re-download all metadata
> if so
this means downloading all the metadata
 
> Point 3 means that the metadata size does not explode, and CLI tools
> like yum don't spend minutes depsolving a much larger set of packages.
> Although this increases the amount of space required on the mirrors
> (by about 15% for fedora-19 by my approximation), the amount of
> bandwidth saved is huge. By my calculations, over the last 7 weeks
> [with ~10 offline updates, and hundreds of 'yum' commands] over 60% of
> my traffic from the mirrors is metadata!
> 
> FWIW; 1,2,3 is what Debian and Ubuntu do. Comments welcome, thanks.
> 
> Richard

with the schedules as they have been I really dont know when Id get any
time to work on the tooling for composing. certainly not before Fedora
21 likely later than that.

Dennis

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iEYEARECAAYFAlINry4ACgkQkSxm47BaWffHQACfQizrrtfRi+qX4+wBK6lQyUQh
jMoAniirXFSRfkcgc63o0TIzISSBrtQI
=s6rn
-----END PGP SIGNATURE-----