Suggestions (how to choose mirrors)

[James Antill]

On Wed, 2010-01-27 at 10:50 +0100, Roberto Ragusa wrote:
> Matt Domsch wrote:
> > Fedora has 244 public mirrors listed at the moment, and hundreds more
> > private mirrors.  We have no way, nor really any desire, to know in
> > real time the load and network capacity of any connection between
> > each mirror and each specific user.  Perhaps Akamai does, but that's
> > not a service we pay for.
> 
> Just an idea: instead of (or in addition to) "blind" planning,
> based on net topology, geography, declared bandwith etc.,
> yum could use an exploration approach:
> 
> 1) choose a few good mirrors candidate
> 2) download one file from each of them (first file from
> first mirror, second file from second mirror, ....)
> 3) gather speed statistics
> 4) reevaluate best mirrors according to statistics for the
> remaining files

 This is fine in theory, but there are a few problems. And more than a
few changes have to be made before we can do it.

> If the downloads are sorted by increasing size, you basically
> use the small ones to sample the mirrors and make a good choice
> for the big ones at the end of the list.

 Except we got significant complaints when we did that sort, so I'm not
dying to do it again (even though I did it, and thought it was better).

> (doing many downloads in parallel would be the real plus,
> so the slow and ugly mirror taking 1 minute for the 40kB file
> will complete while the good mirrors are serving you the
> kernel and openoffice.org)

 Pipelining is on the TODO, and will likely be done sometime this year.
The rest of it is blocked behind that.

> This would also be automatically optimal for local mirrors.

 Unlikely, the problem with local mirrors is that it's often optimal to
download from them as we are atm. ... any attempt to use another mirror
is slower (and sometimes more expensive).
 In fact my local mirror is often so fast that I seriously doubt whether
much improvement could be made due to pipelining etc. (which is not the
case with anything outside my network).

-- 
James Antill - james at fedoraproject.org
http://yum.baseurl.org/wiki/releases
http://yum.baseurl.org/wiki/whatsnew/3.2.26
http://yum.baseurl.org/wiki/YumMultipleMachineCaching