[announce] yum: parallel downloading

[John Reiser]

On 05/18/2012 08:59 AM, Reindl Harald wrote:
> why are making the connections to the SAME mirror at all?
> it would make much more sense to download packages
> parallel and each one from a different mirror

I find that two simultaneous threads to the same one mirror
gives shortest time to completion for an entire list of downloads,
particularly when one thread downloads from smallest to largest,
while the other thread downloads from largest to smallest.
The latency for setup+takedown of a connection for each package
represents lost bytes that could have been transferred.  The other
thread fills that gap much of the time.  When both threads actually
are sending, then the network algorithms (and/or server policies
regarding allocation of resources to the same endpoint) work,
maintaining near-maximal total transfer rate at very low cost.

My connection is Comcast cable modem: advertised as 12 to 15Mbit/s,
often peaking at 25Mbit/s, never more than 30Mbit/s.
[Network buffer bloat _helps_ me when I download larger packages.]
The median package is around 120KByte: setup+takedown is _longer_
than transfer time to me.  As transfer speed increases, then
the setup+takedown becomes an even larger fraction of total time.
Using http://, it can be very advantageous to perform multiple
consecutive GET commands on the same connection (counting bytes to
separate the returned concatenation), bypassing the takedown+setup
for the next file.

Package size of 1MByte (around 1 second of sending time to me)
occurs at position 2303 of 2674 packages in Fedora 17 [smallest
to largest, and not counting @Languages.]  86% of packages are
1MByte or smaller.  Package size of 5MByte is position 2568
of 2674 packages.

--