[announce] yum: parallel downloading

Zdenek Pavlas zpavlas at redhat.com
Mon May 21 10:33:51 UTC 2012


> The number of concurrent users is now lower because, well, each of
> them now completes a "yum update" in one third of the time.

I think Glen's concerns were that the consumed resources
(flow caches, TCP hash entries, sockets) may scale faster
than the aggregated downloading speed.

I am aware of this, and in most cases the downloader in urlgrabber
will make just 1 concurrent connection to a mirror, because:

1) The Nth concurrent connection to the same host is assumed
   to be N times slower than 1st one, so we'll very likely
   not select the same mirror again.

2) maxconnections=1 in every metalink I've seen so far.
   This is a hard limit, we block until a download finishes
   and reuse one connection when the limit is maxed out.

The reason for NOT banning >1 connections to the same host altogether
is that (as John Reiser wrote) 2nd connection does help quite a lot
when downloading many small files and just one mirror is available.
I agree that using strictly 1 connection and HTTP pipelining would 
be even better, but we can't do that with libcurl.

--
Zdenek


More information about the devel mailing list