Proposal: stop holding composes for preupgrade bugs at Alpha and Beta phases

Richard Ryniker ryniker at alum.mit.edu
Tue Apr 10 19:03:46 UTC 2012


>The thread for longer files is more likely to be in the middle of a
>transfer, and thus capable of boosting its throughput, while the other
>thread suffers low throughput due to starting or stopping the transfer of
>an individual file.

Maybe.  Certainly "The thread for longer files is more likely to be in
the middle of a transfer" is true, but "capable of boosting its
throughput" is suspect.  High-volume servers may optimize overall
performance using estimates of the effective bandwidth to each client
system.  Server resources are then managed to deliver data for that
client at no faster rate, if useful work for other clients can be
performed.

The purpose of this strategy is to avoid use of server resources to send
additional data to a client when it is reasonable to believe network
conditions are unlikely to permit any faster delivery than if that data
were sent at a measured effective data rate.  The server does not want
its optimization strategy flummoxed by something like a router's "fair
share" algorithm that may cause significant variations in individual
message processing times.

Of course, network performance is a dynamic thing, and estimates of
effective bandwidth must be updated over time - but this time is likely
to be much longer than the time needed to start or stop an individual
transfer operation.

Even when server resources are not currently strained, optimization
strategies are likely to be in use because (1) they are not believed to
significantly degrade performance under less heavy load and (2) server
load is erratic, therefore it is simpler and better to always have
optimization active than to try to switch it on just when it may be
needed.

That said, it looks to me like dl.fedoraproject.org does not use such an
optimization strategy.  On the other hand, kernel.org does, and takes
about five seconds to reach my maximum bandwidth of 2 Mbytes per second.

Please do not take any of this as criticism of the use of multiple
threads for data transfer operations.  I strongly support that notion,
and would argue more than two threads may be desirable.  While much less
frequent than I used to see, there are still times when I find data
transfer operations seriously degraded by what appears to be packet loss,
and consequent long delays until a system retransmits a message.  Many
threads is not a perfect solution, but it is much more aggreable to see
significant network activity than just the occasional retransmission of a
message to recover from lost packets.




More information about the test mailing list