Updates using idle bandwidth

[Sunil Ghai]

Hi,
We currently have yum-updatesd to check for the updated packages
automatically. Even to download and install them, if marked. Yum has a
throttle option which fixes the upper limit of maximum bandwidth usage. How
about fixing the upper limit dynamically depending upon the current usage of
bandwidth? i.e to download the packages using idle bandwidth.
Those who have limited or slow Internet connection in many developing
countries, need to download packages without feeling noticeable changes in
other applications. e.g in browsing or downloading some another important
file. This way updates would be ready to install without suffering slow
internet connection meanwhile.
I am interested in implementing this, but problem is this would need the
support of repository server(s) to transfer files asynchronously. Will it be
possible?
http://fedoraproject.org/wiki/SummerCoding/2008/Ideas#head-f245bfb86b1ec5...
Regards,
Sunil Ghai

[Seth Vidal]

On Sun, 2008-03-02 at 15:29 -0500, Sunil Ghai wrote:
>  Hi,
>  We currently have yum-updatesd to check for the updated packages
>  automatically. Even to download and install them, if marked. Yum has a
>  throttle option which fixes the upper limit of maximum bandwidth
>  usage. How about fixing the upper limit dynamically depending upon the
>  current usage of bandwidth? i.e to download the packages using idle
>  bandwidth.
>  Those who have limited or slow Internet connection in many developing
>  countries, need to download packages without feeling noticeable
>  changes in other applications. e.g in browsing or downloading some
>  another important file. This way updates would be ready to install
>  without suffering slow internet connection meanwhile.
>  I am interested in implementing this, but problem is this would need
>  the support of repository server(s) to transfer files asynchronously.
>  Will it be possible?
>
> http://fedoraproject.org/wiki/SummerCoding/2008/Ideas#head-f245bfb86b1ec5...
You can have yum-updatesd just download but not install the pkgs - which
may help it.

-sv

[Andrew Farris]

seth vidal wrote:
>  On Sun, 2008-03-02 at 15:29 -0500, Sunil Ghai wrote:
> > Hi,
> > We currently have yum-updatesd to check for the updated packages
> > automatically. Even to download and install them, if marked. Yum has a
> > throttle option which fixes the upper limit of maximum bandwidth
> > usage. How about fixing the upper limit dynamically depending upon the
> > current usage of bandwidth? i.e to download the packages using idle
> > bandwidth.
> > Those who have limited or slow Internet connection in many developing
> > countries, need to download packages without feeling noticeable
> > changes in other applications. e.g in browsing or downloading some
> > another important file. This way updates would be ready to install
> > without suffering slow internet connection meanwhile.
> > I am interested in implementing this, but problem is this would need
> > the support of repository server(s) to transfer files asynchronously.
> > Will it be possible?
> >
> http://fedoraproject.org/wiki/SummerCoding/2008/Ideas#head-f245bfb86b1ec5...
>  
>  You can have yum-updatesd just download but not install the pkgs - which
>  may help it.
>  
>  -sv 
man yum-updatesd.conf does not specify if its possible to set a bandwidth limit 
on this only.  Is it possible to have yum-updatesd use a bandwidth limit but not 
have it set on yum when run manually?

I'm not clear on whether the 'throttle' and 'bandwidth' configuration
would 
apply to yum-updatesd.  It would still not be a dynamic adjustment but if users 
knew how this worked it might be a good enough solution.

-- 
Andrew Farris <lordmorgul(a)gmail.com&gt; www.lordmorgul.net
  gpg 0xC99B1DF3 fingerprint CDEC 6FAD BA27 40DF 707E A2E0 F0F6 E622 C99B 1DF3
No one now has, and no one will ever again get, the big picture. - Daniel Geer
----                                                                       ----

[Stewart Adam]

On Sun, 2008-03-02 at 16:10 -0800, Andrew Farris wrote:
>  
>  man yum-updatesd.conf does not specify if its possible to set a bandwidth limit 
>  on this only.  Is it possible to have yum-updatesd use a bandwidth limit but not 
>  have it set on yum when run manually?
>  
>  I'm not clear on whether the 'throttle' and 'bandwidth' configuration
> would 
>  apply to yum-updatesd.  It would still not be a dynamic adjustment but if users 
>  knew how this worked it might be a good enough solution.
>  
>  -- 
>  Andrew Farris <lordmorgul(a)gmail.com&gt; www.lordmorgul.net
>    gpg 0xC99B1DF3 fingerprint CDEC 6FAD BA27 40DF 707E A2E0 F0F6 E622 C99B 1DF3
>  No one now has, and no one will ever again get, the big picture. - Daniel Geer
>  ----                                                                       ---- 
One
problem I see with this is that bandwidth statistics from are reset when
you reboot the machine. Unless you keep the machine up all the time it will lose track
of the exact amount of bandwidth used up. Maybe another daemon will need to keep track of
this?

Stewart

[Sunil Ghai]

> I'm not clear on whether the 'throttle' and
> 'bandwidth' configuration would
> apply to yum-updatesd.  It would still not be a dynamic adjustment but if 
users
> knew how this worked it might be a good enough solution. 
That would not be the solution in real sense. What if someone wants to
download another important file? A mechanism is needed to stop downloading
the updated packages when the file is being downloaded. When it is done, we
can resume updating the system.

>  We have TCP Low Priority¹ congestion algorithm in kernel.
> yum-updatesd
> should request it on downloading socket via setsockopt(), as per commit
> bc0efe7b46174fa3cadf00ac64e4a751cc4619fd . 
This might help me. Could you please provide me some more pointers?

> One problem I see with this is that bandwidth statistics from are
> reset 
when
> you reboot the machine. Unless you keep the machine up all the time it
> will 
lose track
> of the exact amount of bandwidth used up. Maybe another daemon will
> need to 
keep track of
> this? 
We actually don't need this kind of thing. For
example, if currently 60% of
the bandwidth is in usage, 40% is idle, we can use this idle bandwidth to
download updated packages.Once the list of updated packages has been
prepared, we can start downloading them and if the user shuts down the
machine, next time we can resume downloading the packages where we left off.
In this way user would never feel that the system is actually being updated,
he or she will just get the message that "The system has been updated.."

Problem is this requires server support from where the packages are coming.
Each repository server would require to support asynchronous file
transfer..I am doubtful here..

[Andrew Farris]

Sunil Ghai wrote:
> > I'm not clear on whether the 'throttle' and
> 'bandwidth' configuration would
> > apply to yum-updatesd.  It would still not be a dynamic adjustment but if
>  users
> > knew how this worked it might be a good enough solution.
>  
>  That would not be the solution in real sense. What if someone wants to
>  download another important file? A mechanism is needed to stop downloading
>  the updated packages when the file is being downloaded. When it is done, we
>  can resume updating the system.
>  
> > We have TCP Low Priority� congestion algorithm in kernel. yum-updatesd
> > should request it on downloading socket via setsockopt(), as per commit
> > bc0efe7b46174fa3cadf00ac64e4a751cc4619fd .
>  
>  This might help me. Could you please provide me some more pointers?
>  
> > One problem I see with this is that bandwidth statistics from are reset
>  when
> > you reboot the machine. Unless you keep the machine up all the time it will
>  lose track
> > of the exact amount of bandwidth used up. Maybe another daemon will need to
>  keep track of
> > this?
>  We actually don't need this kind of thing. For example, if currently 60% of
>  the bandwidth is in usage, 40% is idle, we can use this idle bandwidth to
>  download updated packages.Once the list of updated packages has been
>  prepared, we can start downloading them and if the user shuts down the
>  machine, next time we can resume downloading the packages where we left off.
>  In this way user would never feel that the system is actually being updated,
>  he or she will just get the message that "The system has been updated.."
The problem is you need to decide what 40% of 'the bandwidth' actually is, and 
in general you do not know that if you're not paying attention to what maximum 
has been achieved.  Anything else is a guess.  If you really want users to feel 
like all their extra bandwidth is getting used without interfering with traffic 
they want to move you cannot guess there, because tcp will back down on both 
connections as soon as it detects congestion if they aren't prioritized.  You 
would need a mechanism to remember how fast a user's connection really is.

>  Problem is this requires server support from where the packages are
> coming.
>  Each repository server would require to support asynchronous file
>  transfer..I am doubtful here.. 
Won't pretty much all mirrors supporting http handle that.  The mirrors with ftp 
probably won't, so users may need to manually set their chosen mirror to make 
use of what you're trying to do?

-- 
Andrew Farris <lordmorgul(a)gmail.com&gt; www.lordmorgul.net
  gpg 0xC99B1DF3 fingerprint CDEC 6FAD BA27 40DF 707E A2E0 F0F6 E622 C99B 1DF3
No one now has, and no one will ever again get, the big picture. - Daniel Geer
----                                                                       ----

[Sunil Ghai]

> problem is you need to decide what 40% of 'the bandwidth'
> actually is, and
> in general you do not know that if you're not paying attention to what 
maximum
> has been achieved.  Anything else is a guess.  If you really want
> users to 
feel
> like all their extra bandwidth is getting used without interfering
> with 
traffic
> they want to move you cannot guess there, because tcp will back down
> on 
both
> connections as soon as it detects congestion if they aren't
> prioritized. 
 You
> would need a mechanism to remember how fast a user's connection
> really is. 
Yep, setting priority is necessary otherwise the solution will not work and
I think I'll face more problems while implementing but I want to work on
this because me and many people have been facing the same problem for a long
time. Just waiting to get over with my university exams, then I'll start
working, perhaps through Google Summer of Code..

On Mon, Mar 3, 2008 at 4:37 PM, Andrew Farris <lordmorgul(a)gmail.com&gt; wrote:

>  Sunil Ghai wrote:
>  >> I'm not clear on whether the 'throttle' and 'bandwidth'
> configuration
>  would
>  >> apply to yum-updatesd.  It would still not be a dynamic adjustment but
>  if
>  > users
>  >> knew how this worked it might be a good enough solution.
>  >
>  > That would not be the solution in real sense. What if someone wants to
>  > download another important file? A mechanism is needed to stop
>  downloading
>  > the updated packages when the file is being downloaded. When it is done,
>  we
>  > can resume updating the system.
>  >
>  >> We have TCP Low Priority� congestion algorithm in kernel. yum-updatesd
>  >> should request it on downloading socket via setsockopt(), as per commit
>  >> bc0efe7b46174fa3cadf00ac64e4a751cc4619fd .
>  >
>  > This might help me. Could you please provide me some more pointers?
>  >
>  >> One problem I see with this is that bandwidth statistics from are reset
>  > when
>  >> you reboot the machine. Unless you keep the machine up all the time it
>  will
>  > lose track
>  >> of the exact amount of bandwidth used up. Maybe another daemon will
>  need to
>  > keep track of
>  >> this?
>  > We actually don't need this kind of thing. For example, if currently 60%
>  of
>  > the bandwidth is in usage, 40% is idle, we can use this idle bandwidth
>  to
>  > download updated packages.Once the list of updated packages has been
>  > prepared, we can start downloading them and if the user shuts down the
>  > machine, next time we can resume downloading the packages where we left
>  off.
>  > In this way user would never feel that the system is actually being
>  updated,
>  > he or she will just get the message that "The system has been updated.."
>
>  The problem is you need to decide what 40% of 'the bandwidth' actually is,
>  and
>  in general you do not know that if you're not paying attention to what
>  maximum
>  has been achieved.  Anything else is a guess.  If you really want users to
>  feel
>  like all their extra bandwidth is getting used without interfering with
>  traffic
>  they want to move you cannot guess there, because tcp will back down on
>  both
>  connections as soon as it detects congestion if they aren't prioritized.
>   You
>  would need a mechanism to remember how fast a user's connection really is.
>
>  > Problem is this requires server support from where the packages are
>  coming.
>  > Each repository server would require to support asynchronous file
>  > transfer..I am doubtful here..
>
>  Won't pretty much all mirrors supporting http handle that.  The mirrors
>  with ftp
>  probably won't, so users may need to manually set their chosen mirror to
>  make
>  use of what you're trying to do?
>
>  --
>  Andrew Farris <lordmorgul(a)gmail.com&gt; www.lordmorgul.net
>   gpg 0xC99B1DF3 fingerprint CDEC 6FAD BA27 40DF 707E A2E0 F0F6 E622 C99B
>  1DF3
>  No one now has, and no one will ever again get, the big picture. - Daniel
>  Geer
>  ----
>  ----
>
>  --
>  fedora-devel-list mailing list
>  fedora-devel-list(a)redhat.com
>  https://www.redhat.com/mailman/listinfo/fedora-devel-list
>  


-- 
Sunil Ghai

[Bruno Wolff III]

On Tue, Mar 04, 2008 at 15:32:41 -0500,
  Sunil Ghai <sunilkrghai(a)gmail.com&gt; wrote:
>  >problem is you need to decide what 40% of 'the bandwidth'
> actually is, and
>  >in general you do not know that if you're not paying attention to what
>  maximum
>  >has been achieved.  Anything else is a guess.  If you really want users to
>  feel
>  >like all their extra bandwidth is getting used without interfering with
>  traffic
>  >they want to move you cannot guess there, because tcp will back down on
>  both
>  >connections as soon as it detects congestion if they aren't prioritized.
>   You
>  >would need a mechanism to remember how fast a user's connection really is.
>  
>  Yep, setting priority is necessary otherwise the solution will not work and
>  I think I'll face more problems while implementing but I want to work on
>  this because me and many people have been facing the same problem for a long
>  time. Just waiting to get over with my university exams, then I'll start
>  working, perhaps through Google Summer of Code.. 
You might take at the LARTC Howto. It describes ways to can allocate
bandwidth to different things in a way that doesn't force you to not
waste bandwidth if some things don't need it. However for this to work
well you do need some way to estimate what the capacity of the link
is. Also controlling inbound traffic is hard since you can't directly
stop people from sending stuff to you and excess traffic will be dropped
on the other side of the link which isn't under your control.
This would be useful for other things besides updates. Being able to
prioritize interactive traffic over bulk traffic is something a fair
number of people have use for.
When you are looking at the LARTC Howto, note that it is a bit out of
date. IFB interfaces replace IMQ interfaces and there isn't a lot of
documentation on them.

[nodata]

Am Montag, den 03.03.2008, 12:41 -0500 schrieb Sunil Ghai:
>  
>  >I'm not clear on whether the 'throttle' and 'bandwidth'
> configuration
>  would
>  >apply to yum-updatesd.  It would still not be a dynamic adjustment
>  but if users
>  >knew how this worked it might be a good enough solution.
>  
>  That would not be the solution in real sense. What if someone wants to
>  download another important file? 
What if the update is more important than the file the user wants to download?

[Sunil Ghai]

Hi,

Rather than implementing dynamic bandwidth throttling feature in updates
only, how about having a framework possibly in form of headers and libraries
which would implement dynamic throttling and other applications like Package
manager or Firefox would be able to use it. For example, we can add one
option in wget to get files using idle bandwidth only. This will solve
purpose for other applications also.

I need your reviews that how this kind of framework would fit in GNU/Linux
operating system.

Regards,
Sunil Ghai.

[Andrew Farris]

Sunil Ghai wrote:
>  Hi,
>  
>  Rather than implementing dynamic bandwidth throttling feature in updates
>  only, how about having a framework possibly in form of headers and libraries
>  which would implement dynamic throttling and other applications like Package
>  manager or Firefox would be able to use it. For example, we can add one
>  option in wget to get files using idle bandwidth only. This will solve
>  purpose for other applications also.
>  
>  I need your reviews that how this kind of framework would fit in GNU/Linux
>  operating system. 
Isn't this exactly what QOS is intended to do, provided the connections are made 
with priorities assigned?

-- 
Andrew Farris <lordmorgul(a)gmail.com&gt; www.lordmorgul.net
  gpg 0xC99B1DF3 fingerprint CDEC 6FAD BA27 40DF 707E A2E0 F0F6 E622 C99B 1DF3
No one now has, and no one will ever again get, the big picture. - Daniel Geer
----                                                                       ----

[Bruno Wolff III]

On Tue, Mar 11, 2008 at 18:07:02 -0700,
  Andrew Farris <lordmorgul(a)gmail.com&gt; wrote:
>  
>  Isn't this exactly what QOS is intended to do, provided the connections are 
>  made with priorities assigned? 
If you are talking about the qos flags in IP packets, the answer is no.
There aren't that many states and generally you can only describe broad
things about packets (such as I want low latency or I want high throughput)
not detailed bandwidth allocations. Also I tried a few experiments with
testing qos bits and found that they seemed to be getting stripped (I expected
them to be ignored, but that at least they would be preserved) in transit.
I didn't do enough experiments to see where this was happening or to get a
good idea of how common this was.

There are tools for doing this under Linux. You want to use the iproute2
suite to do most of the work and you might need netfilter to do some marking
to allow you to be able to classify packets. You need to know the capacity of
your link. There are also limitations on the effectiveness of shaping on
the wrong side of the link (for incoming traffic). The LARTC Howto is
a good starting point for learning about this, though it is a bit dated.

[Ralf Ertzinger]

Hi.

On Tue, 11 Mar 2008 22:15:47 -0500, Bruno Wolff III wrote:

>  If you are talking about the qos flags in IP packets, the answer is
>  no. There aren't that many states and generally you can only describe
>  broad things about packets (such as I want low latency or I want high
>  throughput) not detailed bandwidth allocations. 
You can do bandwidth allocation based on QoS flags (with the newer DSCP
interpretation of the relevant IP header flags you get 64 possible traffic
classes, which is quite a lot).

>  Also I tried a few
>  experiments with testing qos bits and found that they seemed to be
>  getting stripped (I expected them to be ignored, but that at least
>  they would be preserved) in transit. I didn't do enough experiments
>  to see where this was happening or to get a good idea of how common
>  this was. 
On the other hand QoS is very much an end-to-end process, which means
that all involved devices (as in: routers) have to agree on the QoS
policy (which DSCP flag signifies which traffic class, and what preference
this traffic gets). This is why it is does not work over the internet
as a whole. Your provider usually has a diffenent view on what constitutes
important traffic, and thus strips your classification (or does not strip
it but ignores it).

[Andrew Farris]

Ralf Ertzinger wrote:
>  Hi.
>  
>  On Tue, 11 Mar 2008 22:15:47 -0500, Bruno Wolff III wrote:
>  
> > If you are talking about the qos flags in IP packets, the answer is
> > no. There aren't that many states and generally you can only describe
> > broad things about packets (such as I want low latency or I want high
> > throughput) not detailed bandwidth allocations.
>  
>  You can do bandwidth allocation based on QoS flags (with the newer DSCP
>  interpretation of the relevant IP header flags you get 64 possible traffic
>  classes, which is quite a lot).
>  
> > Also I tried a few
> > experiments with testing qos bits and found that they seemed to be
> > getting stripped (I expected them to be ignored, but that at least
> > they would be preserved) in transit. I didn't do enough experiments
> > to see where this was happening or to get a good idea of how common
> > this was.
>  
>  On the other hand QoS is very much an end-to-end process, which means
>  that all involved devices (as in: routers) have to agree on the QoS
>  policy (which DSCP flag signifies which traffic class, and what preference
>  this traffic gets). This is why it is does not work over the internet
>  as a whole. Your provider usually has a diffenent view on what constitutes
>  important traffic, and thus strips your classification (or does not strip
>  it but ignores it). 
I was thinking more along the lines of just the local machine's behavior with 
different connections having higher or lower priority for outbound (which is 
often what hurts response time the most for slower connections while longer 
running transfers occur).  I really don't know how effective QOS is, so it may 
be a bad way to approach this issue.

If an update connection had low priority for the bandwidth resources, that 
connection should be postponed whenever a higher priority connection wants to 
push outbound traffic.  A browser then would get to send its page requests or 
acks ahead of running transfer packets from the update utility; the result would 
be a much more responsive browser while still using most of the available 
bandwidth.  Whether the QOS flags are being stripped/mangled once the traffic 
leaves the local machine should not really hurt that improvement would it?

I'm just thinking it may not require full end-to-end to enjoy some benefit.  The 
incoming connection would not be slowed or postponed to let the browser respond, 
but by not acking what comes in until the outbound clears up I think it might 
help anyway.

-- 
Andrew Farris <lordmorgul(a)gmail.com&gt; www.lordmorgul.net
  gpg 0xC99B1DF3 fingerprint CDEC 6FAD BA27 40DF 707E A2E0 F0F6 E622 C99B 1DF3
No one now has, and no one will ever again get, the big picture. - Daniel Geer
----                                                                       ----

[Bruno Wolff III]

On Wed, Mar 12, 2008 at 01:59:46 -0700,
  Andrew Farris <lordmorgul(a)gmail.com&gt; wrote:
>  
>  I was thinking more along the lines of just the local machine's behavior 
>  with different connections having higher or lower priority for outbound 
>  (which is often what hurts response time the most for slower connections 
>  while longer running transfers occur).  I really don't know how effective 
>  QOS is, so it may be a bad way to approach this issue. 
You would still need to write the rules that do the shaping. However some
applications set QoS (particularly to distinguish between interactive and
bulk traffic) so it can be useful to look at. For outbound packets you
are OK, for inbound not so much.

>  If an update connection had low priority for the bandwidth resources,
> that 
>  connection should be postponed whenever a higher priority connection wants 
>  to push outbound traffic.  A browser then would get to send its page 
>  requests or acks ahead of running transfer packets from the update utility; 
>  the result would be a much more responsive browser while still using most 
>  of the available bandwidth.  Whether the QOS flags are being 
>  stripped/mangled once the traffic leaves the local machine should not 
>  really hurt that improvement would it? 
It makes it hard to handle inbound traffic which you may also need to
manage. Though in a particular case that may not be a bottleneck. In your
case it looks like you will be needing to throttle inbound traffic, so
this is relevant. The way this shaping is done you either drop some packets
from the connections you want to slow down or you set bits in the
acknowledgement thay say the sender should slow down as if you had dropped
the packet. Not all network stacks support the later feature, but I don't
know what fraction do these days. It might be in practice almost everyone
does.
So you aren't blocking outbound requests in order to prevent applications
from retrieving data. That kind of approach would be a lot different and
have to be customized to each application.
>  
>  I'm just thinking it may not require full end-to-end to enjoy some benefit. 
>  The incoming connection would not be slowed or postponed to let the browser 
>  respond, but by not acking what comes in until the outbound clears up I 
>  think it might help anyway. 
You don't really want to drop all packets, just some. The sender is supposed
to back off with an exponential reduction in send rate until packets stop
getting dropped. If you block all of them, the application will likely assume
the connection has been broken and stop working. Generally throttling and
giving priority to low latency packets should work fairly well.

[Sunil Ghai]

On Wed, Mar 12, 2008 at 11:50 AM, Bruno Wolff III <bruno(a)wolff.to&gt; wrote:

>  On Wed, Mar 12, 2008 at 01:59:46 -0700,
>   Andrew Farris <lordmorgul(a)gmail.com&gt; wrote:
>  >
>  > I was thinking more along the lines of just the local machine's behavior
>  > with different connections having higher or lower priority for outbound
>  > (which is often what hurts response time the most for slower connections
>  > while longer running transfers occur).  I really don't know how
>  effective
>  > QOS is, so it may be a bad way to approach this issue.
>
>  You would still need to write the rules that do the shaping. However some
>  applications set QoS (particularly to distinguish between interactive and
>  bulk traffic) so it can be useful to look at. For outbound packets you
>  are OK, for inbound not so much.
>
>  > If an update connection had low priority for the bandwidth resources,
>  that
>  > connection should be postponed whenever a higher priority connection
>  wants
>  > to push outbound traffic.  A browser then would get to send its page
>  > requests or acks ahead of running transfer packets from the update
>  utility;
>  > the result would be a much more responsive browser while still using
>  most
>  > of the available bandwidth.  Whether the QOS flags are being
>  > stripped/mangled once the traffic leaves the local machine should not
>  > really hurt that improvement would it?
>
>  It makes it hard to handle inbound traffic which you may also need to
>  manage. Though in a particular case that may not be a bottleneck. In your
>  case it looks like you will be needing to throttle inbound traffic, so
>  this is relevant. The way this shaping is done you either drop some
>  packets
>  from the connections you want to slow down or you set bits in the
>  acknowledgement thay say the sender should slow down as if you had dropped
>  the packet. Not all network stacks support the later feature, but I don't
>  know what fraction do these days. It might be in practice almost everyone
>  does.
>  So you aren't blocking outbound requests in order to prevent applications
>  from retrieving data. That kind of approach would be a lot different and
>  have to be customized to each application.
>  >
>  > I'm just thinking it may not require full end-to-end to enjoy some
>  benefit.
>  > The incoming connection would not be slowed or postponed to let the
>  browser
>  > respond, but by not acking what comes in until the outbound clears up I
>  > think it might help anyway.
>
>  You don't really want to drop all packets, just some. The sender is
>  supposed
>  to back off with an exponential reduction in send rate until packets stop
>  getting dropped. If you block all of them, the application will likely
>  assume
>  the connection has been broken and stop working. Generally throttling and
>  giving priority to low latency packets should work fairly well.
>  
In case of dynamic throttling we won't be having any _fixed_ rate at which
the connections assigned for updates will be able receive the packets. It
means packets would be dropped frequently to implement policing.  Isn't this
waste of resources?

Tools like tc and tcng implement queues to control outbound data. Is there
any similar _kind of_ option available for inbound data?
(Obviously we can't have queues because once the packet has been received
must be processed)

-- 
Regards,
Sunil Ghai

[Ahmed Kamal]

lartc mentions ways for "traffic policing" to throttle incoming rates
by throttling outgoing ack packets

2008/3/24 Sunil Ghai <sunilkrghai(a)gmail.com&gt;:
>  On Wed, Mar 12, 2008 at 11:50 AM, Bruno Wolff III <bruno(a)wolff.to&gt; wrote:
>  >
>  > On Wed, Mar 12, 2008 at 01:59:46 -0700,
>  >  Andrew Farris <lordmorgul(a)gmail.com&gt; wrote:
>  > >
>  > > I was thinking more along the lines of just the local machine's behavior
>  > > with different connections having higher or lower priority for outbound
>  > > (which is often what hurts response time the most for slower connections
>  > > while longer running transfers occur).  I really don't know how
>  effective
>  > > QOS is, so it may be a bad way to approach this issue.
>  >
>  > You would still need to write the rules that do the shaping. However some
>  > applications set QoS (particularly to distinguish between interactive and
>  > bulk traffic) so it can be useful to look at. For outbound packets you
>  > are OK, for inbound not so much.
>  >
>  >
>  > > If an update connection had low priority for the bandwidth resources,
>  that
>  > > connection should be postponed whenever a higher priority connection
>  wants
>  > > to push outbound traffic.  A browser then would get to send its page
>  > > requests or acks ahead of running transfer packets from the update
>  utility;
>  > > the result would be a much more responsive browser while still using
>  most
>  > > of the available bandwidth.  Whether the QOS flags are being
>  > > stripped/mangled once the traffic leaves the local machine should not
>  > > really hurt that improvement would it?
>  >
>  > It makes it hard to handle inbound traffic which you may also need to
>  > manage. Though in a particular case that may not be a bottleneck. In your
>  > case it looks like you will be needing to throttle inbound traffic, so
>  > this is relevant. The way this shaping is done you either drop some
>  packets
>  > from the connections you want to slow down or you set bits in the
>  > acknowledgement thay say the sender should slow down as if you had dropped
>  > the packet. Not all network stacks support the later feature, but I don't
>  > know what fraction do these days. It might be in practice almost everyone
>  > does.
>  > So you aren't blocking outbound requests in order to prevent applications
>  > from retrieving data. That kind of approach would be a lot different and
>  > have to be customized to each application.
>  >
>  > >
>  > > I'm just thinking it may not require full end-to-end to enjoy some
>  benefit.
>  > > The incoming connection would not be slowed or postponed to let the
>  browser
>  > > respond, but by not acking what comes in until the outbound clears up I
>  > > think it might help anyway.
>  >
>  > You don't really want to drop all packets, just some. The sender is
>  supposed
>  > to back off with an exponential reduction in send rate until packets stop
>  > getting dropped. If you block all of them, the application will likely
>  assume
>  > the connection has been broken and stop working. Generally throttling and
>  > giving priority to low latency packets should work fairly well.
>  >
>  >
>  >
>  >
>
>  In case of dynamic throttling we won't be having any _fixed_ rate at which
>  the connections assigned for updates will be able receive the packets. It
>  means packets would be dropped frequently to implement policing.  Isn't this
>  waste of resources?
>
>  Tools like tc and tcng implement queues to control outbound data. Is there
>  any similar _kind of_ option available for inbound data?
>  (Obviously we can't have queues because once the packet has been received
>  must be processed)
>
>  --
>  Regards,
>  Sunil Ghai
>  --
>   fedora-devel-list mailing list
>   fedora-devel-list(a)redhat.com
>   https://www.redhat.com/mailman/listinfo/fedora-devel-list
>

[Bruno Wolff III]

>  >
>  > In case of dynamic throttling we won't be having any _fixed_ rate at which
>  > the connections assigned for updates will be able receive the packets. It
>  > means packets would be dropped frequently to implement policing.  Isn't this
>  > waste of resources? 
Yes and a protocol change was made to help. I believe this is the purpose of
ECN (explicit congestion notification).

>  > Tools like tc and tcng implement queues to control outbound
> data. Is there
>  > any similar _kind of_ option available for inbound data?
>  > (Obviously we can't have queues because once the packet has been received
>  > must be processed) 
Shaping on the wrong side of a link is problematic. You can implement queues
on the receiving side which might allow you to better control which flows
get slowed down using IFBs (which replace the older IMQs). While you can't
absolutely prevent the other side from swamping the link with low priority
packets, things should work reasonably with well behaved applications.

[Sunil Ghai]

On Mon, Mar 24, 2008 at 11:14 AM, Bruno Wolff III <bruno(a)wolff.to&gt; wrote:

>  > >
>  > > In case of dynamic throttling we won't be having any _fixed_ rate at
>  which
>  > > the connections assigned for updates will be able receive the packets.
>  It
>  > > means packets would be dropped frequently to implement policing.
>   Isn't this
>  > > waste of resources?
>
>  Yes and a protocol change was made to help. I believe this is the purpose
>  of
>  ECN (explicit congestion notification).
>
>  > > Tools like tc and tcng implement queues to control outbound data. Is
>  there
>  > > any similar _kind of_ option available for inbound data?
>  > > (Obviously we can't have queues because once the packet has been
>  received
>  > > must be processed)
>
>  Shaping on the wrong side of a link is problematic. You can implement
>  queues
>  on the receiving side which might allow you to better control which flows
>  get slowed down using IFBs (which replace the older IMQs). While you can't
>  absolutely prevent the other side from swamping the link with low priority
>  packets, things should work reasonably with well behaved applications.
>
>  
Ingress shaping...sounds good!
Basically it is the way to implement policing efficiently?
-- 
Regards,
Sunil Ghai

[Sunil Ghai]

On Mon, Mar 24, 2008 at 2:14 PM, Sunil Ghai <sunilkrghai(a)gmail.com&gt; wrote:

>  On Mon, Mar 24, 2008 at 11:14 AM, Bruno Wolff III <bruno(a)wolff.to&gt; wrote:
>
>  > > >
>  > > > In case of dynamic throttling we won't be having any _fixed_ rate at
>  > which
>  > > > the connections assigned for updates will be able receive the
>  > packets. It
>  > > > means packets would be dropped frequently to implement policing.
>  >  Isn't this
>  > > > waste of resources?
>  >
>  > Yes and a protocol change was made to help. I believe this is the
>  > purpose of
>  > ECN (explicit congestion notification).
>  >
>  > > > Tools like tc and tcng implement queues to control outbound data. Is
>  > there
>  > > > any similar _kind of_ option available for inbound data?
>  > > > (Obviously we can't have queues because once the packet has been
>  > received
>  > > > must be processed)
>  >
>  > Shaping on the wrong side of a link is problematic. You can implement
>  > queues
>  > on the receiving side which might allow you to better control which
>  > flows
>  > get slowed down using IFBs (which replace the older IMQs). While you
>  > can't
>  > absolutely prevent the other side from swamping the link with low
>  > priority
>  > packets, things should work reasonably with well behaved applications.
>  >
>  >
>  Ingress shaping...sounds good!
>  Basically it is the way to implement policing efficiently?
>  --
>  Regards,
>  Sunil Ghai 

For traffic accepted on an interface, the *ingress* qdisc is traversed. It
means all inbound data is traversed through it. So how do we differentiate
as which inbound packet is for which application?  port numbers? and who
does it..operating system of *filters* attached with *ingress qdisc*?

If we want to implement policing on a particular connection, it's inbound
packets may be dropped. But as *ingress qdisc* is common to an interface so
how do we implement it?


-- 
Regards,
Sunil Ghai

[Bruno Wolff III]

On Tue, Mar 25, 2008 at 04:29:18 -0400,
  Sunil Ghai <sunilkrghai(a)gmail.com&gt; wrote:
>  
>  For traffic accepted on an interface, the *ingress* qdisc is traversed. It
>  means all inbound data is traversed through it. So how do we differentiate
>  as which inbound packet is for which application?  port numbers? and who
>  does it..operating system of *filters* attached with *ingress qdisc*? 
I believe it is possible to do this using netfilter (iptables) to mark
packets. And then you can classify the marked packets. I haven't tried this
though.

There are some limits on what you can do with traffic in an ingress qdisc.
That is why you might want to forward the inbound traffic to an IFB where
you can apply normal egress rules. (Though in the end the rules will be
enforced by packets being dropped when the queue fills up.)

>  If we want to implement policing on a particular connection, it's
> inbound
>  packets may be dropped. But as *ingress qdisc* is common to an interface so
>  how do we implement it? 
You want traffic to be seen together so that you can properly manage it.
So seeing all of the inbound traffic is a good thing. (You'd prefer to
see all of the local network traffic as well, but that isn't going to be
the case unless you are using the Fedora box as your local gateway. And
that is something you'd more likely want to do on a $50 router box running
DDWRT or OpenWRT.

If you continue down this route, keep in mind the possibility of generalizing
this at a later time as a traffic shaping utility. People do have need of
something that will prioritize their ssh or voip traffic over background
downloads (of any kind, not just updates) and your project could potentially
morph into such a tool.

[Tomasz Torcz]

Dnia 11-03-2008, wto o godzinie 20:48 -0400, Sunil Ghai pisze:
>  Hi,
>  
>  Rather than implementing dynamic bandwidth throttling feature in
>  updates only, how about having a framework possibly in form of headers
>  and libraries which would implement dynamic throttling and other
>  applications like Package manager or Firefox would be able to use it.
>  For example, we can add one option in wget to get files using idle
>  bandwidth only. This will solve purpose for other applications also.  
  I write second time about TCP Low Priority. It's a congestion control
algorithm included in Linux kernel. Connections using this algorithm
*should* use only idle bandwidth. In other words, such framework already
exist in form of self-tuning TCP stack.
  Last time I wrote you asked for example code how to use it. Here you
are, in python:

#v+
#!/usr/bin/python

import socket

socket.TCP_CONGESTION = 13

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_CONGESTION, "lp")

#v-

Module tcp_lp.ko will be autoloaded by kernel. So this setsockopt() call
is ready to be put into yum-updatesd, just after socket creation.

-- 
Tomasz Torcz

[James Antill]

On Tue, 2008-03-25 at 14:49 +0100, Tomasz Torcz wrote:
>    I write second time about TCP Low Priority. It's a congestion
> control
>  algorithm included in Linux kernel. Connections using this algorithm
>  *should* use only idle bandwidth. In other words, such framework already
>  exist in form of self-tuning TCP stack.
>    Last time I wrote you asked for example code how to use it. Here you
>  are, in python:
>  
>  #v+
>  #!/usr/bin/python
>  
>  import socket
>  
>  socket.TCP_CONGESTION = 13
>  
>  sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>  sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_CONGESTION, "lp") 
 Ok, that's not a terrible idea _but_:

1. yum-updatesd doesn't have a socket, it calls yum which calls
urlgrabber.

2. yum-updatesd currently holds the yum lock while it downloads.

...so it's far more work than just adding the above two lines to
yum-updatesd.

-- 
James Antill <james(a)fedoraproject.com&gt;
Fedora

[Andrew Farris]

James Antill wrote:
>  ...so it's far more work than just adding the above two lines to
>  yum-updatesd. 
Ok, then since yum-updatesd is calling yum to handle it, adding a normal yum 
parameter to set the congestion level would suffice as long yum gets the ability 
to pass it on to urlgrabber?  It might actually be nice to have the ability to 
do yum --congestion 13 or something like that when manually running an update too.

Tomasz Torcz wrote:
>  Module tcp_lp.ko will be autoloaded by kernel. So this setsockopt()
> call
>  is ready to be put into yum-updatesd, just after socket creation. 
A good question then is how do you set congestion level off and not get the 
kernel module loaded (if its not needed you wouldn't want it loading just to 
have it do nothing).  Does setting TCP_CONGESTION = 0 result in not loading the 
module or does it just not limit traffic?

-- 
Andrew Farris <lordmorgul(a)gmail.com&gt; www.lordmorgul.net
  gpg 0xC99B1DF3 fingerprint CDEC 6FAD BA27 40DF 707E A2E0 F0F6 E622 C99B 1DF3
No one now has, and no one will ever again get, the big picture. - Daniel Geer
----                                                                       ----

[Tomasz Torcz]

Dnia 25-03-2008, wto o godzinie 12:14 -0700, Andrew Farris pisze:
>  Tomasz Torcz wrote:
>   > Module tcp_lp.ko will be autoloaded by kernel. So this setsockopt() call
>   > is ready to be put into yum-updatesd, just after socket creation.
>  
>  A good question then is how do you set congestion level off and not get the 
>  kernel module loaded (if its not needed you wouldn't want it loading just to 
>  have it do nothing).  Does setting TCP_CONGESTION = 0 result in not loading the 
>  module or does it just not limit traffic? 
  That's not how it works. TCP has mechanisms for dealing with congested
links. There is *always* some algorithm present in TCP stack which
manipulates connection parameters to not overflow links. By default its
reno (or cubic), but Linux kernel provides several others to *choose*
from. This isn't a question of turning limiting ON or OFF.

 Anyway, after some more reading I see that TCP Low Priority is
sender-side algorithm. I don not know if sending ACKs is impacted by
congestion algorithms, and this is only thing for client to modify.

-- 
Tomasz Torcz

[Les Mikesell]

Tomasz Torcz wrote:
> >  > Module tcp_lp.ko will be autoloaded by kernel. So this setsockopt() call
> >  > is ready to be put into yum-updatesd, just after socket creation.
> >
> > A good question then is how do you set congestion level off and not get the 
> > kernel module loaded (if its not needed you wouldn't want it loading just to 
> > have it do nothing).  Does setting TCP_CONGESTION = 0 result in not loading the 
> > module or does it just not limit traffic?
>  
>    That's not how it works. TCP has mechanisms for dealing with congested
>  links. There is *always* some algorithm present in TCP stack which
>  manipulates connection parameters to not overflow links. By default its
>  reno (or cubic), but Linux kernel provides several others to *choose*
>  from. This isn't a question of turning limiting ON or OFF.
>  
>   Anyway, after some more reading I see that TCP Low Priority is
>  sender-side algorithm. I don not know if sending ACKs is impacted by
>  congestion algorithms, and this is only thing for client to modify. 
Yes, yum is mostly receiving.  An application can control receive 
bandwidth simply by rate-limiting its reads from the socket, but it will 
have no idea what an appropriate rate might be.

What bandwidth constrained sites with more than one user really need are 
caching proxies, but the fedora mirrorlist configuration will end up 
picking different URLs to get the same file each time instead of using 
copies already in the cache.

--
   Les Mikesell
    lesmikesell(a)gmail.com

[Sunil Ghai]

On Wed, Mar 26, 2008 at 8:48 AM, Les Mikesell <lesmikesell(a)gmail.com&gt; wrote:

>  Tomasz Torcz wrote:
>  >
>  >>  > Module tcp_lp.ko will be autoloaded by kernel. So this setsockopt()
>  call
>  >>  > is ready to be put into yum-updatesd, just after socket creation.
>  >>
>  
>  >> A good question then is how do you set congestion level off
> and not get
>  the
>  >> kernel module loaded (if its not needed you wouldn't want it loading
>  just to
>  >> have it do nothing).  Does setting TCP_CONGESTION = 0 result in not
>  loading the
>  >> module or does it just not limit traffic?
>  >
>  >   That's not how it works. TCP has mechanisms for dealing with congested
>  > links. There is *always* some algorithm present in TCP stack which
>  > manipulates connection parameters to not overflow links. By default its
>  > reno (or cubic), but Linux kernel provides several others to *choose*
>  > from. This isn't a question of turning limiting ON or OFF.
>  >
>  >  Anyway, after some more reading I see that TCP Low Priority is
>  > sender-side algorithm. I don not know if sending ACKs is impacted by
>  > congestion algorithms, and this is only thing for client to modify
>  
>  Yes, yum is mostly receiving.  An application can control receive
>  bandwidth simply by rate-limiting its reads from the socket, but it will
>  have no idea what an appropriate rate might be. 

Won't this *dynamic* rate-limiting be decided by TCP-LP congestion control
mechanism to use only idle bandwidth?  And could someone tell me what
problems I might face if I make it to work with yum?

Thanks.

PS: I am really new to this, please forgive me if any question asked doesn't
make sense. :)

-- 
Regards,
Sunil Ghai

[Les Mikesell]

Sunil Ghai wrote:
>  
>       >> A good question then is how do you set congestion level off and
>      not get the
>       >> kernel module loaded (if its not needed you wouldn't want it
>      loading just to
>       >> have it do nothing).  Does setting TCP_CONGESTION = 0 result in
>      not loading the
>       >> module or does it just not limit traffic?
>  
>      Yes, yum is mostly receiving.  An application can control receive
>      bandwidth simply by rate-limiting its reads from the socket, but it will
>      have no idea what an appropriate rate might be.
>  
>  
>  Won't this *dynamic* rate-limiting be decided by TCP-LP congestion 
>  control mechanism to use only idle bandwidth?  And could someone tell me 
>  what problems I might face if I make it to work with yum?
>  
>  PS: I am really new to this, please forgive me if any question asked 
>  doesn't make sense. :) 
I don't understand what it does well enough to be sure, but normally 
sending and receiving packets are completely independent events and a 
receiver doesn't know whether a packet traversed congested links or not. 
If you are actually dropping packets (which should only happen under 
horrible conditions anyway), a congestion-aware receiver should send a 
duplicate ack for the previous correct packet and the sender may use 
this as an indication to back off.  But when you are actually dropping 
packets, it is way to late to be thinking about using only idle 
bandwidth - you've already backed everything up past its buffer windows.

-- 
  Les Mikesell
    lesmikesell(a)gmail.com

[Benny Amorsen]

Les Mikesell <lesmikesell(a)gmail.com&gt; writes:

>  But when you are actually dropping packets, it is way to late to be
>  thinking about using only idle bandwidth - you've already backed
>  everything up past its buffer windows. 
Nope, you have most likely hit Random Early Drop. If you and the
router support ECN, that will even be Random Early Detect, so you can
throttle without actually losing packets.


/Benny

[Tomasz Torcz]

Dnia 02-03-2008, nie o godzinie 15:29 -0500, Sunil Ghai pisze:
>  Hi,
>  We currently have yum-updatesd to check for the updated packages
>  automatically. Even to download and install them, if marked. Yum has a
>  throttle option which fixes the upper limit of maximum bandwidth
>  usage. How about fixing the upper limit dynamically depending upon the
>  current usage of bandwidth? i.e to download the packages using idle
>  bandwidth. 
 We have TCP Low Priority¹ congestion algorithm in kernel. yum-updatesd
should request it on downloading socket via setsockopt(), as per commit
bc0efe7b46174fa3cadf00ac64e4a751cc4619fd .


¹ http://www-ece.rice.edu/networks/TCP-LP/

-- 
Tomasz Torcz