an up2date idea

[Don]

I use up2date from the command line, with the way it was configured
during a clean install.

I've seen various suggestions to reconfigure up2date to use mirror sites
etc....

I'm not a linux guru by any means, but I do have an active
imagination... :-)

Would it be beneficial for the default web site to basically act as a
traffic director... requests directed to it would be redirected to a
mirror site... either random from a list of synch'd sites, or on a
round-robin basis...

This still allows knowledgeable people to reconfigure as they want, but
the real benefit I see is an improves overall reliability of up2date.

Obviously this isn't a change in Fedora, just a change to the web server
supporting up2date.

If wanted, such redirects could even be decided on based on server
load.... if it's not terribly busy go ahead and service the request.

Well, there's an idea.... :-)
Is it sound?

Don

[Jim Cornette]

Don wrote:
>  I use up2date from the command line, with the way it was configured
>  during a clean install.
>  
>  I've seen various suggestions to reconfigure up2date to use mirror sites
>  etc....
>  
>  I'm not a linux guru by any means, but I do have an active
>  imagination... :-)
>  
>  Would it be beneficial for the default web site to basically act as a
>  traffic director... requests directed to it would be redirected to a
>  mirror site... either random from a list of synch'd sites, or on a
>  round-robin basis...
>  
>  This still allows knowledgeable people to reconfigure as they want, but
>  the real benefit I see is an improves overall reliability of up2date.
>  
>  Obviously this isn't a change in Fedora, just a change to the web server
>  supporting up2date.
>  
>  If wanted, such redirects could even be decided on based on server
>  load.... if it's not terribly busy go ahead and service the request.
>  
>  Well, there's an idea.... :-)
>  Is it sound?
>  
>  Don
>  
>   

I'm speculating, but wouldn't that cause some sort of redirection 
madness? The redirect would try one site, get a slow connection, then do 
the same on another site.

I think that a client side program bit, which tests the speed of mirrors 
available with each category might be a good alternative. Say three 
selected sites for core, three for updates, etc. The client could ping 
the selected servers and choose the best performing selection and stay 
there until download and upgrade is completed.

Up2date is a lot better now with the repositories access for apt and yum 
repositories. One mirror seems to do the trick for me now. In the 
future, this feature would really be nice. Someone is bound to find the 
uncluttered server and bog it down.

Jim

[Don]

On Mon, 2004-01-12 at 20:07, Jim Cornette wrote:

>  I'm speculating, but wouldn't that cause some sort of
> redirection 
>  madness? The redirect would try one site, get a slow connection, then do 
>  the same on another site. 
Not necessarily.... but maybe my description was inaccurate/incomplete.
And perhaps "redirection" is an incorrect term. 

My idea is to move the server selection process out of the client and
onto the network/server. 

What if the default client in Fedora asked a server which server should
it use to get an update from? If a mirror is synching, it would not be
eligible even though the "ping time solution" appears good. As synching
completes, the mirrors are made eligible. This type of dynamic
availability is why I think server-selection should be provided by a
network application instead of a client in the FC system itself.

>  I think that a client side program bit, which tests the speed of
> mirrors 
>  available with each category might be a good alternative. Say three 
>  selected sites for core, three for updates, etc. The client could ping 
>  the selected servers and choose the best performing selection and stay 
>  there until download and upgrade is completed. 
Yes, I think we're talking about the same/similar thing, just different
implementations.

[Alexandre Strube]

Em Ter, 2004-01-13 às 14:06, Don escreveu:

>  > I'm speculating, but wouldn't that cause some sort of
> redirection 
>  > madness? The redirect would try one site, get a slow connection, then do 
>  > the same on another site.
>  
>  Not necessarily.... but maybe my description was inaccurate/incomplete.
>  And perhaps "redirection" is an incorrect term. 
>  
>  My idea is to move the server selection process out of the client and
>  onto the network/server.  
What about torrent-enabling servers to keep their bandwidth a little
lower (well, much lower most of the time)

-- 
[]s

Alexandre Ganso 
500 FOUR vermelha - Diretor Steel Goose Moto Group

[Don]

Em Ter, 2004-01-13 às 14:06, Don escreveu:

> > My idea is to move the server selection process out of the client
> and
> > onto the network/server. 
> What about torrent-enabling servers to keep their bandwidth a little
> lower (well, much lower most of the time) 
The problem I have with bittorrent is it doesn't work well with machines
behind a NAT router.
Or, I don't know how to set it up to work well behind my NAT router. :-)

My understanding of how BT works... to be effective, BT requires other
machines on the public internet to have access to my machine while my
download is in progress. MY machine then acts as another source for files,
effectively increasing the supply of servers for people to download files
from. When I start a BT download, BT also starts a "server" on my machine to
serve up the very file it is downloading.

Since my external firewall blocks most incoming traffic, my machine won't
even see a request from another machine. BT also detects this "non-sharing
attitude" and thinks I don't play nice in the sandbox.

I have multiple machines behind a NAT router.... if I want each of them to
act as BT "sources", how can I do that?

Answer:
BT on each of my machines has to send a periodic "here I am" message to the
main BT server. That "here I am" message will cause a brief (configurable)
opening in my external firewall, and build the needed NAT translation so
somebody "out there" can connect to each of my machines accordingly. As long
as traffic continues, the opening in my firewall remains.

It's doable, but I don't think BT does that now... it just assumes certain
ports are open...

Don

[Alexandre Strube]

Em Ter, 2004-01-13 às 19:12, Don escreveu:

>  My understanding of how BT works... to be effective, BT requires
> other
>  machines on the public internet to have access to my machine while my
>  download is in progress. MY machine then acts as another source for files,
>  effectively increasing the supply of servers for people to download files
>  from. When I start a BT download, BT also starts a "server" on my machine to
>  serve up the very file it is downloading.
>  Since my external firewall blocks most incoming traffic, my machine won't
>  even see a request from another machine. BT also detects this "non-sharing
>  attitude" and thinks I don't play nice in the sandbox.
>  I have multiple machines behind a NAT router.... if I want each of them to
>  act as BT "sources", how can I do that?
>  Answer:
>  BT on each of my machines has to send a periodic "here I am" message to the
>  main BT server. That "here I am" message will cause a brief (configurable)
>  opening in my external firewall, and build the needed NAT translation so
>  somebody "out there" can connect to each of my machines accordingly. As long
>  as traffic continues, the opening in my firewall remains.
>  
>  It's doable, but I don't think BT does that now... it just assumes certain
>  ports are open... 
I don't remember how it works, but I've seen some machines running it
(and sharing) from behind a NAT. 

If it sends a "here I am", the connection could be done from this point
- once it begun inside. I don't know if bt is smart enough though.

-- 
[]s

Alexandre Ganso 
500 FOUR vermelha - Diretor Steel Goose Moto Group

[Don]

> > It's doable, but I don't think BT does that now... it
> just assumes 
certain
> > ports are open...
>
> I don't remember how it works, but I've seen some machines running it
> (and sharing) from behind a NAT.
>
> If it sends a "here I am", the connection could be done from this point
> - once it begun inside. I don't know if bt is smart enough though. 
Yes, exactly. The "here I am" is not required to do the download part. It is
needed to keep a NAT path, and access list open so an incoming request to my
machine will be received.

The "here I am" message basically says "I'm listening on port xxxx at
address 1.2.3.4". Then, when somebody else wants to down load a file and
they go to a BT aware server, that server might tell them... "Yes, go get it
from TCP 1.2.3.4:xxxx".

As long as my NAT translation AND access list still permit that specific
traffic, all should be OK.

Keep in mind the "Here I am" message should not actually embed the port
number nor IP address. Each of those may be changed by NAT along the way.
Instead the BT server will see the incoming address from my machine, and the
port.

As I say, it's doable, I just don't know enough of the internals of BT to
say for certain. Since my firewall access lists are dynamic, in my case,
regardless of NAT

Don

[Don]

{snip} ... I pressed "send too soon" :-(

> As I say, it's doable, I just don't know enough of the
> internals of BT to
> say for certain. Since my firewall access lists are dynamic, in my case,
> regardless of NAT 
Since my firewall access lists are dynamic, regardless of NAT. Access to my
machine will be closed after some period of time (configurable) since the
last traffic on the port.

Don

[Jack Bowling]

On Tue, Jan 13, 2004 at 02:12:18PM -0800, Don wrote:
>  Em Ter, 2004-01-13 às 14:06, Don escreveu:
>  
>  >> My idea is to move the server selection process out of the client and
>  >> onto the network/server.
>  
>  >What about torrent-enabling servers to keep their bandwidth a little
>  >lower (well, much lower most of the time)
>  
>  The problem I have with bittorrent is it doesn't work well with machines
>  behind a NAT router.
>  Or, I don't know how to set it up to work well behind my NAT router. :-)[snip]
The answer to many p2p protocols such as BT is the pattern matching
abilities of a netfilter module such as Layer-7 Packet Classifier (L7
for short) at:

http://l7-filter.sourceforge.net

This module works via the queue target. The BitTorrent module
currently gets a "blue" rating on the site which means that the
authors think it behaves as coded but could use more testing.

-- 
Jack Bowling
mailto: jbinpg(a)shaw.ca