It seems that the route from download-i2 to you is going over commercial peering rather than NCREN/I2.
[mdomsch@download-rdu02 ~]$ traceroute nas1.itc.virginia.edu traceroute to nas1.itc.virginia.edu (128.143.23.141), 30 hops max, 60 byte packets 1 10.11.30.252 (10.11.30.252) 2.719 ms 3.097 ms 3.415 ms 2 10.11.31.252 (10.11.31.252) 0.427 ms 0.497 ms 0.573 ms 3 Vlan10-sw02-core-rdu.redhat.com (66.187.233.250) 1.232 ms 1.355 ms 1.347 ms 4 GigabitEthernet0-2-sw02-core-rdu.redhat.com (66.187.235.3) 1.791 ms 1.824 ms 1.443 ms 5 rrcs-70-62-95-197.midsouth.biz.rr.com (70.62.95.197) 2.534 ms 2.425 ms 2.454 ms 6 ge-2-1-0.rlghncpop-rtr1.southeast.rr.com (24.93.73.62) 7.679 ms 7.526 ms 7.555 ms 7 ae-3-0.cr0.dca10.tbone.rr.com (66.109.6.80) 14.438 ms 107.14.19.20 (107.14.19.20) 14.335 ms 14.310 ms 8 ae-0-0.pr0.dca20.tbone.rr.com (66.109.6.183) 14.799 ms 14.673 ms 14.605 ms 9 xe-4-2-0.edge2.Washington1.Level3.net (4.79.22.57) 20.702 ms xe-4-3-0.edge2.Washington1.Level3.net (4.79.22.49) 93.102 ms xe-4-1-0.edge2.Washington1.Level3.net (4.79.22.53) 59.344 ms 10 ae-1-60.edge1.Washington1.Level3.net (4.69.149.13) 93.529 ms ae-1-80.edge1.Washington1.Level3.net (4.69.149.141) 22.058 ms ae-1-60.edge1.Washington1.Level3.net (4.69.149.13) 93.748 ms 11 UVA-VASCUPP.edge1.Washington1.Level3.net (4.28.124.18) 89.160 ms 93.115 ms 93.179 ms 12 carruthers-6509a-x.misc.Virginia.EDU (128.143.222.92) 89.100 ms 89.362 ms 89.418 ms 13 * * *
For example, here is what one should get going from download-i2 to utexas.edu. See how it exits redhat onto ncren?
[mdomsch@download-rdu02 ~]$ traceroute ftp.utexas.edu traceroute to ftp.utexas.edu (146.6.54.21), 30 hops max, 60 byte packets 1 10.11.30.252 (10.11.30.252) 1.198 ms 1.541 ms 1.886 ms 2 10.11.31.250 (10.11.31.250) 0.506 ms 0.540 ms 0.622 ms 3 Vlan10-sw02-core-rdu.redhat.com (66.187.233.250) 1.269 ms 1.345 ms 1.249 ms 4 GigabitEthernet0-2-sw02-core-rdu.redhat.com (66.187.235.3) 1.149 ms 1.395 ms 1.499 ms 5 ncsu-gw3-to-redhat.ncren.net (128.109.111.33) 1.819 ms 1.974 ms 1.867 ms 6 chlt7600-gw-to-rtp7600-gw.ncren.net (128.109.9.17) 6.365 ms 5.774 ms 5.939 ms 7 chltcrs-gw-to-chlt7600-gw-sec.ncren.net (128.109.9.13) 9.967 ms 8.088 ms 7.156 ms 8 internet2-atl-to-chltcrs-gw.ncren.net (198.86.17.66) 51.686 ms 51.691 ms 51.669 ms 9 xe-1-0-0.0.rtr.hous.net.internet2.edu (64.57.28.112) 35.457 ms 35.304 ms 35.359 ms 10 hstn-hstn-newnet-ge-1-0-2-27-layer3.tx-learn.net (74.200.187.6) 40.296 ms 40.272 ms 40.312 ms 11 ots-hstn-newnet-layer3.tx-learn.net (74.200.187.26) 36.303 ms 36.074 ms 36.085 ms 12 aust-utnoc-core-ge-5-0-0-706.tx-bb.net (192.88.12.50) 39.704 ms 39.620 ms 39.559 ms 13 192.88.12.26 (192.88.12.26) 39.673 ms 39.644 ms 39.595 ms 14 ser10-v702.gw.utexas.edu (128.83.10.1) 39.622 ms 39.501 ms 39.543 ms 15 ser11-v708.gw.utexas.edu (128.83.10.14) 39.682 ms 39.912 ms 39.601 ms 16 * * *
You won't be able to reach the servers if the routes are incorrect. The question is, why is the router at hop 4 sending to you via rr.com instead of ncren? Matt, Will?
-- Matt Domsch Technology Strategist Social Media and Community Professional Dell | Office of the CTO
-----Original Message----- From: Bill Pemberton [mailto:wfp5p@viridian.itc.virginia.edu] Sent: Sunday, August 21, 2011 7:23 PM To: Domsch, Matt Subject: problems with download-i2
It looks like download-i2.fedoraproject.org may be having problems, I've been unable to connect for a few days now.
Date: Sun, 21 Aug 2011 20:43:22 -0500 From: Matt_Domsch@Dell.com To: wfp5p@viridian.itc.virginia.edu, wfoster@redhat.com, mgalgoci@redhat.com Cc: mirror-admin@fedoraproject.org Subject: RE: problems with download-i2
It seems that the route from download-i2 to you is going over commercial peering rather than NCREN/I2.
That is misleading. You need to do the traceroute from the other side to the I2 addresses exposed. The default gateway for the hosts does indeed point out I1, but the natted IPs are I2 addresses, and tcp connections to the I2 addresses will return over I2.
Why the virginia.edu addresses prefer an I2 route I don't know.
You won't be able to reach the servers if the routes are incorrect. The question is, why is the router at hop 4 sending to you via rr.com instead of ncren? Matt, Will?
See above.
Matthew Galgoci writes:
That is misleading. You need to do the traceroute from the other side to the I2 addresses exposed. The default gateway for the hosts does indeed point out I1, but the natted IPs are I2 addresses, and tcp connections to the I2 addresses will return over I2.
Why the virginia.edu addresses prefer an I2 route I don't know.
Ok, something changed. I was going to send a traceroute from nas1 in case that would help, but in the process of drafting this message I noticed the output of traceroute had changed in a big way.... and now I get an answer from rsyncd on download-i2.fedoraproject.org again.
Date: Mon, 22 Aug 2011 09:38:08 -0400 (EDT) From: Bill Pemberton wfp5p@viridian.itc.virginia.edu To: Matthew Galgoci mgalgoci@redhat.com Cc: Matt_Domsch@Dell.com, wfoster@redhat.com, mirror-admin@fedoraproject.org Subject: Re: problems with download-i2
Matthew Galgoci writes:
That is misleading. You need to do the traceroute from the other side to the I2 addresses exposed. The default gateway for the hosts does indeed point out I1, but the natted IPs are I2 addresses, and tcp connections to the I2 addresses will return over I2.
Why the virginia.edu addresses prefer an I2 route I don't know.
Ok, something changed. I was going to send a traceroute from nas1 in case that would help, but in the process of drafting this message I noticed the output of traceroute had changed in a big way.... and now I get an answer from rsyncd on download-i2.fedoraproject.org again.
I didn't touch nuthin'
I didn't touch nuthin'
Ok, neither did I as they won't let me play with routers..... I'll poke our network folks and see if I get anything out of them.
And to update this whole deal, it was on our end. We've apparently had some network problems since last week that our networks didn't feel the need to tell anyone about until after it was fixed..... Sorry to have bugged you guys.
Date: Mon, 22 Aug 2011 18:07:32 -0400 (EDT) From: Bill Pemberton wfp5p@viridian.itc.virginia.edu To: Matthew Galgoci mgalgoci@redhat.com Cc: Matt_Domsch@Dell.com, wfoster@redhat.com, mirror-admin@fedoraproject.org Subject: Re: problems with download-i2
And to update this whole deal, it was on our end. We've apparently had some network problems since last week that our networks didn't feel the need to tell anyone about until after it was fixed..... Sorry to have bugged you guys.
The subset of people affected by an asymmetric route like that is pretty small. It's also very subtle. I can understand why they didn't send an outage notice.
Thanks,
Matt
mirror-admin@lists.fedoraproject.org
-
Bill Pemberton -
Matt Domsch -
Matthew Galgoci