IPv6 addresses and unwanted routing entries

Thu May 14 17:46:42 UTC 2009

On Thu, May 14, 2009 at 11:45:23AM -0400, Richard Achmatowicz wrote:
> Neil Horman wrote:
>> On Wed, May 13, 2009 at 03:11:10PM -0400, Richard Achmatowicz wrote:
>>   
>>> Neil
>>>
>>> Thanks for your speedy reply! Comments in-line.
>>>
>>> Neil Horman wrote:
>>>     
>>>> On Wed, May 13, 2009 at 12:34:36PM -0400, Richard Achmatowicz wrote:
>>>>         
>>>>> Hello
>>>>>
>>>>> I'm using Fedora 8 but I have the same problem on RHEL 5. Before 
>>>>> I   submit this issue as a bug, I wanted to check if my 
>>>>> understanding is not  flawed in some way.
>>>>>
>>>>> For some reason, when I create an IPv6 address (global or 
>>>>> link-local) on  interface eth0, three related routs are added:
>>>>>
>>>>> 3ffe:ffff:100:f101::/64                     *                     
>>>>>                   U     256    0        0 eth0
>>>>> 3ffe:ffff:100:f101::/128                    *                     
>>>>>                   U     0      0        1 lo
>>>>> 3ffe:ffff:100:f101::1/128                   *                     
>>>>>                   U     0      0        1 lo
>>>>>
>>>>> The latter of these, the most specific, drives all datagrams onto 
>>>>> lo  instead of eth0.
>>>>> I'm trying to find out why. I didn't ask for a route to lo to be  
>>>>>  created, so why is it being created?
>>>>>
>>>>> This behavior is causing Sun JDK 6 to behave badly when working 
>>>>> with  IPv6 addresses in certain contexts (Sun bug 6800096).
>>>>>
>>>>> Any ideas appreciated...A full example of what is happening is listed below.
>>>>>
>>>>> Richard
>>>>>
>>>>>             
>>>> This I think looks fairly normal.  Its the mask value that makes all the
>>>> difference.  The first entry says anything going to the 3ffe:ffff:100:f101
>>>> subnet (with a 64 bit netmask) should go out go out eth0.  The second and third
>>>> entries say that anyting going to the addresses 3ffe:ffff:100:f101:: and
>>>> 3ffe:ffff:100:f101::1 should go through lo.  Since those two addresses are local
>>>> to the system, they can be routed through the lo interface.  No other addresses
>>>> on that 64 bit network should match on that route however, since they're both
>>>> masked at 128 bits.  If anything but traffic to your local interfaces is
>>>> matching on those routes, its a bug, but having traffic bound for your local
>>>> addresses go through lo is fine.
>>>>
>>>> What exactly is the behavior that you're seeing which is leading you to think
>>>> that these routes are the cause?
>>>>         
>>> I think I understand now why you have these 128 length prefix rules 
>>> in  the routing table. Thanks for the explanation. But is it really 
>>> right to  equate messages arriving at host X on lo with messages 
>>> arriving at host  X on eth0, which seems to be what the additional lo 
>>> rules seem to  assume? Processes can listen on either interface...and 
>>> if I am listening  on eth0 and messages arrive on lo, i'm not going 
>>> to get them. Which  seems to be what is happening below.
>>>
>>>     
>> I'm glad the explination helped.  To answer your subsequent question, its fine
>> for the routing table to do what its doing above.  You're application should be
>> able to receive them just fine regardless of which interface they arrive on.  If
>> thats not happening, that may be a bug.  I assume that you are binding your
>> applications sockets to INADDR_ANY, or are you selecting a specific address to
>> listen on?  If so, that may be your problem.
>>   
> I'm going to have to think about this one a little more before I answer. :-)
Ok :)  I think your data below is interesting though, and may make this a bit
less relevant.

<snip>

>>>     
>> Thats odd, that should work, can you try to remove them with the iproute2
>> utility?  Something like this should work:
>> /sbin/ip -6 route del 3ffe:ffff:100:f101::/128
>>   
> Getting the same response from iproute2:
>
> [root at localhost nrla]# /sbin/ip -V
> ip utility, iproute2-ss080725
>
> [root at localhost nrla]# route -A inet6
> Kernel IPv6 routing table
> Destination Next Hop Flags Metric Ref Use Iface
> 3ffe:ffff:100:f101::/64 * U 256 0 0 eth0
> fe80::/64 * U 256 0 0 eth0
> localhost6.localdomain6/128 * U 0 1 1 lo
> 3ffe:ffff:100:f101::/128 * U 0 0 1 lo
> 3ffe:ffff:100:f101::1/128 * U 0 0 1 lo
> fe80::/128 * U 0 0 1 lo
> lenovo6/128 * U 0 0 1 lo
> ff00::/8 * U 256 0 0 eth0
>
> [root at localhost nrla]# /sbin/ip -6 route del 3ffe:ffff:100:f101::/128
> RTNETLINK answers: No such process
>
> [root at localhost nrla]# uname -a
> Linux localhost.localdomain 2.6.26.8-57.fc8 #1 SMP Thu Dec 18 19:19:45  
> EST 2008 i686 i686 i386 GNU/Linux
>
> And the routing table givcn by iproute2 looks a lot different from that  
> give by route:
>
> [root at localhost nrla]# ip -6 route show
> unreachable ::/96 dev lo metric 1024 error -101 mtu 16436 advmss 16376  
> hoplimit 4294967295
> unreachable ::ffff:0.0.0.0/96 dev lo metric 1024 error -101 mtu 16436  
> advmss 16376 hoplimit 4294967295
> unreachable 2002:a00::/24 dev lo metric 1024 error -101 mtu 16436 advmss  
> 16376 hoplimit 4294967295
> unreachable 2002:7f00::/24 dev lo metric 1024 error -101 mtu 16436  
> advmss 16376 hoplimit 4294967295
> unreachable 2002:a9fe::/32 dev lo metric 1024 error -101 mtu 16436  
> advmss 16376 hoplimit 4294967295
> unreachable 2002:ac10::/28 dev lo metric 1024 error -101 mtu 16436  
> advmss 16376 hoplimit 4294967295
> unreachable 2002:c0a8::/32 dev lo metric 1024 error -101 mtu 16436  
> advmss 16376 hoplimit 4294967295
> unreachable 2002:e000::/19 dev lo metric 1024 error -101 mtu 16436  
> advmss 16376 hoplimit 4294967295
> 3ffe:ffff:100:f101::/64 dev eth0 metric 256 mtu 1500 advmss 1440  
> hoplimit 4294967295
> unreachable 3ffe:ffff::/32 dev lo metric 1024 error -101 mtu 16436  
> advmss 16376 hoplimit 4294967295
> fe80::/64 dev eth0 metric 256 mtu 1500 advmss 1440 hoplimit 4294967295

So, I'm not sure how the route utility is interpreting the data differently than
the iproute2 package (the both use the rtnetlink socket to gather the data, so
its odd to say the least).  Setting asside the difference in output however I
would say this:
unreachable 3ffe:ffff::/32 dev lo metric 1024 error -101 mtu 16436
advmss 16376 hoplimit 4294967295

Might explain whats going on in part.  If you're matching on that route when
contacting your server, the client is going to be able to contact it at all, as
the kernel will return ENETUNREACH I think.  Of course it shouldn't be doing
that, as your global address route through eth0 should get matched fist I think.
If you issue this command:

ip -6 route del 3ffe:ffff::/32

Does the problem discontinue?

If it does that leaves the questions:

1) Why is the route output different from the ip table output?

2) Where did all these unreachable routes come from?

3) Why aren't you matching on the eth0 rule above like you ought to be?

I'd open a bug for (1), check your system for (2) and see if you're running a
routing daemon or something that might be adding entries to your route table,
and lets see the results of the above test  to figure out how to deal with (3)

Neil