Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

"ip neigh show" not showing arp cache entries?

1,560 views
Skip to first unread message

Chris Friesen

unread,
Dec 10, 2007, 12:40:10 PM12/10/07
to

I'm seeing some strange behaviour on a 2.6.14 ppc64 system. If I run
"ip neigh show" it prints out nothing, but if I run "arp" then I see the
other nodes on the local network.


root@base0-0-0-5-0-11-1:/root> ip neigh show
root@base0-0-0-5-0-11-1:/root> arp -n
Address HWtype HWaddress Flags Mask
Iface
172.24.132.0 ether 00:01:AF:14:E8:DA C
bond0
172.24.132.1 (incomplete)
bond0
172.24.136.0 ether 00:C0:8B:07:B3:7E C
bond0
172.24.132.4 ether 00:01:AF:14:E8:DA C
bond0
172.24.132.2 ether 00:01:AF:14:E8:DA C
bond0


Any ideas what's going on here?

Thanks,

Chris
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Chris Friesen

unread,
Dec 10, 2007, 1:00:20 PM12/10/07
to
Chris Friesen wrote:
>
> I'm seeing some strange behaviour on a 2.6.14 ppc64 system. If I run
> "ip neigh show" it prints out nothing, but if I run "arp" then I see the
> other nodes on the local network.
>
>
> root@base0-0-0-5-0-11-1:/root> ip neigh show
> root@base0-0-0-5-0-11-1:/root> arp -n
> Address HWtype HWaddress Flags Mask Iface
> 172.24.132.0 ether 00:01:AF:14:E8:DA C bond0
> 172.24.132.1 (incomplete) bond0
> 172.24.136.0 ether 00:C0:8B:07:B3:7E C bond0
> 172.24.132.4 ether 00:01:AF:14:E8:DA C bond0
> 172.24.132.2 ether 00:01:AF:14:E8:DA C bond0
>
>
> Any ideas what's going on here?

I've got some further information. If I look for a specific address, it
seems to work:

root@base0-0-0-5-0-11-1:/root> ip neigh show 172.24.136.0
172.24.136.0 dev bond0 lladdr 00:c0:8b:07:b3:7e REACHABLE


In the above scenario, the arp cache lists the device as reachable via
bond0. If I search the arp cache to see whether the address is
reachable from one of bond0's slave devices, should it come back
positive or negative?

Chris Friesen

unread,
Dec 12, 2007, 11:30:14 AM12/12/07
to
I retested it on an x86 machine and am seeing similar problems.

First, "arp" gives the arp table as expected:

root@typhoon-base-unit0:/tftpboot/cnp/0-0-5-0/0-0-5-0> arp -n


Address HWtype HWaddress Flags Mask
Iface

172.24.0.9 ether 00:03:CC:51:06:5E C
bond0
10.41.18.101 ether 00:0E:0C:5E:95:BD C
eth6
172.24.137.0 ether 00:C0:8B:08:E4:88 C

bond0
172.24.136.0 ether 00:C0:8B:07:B3:7E C
bond0

10.41.18.1 ether 00:00:5E:00:01:01 C
eth6
172.24.0.5 ether 00:01:AF:15:E0:6A C
bond0
172.24.0.13 ether 00:0E:0C:85:FD:D2 C
bond0
172.24.0.3 ether 00:01:AF:14:C8:CC C
bond0
172.24.132.1 ether 00:01:AF:14:E9:88 C
bond0
172.24.0.7 ether 00:07:E9:41:4B:B4 C
bond0
192.168.24.81 ether 00:01:AF:14:E9:8A C
bond2

"ip neigh show" gives nothing, but if I search for specific addresses
from the arp table listing they show up:

root@typhoon-base-unit0:/tftpboot/cnp/0-0-5-0/0-0-5-0> ip neigh show
root@typhoon-base-unit0:/tftpboot/cnp/0-0-5-0/0-0-5-0> ip neigh show
172.24.0.9
172.24.0.9 dev bond0 lladdr 00:03:cc:51:06:5e DELAY
root@typhoon-base-unit0:/tftpboot/cnp/0-0-5-0/0-0-5-0> ip neigh show
10.41.18.101
10.41.18.101 dev eth6 lladdr 00:0e:0c:5e:95:bd REACHABLE
root@typhoon-base-unit0:/tftpboot/cnp/0-0-5-0/0-0-5-0> ip neigh show
172.24.137.0
172.24.137.0 dev bond0 lladdr 00:c0:8b:08:e4:88 REACHABLE


Is this expected behaviour?

Thanks,

Eric Dumazet

unread,
Dec 12, 2007, 12:00:19 PM12/12/07
to
Chris Friesen a écrit :
Probably not... Still a 2.6.14 kernel ?

Could you send the result of :

strace ip neigh show

Chris Friesen

unread,
Dec 12, 2007, 1:30:19 PM12/12/07
to
Eric Dumazet wrote:
> Chris Friesen a écrit :

>> Is this expected behaviour?
>
> Probably not... Still a 2.6.14 kernel ?

Yep. Embedded hardware, so I'm unable to test with a more recent kernel.

> Could you send the result of :
>
> strace ip neigh show

I've attached two strace runs, one of "ip neigh show" and one of "ip
neigh show 10.41.18.101".

Chris

trace.txt

Eric Dumazet

unread,
Dec 12, 2007, 2:00:23 PM12/12/07
to
Chris Friesen a écrit :

> Eric Dumazet wrote:
>> Chris Friesen a écrit :
>>> Is this expected behaviour?
>>
>> Probably not... Still a 2.6.14 kernel ?
>
> Yep. Embedded hardware, so I'm unable to test with a more recent kernel.
>

And what is the version of "ip" command you have on this machine ?

ip -V

You may try other versions of this command

http://devresources.linux-foundation.org/dev/iproute2/download/

Chris Friesen

unread,
Dec 12, 2007, 5:00:27 PM12/12/07
to
Eric Dumazet wrote:

> And what is the version of "ip" command you have on this machine ?
>
> ip -V

iproute2-ss051107

> You may try other versions of this command
>
> http://devresources.linux-foundation.org/dev/iproute2/download/

They appear to be numbered by kernel version, and the above version is
the most recent one for 2.6.14. Will more recent ones (for newer
kernels) work with my kernel?

Chris

YOSHIFUJI Hideaki / 吉藤英明

unread,
Dec 12, 2007, 5:10:48 PM12/12/07
to
In article <47605934...@nortel.com> (at Wed, 12 Dec 2007 15:57:08 -0600), "Chris Friesen" <cfri...@nortel.com> says:

> > You may try other versions of this command
> >
> > http://devresources.linux-foundation.org/dev/iproute2/download/
>
> They appear to be numbered by kernel version, and the above version is
> the most recent one for 2.6.14. Will more recent ones (for newer
> kernels) work with my kernel?

It should work; if it doesn't, please make a report. Thanks.

--yoshfuji

Chris Friesen

unread,
Dec 17, 2007, 10:40:19 AM12/17/07
to
YOSHIFUJI Hideaki / 吉藤英明 wrote:
> In article <47605934...@nortel.com> (at Wed, 12 Dec 2007
> 15:57:08 -0600), "Chris Friesen" <cfri...@nortel.com> says:

>>> You may try other versions of this command
>>>
>>> http://devresources.linux-foundation.org/dev/iproute2/download/
>>
>> They appear to be numbered by kernel version, and the above version
>> is the most recent one for 2.6.14. Will more recent ones (for
>> newer kernels) work with my kernel?

> It should work; if it doesn't, please make a report. Thanks.

I downloaded iproute2-2.6.23 and built it for my kernel.

I'm compiling for a different kernel than is actually running on the
build system, so I had to add a line defining KERNEL_INCLUDE to the
Makefile, and I had to add "-I${KERNEL_INCLUDE}" to the CFLAGS
definition. Someone might want to do something about that...

Anyways, the arp entry issue is still there. The "arp" command gives a
bunch of entries:

root@typhoon-base-unit0:/root> arp -n


Address HWtype HWaddress Flags Mask Iface

192.168.24.81 ether 00:01:AF:14:E9:8A C bond2

172.24.132.2 (incomplete) bond0


172.24.136.0 ether 00:C0:8B:07:B3:7E C bond0

172.24.137.0 (incomplete) bond0
172.24.0.9 ether 00:07:E9:41:4B:B4 C bond0


10.41.18.101 ether 00:0E:0C:5E:95:BD C eth6

172.24.0.11 ether 00:03:CC:51:06:5E C bond0


172.24.132.1 ether 00:01:AF:14:E9:88 C bond0

172.24.0.15 ether 00:0E:0C:85:FD:D2 C bond0


172.24.0.3 ether 00:01:AF:14:C8:CC C bond0

172.24.0.5 ether 00:01:AF:15:E0:6A C bond0

The original "ip" command and the new one ("/tmp/ip") both give the same
results--some of the entries are missing.

root@typhoon-base-unit0:/root> ip neigh show all
172.24.137.0 dev bond0 FAILED
172.24.0.9 dev bond0 lladdr 00:07:e9:41:4b:b4 REACHABLE


10.41.18.101 dev eth6 lladdr 00:0e:0c:5e:95:bd REACHABLE

172.24.0.11 dev bond0 lladdr 00:03:cc:51:06:5e STALE
172.24.132.1 dev bond0 lladdr 00:01:af:14:e9:88 REACHABLE
172.24.0.15 dev bond0 lladdr 00:0e:0c:85:fd:d2 STALE
172.24.0.3 dev bond0 lladdr 00:01:af:14:c8:cc REACHABLE
172.24.0.5 dev bond0 lladdr 00:01:af:15:e0:6a STALE

root@typhoon-base-unit0:/root> /tmp/ip neigh show all
172.24.137.0 dev bond0 FAILED
172.24.0.9 dev bond0 lladdr 00:07:e9:41:4b:b4 REACHABLE


10.41.18.101 dev eth6 lladdr 00:0e:0c:5e:95:bd REACHABLE

172.24.0.11 dev bond0 lladdr 00:03:cc:51:06:5e STALE
172.24.132.1 dev bond0 lladdr 00:01:af:14:e9:88 REACHABLE
172.24.0.15 dev bond0 lladdr 00:0e:0c:85:fd:d2 STALE
172.24.0.3 dev bond0 lladdr 00:01:af:14:c8:cc REACHABLE
172.24.0.5 dev bond0 lladdr 00:01:af:15:e0:6a STALE


However, if I specifically try to print out one of the missing entries,
it shows up:

root@typhoon-base-unit0:/root> /tmp/ip neigh show 192.168.24.81
192.168.24.81 dev bond2 lladdr 00:01:af:14:e9:8a REACHABLE


Chris

Patrick McHardy

unread,
Dec 17, 2007, 1:10:11 PM12/17/07
to


From a kernel perspective there are only complete dumps, the
filtering is done by iproute. So the fact that it shows them
when querying specifically implies there is a bug in the
iproute neighbour filter. Does it work if you omit "all"
from the ip neigh show command?

Chris Friesen

unread,
Dec 17, 2007, 3:20:16 PM12/17/07
to
Patrick McHardy wrote:

> From a kernel perspective there are only complete dumps, the
> filtering is done by iproute. So the fact that it shows them
> when querying specifically implies there is a bug in the
> iproute neighbour filter. Does it work if you omit "all"
> from the ip neigh show command?

Omitting "all" gives identical results. It is still missing entries
when compared with the output of "arp".

Chris

Patrick McHardy

unread,
Dec 17, 2007, 7:00:18 PM12/17/07
to
Chris Friesen wrote:
> Patrick McHardy wrote:
>
>> From a kernel perspective there are only complete dumps, the
>> filtering is done by iproute. So the fact that it shows them
>> when querying specifically implies there is a bug in the
>> iproute neighbour filter. Does it work if you omit "all"
>> from the ip neigh show command?
>
> Omitting "all" gives identical results. It is still missing entries
> when compared with the output of "arp".


In that case the easiest way to debug this is probably if you
add some debugging to ip/ipneigh.c:print_neigh() since I'm
unable to reproduce this problem. A printf for all the filter
conditions (=> return 0) at the top should do.

Patrick McHardy

unread,
Dec 17, 2007, 7:40:10 PM12/17/07
to
Thomas Graf wrote:
> * Patrick McHardy <ka...@trash.net> 2007-12-18 00:51

>> Chris Friesen wrote:
>>> Patrick McHardy wrote:
>>>
>>>> From a kernel perspective there are only complete dumps, the
>>>> filtering is done by iproute. So the fact that it shows them
>>>> when querying specifically implies there is a bug in the
>>>> iproute neighbour filter. Does it work if you omit "all"
>>> >from the ip neigh show command?
>>>
>>> Omitting "all" gives identical results. It is still missing entries
>>> when compared with the output of "arp".
>>
>> In that case the easiest way to debug this is probably if you
>> add some debugging to ip/ipneigh.c:print_neigh() since I'm
>> unable to reproduce this problem. A printf for all the filter
>> conditions (=> return 0) at the top should do.
>
> Alternatively, you can download libnl and run
>
> NLCB=debug src/nl-neigh-dump brief
>
> and check if the netlink message is sent by the kenrel for the
> neighbour in question.


It should be, according to Chris, "ip neigh show <ip>" does
show the missing entries, and in case of neighbour entries
all filtering is done in userspace.

Thomas Graf

unread,
Dec 17, 2007, 8:00:16 PM12/17/07
to
* Patrick McHardy <ka...@trash.net> 2007-12-18 00:51
> Chris Friesen wrote:
> >Patrick McHardy wrote:
> >
> >> From a kernel perspective there are only complete dumps, the
> >>filtering is done by iproute. So the fact that it shows them
> >>when querying specifically implies there is a bug in the
> >>iproute neighbour filter. Does it work if you omit "all"
> >>from the ip neigh show command?
> >
> >Omitting "all" gives identical results. It is still missing entries
> >when compared with the output of "arp".
>
>
> In that case the easiest way to debug this is probably if you
> add some debugging to ip/ipneigh.c:print_neigh() since I'm
> unable to reproduce this problem. A printf for all the filter
> conditions (=> return 0) at the top should do.

Alternatively, you can download libnl and run

NLCB=debug src/nl-neigh-dump brief

and check if the netlink message is sent by the kenrel for the
neighbour in question.

Herbert Xu

unread,
Dec 17, 2007, 10:40:08 PM12/17/07
to
Chris Friesen <cfri...@nortel.com> wrote:
>
> However, if I specifically try to print out one of the missing entries,
> it shows up:
>
> root@typhoon-base-unit0:/root> /tmp/ip neigh show 192.168.24.81
> 192.168.24.81 dev bond2 lladdr 00:01:af:14:e9:8a REACHABLE

What about

ip -4 neigh show

Thanks,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <her...@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Chris Friesen

unread,
Dec 18, 2007, 10:00:17 AM12/18/07
to
Herbert Xu wrote:
> Chris Friesen <cfri...@nortel.com> wrote:
>
>>However, if I specifically try to print out one of the missing entries,
>>it shows up:
>>
>>root@typhoon-base-unit0:/root> /tmp/ip neigh show 192.168.24.81
>>192.168.24.81 dev bond2 lladdr 00:01:af:14:e9:8a REACHABLE
>
>
> What about
>
> ip -4 neigh show

Looks like that did it. Why does specifying the family make a difference?

root@typhoon-base-unit0:/root> ip neigh show

10.41.18.1 dev eth6 lladdr 00:00:5e:00:01:01 STALE


172.24.137.0 dev bond0 lladdr 00:c0:8b:08:e4:88 REACHABLE

172.24.0.9 dev bond0 lladdr 00:07:e9:41:4b:b4 REACHABLE
10.41.18.101 dev eth6 lladdr 00:0e:0c:5e:95:bd REACHABLE
172.24.0.11 dev bond0 lladdr 00:03:cc:51:06:5e STALE
172.24.132.1 dev bond0 lladdr 00:01:af:14:e9:88 REACHABLE
172.24.0.15 dev bond0 lladdr 00:0e:0c:85:fd:d2 STALE
172.24.0.3 dev bond0 lladdr 00:01:af:14:c8:cc REACHABLE
172.24.0.5 dev bond0 lladdr 00:01:af:15:e0:6a STALE

root@typhoon-base-unit0:/root> ip -4 neigh show


192.168.24.81 dev bond2 lladdr 00:01:af:14:e9:8a REACHABLE

172.24.132.2 dev bond0 FAILED
172.24.136.0 dev bond0 lladdr 00:c0:8b:07:b3:7e REACHABLE
10.41.18.1 dev eth6 lladdr 00:00:5e:00:01:01 STALE


172.24.137.0 dev bond0 lladdr 00:c0:8b:08:e4:88 REACHABLE

172.24.0.9 dev bond0 lladdr 00:07:e9:41:4b:b4 REACHABLE
10.41.18.101 dev eth6 lladdr 00:0e:0c:5e:95:bd REACHABLE
172.24.0.11 dev bond0 lladdr 00:03:cc:51:06:5e STALE
172.24.132.1 dev bond0 lladdr 00:01:af:14:e9:88 REACHABLE
172.24.0.15 dev bond0 lladdr 00:0e:0c:85:fd:d2 STALE
172.24.0.3 dev bond0 lladdr 00:01:af:14:c8:cc REACHABLE
172.24.0.5 dev bond0 lladdr 00:01:af:15:e0:6a STALE

Chris

Herbert Xu

unread,
Dec 18, 2007, 8:20:11 PM12/18/07
to
On Tue, Dec 18, 2007 at 08:52:01AM -0600, Chris Friesen wrote:
>
> Looks like that did it. Why does specifying the family make a difference?

Because this is the only parameter that changes kernel behaviour.

Next step is to strace both commands with -s 16384 to see exactly
what the kernel reply looks like to determine the problem.

BTW my emails to you are bouncing so you might want to fix that.

Thanks,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <her...@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Herbert Xu

unread,
Dec 24, 2007, 8:10:09 PM12/24/07
to
On Wed, Dec 19, 2007 at 08:28:49AM -0600, Chris Friesen wrote:
>
> I've included the traces below, they're pretty long. I don't know
> enough about the netlink format to trace the responses, but I did notice
> one interesting thing.
>
> In the "ip neigh show" case the "bond2" entry isn't printed out, but I
> can see the string "bond2" in the recvmsg() call so it looks like the

Actually bond2 isn't in the recvmsg, it's just that strace is
printing out the complete content of the buffer which happenes
to include bond2.

I've verified that the first recvmsg indeed does not contain the
bond2 address at all. Instead of the first 3 IPv4 records it has
3 IPv6 records. So we seem to have a kernel problem here.

However, too many changes have been made in this area since 2.6.14
for it to be worthwhile for me to debug this any further until you
manage to reproduce it with the current kernel.

Cheers,

0 new messages