unable to ping outside thread network

[Sathish guru]

Hello All,

Trying to setup a thread network and communicate to the outside thread network (IP V6) through border router.

• Both border router and end device runs on RPi in NCP mode.
• Setup the network with on-mesh prefix fd11:22:db8:2::
• Host and border router has been added with ipv6 with fd11:22:db8:1::2 and fd11:22:db8:1::3
• Added a route on the host network for  fd11:22:db8:2:: to go via gw
  

When a native commissioner is started and end device is joined the network, end device is able to reach the host without any issues.

But when external commissioner is used instead of native commissioner, end device is able to reach only the border router thread ip and not able to reach the host ip or border router's eth ip (fd11:22:db8:1::3)

Any insight on when the native commissioner the end device is able to reach the host ipv6 but not able to ping when external commissioner is used ? Does during commissioning process, any off-mesh route details shared to the end devices? Even when off-mesh is added manually still the host is not reachable. But the moment native commissioner is started communication is established.

Thanks,

Sathish

[Jonathan Hui]

Whether the commissioner is on-mesh or external should not affect IPv6 routing within the Thread network.

On the end device, can you provide the output of `wpanctl get Thread:OnMeshPrefixes` when ping is working and also when ping is not working?

On the end device and border router, can you use `tcpdump` to see where the IPv6 packets are getting forwarded?

--
Jonathan Hui

--
You received this message because you are subscribed to the Google Groups "openthread-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openthread-use...@googlegroups.com.
To post to this group, send email to openthre...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openthread-users/c9b97818-bd44-4220-957b-987754e736f7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sathish guru]

Thanks Jonathan for the reply.

Please find the details below

Couldnt see any difference in the Thread:OnThreadMeshPrefixes.

External Commissioner started and end device joined: Ping unsuccessful to Non-thread IPv6 network.

End device Thread:OnMeshPrefixes

wpanctl:wpan0> getprop Thread:OnMeshPrefixes
Thread:OnMeshPrefixes = [
    "fd11:22:db8:2::        prefix_len:64   origin:ncp      stable:yes flags:0x33 [on-mesh:1 def-route:1 config:0 dhcp:0 slaac:1 pref:1 prio:med] rloc:0xdc00"
]

External Commissioner still running. On Mesh native commissioner also started. Ping successful to Non-thread IPV6 network.

End device Thread:OnMeshPrefixes

Thread:OnMeshPrefixes = [
    "fd11:22:db8:2::        prefix_len:64   origin:ncp      stable:yes flags:0x33 [on-mesh:1 def-route:1 config:0 dhcp:0 slaac:1 pref:1 prio:med] rloc:0xdc00"
]

Attached the sniffer wireshark dump before the on-mesh commissioner start and after on-mesh commissioner start.

Couldnt see any major difference except when on-mesh commissioner is started border router starts accepting the packet.

Thanks,

Sathish G

[Jonathan Hui]

I'm not able to reproduce the issue.

Can you provide the exact steps and commands that you used so we can reproduce this?

--

Jonathan Hui

[Sathish guru]

Hi Jonathan,  Here are the steps being carried out

1. Start the OTBR (Rpi + Atmel).
   
2. Add Ipv6 to eth0 in OTBR as fd11:22:db8:1::2
3. Add Ipv6 to eth0 in Host machine as fd11:22:db8:1::3
4. Add route in Host machine for fd11:22:db8:2:: via fd11:22:db8:1::2
   
5. Form the thread network with on-mesh prefix: fd11:22:db8:2:: with default route checked.
6. Start the OT-FTD in the end-device
7. Set the panid, channel and ifconfig up
8. Start the external commissioner app and connect to the OTBR wifi AP
9. Generate QR code for the end device and scan in the app
10. Start the joiner command in the end device and join the network
11. Now ping the host machine ip (fd11:22:db8:1::3) or OTBR eth0 ip(fd11:22:db8:1::2) from end-device - PING Unsuccessful
12. Now stop the external commissioner app or stop/kill OTBR-agent in OTBR or start native commissioner in OTBR - PING Successful.

Current inference: When OTBR agent is running or external commissioner is running communication to outside thread network is not happening. After commissioning if the external commissioner is stopped then non-thread IPV6 communication works.

To view this discussion on the web visit https://groups.google.com/d/msgid/openthread-users/CAGwZUDsFa-WDDWgc_OX7UY49%3D7dPuPagtCDy74bA%2B%2BghbcHUpg%40mail.gmail.com.

[Jonathan Hui]

Is it possible for you to share the Thread Network Key in order to decrypt the messages in the pcap files?

Also, can you generate a single sniffer trace that captures:

1. Starts before commissioning the end device
2. Start External Commissioner
3. Commissioning of the end device
4. End device attaching to Thread network
5. End device trying to ping external address (and fails)
6. Stop External Commissioner
7. End device trying to ping external address (and works)

At the same time, can you share Linux-side pcaps of the wpan0 interfaces as well?

--

Jonathan Hui

[Vijay Annamalaisamy]

Hi Jonathan,

The key we used was "00112233445566778899aabbccddeeff". External commissioner used was Android Thread APP

Attached pcap files taken in borderrouter for wpan0 side and with USB sniffer attached to a laptop.

Also could you please confirm us the below

1. As per the thread spec, section 8.4.2.2 in page 214, Fig 8-2, once commissioner authentication is completed, commissioner needs to send MGMT_SET and commissioner dataset. Is this mandatory? We are using external commissioner code from https://github.com/openthread/ot-br-posix. In the function Commissioner::CommissionerSet in the file src/commissioner/commissioner.cpp, only commissioner session ID and steering data is sent. Border agent locator and joinerUdpport is not sent as part of this. Is this causing the issue?

Please let us know your comments

Regards,

Vijay A.

[Jonathan Hui]

Looking at the wpan0 pcap file, it appears the ICMPv6 Echo Request messages are reaching the NCP (see packets 91 through 96).

I can't tell anything wrong on the Thread network side. The Thread Network Data shows the "fd11:22::/64" prefix with a default route. The Thread end device is properly forwarding ICMPv6 Echo Requests to the border router's NCP.

Can you also capture what's coming out of the wpan0 interface using tcpdump? "e.g. with `sudo tcpdump -i wpan0`"

To answer your question, after establishing a DTLS session with the Border Router, the Commissioner is only required to set the Steering Data to open the network for joining.

--

Jonathan Hui

To view this discussion on the web visit https://groups.google.com/d/msgid/openthread-users/6d831a04-d208-4e01-99f2-12fb6bdb10df%40googlegroups.com.

[Sathish guru]

One thing we noticed is the moment otbr-agent is stopped in otbr, packets reach the host machine. 

To view this discussion on the web visit https://groups.google.com/d/msgid/openthread-users/CAGwZUDufAtfD65i2D2O5wO2yJiEiNWx4qR9%2BHOWMyKnD8Odx3Q%40mail.gmail.com.

[Jonathan Hui]

Just curious - after you are able to successfully ping, if you use the Thread Commissioning App to connect to the Border Router again, do you see the pings stop working?

--
Jonathan Hui

[Sathish guru]

Yes.Ping stops immediately when the Thread commissioning app connects back to otbr.

[Jonathan Hui]

Are you able to provide a capture of packets coming out of the wpan0 interface to the Linux host using tcpdump?

--

Jonathan Hui

[Vijay Annamalaisamy]

Hi Jonathan,

Attached ping.pcap is the capture taken from wpan0 using tcpdump. In this capture, in the non working case there were no ping requests received. In the working case, ping request and ping reply packets are there at the end of the capture.

Corresponding Sniffer capture is also attached.

Other Observations:

1. Whenever, date is changed in border router(using date -s command), it triggers "Data Response" most of the times which makes ping successful even with external commissioner

2. Whenever ping works in any case, address query(a/aq) and address notification(a/an) coap msgs are getting exchanged which is not happening in not working case. This could be seen in the attached sniffer capture

3. Whenever external commissioner is disconnected, after sometime this event triggers "Data Response" which makes ping successful.

So we feel some of the parameter in data set(from Data Response) is responsible for the ping to work. We are trying to find that out. Please let us know your thoughts on these observations. It will be very helpful.

Thanks,

Vijay.

[Jonathan Hui]

Thanks for the additional information.

It seems the end device is properly sending IPv6 Echo Requests to the NCP, but the NCP is not forwarding them to the host. The logic that forwards packets to the host is in src/core/net/ip6.cpp. If possible, I would suggest adding logs/breakpoints to see why packets are not getting forwarded to host.

The Address Query/Notification messages are triggered when trying to forward the ICMPv6 Echo Reply message, so that does not really indicate the problem.

The Data Response messages are sent whenever there is a change to the Thread Network Data. However, it seems the NCP has on-mesh prefix with default route at all times (the Child ID Response and Data Response messages have the same on-mesh prefix information).

--
Jonathan Hui

To view this discussion on the web visit https://groups.google.com/d/msgid/openthread-users/fe541cb6-5db5-436b-9b69-c469fc305838%40googlegroups.com.

[Vijay Annamalaisamy]

Hi Jonathan,

We added logs in src/core/net/ip6.cpp(attached with logs) and below are the observations

1. In non working case(with externa commissioner), GetOnLinkNetif()  is returning value greater than 0. Below are the logs. Please check with the attached file.

Mar 15 13:19:32 raspberrypi wpantund[18704]: IP6 - Unicast..

Mar 15 13:19:32 raspberrypi wpantund[18704]: IP6 - Linklocal Unicast..

Mar 15 13:19:32 raspberrypi wpantund[18704]: IP6 - forward1..

Mar 15 13:19:32 raspberrypi wpantund[18704]: DestIP - [2001:22:db8:1:0:0:0:2]

Mar 15 13:19:32 raspberrypi wpantund[18704]: NetifIP - [fd11:1111:1122:0:0:ff:fe00:fc37]

Mar 15 13:19:32 raspberrypi wpantund[18704]: PrefixLength is 0

Mar 15 13:19:32 raspberrypi wpantund[18704]: iForLoop is 1

Mar 15 13:19:32 raspberrypi wpantund[18704]: oForLoop is 1

Mar 15 13:19:32 raspberrypi wpantund[18704]: rval is 1

Mar 15 13:19:32 raspberrypi wpantund[18704]: IP6 - Iface 3 global..

Mar 15 13:19:32 raspberrypi wpantund[18704]: Fwd Iface ID  - 1

Mar 15 13:19:32 raspberrypi wpantund[18704]: IP6 - forward3..

Mar 15 13:19:32 raspberrypi wpantund[18704]: IP6 - forward5..

Mar 15 13:19:32 raspberrypi wpantund[18704]: IP6 - exit..

Mar 15 13:19:32 raspberrypi wpantund[18704]: IP6 - Unicast..

Mar 15 13:19:32 raspberrypi wpantund[18704]: IP6 - 1 Unicast..

Mar 15 13:19:32 raspberrypi wpantund[18704]: IP6 - Receive 1..

Mar 15 13:19:32 raspberrypi wpantund[18704]: IP6 - exit..

In working case(When native commissioner is started), GetOnLinkNetif()  is returning value lesser than 0. Below are the logs.

Mar 15 13:44:29 raspberrypi wpantund[18704]: IP6 - Unicast..

Mar 15 13:44:29 raspberrypi wpantund[18704]: IP6 - Linklocal Unicast..

Mar 15 13:44:29 raspberrypi wpantund[18704]: IP6 - forward1..

Mar 15 13:44:29 raspberrypi wpantund[18704]: DestIP - [2001:22:db8:1:0:0:0:2]

Mar 15 13:44:29 raspberrypi wpantund[18704]: NetifIP - [fd11:22:0:0:7428:71ab:ebbc:3358]

Mar 15 13:44:29 raspberrypi wpantund[18704]: PrefixLength is 64

Mar 15 13:44:29 raspberrypi wpantund[18704]: NetifIP - [fd11:1111:1122:0:0:ff:fe00:fc00]

Mar 15 13:44:29 raspberrypi wpantund[18704]: PrefixLength is 128

Mar 15 13:44:29 raspberrypi wpantund[18704]: NetifIP - [fd11:1111:1122:0:0:ff:fe00:8c00]

Mar 15 13:44:29 raspberrypi wpantund[18704]: PrefixLength is 64

Mar 15 13:44:29 raspberrypi wpantund[18704]: NetifIP - [fd11:1111:1122:0:92e:db17:2800:cefe]

Mar 15 13:44:29 raspberrypi wpantund[18704]: PrefixLength is 64

Mar 15 13:44:29 raspberrypi wpantund[18704]: NetifIP - [fe80:0:0:0:7428:71ab:ebbc:3358]

Mar 15 13:44:29 raspberrypi wpantund[18704]: PrefixLength is 64

Mar 15 13:44:29 raspberrypi wpantund[18704]: iForLoop is 5

Mar 15 13:44:29 raspberrypi wpantund[18704]: oForLoop is 1

Mar 15 13:44:29 raspberrypi wpantund[18704]: rval is -1

Mar 15 13:44:29 raspberrypi wpantund[18704]: IP6 - Iface 5..

Mar 15 13:44:29 raspberrypi wpantund[18704]: Fwd Iface ID  - 0

Mar 15 13:44:29 raspberrypi wpantund[18704]: IP6 - forward2..

Mar 15 13:44:29 raspberrypi wpantund[18704]: IP6 - exit..

Mar 15 13:44:29 raspberrypi wpantund[18704]: IP6 - Unicast..

Mar 15 13:44:29 raspberrypi wpantund[18704]: IP6 - Linklocal Unicast..

Mar 15 13:44:29 raspberrypi wpantund[18704]: IP6 - forward1..

Mar 15 13:44:29 raspberrypi wpantund[18704]: DestIP - [fd11:22:0:0:a48:3f8:57eb:4b70]

Mar 15 13:44:29 raspberrypi wpantund[18704]: NetifIP - [fd11:22:0:0:7428:71ab:ebbc:3358]

Mar 15 13:44:29 raspberrypi wpantund[18704]: PrefixLength is 64

Mar 15 13:44:29 raspberrypi wpantund[18704]: iForLoop is 1

Mar 15 13:44:29 raspberrypi wpantund[18704]: oForLoop is 1

Mar 15 13:44:29 raspberrypi wpantund[18704]: rval is 1

Mar 15 13:44:29 raspberrypi wpantund[18704]: IP6 - Iface 3 global..

Mar 15 13:44:29 raspberrypi wpantund[18704]: Fwd Iface ID  - 1

Mar 15 13:44:29 raspberrypi wpantund[18704]: IP6 - forward5..

Mar 15 13:44:29 raspberrypi wpantund[18704]: IP6 - exit..

Observations:

===========

1. In non working case, commissioner ALOC alone is present as part of mNetifListHead whereas in working case, ip addresses of wpan0, leader ip address, leader aloc address are present in mNetifListHead 

2. In a scenario we tested, first with native commissioner started and it works fine with mNetifListHead containing all IPs. The same exists even when native commissioner is stopped. But  the moment when external commissioner is started, mNetifListHead  contains only commissioner ALOC.

Please let us know your thoughs on these observations

Thanks.

[Sathish guru]

Jonathan, Any insights or directions welcome based on the latest details provided below.

[Jonathan Hui]

Thanks for the additional logging. The fact that the Commissioner ALOC is added with zero prefix length indicates the bug.

This issue was raised in #3322 and fixed in #3328. I would suggest updating to the latest master to pull in the fix.

Hope that helps.

--
Jonathan Hui

To view this discussion on the web visit https://groups.google.com/d/msgid/openthread-users/53ad7b54-704e-407b-acae-1b6b470ce00f%40googlegroups.com.