Application architecture

[ka...@tokateam.com]

Hello,

I'm recently analyzing the Thread protocol and OpenThread implementation trying to understand what whould be the best way to implement some application services which I need. While I got most of the basic details sorted out, there are few questions to which I don't know the good answer yet.

First, few words about what I'm trying to do, as a context for the questions: First, lets assume that an application has some software running on Thread mesh, with 2 types of nodes: regular ones and border routers. Software on both nodes are different, as border routers can have more features (e.g. keep connecting to some external services, e.g. by means of MQTT). I do need to connect from node to outside service(s) and outside servers being able to send some data (e.g. parameters update) to specific node as well. This connectivity do not need to be direct (e.g. going through some proxy software being installed on border router is OK), and most likely will need to use tcp and something else than coap. Second, I'm looking for some kind service discovery mechanism inside the mesh. 

Now, the questions. Since it seems that the only 'native' way of communicating between node and external world would be udp + coap, and I need to use tcp and other services (e.g. something like MQTT), it seems that the host part of the border router (which I plan to implement using some linux host, e.g. RaspberryPI) will need to run some 'proxy' software, e.g. doing MQTT over tcp from the open internet side and coap+udp from the mesh side. I'm assuming here some dedicated chip like Nordic nRF52840, so no linux on nodes, FreeRTOS at most. FreeRTOS has some TCP implementation, though I'd prefer to avoid it, if there are other good solutions.

Having a dedicated software running on a border router means, that each node will need to contact the closest border router in the mesh. How to find out the closest border router from the node, and send data to it, directly? I mean, if I will know rloc16, or its EID or other ID, sending data there is not an issue. But, using plain OpenThread API without any extra discovery services, can I easily locate the closest border router?

If we will have some service discovery, the problem above can be solved just by that mechanism. It seems, that Thread specification do have `a/sd` endpoint accepting Server TLV and Service TLV, so each node could register itself to leader as a server (and it mentions that mesh can route packets to nearest server for given service). Yet, OpenThread in https://github.com/openthread/openthread/blob/master/src/core/thread/network_data_tlvs.hpp#L71-L82 and https://github.com/openthread/openthread/blob/master/src/core/thread/network_data_leader_ftd.cpp#L603-L627 seems to not implement this. Is that correct? Can this mechanism be used as a service discovery as I need it (i.e. my app to define some services, and then nodes will contact the closest server for given service), and I just should wait for OpenThread implementation? Or I should use something else?

I do not want to use some centralized way of service discovery, like single node being DNS server or anything like that - I'm looking for distributed approaches, preferably to be distributed like leader's network data structure.

Other question: it seems, that application should use EID when talking from one node to another, and map it to current RLOC16 every time (either by cache or address query). When some external service want to talk to specific node, my plan is for it to contact the proxy software running on border router, and then go via coap. But, external software only knows MAC address or some serial number of the destination node. So, I need to map that somehow to EID or RLOC16. Is there some existing feature / implementation which could help me, or I need to implement it from scratch myself, e.g. by doing broadcasts from border router to discover all necessary data about connected nodes? 

Thanks in advance.

[Jonathan Hui]

Thanks for you detailed email.  See responses below:

On Fri, Aug 18, 2017 at 5:00 AM, <ka...@tokateam.com> wrote:

I'm recently analyzing the Thread protocol and OpenThread implementation trying to understand what whould be the best way to implement some application services which I need. While I got most of the basic details sorted out, there are few questions to which I don't know the good answer yet.

First, few words about what I'm trying to do, as a context for the questions: First, lets assume that an application has some software running on Thread mesh, with 2 types of nodes: regular ones and border routers. Software on both nodes are different, as border routers can have more features (e.g. keep connecting to some external services, e.g. by means of MQTT). I do need to connect from node to outside service(s) and outside servers being able to send some data (e.g. parameters update) to specific node as well. This connectivity do not need to be direct (e.g. going through some proxy software being installed on border router is OK), and most likely will need to use tcp and something else than coap. Second, I'm looking for some kind service discovery mechanism inside the mesh. 

Now, the questions. Since it seems that the only 'native' way of communicating between node and external world would be udp + coap, and I need to use tcp and other services (e.g. something like MQTT), it seems that the host part of the border router (which I plan to implement using some linux host, e.g. RaspberryPI) will need to run some 'proxy' software, e.g. doing MQTT over tcp from the open internet side and coap+udp from the mesh side. I'm assuming here some dedicated chip like Nordic nRF52840, so no linux on nodes, FreeRTOS at most. FreeRTOS has some TCP implementation, though I'd prefer to avoid it, if there are other good solutions.

As you noted, Thread and OpenThread transport IPv6 datagrams and do not preclude the use of TCP.  However, as you also noted, TCP may not be desirable in low-power, mesh networks for a variety of reasons.

Having a dedicated software running on a border router means, that each node will need to contact the closest border router in the mesh. How to find out the closest border router from the node, and send data to it, directly? I mean, if I will know rloc16, or its EID or other ID, sending data there is not an issue. But, using plain OpenThread API without any extra discovery services, can I easily locate the closest border router?

The Border Router information is encoded in the Thread Network Data.  Determining the closest border router is possible to do with a little bit of work:

1) OpenThread provides an API to extract and iterate through the Border Router entries with otBorderRouterGetNextOnMeshPrefix().

2) OpenThread provides an API to extract routing info with otThreadGetRouterInfo().

If we will have some service discovery, the problem above can be solved just by that mechanism. It seems, that Thread specification do have `a/sd` endpoint accepting Server TLV and Service TLV, so each node could register itself to leader as a server (and it mentions that mesh can route packets to nearest server for given service). Yet, OpenThread in https://github.com/openthread/openthread/blob/master/src/core/thread/network_data_tlvs.hpp#L71-L82 and https://github.com/openthread/openthread/blob/master/src/core/thread/network_data_leader_ftd.cpp#L603-L627 seems to not implement this. Is that correct? Can this mechanism be used as a service discovery as I need it (i.e. my app to define some services, and then nodes will contact the closest server for given service), and I just should wait for OpenThread implementation? Or I should use something else?

Thanks for pointing this out.  We can work to add support for Service and Server TLVs.  Unfortunately, the Thread Certification Test Plan does not cover Service and Server TLVs, so this escaped the Thread Group testing process.

I do not want to use some centralized way of service discovery, like single node being DNS server or anything like that - I'm looking for distributed approaches, preferably to be distributed like leader's network data structure.

Other question: it seems, that application should use EID when talking from one node to another, and map it to current RLOC16 every time (either by cache or address query). When some external service want to talk to specific node, my plan is for it to contact the proxy software running on border router, and then go via coap. But, external software only knows MAC address or some serial number of the destination node. So, I need to map that somehow to EID or RLOC16. Is there some existing feature / implementation which could help me, or I need to implement it from scratch myself, e.g. by doing broadcasts from border router to discover all necessary data about connected nodes? 

Thread does not provide a device discovery service.  This is typically an application-layer service.

--

Jonathan Hui

 

[ka...@tokateam.com]

Thanks for your response, it is very valuable! See for comments below.

W dniu piątek, 18 sierpnia 2017 18:51:36 UTC+2 użytkownik Jonathan Hui napisał:

Thanks for you detailed email.  See responses below:

On Fri, Aug 18, 2017 at 5:00 AM, <ka...@tokateam.com> wrote:

I'm recently analyzing the Thread protocol and OpenThread implementation trying to understand what whould be the best way to implement some application services which I need. While I got most of the basic details sorted out, there are few questions to which I don't know the good answer yet.

First, few words about what I'm trying to do, as a context for the questions: First, lets assume that an application has some software running on Thread mesh, with 2 types of nodes: regular ones and border routers. Software on both nodes are different, as border routers can have more features (e.g. keep connecting to some external services, e.g. by means of MQTT). I do need to connect from node to outside service(s) and outside servers being able to send some data (e.g. parameters update) to specific node as well. This connectivity do not need to be direct (e.g. going through some proxy software being installed on border router is OK), and most likely will need to use tcp and something else than coap. Second, I'm looking for some kind service discovery mechanism inside the mesh. 

Now, the questions. Since it seems that the only 'native' way of communicating between node and external world would be udp + coap, and I need to use tcp and other services (e.g. something like MQTT), it seems that the host part of the border router (which I plan to implement using some linux host, e.g. RaspberryPI) will need to run some 'proxy' software, e.g. doing MQTT over tcp from the open internet side and coap+udp from the mesh side. I'm assuming here some dedicated chip like Nordic nRF52840, so no linux on nodes, FreeRTOS at most. FreeRTOS has some TCP implementation, though I'd prefer to avoid it, if there are other good solutions.

As you noted, Thread and OpenThread transport IPv6 datagrams and do not preclude the use of TCP.  However, as you also noted, TCP may not be desirable in low-power, mesh networks for a variety of reasons.

Having a dedicated software running on a border router means, that each node will need to contact the closest border router in the mesh. How to find out the closest border router from the node, and send data to it, directly? I mean, if I will know rloc16, or its EID or other ID, sending data there is not an issue. But, using plain OpenThread API without any extra discovery services, can I easily locate the closest border router?

The Border Router information is encoded in the Thread Network Data.  Determining the closest border router is possible to do with a little bit of work:

1) OpenThread provides an API to extract and iterate through the Border Router entries with otBorderRouterGetNextOnMeshPrefix().

2) OpenThread provides an API to extract routing info with otThreadGetRouterInfo().

This one was extremely helpful, as it basically solves one of the problems I was facing with.

 

If we will have some service discovery, the problem above can be solved just by that mechanism. It seems, that Thread specification do have `a/sd` endpoint accepting Server TLV and Service TLV, so each node could register itself to leader as a server (and it mentions that mesh can route packets to nearest server for given service). Yet, OpenThread in https://github.com/openthread/openthread/blob/master/src/core/thread/network_data_tlvs.hpp#L71-L82 and https://github.com/openthread/openthread/blob/master/src/core/thread/network_data_leader_ftd.cpp#L603-L627 seems to not implement this. Is that correct? Can this mechanism be used as a service discovery as I need it (i.e. my app to define some services, and then nodes will contact the closest server for given service), and I just should wait for OpenThread implementation? Or I should use something else?

Thanks for pointing this out.  We can work to add support for Service and Server TLVs.  Unfortunately, the Thread Certification Test Plan does not cover Service and Server TLVs, so this escaped the Thread Group testing process.

Ah. I was wondering whether I do understand it correctly, or I'm missing something ;)

 

I do not want to use some centralized way of service discovery, like single node being DNS server or anything like that - I'm looking for distributed approaches, preferably to be distributed like leader's network data structure.

Other question: it seems, that application should use EID when talking from one node to another, and map it to current RLOC16 every time (either by cache or address query). When some external service want to talk to specific node, my plan is for it to contact the proxy software running on border router, and then go via coap. But, external software only knows MAC address or some serial number of the destination node. So, I need to map that somehow to EID or RLOC16. Is there some existing feature / implementation which could help me, or I need to implement it from scratch myself, e.g. by doing broadcasts from border router to discover all necessary data about connected nodes? 

Thread does not provide a device discovery service.  This is typically an application-layer service.

Now, isn't Service TLV + Server TLV meant as kind of service discovery mechanism? The way I think it is intended to work is:

a) node registers itself as a server

b) leader distributes service information in network data

c) any other node can then easily connect to the server

If I understand it properly, also following remarks are true:

- service ALOC is being advertised in the network data, regardless of the actual server addresses. So, if the server node dies and another will register, all clients can still use the same ALOC

- some part of the Thread spec (cant remember now which one) mentioned that the requests from client are being routed to the closest server

If the above is true (the semantic of ALOC is still not 100% clear to me), the above would be exactly what I need. Like, on a client I would love to get some API like repetitively calling otServerGetNext() (similarly to iterate the border routers with otBorderRouterGetNextOnMeshPrefix()) to receive all service descriptions, and then just send the message to the server which client selected based on those data. Plus, of course some otServerRegisterData() kind of call for easy registration of local services with the leader.

Or my understanding is completely wrong and this mechanism is not supposed to do stuff like this?

Btw. if my understanding of the above is actually correct, I'm also considering doing the implementation (or help with it) in my team, the submitting some pull request. Some core team help will probably be necessary, but hopefully not so much. It will still probably be simpler for my team to extend OpenThread by this feature rather than develop whole new protocol just for this purpose.

 

--

Jonathan Hui

 

[Jonathan Hui]

See response below:

Yes, I believe your understanding is correct.  You've done your homework! :)

Btw. if my understanding of the above is actually correct, I'm also considering doing the implementation (or help with it) in my team, the submitting some pull request. Some core team help will probably be necessary, but hopefully not so much. It will still probably be simpler for my team to extend OpenThread by this feature rather than develop whole new protocol just for this purpose.

Contributions are always welcome!  I'll assign Issue #2120 to you.  Looking forward to your PR!

--

Jonathan Hui

[ka...@tokateam.com]

Yay, ideas are usually being punished by having to execute them, thanks! ;)

I'll move the related discussion to the Github issue.