leaf nodes for an IoT system

276 views
Skip to first unread message

Cliff Brake

unread,
Apr 27, 2021, 11:00:07 AM4/27/21
to nats
Hi,

I'm working on an IoT project where I plan to run multiple instances in different locations (cloud, edge, etc) and connect them via NATS. Each instance has its own database (currently Genji, which is embedded -- similar to sqlite). Most of the data is structured as a tree of nodes and each node contains a number of points that encapsulate configuration and status information. The structure of the tree is used to manage groups, permissions, what data gets processed by rules, etc.

The current NATS API:


The data flow is structured so that all write data goes through NATS and then hits the database. This is working well and makes it easy to add functionality.

A single instance is working pretty well, but now need to implement synchronization between device/cloud instances. My current thought is that node tree in an edge device would be fully synchronized to an upstream cloud instance and become part of a larger tree in the cloud as shown in the below diagram:


Pondering how to use NATS to synchronize the nodes between the cloud and edge devices ...  I've been looking into leaf nodes, but it is not obvious to me yet how to set up the topic namespace for this. Currently the NATS topic namespace (see API link above) does not reflect the node tree structure -- maybe it should. One puzzle is the edge nodes would only have part of the cloud tree. So the tree location for a node in the cloud might be:

/root/groupA/subgroupB/device1/modbus/io-1

On the edge device, it would be:

/device1/modbus/io-1

Nodes can be mirrored to multiple places in the cloud tree -- for instance, one edge device might be part of multiple groups so it can appear in multiple places in the cloud node tree.

/root/groupA/subgroupB/device1/modbus/io-1
/root/groupC/device1/modbus/io-1

So with a leaf setup if the topic paths reflected the node tree structure, would I need to know the full path of the nodes in the cloud?


I really like the simplicity of the current NATS api using UUIDs to identify nodes because the UUID for a node will never change. Descriptions and locations in the tree may change.

My initial thought for a simple implementation (that does not use NATS leaf node functionality) is that the edge instances would open a NATS client connection to the cloud instance and simply subscribe to all the nodes present on the device, and publish all messages upstream. Because a device typically has a limited number of nodes (perhaps a dozen or so), the number of subscriptions would be relatively low. Data could be cached/rate limited to conserve bandwidth if needed. Most edge devices are connected via low bandwidth Cat-M modems, so we don't have unlimited data bandwidth.

There would also need to be a mechanism for synchronizing changes in node structure -- NATS topic namespace that reflected the tree structure might make this easier. However, as the structure changes, it seems this could get complex quickly.

Mostly thinking out loud, but appreciate any ideas on this.

Thanks,
Cliff




R.I.Pienaar

unread,
Apr 28, 2021, 4:38:46 AM4/28/21
to natsio
Hello,

The way to think about leafnodes is that they just extend nats in spokes away from the central cluster. So at a high level whatever is possible in core is also possible on a leafnode - its just a lower latency extension point nearer to your nodes.

Past that there are several features that might be handy for you, the specifics of how these might apply will require some experimentation:

  • each leafnode can have isolation by accounts, if you needed isolation (which I think you dont) leafnodes can be islands incapble of seeing other islands and only have access to core things when needed
  • building on that feature we have subject remapping, so you could by configuration change subjects so that when a device accesses subject device.io-1 underneath it might access a longer subject name that you control by config.  So you could simplify the device code and reconfigure the network to move them about in your structure
  • they might also off-load TLS handling from your small clients.  A nearby leafnode - perhaps on a trusted network - can be connected to in plain but the leafnode having more resources than the devices can maintain TLS to the central network
Other than that, they just pass messages on the same rules as you already are familiar with.  You could look at JetStream for persistence and store-and-forward but it requires a lot more resources and might not be a good fit for edge devices like you describe
--
You received this message because you are subscribed to the Google Groups "nats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to natsio+un...@googlegroups.com.

--
R.I.Pienaar / www.devco.net / @ripienaar

Cliff Brake

unread,
Apr 28, 2021, 10:25:43 AM4/28/21
to nats
Thanks for the information -- very helpful!

Can you point me to any information on the subject remapping? This sounds very useful as I can see it being fairly common to need to map smaller namespaces into larger ones.

Thanks,
Cliff



derek

unread,
Apr 28, 2021, 10:30:23 AM4/28/21
to nats
Not sure the docs have been updated on this feature as of yet. However we have quite a few tests. This one shows what a server config looks like and could be helpful.

Reply all
Reply to author
Forward
0 new messages