Hi,
I'm working on an IoT project where I plan to run multiple instances in different locations (cloud, edge, etc) and connect them via NATS. Each instance has its own database (currently Genji, which is embedded -- similar to sqlite). Most of the data is structured as a tree of nodes and each node contains a number of points that encapsulate configuration and status information. The structure of the tree is used to manage groups, permissions, what data gets processed by rules, etc.
The current NATS API:
The data flow is structured so that all write data goes through NATS and then hits the database. This is working well and makes it easy to add functionality.
A single instance is working pretty well, but now need to implement synchronization between device/cloud instances. My current thought is that node tree in an edge device would be fully synchronized to an upstream cloud instance and become part of a larger tree in the cloud as shown in the below diagram:
Pondering how to use NATS to synchronize the nodes between the cloud and edge devices ... I've been looking into leaf nodes, but it is not obvious to me yet how to set up the topic namespace for this. Currently the NATS topic namespace (see API link above) does not reflect the node tree structure -- maybe it should. One puzzle is the edge nodes would only have part of the cloud tree. So the tree location for a node in the cloud might be:
/root/groupA/subgroupB/device1/modbus/io-1
On the edge device, it would be:
/device1/modbus/io-1
Nodes can be mirrored to multiple places in the cloud tree -- for instance, one edge device might be part of multiple groups so it can appear in multiple places in the cloud node tree.
/root/groupA/subgroupB/device1/modbus/io-1
/root/groupC/device1/modbus/io-1
So with a leaf setup if the topic paths reflected the node tree structure, would I need to know the full path of the nodes in the cloud?
I really like the simplicity of the current NATS api using UUIDs to identify nodes because the UUID for a node will never change. Descriptions and locations in the tree may change.
My initial thought for a simple implementation (that does not use NATS leaf node functionality) is that the edge instances would open a NATS client connection to the cloud instance and simply subscribe to all the nodes present on the device, and publish all messages upstream. Because a device typically has a limited number of nodes (perhaps a dozen or so), the number of subscriptions would be relatively low. Data could be cached/rate limited to conserve bandwidth if needed. Most edge devices are connected via low bandwidth Cat-M modems, so we don't have unlimited data bandwidth.
There would also need to be a mechanism for synchronizing changes in node structure -- NATS topic namespace that reflected the tree structure might make this easier. However, as the structure changes, it seems this could get complex quickly.
Mostly thinking out loud, but appreciate any ideas on this.
Thanks,
Cliff