Improving mote resilience and offline operation by adding fields to Model_DeviceAlert

[Andre Maree]

In order to improved the reliability and availability of our motes we are trying to enable the buffering of significant numbers of device measurements and device alerts in times when communications is disrupted. Our ability to buffer this information for uploading once communications has been reestablished is directly proportional to the amount of free SRAM available in the mote.

Since all communications happen using GPB as the encoding protocol, and since GPB is generally very efficient in the encoding process we would like to create the actual packets to be sent, fully GPB encoded, and queue these for transmission once the comms has been restored. However, the previous statement is only true for normal defined fields and NOT for metadata K:V pairs.

For example, sending a rule version (uint32_t) as metadata take at least 10-14 bytes (with a very short name) as a defined field max 5. Looking at our worst case, if I can add the 7 metadata fields as defined fields we can reduce our packet size from max 591 bytes to around 200 bytes resulting in a 200% improvement in the number of alerts that can be buffered. And since alerts, together with measurements, form the heart of any/most/all IoT projects the benefit will be significant.

All assistance and effort appreciated, considering the limited SRAM resources on most MCUs this could become a make or break case for some projects..

Thanks

Andre

[Derek Adams]

Hi Andre,

The default SiteWhere protobuf protocol is designed as a generic means of moving data into the system, but it's definitely not as efficient as using your own binary format. SiteWhere supports the use of custom binary decoders, so you could create a your own protobuf representation, then use a Groovy script decoder on the event source to parse the inbound payload into SiteWhere events. There's not much in the documentation about this so far (working on it for the 2.0 release), but I will provide an example ASAP.

Thanks,

Derek

[Andre Maree]

Hi Derek,  thanks for the response, but I just need to clarify my bad explanation.

We are not using, nor do we want to use, a custom/proprietary binary format, we absolutely want to stay with standard GPB definition. The 200% savings comes in with the change from encoding fields as metadata vs normal defined binary GPB fields.

 I think the ideal situation would be the ability to change/extend the default template for the Device Alert & Registration message to include specific defined fields as required by the application. The default fields required by SW will remain exactly as they are for SW to use, and the additional fields would be left to the application to do with what it wants.

The inefficiency that we are trying to remove lies in how GPB has to treat normal defined fields (compacted, binary, default values etc) vs metadata text strings describing the key and value. And hopefully this is something that could be achieved without a move to EA v2.0.

Thanks

Andre