Packet timing across multiple sources

[Clay Jackson]

Hi - l have an "interesting" use case. 

I have a WeatherFlow Tempest for a station, and am using the WeatherFlowUDP driver for my "main" sensors.  I ALSO have a bunch of 433 Mhz sensors that I'm reading via MQTTSubscribe and a PurpleAir AQI sensor read by the Purple extension.

I'm having a problem that I'm already discussing with Rich Bell (author of MQTTSubscribe) where one of the MQTT packets arrives with a timestamp that is 1 second BEFORE a previously recorded timestamp and gets rejected.   The way I read this  Accumulators · weewx/weewx Wiki; what's happening is that the prior packet is, or could be, outside the timespan of the current accumulator.  Here's the error:
2026-05-14T20:21:21.223719-07:00 weather weewxd[37633]: ERROR user.MQTTSubscribe: (Service-37633) 1778815500 Ignoring packet has dateTime of 1778815277.000000 which is prior to previous packet 1778815278.000000

First, is my assumption about the cause of the error correct?  And then second, is there any way to avoid this when using multiple asynchronous data sources?

Thanks!

Clay Jackson

[bell...@gmail.com]

Some more background:

On a new loop packet event, the first thing that MQTTSubscribeService does is check if the timestamp of the current packet is greater than the timestamp of the the previous packet. If it is, MQTTSubscribeService logs an error and ignores the packet.

I’ll be the first to admit that the time processing of MQTTSubscribeService needs to be reworked. But, based on this conversation, https://groups.google.com/g/weewx-user/c/_zILpHe6ptQ/m/Gi557PJ0AAAJ; while it is ‘legal’ for a driver to submit out of order packets it is probably not a good idea and depending on the data may cause problems.

I’ve thought a bit more about this and looked a bit more at MQTTSubscribeService’s code. At this time I think the best I can do is suppress this message and ignore the packet. At some point a deep dive into the time processing and perhaps add an option to ignore timestamps in loop packets (aka process out of time order packets).

rich

[Karen K]

You could receive the MQTT packets of different sources using different WeeWX instances and write them to different databases. In your "main" WeeWX instance you can then include those databases as additional databases and that way use them in skin pages. 

[bell...@gmail.com]

Some (hopefully) clarifying information.

Current when MQTTSubscribeService is ‘given’ a loop packet to augment with MQTT data, if the timestamp in the loop packet is less than the previous one.

- MQTTSubscribeService logs it as an error.

- MQTTSubscribeService ‘ignores’ the loop packet. Does  not attempt to augment it with any MQTT data and ‘passes it back’ to WeeWX unchanged.

- WeeWX continues to process the loop packet as-is.

When MQTTSubscribeService (finally) receives a loop packet with a timestamp greater than or equal to the previous timestamp. It will augment it with any MQTT data received since the previous ‘good’ packet.

I’m looking at adding an option that would allow MQTTSubscribeService to update packets with timestamps less than the previous one. This would give the processing choice to the user of MQTTSubscribeService.

rich

[Graham Eddy]

needs to be clever applying new timestamps to allow for multiple out of sequence packets

⊣GE⊢

[Graham Eddy]

interesting post at https://github.com/peterhinch/micropython-mqtt/issues/95 dated 06 October 2022:

After spending some (a lot) more time investigating this, we finally found some answers. We did some tests with Wireshark packets of predefined length, so we could recognize them in the traffic with Wireshark. Whenever there are multiple MQTT messages in the same TLS packet/frame, the MQTT would arrive out of order on the broker side. This seems to be an error in the processing of our broker. It seems that the esp32 (and maybe TLS, I am not familiar with it) prefers packets sizes of around 1500 bytes, and this seems to cause some issues with our broker. I am uncertain if this impacts any other users, as I do not know what AWS IoT core uses under the hood.

so, maybe the sequencing issue is actually caused by how paho mqtt library demuxes single tcp packet into multiple mqtt messages when queuing..? (if multiple messages from same device but with diff timestamps can end up in same tcp packet)

⊣GE⊢

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/weewx-user/A56895AD-323A-4003-8862-DCC48251C683%40geddy.au.

[Karen K]

The general question is how to aggregate measurements from several sources into one WeeWX instance. Especially, if data does not arrive in time. WeeWX is not designed for that. It is designed for one base data source and some additional data, that arrive without their own timestamp. 

The only solution I see is to write an extension that writes into a separate database, while using WeeWX's database module for that. So you can use those measurements in skins, but you can also write different timestamps.

[bell...@gmail.com]

Interesting read. But this has nothing to do with MQTT. The driver is generating out of order loop packets. What should extensions do that modify/interact with loop packets? Currently the extension “ignores” out of order loop packets and waits for the next “correct” one. It logs this as an error. This is the behavior that I prefer. I am investigating processing/augmenting the out of order loop packets. Right now I am not fully sure of the ramifications.

rich

[bell...@gmail.com]

I can see how you arrived at the conclusion. But, we wouldn’t be having this discussion if the driver did not generate  out of order loop packets. So for my perspective, I’d like to determine how an extension should deal with this specific use case.

Some additional information, https://github.com/weewx-mqtt/subscribe/issues/10