Inform front end about certain events

[michael.k...@gmx.at]

I want to inform the front end a certain event occurs on the backend, to be more exact: when a report is finished. Does something like this exist?

If not, my ideas:

My front end is running a MQTT client and can be configured to listen to such events. My idea is to hook into a report and emit a MQTT message when finished. Is this possible, if yes, how? Something like this:

[StdReport]
    [[FTP]]
        skin = Ftp
        enable = true
        user = replace_me
        password = replace_me
        server = replace_me
        path = replace_me
        secure_ftp = False
        #HTML_ROOT = /var/www/html/weewx
        port = 21
        passive = 1
       
        afterReport = run_something

The other idea is to create a report and placing it after the report I want to trigger the message, assuming Reports in[StdReport] are run sequentially:

[StdReport]
    [[FTP]]
        ...
       
    [[MQTTInform]]
        skin = mqttInform
        topic = weather/reportFinished
        ...

[Tom Keffer]

Your second approach would be more "WeeWX-like". It's likely to be a very simple "report." Use the "CopyGenerator" as an example to follow.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/1eda5109-e33d-456f-8671-5ef57bea4f8bn%40googlegroups.com.

[michael.k...@gmx.at]

Thank you Tom.

I am still not so sure if this i a good idea, since it isn't self contained in any way. Having a web page generated by a report (a skin) uploaded by a second report (FTP) being informed by a third report isn't really straight forward :)

[vince]

On Thu, Aug 17, 2023 at 12:58 AM michael.k...@gmx.at <michael.k...@gmx.at> wrote:

I want to inform the front end a certain event occurs on the backend, to be more exact: when a report is finished. Does something like this exist?

When StdReport runs, it changes the date+time last modified for the output directory contents of the skin, so simply watch the timestamp on any file the skin writes out.

You will likely need to externally monitor the file with a python script or something and then do a MQTT publish to your frontend.  How to do the monitoring depends on how many skins you want to monitor and how realtime you want the notification to be.

[michael.k...@gmx.at]

I'll take a step back and describe the challenge more detailed:

I have a skin (Bootstrap) which is showing live data in charts. One design goal is to have the most accurate and most up to date values shown in the front end. The charts are updated with every loop value (received by the MQTT-JS client in the browser). One challenge is to keep the charts in sync with the data in the backend. They run out of sync quickly for various reasons. One very common reason is inactivity: the device which runs the browser is inactive, the browser tab running the web page is inactive, and so on. A mechanism in the web page checks regularly for updated backend data and asynchronously reloads backend data and refreshes the charts. Such a call for new data is done in two steps: the first step is to get a small JS file containing the timestamp of the latest backend data, and is only done when such new data can be expected (after the next archive_interval). If this timestamp is newer than the last known timestamp, the second step, fetching the new backend data and reloading the charts, is done.

The idea is, or better was, to inform the front end when new backend data was uploaded, to get the latest backend data immediately.

But thinking a little bit further, this doesn't really fix or improve the issue I have with this mechanism, which is: 

The new backend data is uploaded quite a while after the preceding archive_interval has finished. Depending on the number and complexity of reports run, and the backend machine's capabilities, this takes the one or the other minute. While in most cases this gap isn't an issue, in some cases you miss or lose events and readings up until the next archive_interval's backend data is available, for instance:

Imagine, rain is pouring down like crazy and your gauge goes up every couple of seconds. At the top of the archive_interval the live data is constantly updates by the MQTT messages coming in. After a minute or so, the backend has finished storing the new archive value, generated the reports and uploaded the fresh backend data. The front end now fetches the backend data and reloads the charts, "losing" all loop data in between. "Losing", because this data will show up again, but you will have to wait for the next archive_interval and the reports and uploads to be done. Or imagine a very strong gust happening in this gap: it will disappear in the chart and reappear after the next archive_interval.

So all in all, this is not a big deal. But from the above design goals point of view, it is an issue.

To resolve the issue, informing the front end isn't a good approach. The gap is still there. To solve my issue, I need to keep all loop data with timestamps after the top of the newest archive interval remaining in the front end, after syncing it with the newest backend data.

[Karen K]

michael.k...@gmx.at schrieb am Montag, 21. August 2023 um 14:48:30 UTC+2:

I'll take a step back and describe the challenge more detailed:

I am not sure I understand the problem.

In MQTT you can distinguish between LOOP and ARCHIVE packets by the existence of the dateTime and interval key. If you want to always see the very last reading you want to make sure to process LOOP packets in MQTT only. 

In case there are no values to process that are included in the ARCHIVE packet only, you can configure MQTT to only send LOOP packets.

 

[Karen K]

One fix: "interval" ("interval_minute" to be exact) is the only indicator for ARCHIVE records within the MQTT data. Or you compare the timestamps in dateTime.

[michael.k...@gmx.at]

Thanks for your input Karen, what I've been talking about is very skin specific. The skin produces a JS File which is uploaded (This one: https://www.kainzbauer.net/weather/Rif/de/ts.json) and the front end handles everything else in javascript. If the front end finds an updated timestamp (which is a very lightweight call) it fetches the updated data (transfers several hundred kB or more, depending on the individual setup). It's all about improving the watching-live-weather-data experience. The way it is implemented now, is not "seamless" and I want to improve that.

[vince]

Belchertown supports close to realtime measurements if you enable its websockets feature.  Why isn't that mechanism good enough ?

[michael.k...@gmx.at]

I don't know Belchertown's mechanism so I cannot answer this question. The Bootstrap skin also supports close real time measurements, if you enable it's MQTT feature, which I, the author, consider good enough, but still improvable. 

[Karen K]

michael.k...@gmx.at schrieb am Dienstag, 22. August 2023 um 05:37:25 UTC+2:

I don't know Belchertown's mechanism so I cannot answer this question. The Bootstrap skin also supports close real time measurements, if you enable it's MQTT feature, which I, the author, consider good enough, but still improvable. 

I think it is very similar. Belchertown creates a JSON file (weewx_data.json) at every archive interval, and it receives the very last readings by MQTT. 

[michael.k...@gmx.at]

I've just looked at belchertown's code, and decided the dive is to deep to find out, if it has a similar issue. I now have an approach for fixing it in Bootstrap. 

Bottom line: my "dumb question" above was answered, but it wasn't really related to the real issue. But it lead to an approach solving the issue. Thanks everyone.