Issue with MQTT extension

[Olivier Payrastre]

Hi, I've found this group, and hope someone can help me.

I've belchertown skin installed on my weewx software, and i enable MQTT extension from here https://github.com/weewx/weewx/wiki/mqtt.

I precise I use 3.9.1 and wmr200 driver.

But sometime, i see that that the seconds diplayed was coming back to 00 and comes good after. This produce each minute.

You can get a look here

I see to the log and i see that this was appening when data were archived to the database.

Here's a bit of my log file : green is ok, red is abnormal, second shouldnt be 00...


Jun  9 20:52:43 srvmeteo weewx[21474]: ftpgenerator: ftp'd 20 files in 12.90 seconds
Jun  9 20:52:48 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:52:47 CEST (1560106367)
Jun  9 20:52:49 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:52:48 CEST (1560106368)
Jun  9 20:52:51 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:52:50 CEST (1560106370)
Jun  9 20:52:52 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:52:51 CEST (1560106371)
Jun  9 20:52:54 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:52:53 CEST (1560106373)
Jun  9 20:52:56 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:52:53 CEST (1560106373)
Jun  9 20:52:57 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:52:57 CEST (1560106377)
Jun  9 20:52:58 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:52:57 CEST (1560106377)
Jun  9 20:53:07 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:53:07 CEST (1560106387)
Jun  9 20:53:08 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:53:07 CEST (1560106387)
Jun  9 20:53:15 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:53:15 CEST (1560106395)
Jun  9 20:53:15 srvmeteo weewx[21474]: manager: Added record 2019-06-09 20:53:00 CEST (1560106380) to database 'weewx.sdb'
Jun  9 20:53:15 srvmeteo weewx[21474]: manager: Added record 2019-06-09 20:53:00 CEST (1560106380) to daily summary in 'weewx.sdb'
Jun  9 20:53:15 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:53:00 CEST (1560106380)
Jun  9 20:53:16 srvmeteo weewx[21474]: history.pyc: Generated 6 tables in 0.41 seconds
Jun  9 20:53:17 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:53:16 CEST (1560106396)
Jun  9 20:53:20 srvmeteo weewx[21474]: cheetahgenerator: Generated 12 files for report Belchertown in 4.80 seconds
Jun  9 20:53:20 srvmeteo weewx[21474]: copygenerator: copied 2 files to /var/www/html/weewx
Jun  9 20:53:22 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:53:21 CEST (1560106401)
Jun  9 20:53:27 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:53:26 CEST (1560106406)
Jun  9 20:53:37 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:53:37 CEST (1560106417)
Jun  9 20:53:41 srvmeteo weewx[21474]: ftpgenerator: ftp'd 20 files in 13.04 seconds
Jun  9 20:53:44 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:53:44 CEST (1560106424)
Jun  9 20:53:45 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:53:45 CEST (1560106425)
Jun  9 20:53:46 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:53:47 CEST (1560106427)
Jun  9 20:53:47 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:53:47 CEST (1560106427)
Jun  9 20:53:48 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:53:49 CEST (1560106429)
Jun  9 20:53:49 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:53:49 CEST (1560106429)
Jun  9 20:53:50 srvmeteo weewx[21474]: restx: MQTT: Published record 2019-06-09 20:53:50 CEST (1560106430)

Any idea why ?

Thanks for all,

Olivier from South France.

[Andrew Milner]

I would imagine it is just that the green mqtt records are LOOP packets and the red mqtt record is a REC packet

LOOP records will continue to be being processed during the processing of the REC packet.  Processing of the REC takes a while as database tabes are updated and reports generated - but the time or the REC record will be the time at which processing of the REC commenced.  The result can be slightly out of sequence in that the REC processing for MQTT I suspect takes place after some other REC processing, so the time will appear to go backwards (as indeed it will have done) since the time of the REC packet will precede other LOOP packets processed dusing the processing of the REC.

Well that is my guess as to what may happening anyway!!  Personally I prefer to ignore LOOP packets and just deal with REC packets - which are what end up in the database anyway since LOOP data is not archived by weewx - only REC.

[Pat]

This is it exactly. The REC packet (or archive packet) is what you're seeing with that timestamp. This is nothing that's broken. My site is doing the same thing. 

For the Belchertown skin, you would want both LOOP and REC packets for all the info to be updated. 

[Olivier Payrastre]

Thanks for the explanations.

The trouble in belchertown skin is that the rec packet is displayed beetween two loops packets and its not chronological. In this example Belchertown Skin displayed :

1st :          Updated on  2019-06-09 20:53:15

2nd :         Updated on 2019-06-09 20:53:00

and

3rd :          Updated on 2019-06-09 20:53:16

Pat, you can have a look here, https://meteo.payrastre.fr It happen each minute. I see your site for some minutes I dont see this.

I Don't know if possible, but may we not display this packets in belchertown skin ? 

[Andrew Milner]

maybe you could try a 5 minute archive interval.  Do you use hardware record generation or software - that may be something different to Pat's setup also, as indeed the station type may be having an effect too!!  So many variables can get involved!!

[Pat]

I understand. This happens on my site as well. My archive interval is set to 5 minutes because I don't want my database to become so large so fast. So if you look at my site on the 5th minute interval (:00, :05, :10, etc) you will see it happen about 15 seconds after that minute. 

The REC packet is needed to update a lot of items in the skin. Specifically the charts, all weewx data, forecasts, earthquake and images. If you disable it, then the only thing that would ever get updated is the temperature and the box next to the radar. 

At that point you may as well disable the MQTT option :)

[Pat]

I guess what I could do is move a couple things around so that if it's a REC packet, then to not update the timestamp in the header, but still use that data for updating the site. 

This could be good for those who have LOOP packets at longer intervals where this REC packet's timestamp would be more noticeable. I'll add that to the 1.1 beta that I'm working on. 

[Olivier Payrastre]

Thanks a lot Pat. Now the site is working very well with the last releases. Hope this will help other users.