Weewx 4.1.1 silently stops updating DB and Graphs

[Russell Cutcliffe]

Hi,

I have a long standing issue with Weewx 4.x, where the process will spontaneously stop adding database records and updating the graphs.

The attached syslog snippet should contain enough detail to show that everything looks okay, except that the report generation just seems to stop.

For completeness, I should point out that the process will generally stop just after midnight local time, or occasionally around 3am/pm.

The system is now running on a dedicated Ubuntu 20.04 VM, with its own Mariadb instance.  The install was via the Ubuntu repo (now 4.1.1), and is running with Python 3.8.2, also from the repo.

Data comes in via an interceptor, listening on port 80 and being fed from a custom WU protocol feed from my WH2905.

 I've been looking for some external influence, but I'm at my wit's end.  Certainly syslog never shows anything.  I've turned off unattended-updates after I caught it restarting my VM, but the problem remains.

I've gotten to the point where I have Node Red watching the database and restarting Weewx when updates stop, so I'm not completely desperate, but the gaps in the graphs are a bit annoying.

From my reading, it appears that I'm alone with this particular issue, so any fault finding hints would be appreciated.

Russell C

[gjr80]

Hi,

Here is what is causing the strange record generation/reporting behaviour with WeeWX. The 15:02:54 packet has a timestamp of 1592197317 (derived from dateutc=2020-06-15%2005:01:57 - 2020-06-15 05:01:57):

Jun 15 15:02:54 pete weewx[11155] DEBUG user.interceptor: GET: ID=number10&PASSWORD=XXXX&indoortempf=73.6&tempf=74.3&dewptf=62.6&windchillf=74.3&indoorhumidity=63&humidity=67&windspeedmph=0.0&windgustmph=0.0&winddir=137&absbaromin=30.080&baromin=30.024&rainin=0.000&dailyrainin=0.008&weeklyrainin=1.268&monthlyrainin=1.480&solarradiation=283.69&UV=3&dateutc=2020-06-15%2005:01:57&softwaretype=EasyWeatherV1.4.9&action=updateraw&realtime=1&rtfreq=5

That packet is properly decoded and a loop packet created (the mapped packet at 15:02:54):

Jun 15 15:02:54 pete weewx[11155] DEBUG user.interceptor: mapped packet: {'dateTime': 1592197317, 'usUnits': 1, 'pressure': 30.08, 'barometer': 30.024, 'outHumidity': 67.0, 'inHumidity': 63.0, 'outTemp': 74.3, 'inTemp': 73.6, 'windSpeed': 0.0, 'windGust': 0.0, 'windDir': 137.0, 'radiation': 283.69, 'dewpoint': 62.6, 'windchill': 74.3, 'rain': 0.0, 'UV': 3.0}

If you look at the next packet at 15:03:58 you will see the packet has a timestamp of 1592229600 (derived from dateutc=2020-06-15%2014:00:00 - 2020-06-15 14:00:00), some nine hours later:

Jun 15 15:03:58 pete weewx[11155] DEBUG user.interceptor: GET: ID=number10&PASSWORD=XXXX&indoortempf=73.6&tempf=74.5&dewptf=62.1&windchillf=74.5&indoorhumidity=63&humidity=65&windspeedmph=1.3&windgustmph=2.2&winddir=211&absbaromin=30.089&baromin=30.033&rainin=0.000&dailyrainin=0.008&weeklyrainin=1.268&monthlyrainin=1.480&solarradiation=284.06&UV=3&dateutc=2020-06-15%2014:00:00&softwaretype=EasyWeatherV1.4.9&action=updateraw&realtime=1&rtfreq=5

Such a jump in timestamps has the effect of causing WeeWX to synthesise an archive record at 15:03:58. The record is saved to database and the report cycle initiated. Subsequent packets jump back to the correct time but because WeeWX has seen the much later packet WeeWX effectively ignores the subsequent data (in truth it is probably being accumulated but WeeWX will not synthesise another archive record until it sees a loop packet with a timestamp of at least 1592229900 (which will be five minutes after midnight on 16 June local time)). The report cycle does not occur as the report cycle is kicked off when an archive record is produced and no archive record is being produced.

Chances are if you left well enough alone an archive record will be synthesised at five minutes after midnight on 16 June (provided no more future dated packets are recieved) and the recport cycle initiated but that is not really an acceptable situation.

So that is the mechanism that causes the odd behaviour but what is the fundamental cause? The WeeWX interceptor driver dumps the raw GET data it picks up from your station to log, the interceptor does not adjust/decode times when displaying the raw GET data so we have to assume your station emitted the packet with the future dated timestamp. I have no idea why, your config file seems reasonable. I have not used/seen the EasyWeather software so I don't know if there are any configuration changes there that might help. I do get toey when i see things like rtfreq=5 (at the end of each GET data/raw packet) as that indicates to me WeatherUnderground rapidfire (I have a natural suspicion of WU and/or rapidfire) is being used and this implies an update every 5 seconds, but we are only seeing packets being emitted every 60 odd seconds (granted there are some empty queue log entries but not every five seconds). Perhaps that is something worth looking at - can the update frequency be set within EasyWeather?

You mention a 'custom WU protocol feed' - exactly what do you mean by that?

Gary

[Russell Cutcliffe]

Interesting...

I'll check further into that. 

As to the custom feed, check the attached screenshot from my phone screen.

This is a newish feature that rolled out to these weather stations, and seems to work (with maybe some time issues ;^).

Russell C

[gjr80]

Might be worth giving the ecowitt format a go. You would need to set device_type = ecowitt-client under [Interceptor] in weewx.conf.

Gary

[NanoG5Kite]

As it works for me,

[Interceptor]
    # This section is for the network traffic interceptor driver.
   
    # The driver to use:
    driver = user.interceptor
   
    # Specify the hardware device to capture.  Options include:
    #   acurite-bridge - acurite internet bridge, smarthub, or access
    #   observer - fine offset WH2600/HP1000/HP1003, ambient WS2902
    #   lw30x - oregon scientific LW301/LW302
    #   lacrosse-bridge - lacrosse GW1000U/C84612 internet bridge
    #   ecowitt-client - any hardware that uses the ecowitt protocol
    #   wu-client - any hardware that uses the weather underground protocol
    # device_type = fineoffset-bridge
    device_type = ecowitt-client
    port = 8000 (take your port)

And possible the patched interceptor.py - enclosed...

Pattched according to the one from Oliver:

https://github.com/matthewwall/weewx-interceptor/issues/69

[NanoG5Kite]

by the way, the interceptor.py I enclosed includes also this patch:

https://github.com/matthewwall/weewx-interceptor/pull/64

Regards,

Matthias