v5.0 main loop exiting/restarting constantly and very slow/misses readings

[Richard Whitcombe]

I accidentally updated to v5.0 (Debian apt update) and since then my install has basically been broken and i cant seem to work out why.

Running on a RaPi for years with no issue, using Belchertown skin.

What im seeing now is the service seems to start fine, but after each 5 min cycle the entire thread seems to kill itself and reload/reinitialise from scratch.

In logs:

INFO weewx.cheetahgenerator: Generated 12 files for report StandardReport in 188.04 seconds
Jan 18 10:03:28 <SERVER> weewxd[420]: INFO weewx.reportengine: Copied 39 files to /var/www/html/weewx
Jan 18 10:05:24 <SERVER> weewxd[420]: INFO weewx.manager: Added record 2024-01-18 10:05:00 GMT (1705572300) to database 'weewx.sdb'
Jan 18 10:05:24 <SERVER> weewxd[420]: INFO weewx.manager: Added record 2024-01-18 10:05:00 GMT (1705572300) to daily summary in 'weewx.sdb'
Jan 18 10:05:29 <SERVER> weewxd[420]: INFO weewx.engine: Main loop exiting. Shutting engine down.
Jan 18 10:05:29 <SERVER> weewxd[420]: INFO weewx.engine: Shutting down StdReport thread
Jan 18 10:05:49 <SERVER> weewxd[420]: INFO user.interceptor: shutting down server thread
Jan 18 10:05:50 <SERVER> weewxd[420]: INFO weewx.cheetahgenerator: Generated 12 files for report Belchertown in 136.37 seconds

Jan 18 10:07:49 <SERVER> weewxd[420]: INFO __main__: retrying...
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO weewx.engine: Loading station type Interceptor (user.interceptor)
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO user.interceptor: driver version is 0.60
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO user.interceptor: device type: observer
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO user.interceptor: hardware name: weatherstation via interceptor
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO user.interceptor: mode is listen
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO user.interceptor: listen on :8090
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO user.interceptor: sensor map: {'pressure': 'pressure', 'barometer': 'barometer', 'outHumidity': 'humidity_out', 'inHumidity': 'humidity_in', 'outTemp': 'temperature_out', 'inTemp': 'temperature_in', 'windSpeed': 'wind_speed', 'windGust': 'wind_gust', 'windDir': 'wind_dir', 'windGustDir': 'wind_gust_dir', 'radiation': 'solar_radiation', 'dewpoint': 'dewpoint', 'windchill': 'windchill', 'rain': 'rain', 'rainRate': 'rain_rate', 'UV': 'uv', 'txBatteryStatus': 'battery', 'extraTemp1': 'temperature_1', 'extraTemp2': 'temperature_2', 'extraTemp3': 'temperature_3', 'extraHumid1': 'humidity_1', 'extraHumid2': 'humidity_2', 'soilTemp1': 'soil_temperature_1', 'soilTemp2': 'soil_temperature_2', 'soilMoist1': 'soil_moisture_1', 'soilMoist2': 'soil_moisture_2', 'soilMoist3': 'soil_moisture_3', 'soilMoist4': 'soil_moisture_4', 'leafWet1': 'leafwetness_1', 'leafWet2': 'leafwetness_2', 'pm2_5': 'pm2_5', 'extraTemp4': 'temperature_4', 'extraTemp5': 'temperature_5', 'extraTemp6': 'temperature_6', 'extraTemp7': 'temperature_7', 'extraTemp8': 'temperature_8', 'extraHumid3': 'humidity_3', 'extraHumid4': 'humidity_4', 'extraHumid5': 'humidity_5', 'extraHumid6': 'humidity_6', 'extraHumid7': 'humidity_7', 'extraHumid8': 'humidity_8', 'soilTemp3': 'soil_temperature_3', 'soilTemp4': 'soil_temperature_4'}
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO weewx.engine: StdConvert target unit is 0x1
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO weewx.wxservices: StdWXCalculate will use data binding wx_binding
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO weewx.engine: Archive will use data binding wx_binding
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO weewx.engine: Record generation will be attempted in 'hardware'
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO weewx.engine: Using archive interval of 300 seconds (specified in weewx configuration)
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO weewx.restx: StationRegistry: Station will be registered.
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO weewx.restx: Wunderground-PWS: Data for station XXXXXXXX will be posted
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO weewx.restx: PWSweather: No config info. Skipped.
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO weewx.restx: CWOP: Posting not enabled.
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO weewx.restx: WOW: Data for station XXXXXXXXXXXXXX will be posted
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO weewx.restx: AWEKAS: Posting not enabled.
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO user.windy: version is 0.7
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO user.windy: Data will be uploaded to https://stations.windy.com/pws/update
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO weewx.engine: 'pyephem' detected, extended almanac data is available
Jan 18 10:07:49 <SERVER> weewxd[420]: INFO __main__: Starting up weewx version 5.0.0

The end result of this is it isnt getting regular, reliable data at all, the graphs have huge gaps and rsync to the server hosting the external webpage only seem to work sporadically (every 15 mins or so)

One other thing that stands out is the huge is the time to create StandardReport - near 3minutes.

This effectively means it takes most of the 5 minute cycle after reloading itself doing this and then crashes and repeats again.

Any idea whats broken, how to fix this or how i can go back to v4 (Using APT) ?

[Tom Keffer]

You can revert to V4.10.2 by removing weewx, then reinstalling with the desired version:

sudo apt remove weewx

sudo apt install weewx=4.10.2-1

But before you do, we have been getting several reports of slow report times using the Belchertown skin, but can't reproduce it. Perhaps you could answer a couple of questions?

- Please post the [Belchertown] section of your weewx.conf

- How large is your database (measured in either records or years)?

- Any custom modifications of the Belchertown skin?

thanks

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/c6706978-ff7d-4efe-9ef2-c118de8d4814n%40googlegroups.com.

[Cameron D]

There might be two separate issues - I had the slow reports, but not associated with restarting  weewx.

What happens if you disable the reports ? does weewx runs happily or still restart?

[vince]

On Thursday, January 18, 2024 at 2:14:23 AM UTC-8 Richard Whitcombe wrote:

What im seeing now is the service seems to start fine, but after each 5 min cycle the entire thread seems to kill itself and reload/reinitialise from scratch.

INFO weewx.cheetahgenerator: Generated 12 files for report StandardReport in 188.04 seconds
Jan 18 10:03:28 <SERVER> weewxd[420]: INFO weewx.reportengine: Copied 39 files to /var/www/html/weewx
Jan 18 10:05:24 <SERVER> weewxd[420]: INFO weewx.manager: Added record 2024-01-18 10:05:00 GMT (1705572300) to database 'weewx.sdb'
Jan 18 10:05:24 <SERVER> weewxd[420]: INFO weewx.manager: Added record 2024-01-18 10:05:00 GMT (1705572300) to daily summary in 'weewx.sdb'
Jan 18 10:05:29 <SERVER> weewxd[420]: INFO weewx.engine: Main loop exiting. Shutting engine down.
Jan 18 10:05:29 <SERVER> weewxd[420]: INFO weewx.engine: Shutting down StdReport thread
Jan 18 10:05:49 <SERVER> weewxd[420]: INFO user.interceptor: shutting down server thread
Jan 18 10:05:50 <SERVER> weewxd[420]: INFO weewx.cheetahgenerator: Generated 12 files for report Belchertown in 136.37 seconds

Looks like the same thing one or two others have reported. Try temporarily setting enabled=false for Belchertown, restart weewx, and see if the problem goes away.

If you run 'htop' you will likely see two threads running and the cpu pegged for both.

What model pi are you running on ?

 

[Richard Whitcombe]

Disabling Belchertown or selecting Standard/Sofaskin and it all appears normal - report generation is 3 seconds or so as it used to be.

Re-enabling Belchertown and problem returns - 3+ minutes, pegged out performance, main loop crashing.
FWIW whatever causes this is new in 5.0 as its been fine on all previous versions.

Its running on a Pi4 / 2gb under Raspbian.

[Tom Keffer]

Please see this thread for an analysis of the problem and a solution.

https://groups.google.com/g/weewx-user/c/yfqGGuFw-L8/m/8Ba9Kd86AAAJ

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/5d2ff79c-7ae8-4291-af56-31c15f359e5bn%40googlegroups.com.

[Richard Whitcombe]

Should add yes, to confirm i see 2 threads of weewxd.py at 100% CPU at this point.