Cheetah generator does not generate SummaryByMonth and other files

[mihec]

My weewx version is 3.9.2 and have used the "old" 'neowx' skin for a couple of years. I've upgraded the skin to the latest neowx version now and have trouble with NOAA and summary html pages.

Even though the templates are listed in the skin.conf they are obviously not used to generate archive data htmls. No error is reported in the log file.

I am using this same neowx skin in two other setups (with newer weewx version) where they work normally.

Is there anything I should check or set? What could be the issue?

[Tom Keffer]

I don't know anything about the neowx skin, but if it acts like any other skin, you should be able to delete all the HTML pages and WeeWX will regenerate them at the next reporting cycle. 

If that doesn't help, we'd have to see the logs. I know you said that "no error" was reported, but let's be sure. It will also tell us what hardware you are using (which you didn't mention).

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/weewx-user/a214a7aa-7ae2-4f9b-b401-f795be24232bn%40googlegroups.com.

[Graham Eddy]

the skin was running on weewx 3.9.2, then the skin updated to latest. could be python2 issues… really should look to upgrading to weewx 5 for bug fixes and stability

⊣GE⊢

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEA5_N5%2BUS3275dVXX3zUv5%3D3hLSA6cGPs6Q4nzwHu0MhQ%40mail.gmail.com.

[vince]

Which neowx skin and version precisely? Have a download link?

Weewx 3.9.2 is almost 5.5 years old. If you can do so, upgrading to current v5 is recommended, but you need python 3 to do so…

[mihec]

The latest neowx skin is here: https://neoground.com/projects/neowx-material#section-download

I hesitate with the upgrade since everything has been working well for years now. The hardware is Davis Vantage Vue with USB data logger. The OS is buster (no GUI), I see that I have python2 and python3.7 installed in the system.

This is an excerpt from the log file, related to cheetah.

NOTE: The 'NeoWX' is the old neowx skin and I have it disabled. The new neowx skin is copied to a "Standard" skin.

Dec 20 10:40:15 vreme-Solkan-master weewx[30623]: vantage: Catch up complete.
Dec 20 10:40:15 vreme-Solkan-master weewx[30623]: reportengine: Running reports for latest time in the database.
Dec 20 10:40:15 vreme-Solkan-master weewx[30623]: vantage: Requesting 200 LOOP packets.
Dec 20 10:40:15 vreme-Solkan-master weewx[30623]: reportengine: Report 'SeasonsReport' not enabled. Skipping.
Dec 20 10:40:15 vreme-Solkan-master weewx[30623]: reportengine: Report 'NeoWX' not enabled. Skipping.
Dec 20 10:40:15 vreme-Solkan-master weewx[30623]: reportengine: Running report 'SteelSeries'
Dec 20 10:40:15 vreme-Solkan-master weewx[30623]: vantage: Gentle wake up of console successful
Dec 20 10:40:15 vreme-Solkan-master weewx[30623]: reportengine: Found configuration file /etc/weewx/skins/ss/skin.conf for report 'SteelSeries'
Dec 20 10:40:15 vreme-Solkan-master weewx[30623]: cheetahgenerator: using search list ['weewx.cheetahgenerator.Almanac', 'weewx.cheetahgenerator.Station', 'weewx.cheetahgenerator.Current', 'weewx.cheetahgenerator.Stats', 'weewx.cheetahgenerator.UnitInfo', 'weewx.cheetahgenerator.Extras']
Dec 20 10:40:15 vreme-Solkan-master weewx[30623]: manager: Daily summary version is 2.0
Dec 20 10:40:15 vreme-Solkan-master weewx[30623]: reportengine: Report 'SmartphoneReport' not enabled. Skipping.
Dec 20 10:40:15 vreme-Solkan-master weewx[30623]: reportengine: Report 'MobileReport' not enabled. Skipping.
Dec 20 10:40:15 vreme-Solkan-master weewx[30623]: reportengine: Running report 'StandardReport'
Dec 20 10:40:15 vreme-Solkan-master weewx[30623]: reportengine: Found configuration file /etc/weewx/skins/Standard/skin.conf for report 'StandardReport'
Dec 20 10:40:15 vreme-Solkan-master weewx[30623]: cheetahgenerator: using search list ['weewx.cheetahgenerator.Almanac', 'weewx.cheetahgenerator.Station', 'weewx.cheetahgenerator.Current', 'weewx.cheetahgenerator.Stats', 'weewx.cheetahgenerator.UnitInfo', 'weewx.cheetahgenerator.Extras']
Dec 20 10:40:15 vreme-Solkan-master weewx[30623]: manager: Daily summary version is 2.0
Dec 20 10:40:17 vreme-Solkan-master weewx[30623]: restx: Wunderground-PWS: Published record 2024-12-20 10:40:00 CET (1734687600)
Dec 20 10:40:47 vreme-Solkan-master weewx[30623]: reportengine: Running report 'FTP'

petek, 20. december 2024 ob 03:41:21 UTC+1 je oseba vince napisala:

[Tom Keffer]

You didn't show much of the log, but seeing as how you have a Vantage Vue, there's a strong possibility that you're suffering from corrupted logger memory. 

Try the suggestions in the section Corrupted station memory in the Wiki.

-tk

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/2e43c24b-b4c5-433a-80dd-ba7235880e95n%40googlegroups.com.

[mihec]

Thank you for your suggestion, I will clear the logger memory. However, everything else works - daily (current) data, weekly, monthly, yearly. Only the specific reports are not generated (NOAA). And those are the ones with %Y and %Y-%m in the filename. When I use the previous skin, these files are generated normally.

I will make a diff to my other stations' skin.conf files and also try to upgrade the weewx's version.

petek, 20. december 2024 ob 13:49:41 UTC+1 je oseba Tom Keffer napisala:

[Tom Keffer]

In that case, let's make it easier to debug. You do not have neowx enabled, but you said it is "copied to a Standard skin." Please unravel whatever you did and set things back to a more canonical setup. It's hard to debug when we don't know what we're looking at.

Also, it appears that you are also running the "SteelSeries" skin. Is that right? Has this also been replaced by something else?

When you've got things back to a more normal form, if things are still not working, set debug=1, restart weewx, post the log from startup through the first reporting cycle. Don't skimp on the log!

-tk

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/4ffafdb3-c85a-4500-8ce3-72f525e28bebn%40googlegroups.com.

[mihec]

I have moved the neowx skin to the "NeoWX" directory so there is no confusion with naming, etc. Yes, I do have SteelSeries enabled, too. This configuration has worked with no issue for over 6 years.

My suspicion is that weewx is not handling the %Y and %Y-%m template files properly. I've been monitoring the file generation through a terminal. It does not even attempt to run the generation over years and months but only generates the present one.

Example: it generates (namely) 'year-%Y.html', then it obviously copies or regenerates a 'year.html' file.

-rw-r--r-- 1 root root 153002 dec 20 15:20 year.html
-rw-r--r-- 1 root root 177695 dec 20 15:20 year-%Y.html

Normally, I would expect 'year-2024.html', 'year-2023.html', etc. Same for the months, e.g. 'month-2024-11.html', 'month-2024-10.html' , etc.

I have "debug=1" enabled in the weewx.conf already, the log I shared was collected using debug=1.

Log records immediately after restart.

Dec 20 15:06:34 vreme-Solkan-master systemd[1]: Starting LSB: weewx weather system...
Dec 20 15:06:34 vreme-Solkan-master weewx[26575]: engine: Initializing weewx version 3.9.2
Dec 20 15:06:34 vreme-Solkan-master weewx[26575]: engine: Using Python 2.7.16 (default, Apr  6 2019, 01:42:57) #012[GCC 8.2.0]
Dec 20 15:06:34 vreme-Solkan-master weewx[26575]: engine: Platform Linux-4.19.66-v7+-armv7l-with-debian-10.0
Dec 20 15:06:34 vreme-Solkan-master weewx[26575]: engine: Locale is 'sl_SI.UTF-8'
Dec 20 15:06:34 vreme-Solkan-master weewx[26575]: engine: pid file is /var/run/weewx.pid
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Using configuration file /etc/weewx/weewx.conf
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Debug is 1
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Initializing engine
Dec 20 15:06:34 vreme-Solkan-master weewx[26565]: Starting weewx weather system: weewx.
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Loading station type Vantage (weewx.drivers.vantage)
Dec 20 15:06:34 vreme-Solkan-master systemd[1]: Started LSB: weewx weather system.
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: vantage: Driver version is 3.1.1
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: vantage: Opened up serial port /dev/ttyUSB0; baud 19200; timeout 4.00
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: vantage: Gentle wake up of console successful
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: vantage: Hardware type is 17
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: vantage: ISS ID is 1
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: vantage: Hardware name: Vantage Vue
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Loading service weewx.engine.StdTimeSynch
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Finished loading service weewx.engine.StdTimeSynch
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Loading service user.strele.streleservice
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: strele: using /home/pi/weewx/html/strele2.txt
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Finished loading service user.strele.streleservice
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Loading service user.tempjar.tempjarservice
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: tempJar: using /home/pi/weewx/html/extraTemp1.txt
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Finished loading service user.tempjar.tempjarservice
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Loading service user.solarjar.solarjarservice
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: solarJar: using /home/pi/weewx/html/solar.txt
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Finished loading service user.solarjar.solarjarservice
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Loading service user.aqi.AQIservice
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: AQI: using /home/pi/weewx/html/aqi.txt
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Finished loading service user.aqi.AQIservice
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Loading service user.soil.Soilservice
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: Temperatura tal: using /home/pi/weewx/html/soiltemp.txt
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Finished loading service user.soil.Soilservice
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Loading service user.temp_pisarna.Pisarnaservice
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: Temperatura v pisarni: /home/pi/data/ITtemp.txt
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Finished loading service user.temp_pisarna.Pisarnaservice
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Loading service weewx.engine.StdConvert
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: StdConvert target unit is 0x1
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Finished loading service weewx.engine.StdConvert
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Loading service weewx.engine.StdCalibrate
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Finished loading service weewx.engine.StdCalibrate
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Loading service weewx.engine.StdQC
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Finished loading service weewx.engine.StdQC
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Loading service weewx.wxservices.StdWXCalculate
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: wxcalculate: The following values will be calculated: barometer=prefer_hardware, windchill=prefer_hardware, dewpoint=prefer_hardware, appTemp=prefer_hardware, rainRate=prefer_hardware, windrun=prefer_hardware, heatindex=prefer_hardware, maxSolarRad=prefer_hardware, humidex=prefer_hardware, pressure=prefer_hardware, inDewpoint=prefer_hardware, ET=prefer_hardware, altimeter=prefer_hardware, cloudbase=prefer_hardware
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: wxcalculate: The following algorithms will be used for calculations: altimeter=aaNOAA, maxSolarRad=RS
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Finished loading service weewx.wxservices.StdWXCalculate
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Loading service weewx.engine.StdArchive
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Archive will use data binding wx_binding
Dec 20 15:06:34 vreme-Solkan-master weewx[26579]: engine: Record generation will be attempted in 'hardware'
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Using archive interval of 60 seconds (specified by hardware)
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Use LOOP data in hi/low calculations: 1
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: manager: Daily summary version is 2.0
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Using binding 'wx_binding' to database 'weewx.sdb'
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: manager: Starting backfill of daily summaries
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Finished loading service weewx.engine.StdArchive
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Loading service weewx.restx.StdStationRegistry
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: restx: StationRegistry: Registration not requested.
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Finished loading service weewx.restx.StdStationRegistry
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Loading service weewx.restx.StdWunderground
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: restx: WU essentials: {}
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: restx: Wunderground-PWS: Data for station INOVAGOR14 will be posted
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Finished loading service weewx.restx.StdWunderground
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Loading service weewx.restx.StdPWSweather
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: restx: PWSweather: Posting not enabled.
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Finished loading service weewx.restx.StdPWSweather
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Loading service weewx.restx.StdCWOP
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: restx: CWOP: Posting not enabled.
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Finished loading service weewx.restx.StdCWOP
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Loading service weewx.restx.StdWOW
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: restx: WOW: Posting not enabled.
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Finished loading service weewx.restx.StdWOW
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Loading service weewx.restx.StdAWEKAS
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: restx: AWEKAS: Posting not enabled.
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Finished loading service weewx.restx.StdAWEKAS
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Loading service weewx.engine.StdPrint
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Finished loading service weewx.engine.StdPrint
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Loading service weewx.engine.StdReport
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Finished loading service weewx.engine.StdReport
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Starting up weewx version 3.9.2
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: vantage: Gentle wake up of console successful
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: manager: Daily summary version is 2.0
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: engine: Clock error is -1.26 seconds (positive is fast)
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: vantage: Getting archive packets since 2024-12-20 15:05:00 CET (1734703500)
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: vantage: Gentle wake up of console successful
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: vantage: Retrieving 1 page(s); starting index= 3
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: strele: found value of 0
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: tempJar: found value of 55.63
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: Soncno obsevanje WF: 88.0 W/m2
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: UV indeks WF       : 0.3
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: AQI PM1           : 0.84
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: AQI PM2.5         : 1.55
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: AQI PM10          : 1.55
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: AQI PM2.5 nowcast : 1.43
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: AQI PM10 nowcast  : 1.86
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: Temperatura zraka, senzor 1/2: 39.2 C / 40.55 C
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: Povprecna temperatura: 39.9 C
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: Temperatura tal -5 cm  : found value of 39.2
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: Temperatura tal -20 cm : found value of 40.55
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: Temperatura v pisarni vzhod: 71.654 C
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: Temperatura v pisarni zahod: 73.76 C
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: manager: Added record 2024-12-20 15:06:00 CET (1734703560) to database 'weewx.sdb'
Dec 20 15:06:35 vreme-Solkan-master weewx[26579]: manager: Added record 2024-12-20 15:06:00 CET (1734703560) to daily summary in 'weewx.sdb'
Dec 20 15:06:36 vreme-Solkan-master weewx[26579]: vantage: DMPAFT complete: page timestamp 2024-12-18 20:27:00 CET (1734550020) less than final timestamp 2024-12-20 15:06:00 CET (1734703560)
Dec 20 15:06:36 vreme-Solkan-master weewx[26579]: vantage: Catch up complete.
Dec 20 15:06:36 vreme-Solkan-master weewx[26579]: engine: Starting main packet loop.
Dec 20 15:06:36 vreme-Solkan-master weewx[26579]: vantage: Gentle wake up of console successful
Dec 20 15:06:36 vreme-Solkan-master weewx[26579]: vantage: Requesting 200 LOOP packets.
Dec 20 15:06:36 vreme-Solkan-master weewx[26579]: vantage: Gentle wake up of console successful
Dec 20 15:06:36 vreme-Solkan-master weewx[26579]: restx: Wunderground-PWS: Published record 2024-12-20 15:06:00 CET (1734703560)

petek, 20. december 2024 ob 14:53:12 UTC+1 je oseba Tom Keffer napisala:

[Tom Keffer]

The log was cut off just when it started getting interesting! It should show information about the reports that are run. 

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/287c6189-bfd7-48a0-b644-b942bac470cdn%40googlegroups.com.

[vince]

"My suspicion is that weewx is not handling the %Y and %Y-%m template files properly."

Tom - it's been literally years, but I remember some kind of breaking change that happened along those lines way back when and the fix was to rename filenames on the template files in pre-existing skins.

FWIW - surfing the neowx github site(s), the old original neowx skin file names in the format of  "NOAA-YYYY-MM.txt.tmpl" and the like.   The new ones uses the % syntax for year/month in the filename name.   My very hazy recollection is that at some point in time I had to rename things in old skins to use the new syntax.

My guess (guess) is that to use the new neowx skin on the old weewx he has to rename the neowx templates the other way to use the old syntax for file naming.

From the install.py file for the current neowx:

'skins/neowx-material/archive/NOAA-%Y.txt.tmpl', 'skins/neowx-material/archive/NOAA-%Y-%m.txt.tmpl'

For the new skin on old ancient weewx, I'd try renaming them to the old filename syntax ala:

'skins/neowx-material/archive/NOAA-YYYY.txt.tmpl', 'skins/neowx-material/archive/NOAA-YYYY-MM.txt.tmpl'

Worst that happens is it doesn't work :-)

[Tom Keffer]

You remember right, except that the old 'YYYY', 'MM', etc., names are still honored. 

Besides, he has not upgraded his copy of WeeWX. He's still on V3.9.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/998b9ae9-cf90-47c5-91c0-587f532d151dn%40googlegroups.com.

[vince]

Sure, but 3.9 doesn't know of the newer %Y syntax that the current (new) skin that is failing uses.

He can:

• hack the new/current version of the skin to use the old YYYY format that 3.9 knows of
• Or go back to running the old skin on the old weewx
• Or better yet get current and run v5 weewx and the new skin that matches it

But old weewx and new format template naming isn't gonna work.   Weewx is backward compatible, but it's not forward compatible.

[Tom Keffer]

Ah, yes! That would explain the symptoms. I forgot that Peter had upgraded neowx.

Peter: I think Vince is right: the modern neowx skin depends on a modern version of WeeWX.

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/8dd66ca8-0222-4a2b-a1ad-32f16f03bf63n%40googlegroups.com.

[mihec]

Thanks both for your advice. I decided to do a weewx upgrade before I read you last two posts... It has not progressed OK and now I'm stuck with some strange error due to modified schema in the weewx.sdb file. I moved to weewx 4.10. I'm trying to solve this now...

petek, 20. december 2024 ob 20:04:39 UTC+1 je oseba Tom Keffer napisala:

[mihec]

Latest status: I've upgraded to weewx 5.1 and set python3 as default. I think everything should be operational except the weewx does not have write permissions to the existing weewx.sdb. I've attached complete startup log.
I really appreciate your help since the station is now not operational... Thanks!
Dec 23 10:34:11 vreme-Solkan-master systemd[1]: Started WeeWX.
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: INFO __main__: Initializing weewxd version 5.1.0
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: INFO __main__: Command line: /usr/share/weewx/weewxd.py /etc/weewx/weewx.conf
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: INFO __main__: Using Python: 3.7.3 (default, Apr  3 2019, 05:39:12) #012[GCC 8.2.0]
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: INFO __main__: Located at:   /usr/bin/python3
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: INFO __main__: Platform:     Linux-4.19.66-v7+-armv7l-with-debian-10.0
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: INFO __main__: Locale:       'sl_SI.UTF-8'
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: INFO __main__: Entry path:   /usr/share/weewx/weewxd.py
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: INFO __main__: WEEWX_ROOT:   /etc/weewx
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: INFO __main__: Config file:  /etc/weewx/weewx.conf
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: INFO __main__: User module:  /etc/weewx/bin/user
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: INFO __main__: Debug:        1
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: INFO __main__: User:         weewx
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: INFO __main__: Group:        weewx
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: INFO __main__: Groups:       dialout weewx
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG __main__: loop_on_init: False
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG __main__: Initializing engine
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: INFO weewx.engine: Loading station type Vantage (weewx.drivers.vantage)
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.drivers.vantage: Driver version is 3.6.2
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.drivers.vantage: Option loop_request=1
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.drivers.vantage: Opened up serial port /dev/ttyUSB0; baud 19200; timeout 4.00
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.drivers.vantage: Successfully woke up Vantage console
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.drivers.vantage: Hardware type is 17
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.drivers.vantage: ISS ID is 1
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.drivers.vantage: Hardware name: Vantage Vue
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Loading service weewx.engine.StdTimeSynch
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Finished loading service weewx.engine.StdTimeSynch
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Loading service user.strele.streleservice
Dec 23 10:34:12 vreme-Solkan-master /weewxd.py: strele: using /home/pi/weewx/html/strele2.txt
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Finished loading service user.strele.streleservice
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Loading service user.tempjar.tempjarservice
Dec 23 10:34:12 vreme-Solkan-master /weewxd.py: tempJar: using /home/pi/weewx/html/extraTemp1.txt
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Finished loading service user.tempjar.tempjarservice
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Loading service user.solarjar.solarjarservice
Dec 23 10:34:12 vreme-Solkan-master /weewxd.py: solarJar: using /home/pi/weewx/html/solar.txt
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Finished loading service user.solarjar.solarjarservice
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Loading service user.aqi.AQIservice
Dec 23 10:34:12 vreme-Solkan-master /weewxd.py: AQI: using /home/pi/weewx/html/aqi.txt
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Finished loading service user.aqi.AQIservice
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Loading service user.soil.Soilservice
Dec 23 10:34:12 vreme-Solkan-master /weewxd.py: Temperatura tal: using /home/pi/weewx/html/soiltemp.txt
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Finished loading service user.soil.Soilservice
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Loading service user.temp_pisarna.Pisarnaservice
Dec 23 10:34:12 vreme-Solkan-master /weewxd.py: Temperatura v pisarni: /home/pi/data/ITtemp.txt
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Finished loading service user.temp_pisarna.Pisarnaservice
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Loading service weewx.engine.StdConvert
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: INFO weewx.engine: StdConvert target unit is 0x1
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Finished loading service weewx.engine.StdConvert
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Loading service weewx.engine.StdCalibrate
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Finished loading service weewx.engine.StdCalibrate
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Loading service weewx.engine.StdQC
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Finished loading service weewx.engine.StdQC
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.engine: Loading service weewx.wxservices.StdWXCalculate
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: INFO weewx.wxservices: StdWXCalculate will use data binding wx_binding
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.manager: Daily summary version is 2.0
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: INFO weewx.manager: Daily summaries at V2.0. Patching to V4.0
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: INFO weewx.manager: recalculate_weights: Using database 'weewx.sdb'
Dec 23 10:34:12 vreme-Solkan-master weewxd[12090]: DEBUG weewx.manager: recalculate_weights: Tranche size 100
Dec 23 10:34:15 vreme-Solkan-master weewxd[12090]: CRITICAL __main__: Database OperationalError exception: attempt to write a readonly database
Dec 23 10:34:15 vreme-Solkan-master weewxd[12090]: CRITICAL __main__:     ****  Waiting 2 minutes then retrying...

This is my database file:

-rwxrwxrwx 1 weewx weewx 682086400 dec 23 08:57 weewx.sdb

I also tried with pi:root or root:root but none of the options worked.

ponedeljek, 23. december 2024 ob 09:34:05 UTC+1 je oseba mihec napisala:

[michael.k...@gmx.at]

The user which is running weewx has to have write access both to the database file and the containing folder. Furthermore, no other process can have the database open for writing at the same time.

[mihec]

I set all directories' permissions to '777' and the error has gone. The next issue is the weewx does not log the data from the station at all. No message in the log, no WU update, no FTP update, no database write attempt. I'm completely lost.

ponedeljek, 23. december 2024 ob 12:49:09 UTC+1 je oseba michael.k...@gmx.at napisala:

[Tom Keffer]

Without a log, so are we.

See How to get a good, useful log.

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/2d2a6873-7e20-44b9-80fc-60372a12e4d3n%40googlegroups.com.

[mihec]

Sorry, this is the log content when I restart weewx after I set the permissions on directories and database file:

Dec 23 13:50:26 vreme-Solkan-master systemd[1]: Started WeeWX.
Dec 23 13:50:26 vreme-Solkan-master weewxd[32632]: DEBUG __main__:     ****      raise Terminate
Dec 23 13:50:26 vreme-Solkan-master weewxd[32632]: DEBUG __main__:     ****  Terminate
Dec 23 13:50:26 vreme-Solkan-master weewxd[1302]: INFO __main__: Initializing weewxd version 5.1.0
Dec 23 13:50:26 vreme-Solkan-master weewxd[1302]: INFO __main__: Command line: /usr/share/weewx/weewxd.py /etc/weewx/weewx.conf
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: INFO __main__: Using Python: 3.7.3 (default, Apr  3 2019, 05:39:12) #012[GCC 8.2.0]
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: INFO __main__: Located at:   /usr/bin/python3
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: INFO __main__: Platform:     Linux-4.19.66-v7+-armv7l-with-debian-10.0
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: INFO __main__: Locale:       'sl_SI.UTF-8'
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: INFO __main__: Entry path:   /usr/share/weewx/weewxd.py
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: INFO __main__: WEEWX_ROOT:   /etc/weewx
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: INFO __main__: Config file:  /etc/weewx/weewx.conf
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: INFO __main__: User module:  /etc/weewx/bin/user
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: INFO __main__: Debug:        1
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: INFO __main__: User:         weewx
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: INFO __main__: Group:        weewx
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: INFO __main__: Groups:       dialout weewx
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG __main__: loop_on_init: False
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG __main__: Initializing engine
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: INFO weewx.engine: Loading station type Vantage (weewx.drivers.vantage)
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.drivers.vantage: Driver version is 3.6.2
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.drivers.vantage: Option loop_request=1
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.drivers.vantage: Opened up serial port /dev/ttyUSB0; baud 19200; timeout 4.00
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.drivers.vantage: Successfully woke up Vantage console
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.drivers.vantage: Hardware type is 17
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.drivers.vantage: ISS ID is 1
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.drivers.vantage: Hardware name: Vantage Vue
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Loading service weewx.engine.StdTimeSynch
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Finished loading service weewx.engine.StdTimeSynch
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Loading service user.strele.streleservice
Dec 23 13:50:27 vreme-Solkan-master /weewxd.py: strele: using /home/pi/weewx/html/strele2.txt
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Finished loading service user.strele.streleservice
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Loading service user.tempjar.tempjarservice
Dec 23 13:50:27 vreme-Solkan-master /weewxd.py: tempJar: using /home/pi/weewx/html/extraTemp1.txt
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Finished loading service user.tempjar.tempjarservice
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Loading service user.solarjar.solarjarservice
Dec 23 13:50:27 vreme-Solkan-master /weewxd.py: solarJar: using /home/pi/weewx/html/solar.txt
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Finished loading service user.solarjar.solarjarservice
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Loading service user.aqi.AQIservice
Dec 23 13:50:27 vreme-Solkan-master /weewxd.py: AQI: using /home/pi/weewx/html/aqi.txt
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Finished loading service user.aqi.AQIservice
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Loading service user.soil.Soilservice
Dec 23 13:50:27 vreme-Solkan-master /weewxd.py: Temperatura tal: using /home/pi/weewx/html/soiltemp.txt
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Finished loading service user.soil.Soilservice
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Loading service user.temp_pisarna.Pisarnaservice
Dec 23 13:50:27 vreme-Solkan-master /weewxd.py: Temperatura v pisarni: /home/pi/data/ITtemp.txt
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Finished loading service user.temp_pisarna.Pisarnaservice
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Loading service weewx.engine.StdConvert
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: INFO weewx.engine: StdConvert target unit is 0x1
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Finished loading service weewx.engine.StdConvert
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Loading service weewx.engine.StdCalibrate
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Finished loading service weewx.engine.StdCalibrate
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Loading service weewx.engine.StdQC
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Finished loading service weewx.engine.StdQC
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.engine: Loading service weewx.wxservices.StdWXCalculate
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: INFO weewx.wxservices: StdWXCalculate will use data binding wx_binding
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.manager: Daily summary version is 2.0
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: INFO weewx.manager: Daily summaries at V2.0. Patching to V4.0
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: INFO weewx.manager: recalculate_weights: Using database 'weewx.sdb'
Dec 23 13:50:27 vreme-Solkan-master weewxd[1302]: DEBUG weewx.manager: recalculate_weights: Tranche size 100
Dec 23 13:51:01 vreme-Solkan-master CRON[1333]: (pi) CMD (/home/pi/scripts/readITtemp.sh)
Dec 23 13:51:01 vreme-Solkan-master CRON[1335]: (pi) CMD (/home/pi/weewx/scripts/loadStrele.sh)
Dec 23 13:51:01 vreme-Solkan-master CRON[1341]: (pi) CMD (/home/pi/weewx/scripts/loadSoilTemp.sh)
Dec 23 13:51:01 vreme-Solkan-master CRON[1345]: (pi) CMD (/home/pi/weewx/scripts/obeti.sh)
Dec 23 13:51:01 vreme-Solkan-master CRON[1351]: (pi) CMD (/usr/bin/python /home/pi/scripts/readDALSolkan.py)
Dec 23 13:51:01 vreme-Solkan-master CRON[1353]: (pi) CMD (/usr/bin/python /home/pi/scripts/parseWForiginal.py)
Dec 23 13:51:01 vreme-Solkan-master CRON[1355]: (pi) CMD (/home/pi/weewx/scripts/solar.sh)
Dec 23 13:51:01 vreme-Solkan-master CRON[1312]: (CRON) info (No MTA installed, discarding output)
Dec 23 13:51:01 vreme-Solkan-master CRON[1313]: (CRON) info (No MTA installed, discarding output)
Dec 23 13:52:01 vreme-Solkan-master CRON[1504]: (pi) CMD (/home/pi/weewx/scripts/loadStrele.sh)
Dec 23 13:52:01 vreme-Solkan-master CRON[1505]: (pi) CMD (/home/pi/weewx/scripts/solar.sh)
Dec 23 13:52:01 vreme-Solkan-master CRON[1509]: (pi) CMD (/home/pi/weewx/scripts/loadSoilTemp.sh)
Dec 23 13:52:01 vreme-Solkan-master CRON[1517]: (pi) CMD (/usr/bin/python /home/pi/scripts/parseWForiginal.py)
Dec 23 13:52:01 vreme-Solkan-master CRON[1512]: (pi) CMD (/usr/bin/python /home/pi/scripts/readDALSolkan.py)
Dec 23 13:52:01 vreme-Solkan-master CRON[1520]: (pi) CMD (/home/pi/scripts/readITtemp.sh)
Dec 23 13:52:01 vreme-Solkan-master CRON[1527]: (pi) CMD (/home/pi/weewx/scripts/obeti.sh)
Dec 23 13:52:01 vreme-Solkan-master CRON[1484]: (CRON) info (No MTA installed, discarding output)
Dec 23 13:52:01 vreme-Solkan-master CRON[1483]: (CRON) info (No MTA installed, discarding output)
Dec 23 13:53:01 vreme-Solkan-master CRON[1690]: (pi) CMD (/home/pi/scripts/readITtemp.sh)
Dec 23 13:53:01 vreme-Solkan-master CRON[1696]: (pi) CMD (/home/pi/weewx/scripts/obeti.sh)
Dec 23 13:53:01 vreme-Solkan-master CRON[1703]: (pi) CMD (/home/pi/weewx/scripts/solar.sh)
Dec 23 13:53:01 vreme-Solkan-master CRON[1702]: (pi) CMD (/home/pi/weewx/scripts/loadSoilTemp.sh)
Dec 23 13:53:01 vreme-Solkan-master CRON[1706]: (pi) CMD (/usr/bin/python /home/pi/scripts/readDALSolkan.py)
Dec 23 13:53:01 vreme-Solkan-master CRON[1713]: (pi) CMD (/home/pi/weewx/scripts/loadStrele.sh)
Dec 23 13:53:01 vreme-Solkan-master CRON[1717]: (pi) CMD (/usr/bin/python /home/pi/scripts/parseWForiginal.py)
Dec 23 13:53:01 vreme-Solkan-master CRON[1671]: (CRON) info (No MTA installed, discarding output)
Dec 23 13:53:01 vreme-Solkan-master CRON[1672]: (CRON) info (No MTA installed, discarding output)

pi@vreme-Solkan-master:~ $ ls -la weewx/database/
skupno 666208
drwxrwxrwx 3 weewx weewx      4096 dec 23 13:54 .
drwxrwxrwx 8 weewx weewx      4096 dec 14 06:31 ..
drwxrwxrwx 2 root  root       4096 sep 11  2019 backup
-rwxrwxrwx 1 weewx weewx 682086400 dec 23 13:48 weewx.sdb
-rwxrwxrwx 1 weewx weewx     90800 dec 23 13:56 weewx.sdb-journal

The /home/pi/weewx/ and /home/pi/weewx/database/ directories are both set to weewx:weewx owners.

The 'weewx.sdb-journal' appeared with weewx 5.1.0. The time of last change on the 'weewx.sdb' is the time when I re-set ownership to weewx:weewx.

Now it looks like the weewx doesn't try to read the data from the logger and run the tasks (WU, FTP).

ponedeljek, 23. december 2024 ob 13:51:30 UTC+1 je oseba Tom Keffer napisala:

[mihec]

My suspicion is now in the [Engine] section. I'm almost sure there is some issue there.

[Engine]
    # The following section specifies which services should be run and in what order.
    [[Services]]
        prep_services = weewx.engine.StdTimeSynch
#        data_services = user.strele.streleservice, user.tempjar.tempjarservice, user.solarjar.solarjarservice, user.aqi.AQIservice, user.soil.Soilservice, user.temp_pisarna.Pisarnaservice
        data services = ,
        process_services = weewx.engine.StdConvert, weewx.engine.StdCalibrate, weewx.engine.StdQC, weewx.wxservices.StdWXCalculate
        xtype_services = weewx.wxxtypes.StdWXXTypes, weewx.wxxtypes.StdPressureCooker, weewx.wxxtypes.StdRainRater, weewx.wxxtypes.StdDelta
        archive_services = weewx.engine.StdArchive
        restful_services = weewx.restx.StdStationRegistry, weewx.restx.StdWunderground, weewx.restx.StdPWSweather, weewx.restx.StdCWOP, weewx.restx.StdWOW, weewx.restx.StdAWEKAS
        report_services = weewx.engine.StdPrint, weewx.engine.StdReport

Reason: I use a few data services where my python files read various data and log them to the database file. Usually, after the weewx was run, a .pyc has been generated. Now, none of these services run as they don't provide any log to the syslog file. Even if I use 'data services = , ', it does not help.

Another observation: weewx would normally attempt to read archive records from the USB logger after restart. Now, it does nothing.

My apologies for posting so much but I need to resolve this.

ponedeljek, 23. december 2024 ob 14:00:13 UTC+1 je oseba mihec napisala:

[Graham Eddy]

your weewx.conf [Services] shows user.strele.streleservice is not to be loaded, yet your log shows that it is loaded ➔ you must be looking at wrong weewx.conf file

⊣GE⊢

On 24 Dec 2024, at 12:29 am, mihec <pete...@gmail.com> wrote:

    [[Services]]
        prep_services = weewx.engine.StdTimeSynch
#        data_services = user.strele.streleservice, user.tempjar.tempjarservice, user.solarjar.solarjarservice, user.aqi.AQIservice, user.soil.Soilservice, user.temp_pisarna.Pisarnaservice
        data services = ,
        process_services = weewx.engine.StdConvert, weewx.engine.StdCalibrate, weewx.engine.StdQC, weewx.wxservices.StdWXCalculate
        xtype_services = weewx.wxxtypes.StdWXXTypes, weewx.wxxtypes.StdPressureCooker, weewx.wxxtypes.StdRainRater, weewx.wxxtypes.StdDelta
        archive_services = weewx.engine.StdArchive
        restful_services = weewx.restx.StdStationRegistry, weewx.restx.StdWunderground, weewx.restx.StdPWSweather, weewx.restx.StdCWOP, weewx.restx.StdWOW, weewx.restx.StdAWEKAS
        report_services = weewx.engine.StdPrint, weewx.engine.StdReport

[Tom Keffer]

It looks like your database was never patched to fix a bug in the daily summaries. See the Upgrade Guide for V3.7. This patching process takes some time. Unfortunately, it was stopped prematurely.

Try patching it explicitly. This has the advantage that you can watch its progress.

weectl database upgrade

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/b12fabdb-1fe3-4dea-9279-3dc0fb5d41f0n%40googlegroups.com.

[mihec]

OK, I did a fresh OS install (bullseye 32-bit) and installed version 5.1.0. The issue with "nothing happening" remained. Then, I let weewx create a fresh database and that worked fine. So it seemed the issue might be related to the database. I added three more columns to the original weewx.sdb (when I started with weewx years ago) so I didn't / don't want to lose the data.

Thanks for another hint, Tom. My RPi 3B+ is doing database update now. It seems it will take several hours to finish (takes 1 minute for 25 days). I hope this will fix the database so I can use it from this point on.

Will weewx know how many archive records to read from the datalogger once it comes online again?

ponedeljek, 23. december 2024 ob 16:27:56 UTC+1 je oseba Tom Keffer napisala:

[mihec]

The database update failed, unfortunately...

pi@vreme-Solkan-master:~ $ sudo weectl database update
Using configuration file /etc/weewx/weewx.conf
The update process does not affect archive data, but does alter the database.
Continue (y/n)? y
Preparing interval weighting fix...

Daily summaries at V2.0. Patching to V4.0

Traceback (most recent call last):0:00 CEST (1628719200)
  File "/usr/share/weewx/weedb/sqlite.py", line 38, in guarded_fn
    return fn(*args, **kwargs)
  File "/usr/share/weewx/weedb/sqlite.py", line 233, in execute
    return sqlite3.Cursor.execute(self, *args, **kwargs)
sqlite3.OperationalError: no such column: inf

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/weewx/weectl.py", line 75, in <module>
    main()
  File "/usr/share/weewx/weectl.py", line 67, in main
    namespace.func(namespace)
  File "/usr/share/weewx/weectllib/__init__.py", line 90, in dispatch
    namespace.action_func(config_dict, namespace)
  File "/usr/share/weewx/weectllib/database_cmd.py", line 406, in update_database
    weectllib.database_actions.update_database(config_dict,
  File "/usr/share/weewx/weectllib/database_actions.py", line 559, in update_database
    with weewx.manager.open_manager_with_config(config_dict, db_binding) as dbm:
  File "/usr/share/weewx/weewx/manager.py", line 938, in open_manager_with_config
    return open_manager(manager_dict, initialize)
  File "/usr/share/weewx/weewx/manager.py", line 928, in open_manager
    return manager_cls.open(manager_dict['database_dict'],
  File "/usr/share/weewx/weewx/manager.py", line 187, in open
    dbmanager = cls(connection, table_name)
  File "/usr/share/weewx/weewx/manager.py", line 1076, in __init__
    self.patch_sums()
  File "/usr/share/weewx/weewx/manager.py", line 1531, in patch_sums
    self.recalculate_weights(start_d=datetime.date(2020, 6, 1))
  File "/usr/share/weewx/weewx/manager.py", line 1451, in recalculate_weights
    self._do_tranche(mark_d, end_of_tranche_d, weight_fn, progress_fn)
  File "/usr/share/weewx/weewx/manager.py", line 1490, in _do_tranche
    self._set_day_sums(day_accum, cursor)
  File "/usr/share/weewx/weewx/manager.py", line 1515, in _set_day_sums
    cursor.execute(update_sql, (day_accum.timespan.start,))
  File "/usr/share/weewx/weedb/sqlite.py", line 50, in guarded_fn
    raise weedb.NoColumnError(e)
weedb.NoColumnError: no such column: inf

ponedeljek, 23. december 2024 ob 20:46:36 UTC+1 je oseba mihec napisala:

[Tom Keffer]

No idea what would cause that.

The simplest way to proceed is to just drop the daily summaries, then rebuild them.

weectl database drop-daily

weectl database rebuild-daily

-tk

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/d3923af6-765e-42f0-a27f-e9fe72c28cc9n%40googlegroups.com.

[Graham Eddy]

a few years ago i struck a case in weewx where a mass database update was taking many, many hours to run in test environment - too long for live database - seemed to be a weird i/o bottleneck corner case.

my solution was to move the database to ram disk (symlink’ed to minimise weewx config changes), complete the update, then move it back - execution time a few minutes.

⊣GE⊢

[mihec]

Hi,

I then used a desktop PC to do database update (which went much faster) but it fails with same error at same timestamp. I also tried with drop-daily and rebuild-daily options but it failed at same timestamp, too.

I think there must be an invalid entry in my database (Aug 12, 2021). My SQLite database management knowledge is basically zero, so I'm looking for advice how to transfer portions of my database (e.g. from-to specific dates) to a fresh database. This way I'd avoid invalid dates (I hope?).

For the new database I'd like to add at least one new column where I keep some specific data (lightning strikes) and copy it from my original one.

Keeping in mind I started a fresh (temporary) database yesterday, I hope to add this latest data, too - if possible.

Thank you for your instant help so far, I really appreciate.

torek, 24. december 2024 ob 01:58:11 UTC+1 je oseba Graham Eddy napisala:

[Auchtermuchty Weather]

I wonder if you've managed to get a duplicate timestamp?

select datetime(dateTime, 'auto'), count(*) from archive group by 1 having count(*) > 1;

[mihec]

I did a couple of checks and no, there is no duplicate timestamp and I also checked for the +/-INF and +/-inf field values and confirmed there is none.

torek, 24. december 2024 ob 08:52:29 UTC+1 je oseba Auchtermuchty Weather napisala:

[Graham Eddy]

weewx used to include a tool for picking up things like a string where a number should be. maybe `weectl database check’ still does this

⊣GE⊢

[Auchtermuchty Weather]

That's not what it does now.

" Databases created earlier than 3.7.0 (released 11-March-2017) have a flaw that prevents them from being used with archive intervals that change with time. This utility check whether your database is affected."

$ weectl database check
Using configuration file /etc/weewx/weewx.conf
Checking daily summary tables version...
Daily summary tables are at version 4.0.
Interval Weighting Fix is not required.
$

[mihec]

Well, I've checked also the weectl database check which fails at the same point as the update.

torek, 24. december 2024 ob 12:31:36 UTC+1 je oseba Auchtermuchty Weather napisala:

[Tom Keffer]

Let's put aside the database update and just focus on rebuilding the daily summaries. 

Could you please include the stack trace when doing "weectl database rebuild-daily"? Once I see where it fails, I can send you an instrumented version of the database code so we can see exactly what is happening.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/e60483cc-3910-45f8-aff4-7b4e95939cd9n%40googlegroups.com.

[mihec]

Hi, first I want to wish you Merry Christmas and Happy Holidays!

This is the output of the "weectl database rebuild-daily". Please note that I've copied the weewx.sdb database file to a desktop PC to make it run much faster. I've replaced the real username and hostname with something generic. The weewx's version is same as in RPi.

When executed, the 'rebuild' process starts around June 2020 and goes on normally until it fails on August 12, 2021 (epoch 1628719200). In the output below, you only see the status when it fails already. I can provide with a screenshot if that helps.

The thing is that any command on the database (update, check or rebuild) fails on the same date (record?).

username@host:~$ ls -la weewx.sdb
-rwxrwxr-x 1 username username 682086400 dec 25 18:18 weewx.sdb
username@host:~$ weewxd -v
5.1.0
username@host:~$ weectl -v
weectl 5.1.0
username@host:~$ sudo weectl database rebuild-daily
[sudo] password for username:
Using configuration file /etc/weewx/weewx.conf
All daily summaries will be rebuilt.
Rebuild the daily summaries in the database 'weewx.sdb' (y/n)? y
Rebuilding daily summaries in database 'weewx.sdb' ...

Daily summaries at V2.0. Patching to V4.0

Traceback (most recent call last):0:00 CEST (1628719200)
  File "/usr/share/weewx/weedb/sqlite.py", line 38, in guarded_fn
    return fn(*args, **kwargs)
  File "/usr/share/weewx/weedb/sqlite.py", line 233, in execute
    return sqlite3.Cursor.execute(self, *args, **kwargs)
sqlite3.OperationalError: no such column: inf

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/weewx/weectl.py", line 75, in <module>
    main()
  File "/usr/share/weewx/weectl.py", line 67, in main
    namespace.func(namespace)
  File "/usr/share/weewx/weectllib/__init__.py", line 90, in dispatch
    namespace.action_func(config_dict, namespace)

  File "/usr/share/weewx/weectllib/database_cmd.py", line 329, in rebuild_daily
    weectllib.database_actions.rebuild_daily(config_dict,
  File "/usr/share/weewx/weectllib/database_actions.py", line 134, in rebuild_daily
    with weewx.manager.open_manager(manager_dict, initialize=True) as dbm:
  File "/usr/share/weewx/weewx/manager.py", line 924, in open_manager
    return manager_cls.open_with_create(manager_dict['database_dict'],
  File "/usr/share/weewx/weewx/manager.py", line 244, in open_with_create
    dbmanager = cls(connection, table_name=table_name, schema=schema)

  File "/usr/share/weewx/weewx/manager.py", line 1076, in __init__
    self.patch_sums()
  File "/usr/share/weewx/weewx/manager.py", line 1531, in patch_sums
    self.recalculate_weights(start_d=datetime.date(2020, 6, 1))
  File "/usr/share/weewx/weewx/manager.py", line 1451, in recalculate_weights
    self._do_tranche(mark_d, end_of_tranche_d, weight_fn, progress_fn)
  File "/usr/share/weewx/weewx/manager.py", line 1490, in _do_tranche
    self._set_day_sums(day_accum, cursor)
  File "/usr/share/weewx/weewx/manager.py", line 1515, in _set_day_sums
    cursor.execute(update_sql, (day_accum.timespan.start,))
  File "/usr/share/weewx/weedb/sqlite.py", line 50, in guarded_fn
    raise weedb.NoColumnError(e)
weedb.NoColumnError: no such column: inf

username@host:~$

torek, 24. december 2024 ob 15:15:16 UTC+1 je oseba Tom Keffer napisala:

[p q]

Have you tried browsing the data using https://sqlitebrowser.org/ or similar tool?

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/e92cf44c-d06f-4de1-ba04-440bceed755fn%40googlegroups.com.

[vince]

If you can give us a database dump of the archive table and put a compressed version of that online someplace we can pull it from, it might be easier for everybody.

The command should be something like:

         echo ".dump archive" | sqlite3 /path/to/your/database/mydatabase.sdb | gzip - > db.dump.gz

On my m1 mac mini it takes 33 seconds for my database that is 471,695,360 in size, so about 2/3 of your unusually large database.  It would take a long time on a raspi.

Please don't post your whole db here, it's far too big.  Put it online someplace we can download it from.

[Tom Keffer]

Thanks. That tells me what I needed to know to create a useful instrumented version of manager.py. Replace the one you have with the attached version. It will give more information about exactly what created the error. Post the log.

-tk

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/e92cf44c-d06f-4de1-ba04-440bceed755fn%40googlegroups.com.

[mihec]

I replaced the original manager.py with attached one. That file is in the /usr/share/weewx/weewx/ directory.

I will prepare the dump of the database.

username@hostname:~$ sudo weectl database rebuild-daily

  File "/usr/share/weewx/weewx/manager.py", line 1537, in patch_sums

    self.recalculate_weights(start_d=datetime.date(2020, 6, 1))
  File "/usr/share/weewx/weewx/manager.py", line 1451, in recalculate_weights
    self._do_tranche(mark_d, end_of_tranche_d, weight_fn, progress_fn)
  File "/usr/share/weewx/weewx/manager.py", line 1490, in _do_tranche
    self._set_day_sums(day_accum, cursor)

  File "/usr/share/weewx/weewx/manager.py", line 1516, in _set_day_sums

    cursor.execute(update_sql, (day_accum.timespan.start,))
  File "/usr/share/weewx/weedb/sqlite.py", line 50, in guarded_fn
    raise weedb.NoColumnError(e)
weedb.NoColumnError: no such column: inf

sreda, 25. december 2024 ob 21:13:17 UTC+1 je oseba Tom Keffer napisala:

[mihec]

This is the dump: https://we.tl/t-X3r6gbimBx

sreda, 25. december 2024 ob 22:48:59 UTC+1 je oseba mihec napisala:

[vince]

Thank you.   I built a db from it and ran rebuild-daily and it worked just fine with no errors in 2.5 minutes.   No errors running database check.  No errors running database update.

Computer was a m1 mac mini with 16 GB of RAM.   I wonder if you're running out of memory and need to split your rebuild-daily up into pieces with the --from and --to options.  Just a guess.

[Tom Keffer]

I was thinking the same.

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/124fd290-b296-4c56-a4d4-43e7b30ae2d7n%40googlegroups.com.

[mihec]

Thanks for checking. I ran the command again and monitored memfree but it was nothing to be worried about.

Here is how I ran the command (note the from-to and where the processing was when I captured this status):

username@hostname:~$ sudo weectl database rebuild-daily --from=2021-01-01 --to=2022-01-01
Using configuration file /etc/weewx/weewx.conf
Daily summaries from 2021-01-01 through 2022-01-01, inclusive, will be rebuilt.

Rebuild the daily summaries in the database 'weewx.sdb' (y/n)? y
Rebuilding daily summaries in database 'weewx.sdb' ...
Daily summaries at V2.0. Patching to V4.0

Processed through: 2020-08-28 00:00:00 CEST (1598565600)

It again fails at the same point. I'm sorry to bother you so much with this.

I just tried on a laptop where I run weewx 4.10. The 'wee_database --rebuild-daily' fails with same error at same point as on the other two systems.

For the record, the RPi is 3B+ running bullseye 32-bit, my desktop is old i7-6700 3.4GHz with 8 GB, running Ubuntu 22.04.1. Laptop is i7-7500 2.70 GHz with 8 GB running Ubuntu 22.04.1.

sreda, 25. december 2024 ob 23:12:39 UTC+1 je oseba Tom Keffer napisala:

[Tom Keffer]

I am not seeing the log entries from the instrumented code that I included. It will tell us much more about what is going on. Please include it.

For the record, I was also able to rebuild the daily summaries uneventfully using your dumped archive table.

But, it occurred to me, midnight here on the west coast of the USA (UTC-7) is a different time zone from you, which I assume is CEST. I set up a VM using CEST but, again, it rebuilt the summaries uneventfully.

Really need to see the log.

-tk

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/8fe396c2-d6b2-4aac-a8c8-fc014e9d2df8n%40googlegroups.com.

[mihec]

Tom, apologies for my misunderstanding. You were looking for the syslog I believe...

Here it is a fresh one.

Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Initializing weectl version 5.1.0
Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Command line: /usr/share/weewx/weectl.py database rebuild-daily
Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Using Python: 3.10.12 (main, Nov  6 2024, 20:22:13) [GCC 11.4.0]
Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Located at:   /usr/bin/python3
Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Platform:     Linux-6.8.0-47-generic-x86_64-with-glibc2.35
Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Locale:       'LC_CTYPE=en_US.UTF-8;LC_NUMERIC=sl_SI.UTF-8;LC_TIME=sl_SI.UTF-8;LC_COLLATE=en_US.UTF-8;LC_MONETARY=sl_SI.UTF-8;LC_MESSAGES=en_US.UTF-8;LC_PAPER=sl_SI.UTF-8;LC_NAME=sl_SI.UTF-8;LC_ADDRESS=sl_SI.UTF-8;LC_TELEPHONE=sl_SI.UTF-8;LC_MEASUREMENT=sl_SI.UTF-8;LC_IDENTIFICATION=sl_SI.UTF-8'
Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Entry path:   /usr/share/weewx/weectl.py
Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: WEEWX_ROOT:   /etc/weewx
Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Config file:  /etc/weewx/weewx.conf
Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: User module:  /etc/weewx/bin/user
Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Debug:        0
Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: User:         root
Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Group:        root
Dec 26 07:50:21 hostname weectl[309641]: INFO weectllib: Groups:       root
Dec 26 07:50:21 hostname weectl[309641]: INFO weectl-database: All daily summaries will be rebuilt.
Dec 26 07:50:23 hostname weectl[309641]: INFO weectl-database: Rebuilding daily summaries in database 'weewx.sdb' ...
Dec 26 07:50:23 hostname weectl[309641]: INFO weewx.manager: Daily summaries at V2.0. Patching to V4.0
Dec 26 07:50:23 hostname weectl[309641]: INFO weewx.manager: recalculate_weights: Using database 'weewx.sdb'
Dec 26 07:51:34 hostname weectl[309641]: ERROR weewx.manager: No column error: no such column: inf
Dec 26 07:51:34 hostname weectl[309641]: ERROR weewx.manager: ***  sql statement: 'UPDATE archive_day_soilTemp1 SET sum=inf, count=1434, wsum=inf, sumtime=86040.0 WHERE dateTime = ?;'
Dec 26 07:51:34 hostname weectl[309641]: ERROR weewx.manager: ***  start time=1628719200.0

četrtek, 26. december 2024 ob 01:35:53 UTC+1 je oseba Tom Keffer napisala:

[Tom Keffer]

That gave us the information that we needed. The problem is that starting at dateTime=1628754300 field soilTemp1 and soilTemp2 have entries 'Inf'. 

This is 3+ years ago, around 2021-08-21 0900. Any idea what happened then?

To fix (make a backup first):

sudo sqlite3 /var/lib/weewx/weewx.sdb

sqlite> update archive set soilTemp1=null where soilTemp1=9e999;

sqlite> update archive set soilTemp2=null where soilTemp2=9e999;

sqlite> .quit

The number "9e999" serves as value "Inf".

Then do your database update.

-tk

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/bb0c8c81-d7b4-401f-8758-5561660d001fn%40googlegroups.com.

[mihec]

Great! I confirm this helped! Many thanks, Tom!

Regarding corrupt (?) data: I have a Vantage Vue station but as you noticed I combine data from other sources, too. One of them is soil temperature coming from a Wemos D1. It might have been around 2021 when I started with this and maybe entered garbage data. Honestly, I don't remember.

I have updated the database file on the desktop PC now. If ain't too much risky, can I merge this database which has last record logged on December 23 with the temporary one (which I started that same day)? Is the procedure described anywhere in the weewx docs perhaps?

I am very happy for your support and patience, thank you.

četrtek, 26. december 2024 ob 20:49:52 UTC+1 je oseba Tom Keffer napisala: