outTemp - Caught unrecoverable exception in engine
128 views
Skip to first unread message
Roberto Botero
Dec 13, 2018, 6:00:18 AM12/13/18
to weewx-user
Good morning All
I have had a Vantage Vue station running since 2014 with Weewx without many problems. Over the last couple of days I Weewx has been unable to get going and the log reports a KeyError issue with outTemp (log below). I have scanned weewx.sdb for out of range values in outTemp using the instructions here: https://github.com/weewx/weewx/wiki/Cleaning-up-old-'bad'-data and replaced three outTemp= -90 values with NULL using that procedure (those records were all recent). A subsequent scan reports no out of range outTemp records, yet Weewx cannot start. I would appreciate any insight on what else the log below is telling me?
Dec 13 10:34:17 RPi-Weatherstation weewx[13471]: vantage: Retrieving 9 page(s); starting index= 0
Dec 13 10:34:17 RPi-Weatherstation weewx[13471]: manager: Daily summary version is 2.0
Dec 13 10:34:17 RPi-Weatherstation weewx[13471]: engine: Main loop exiting. Shutting engine down.
Dec 13 10:34:17 RPi-Weatherstation weewx[13471]: restx: Shut down OWM thread.
Dec 13 10:34:17 RPi-Weatherstation weewx[13471]: restx: Shut down WOW thread.
Dec 13 10:34:17 RPi-Weatherstation weewx[13471]: restx: Shut down Wunderground-PWS thread.
Dec 13 10:34:17 RPi-Weatherstation weewx[13471]: restx: Shut down StationRegistry thread.
Dec 13 10:34:18 RPi-Weatherstation weewx[13471]: engine: Caught unrecoverable exception in engine:
Dec 13 10:34:18 RPi-Weatherstation weewx[13471]: **** 'outTemp'
Dec 13 10:34:18 RPi-Weatherstation weewx[13471]: **** Traceback (most recent call last):
Dec 13 10:34:18 RPi-Weatherstation weewx[13471]: **** File "/home/weewx/bin/weewx/engine.py", line 877, in main
Dec 13 10:34:18 RPi-Weatherstation weewx[13471]: **** engine.run()
Dec 13 10:34:18 RPi-Weatherstation weewx[13471]: **** File "/home/weewx/bin/weewx/engine.py", line 160, in run
Dec 13 10:34:18 RPi-Weatherstation weewx[13471]: **** self.dispatchEvent(weewx.Event(weewx.STARTUP))
Dec 13 10:34:18 RPi-Weatherstation weewx[13471]: **** File "/home/weewx/bin/weewx/engine.py", line 224, in dispatchEvent
Dec 13 10:34:18 RPi-Weatherstation weewx[13471]: **** callback(event)
Dec 13 10:34:18 RPi-Weatherstation weewx[13471]: **** File "/home/weewx/bin/weewx/engine.py", line 509, in startup
Dec 13 10:34:18 RPi-Weatherstation weewx[13471]: **** self._catchup(self.engine.console.genStartupRecords)
Dec 13 10:34:18 RPi-Weatherstation weewx[13471]: **** File "/home/weewx/bin/weewx/engine.py", line 623, in _catchup
Dec 13 10:34:18 RPi-Weatherstation weewx[13471]: **** origin='hardware'))
Dec 13 10:34:18 RPi-Weatherstation weewx[13471]: **** File "/home/weewx/bin/weewx/engine.py", line 224, in dispatchEvent
Dec 13 10:34:18 RPi-Weatherstation weewx[13471]: **** callback(event)
Dec 13 10:34:18 RPi-Weatherstation weewx[13471]: **** File "/home/weewx/bin/user/weewxwd3.py", line 170, in new_archive_record
Dec 13 10:34:18 RPi-Weatherstation weewx[13471]: **** wd_data['humidex'] = weewx.wxformulas.humidexC(data_metricwx['outTemp'],
Dec 13 10:34:18 RPi-Weatherstation weewx[13471]: **** KeyError: 'outTemp'
Dec 13 10:34:18 RPi-Weatherstation weewx[13471]: **** Exiting.
And the three records fixed:
1544516100|1|5|30.298|||66.6|-90.0|47.0|100.0|||0.0||0.0|0.0||||0.0||||||-88.0|-82.0|-88.0|-82.0|-88.0|-82.0|||2.0|8.0|2.0|36.0|2.0|8.0|0.0||||||||||||
1544592300|1|5|30.198|||65.2|-90.0|50.0|93.0|1.0|90.0|4.0|90.0|0.0|0.0|-90.9160305931307|-90.0|-90.0|0.0||||||-86.0|-82.0|-86.0|-82.0|-82.0|-82.0|||2.0|8.0|2.0|36.0|8.0|8.0|31.6041666666667||||||||||||
1544592600|1|5|30.197|||65.9|-90.0|50.0|93.0|||0.0||0.0|0.0||||0.0||||||-86.0|-82.0|-86.0|-82.0|-82.0|-82.0|||2.0|8.0|4.0|36.0|8.0|8.0|0.0||||||||||||
Many thanks
Roberto
Roberto Botero
Dec 13, 2018, 6:03:21 AM12/13/18
to weewx-user
And for completeness the NOAA data for December this year shows the offending dates. Is there a way to just delete those records from the database and get things up and running again?
MONTHLY CLIMATOLOGICAL SUMMARY for Dec 2018
NAME: Petts Wood, Kent, England
ELEV: 79 meter LAT: 51-23.38 N LONG: 000-04.95 E
TEMPERATURE (°C), RAIN (mm), WIND SPEED (m/s)
HEAT COOL AVG
MEAN DEG DEG WIND DOM
DAY TEMP HIGH TIME LOW TIME DAYS DAYS RAIN SPEED HIGH TIME DIR
---------------------------------------------------------------------------------------
01 10.2 12.6 16:25 6.9 01:40 4.3 0.0 4.3 0.6 6.3 12:20 254
02 13.4 14.8 12:25 11.8 00:05 2.6 0.0 2.3 1.2 7.2 09:55 252
03 10.3 13.7 11:55 3.6 00:00 4.3 0.0 3.0 0.8 8.5 12:05 251
04 4.5 7.6 14:10 0.8 08:15 7.5 0.0 1.0 0.0 1.8 20:25 248
05 10.1 12.3 13:55 5.5 00:30 4.4 0.0 3.6 0.4 5.4 12:50 236
06 11.6 12.8 23:15 10.0 02:55 3.5 0.0 0.0 0.9 7.2 16:35 258
07 9.9 13.0 02:30 6.5 20:25 4.5 0.0 6.3 1.4 9.4 06:25 253
08 9.4 11.4 14:10 7.6 00:05 4.8 0.0 4.6 1.8 8.9 02:50 257
09 8.5 9.8 06:25 5.3 22:25 5.3 0.0 1.5 0.6 8.9 03:15 252
10 7.0 8.7 13:15 5.6 00:20 6.1 0.0 0.0 0.1 3.1 01:45 245
11 4.0 8.5 13:35 -67.8 08:15 7.8 0.0 0.3 0.0 2.2 22:25 105
12 4.1 6.8 02:00 -67.8 05:25 7.7 0.0 0.0 0.2 4.5 14:05 103
13 2.2 2.5 01:05 1.5 02:05 8.8 0.0 0.0 0.3 3.1 00:05 112
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
---------------------------------------------------------------------------------------
8.6 14.8 02 -67.8 11 71.6 0.0 26.9 0.7 9.4 07 253Thanks
Roberto
Andrew Milner
Dec 13, 2018, 7:49:30 AM12/13/18
to weewx-user
did you drop daily and rebuild daily after updating the archive??
if you did not do that try it (stop weewx first of course!!)
Roberto Botero
Dec 13, 2018, 8:25:19 AM12/13/18
to weewx-user
Hi Andrew
Yes, have done both --drop and --rebuild as per the procedure in the link I provided. I am going through the replies in this post of earlier https://groups.google.com/forum/#!topic/weewx-user/fZtyq_cTyZ4 where you helped a lot!
I think I have the same issue.
Roberto
Message has been deleted
Roberto Botero
Dec 13, 2018, 9:29:33 AM12/13/18
to weewx-user
Andrew
I used some of the posts in the thread I just cited and went looking for very negative temperature minima in other tables in addition to outTemp: archive_day_windchill and archive_day_dewpoint. I found records with bad minima in both and replaced them NULL.
I have dropped the daily tables, checked that they were empty and am now rebuilding them.
Let's see if this works.
Roberto
Andrew Milner
Dec 13, 2018, 9:41:46 AM12/13/18
to weewx-user
Am intrigued to know the result. After rereading the other thread you referenced I assume that this time you used sudo for drop daily whereas on the previous occasion you did not - is that correct??
Roberto Botero
Dec 13, 2018, 9:51:40 AM12/13/18
to weewx-user
Andrew
I am not the same guy that posted on the other thread. I think I have the same (original) problem as he did with corrupt minimum temperatures throwing off the weewx engine. I hope it gets resolved by the instructions you gave. On the sudo issue, I have been working as root all along.
Roberto
Roberto Botero
Dec 13, 2018, 12:24:05 PM12/13/18
to weewx-user
Well after cleaning columns outTemp, dewpoint, windchill, heatindex in the archive table of weewx.sdb, --droping the dailies and rebuilding them, I still get the same error message when I restart weewx. I am going to have to wipe out my sdcard and boot from a backup I did last week. Fortunately for me the logger in my Vantage Vue has data going back to the day before the backup. What worries me is that the same corrupted data will be in the logger. If I set more stringent stdQC parameters, will the import avoid the problems I am having?
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: vantage: Retrieving 25 page(s); starting index= 0
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: engine: Main loop exiting. Shutting engine down.
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: restx: Shut down OWM thread.
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: restx: Shut down WOW thread.
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: restx: Shut down Wunderground-PWS thread.
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: restx: Shut down StationRegistry thread.
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: engine: Caught unrecoverable exception in engine:
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: **** 'outTemp'
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: **** Traceback (most recent call last):
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: **** File "/home/weewx/bin/weewx/engine.py", line 877, in main
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: **** engine.run()
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: **** File "/home/weewx/bin/weewx/engine.py", line 160, in run
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: **** self.dispatchEvent(weewx.Event(weewx.STARTUP))
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: **** File "/home/weewx/bin/weewx/engine.py", line 224, in dispatchEvent
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: **** callback(event)
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: **** File "/home/weewx/bin/weewx/engine.py", line 509, in startup
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: **** self._catchup(self.engine.console.genStartupRecords)
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: **** File "/home/weewx/bin/weewx/engine.py", line 623, in _catchup
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: **** origin='hardware'))
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: **** File "/home/weewx/bin/weewx/engine.py", line 224, in dispatchEvent
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: **** callback(event)
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: **** File "/home/weewx/bin/user/weewxwd3.py", line 170, in new_archive_record
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: **** wd_data['humidex'] = weewx.wxformulas.humidexC(data_metricwx['outTemp'],
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: **** KeyError: 'outTemp'
Dec 13 17:19:28 RPi-Weatherstation weewx[20968]: **** Exiting.
Thomas Keffer
Dec 13, 2018, 8:23:29 PM12/13/18
to weewx-user
This appears to be a flaw in the weewx-wd extension. It does not check for missing values before attempting to calculate humidex.
You'll have to check with the author of the extension.
-tk
--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
gjr80
Dec 13, 2018, 8:54:53 PM12/13/18
to weewx-user
Indeed weeWX-WD is not checking the packet before calculating derived obs. The fix is to download and install weewxwd3.py v1.04. I don't have a packaged v1.04 extension yet so you can just update weewxwd3.py as follows:
1. move the existing weewxwd3.py aside:
$ sudo mv /home/weewx/bin/user/weewxwd3.py /home/weewx/bin/user/weewxwd3_1_03.py2. download weewxwd3.py v1.04:
$ sudo wget -P /home/weewx/bin/user https://raw.githubusercontent.com/gjr80/weewx-weewx-wd/master/bin/user/weewxwd3.py3. restart weeWX
4. monitor the log to ensure there are no further problems.
It's worth noting the issue was nothing to do with your database contents, rather weeWX was running in catchup mode to pull any unarchived records from your console and archive them. These archive records were missing weeWX field outTemp. This is somewhat unusual as Vantage stations include all obs in each archive record and therefore you would normally expect to see outTemp in every archive record. You might want to keep an eye on your station or run weeWX directly to ensure everything is working as it should.
Gary
Roberto Botero
Dec 14, 2018, 3:54:40 AM12/14/18
to weewx-user
Tom, Gary
Thank you! Gary, following your instructions weewx is downloading data from the station and publishing to WOW, OWM and WU but something else is still not correct:
Dec 14 08:52:53 RPi-Weatherstation weewx[1428]: vantage: LOOP try #3; error: Expected to read 99 chars; got 0 instead
Dec 14 08:52:53 RPi-Weatherstation weewx[1428]: vantage: Requesting 200 LOOP packets.
Dec 14 08:52:54 RPi-Weatherstation weewx[1428]: vantage: Retry #0 failed
Dec 14 08:52:54 RPi-Weatherstation weewx[1428]: vantage: Rude wake up of console successful
Dec 14 08:52:59 RPi-Weatherstation weewx[1428]: vantage: LOOP try #4; error: Expected to read 99 chars; got 0 instead
Dec 14 08:52:59 RPi-Weatherstation weewx[1428]: vantage: LOOP max tries (4) exceeded.
Dec 14 08:52:59 RPi-Weatherstation weewx[1428]: engine: Main loop exiting. Shutting engine down.
Dec 14 08:52:59 RPi-Weatherstation weewx[1428]: restx: Shut down OWM thread.
Dec 14 08:52:59 RPi-Weatherstation weewx[1428]: restx: Shut down WOW thread.
Dec 14 08:52:59 RPi-Weatherstation weewx[1428]: restx: Shut down Wunderground-PWS thread.
Dec 14 08:52:59 RPi-Weatherstation weewx[1428]: restx: Shut down StationRegistry thread.
Dec 14 08:52:59 RPi-Weatherstation weewx[1428]: engine: Caught WeeWxIOError: Max tries exceeded while getting LOOP data.
Dec 14 08:52:59 RPi-Weatherstation weewx[1428]: **** Waiting 60 seconds then retrying...
I tried running all historic reports using wee_reports and it did but produced this error in the log:
root@RPi-Weatherstation:/home/weewx# ./bin/wee_reports weewx.conf
Using configuration file weewx.conf
Generating for all time
Traceback (most recent call last):
File "/home/weewx/bin/weewx/reportengine.py", line 239, in run
obj.start()
File "/home/weewx/bin/weewx/reportengine.py", line 273, in start
self.run()
File "/home/weewx/bin/weewx/cheetahgenerator.py", line 158, in run
ngen = self.generate(gen_dict[section_name], self.gen_ts)
File "/home/weewx/bin/weewx/cheetahgenerator.py", line 232, in generate
ngen += self.generate(section[subsection], gen_ts)
File "/home/weewx/bin/weewx/cheetahgenerator.py", line 232, in generate
ngen += self.generate(section[subsection], gen_ts)
File "/home/weewx/bin/weewx/cheetahgenerator.py", line 320, in generate
default_binding)
File "/home/weewx/bin/weewx/cheetahgenerator.py", line 372, in _getSearchList
searchList += obj.get_extension_list(timespan, db_lookup)
File "/home/weewx/bin/user/wdSearchX3.py", line 1030, in get_extension_list
avwind120_vh = ValueHelper((avwind120_vt[0][0], avwind120_vt[1], avwind120_vt[2]),
IndexError: list index out of range
Roberto Botero
Dec 14, 2018, 6:33:57 AM12/14/18
to weewx-user
Well after restarting the raspberry a couple of times it seems I'm up and running. My webpage still has those wrong outTemp values in the weekly summary which I thought I had cleaned in the database but current and NOAA tables are correct.
Thank you everyone for your help!
Roberto
Reply all
Reply to author
Forward
0 new messages