Get data after power outage

112 views

Skip to first unread message

pligg...@gmail.com

unread,

Jun 26, 2021, 3:14:08 PM6/26/21

to weewx-user

Hi!

Trying to figure out how to get data to my database after a power outage. Can't find any solution to it. I had a power outage during a thunderstorm and I now want to get the rain under that hour into my database. Can this be done with any wee_ command?

Running weewx 4.0.0 on a raspberry pi 3, weather station wh1080.

/Mikael

vince

unread,

Jun 26, 2021, 3:59:27 PM6/26/21

to weewx-user

Does the WH1080 have a logger that works if the power is out ? If so, weewx should gracefully catch up.

FWIW, a Davis VP2 has a battery backed up console which is where the logger is physically, so the console has to have good batteries (or UPS) for data to be saved while the power is out. Sorry but I don't know enough about the WH1080 to help more.

pligg...@gmail.com

unread,

Jun 26, 2021, 4:22:27 PM6/26/21

to weewx-user

Vince,

The wh1080 has a battery backup and runs if it's out of power via usb. So I'll wait and see if the rain gets into the database!

Do you know how this is done? And when?

Thanks for your answer!

/Mikael

gjr80

unread,

Jun 26, 2021, 7:47:02 PM6/26/21

to weewx-user

The catch up is built in to WeeWX and automatically occurs when WeeWX starts up. There are a few ifs buts and maybes though.

First up your driver and hardware needs to support it. I assume you are using the fousb driver which does support catchup, your station also appears to support it. What WeeWX does during the catchup is ask the driver to provide any records since the timestamp of the last good record in your archive. The driver then does what it needs to obtain these records from your station. If the missing data in your database has good records after the period of missing data (eg you have no data from 9am to 10am but there is data from 10am onwards) then that period of missing data will not be filled, even if the records are in the stations logger. So generally speaking there is one chance to catchup, and that is on the first WeeWX startup after an outage. Once WeeWX starts adding records after the outage backfill of that outage will not occur. If you look at the log during startup you will see the catchup just before startup is complete. I don't believe there is any detail of the outcome in the log though.

That fairly much covers the automatic catchup. There are a few things you can try to force a backfill. Firstly, if there are records in the archive after the period of the outage then try stopping WeeWX, backup you database just in case, delete the records after the outage and then restart WeeWX. This will trigger the automatic backfill again and you may or may not have success. Check the archive afterwards to see if the data was restored.

The other thing you can try is using wee_device to somehow dump the contents of your logger. wee_device capabilities in this regard are dependent on the capabilities of your driver and station. When used with the vantage driver wee_device has the --dump command line option that dumps all logger records to database, the result is that any gaps in the archive are filled if the corresponding record is in the logger. Looking at wee_device, when used with the fousb driver the --dump option is not available but the --history option is. --history appears to dump the logger contents to console not the database, not sure why not, that is one for the fousb driver author. At least this will let you see what is in your stations logger. If the missing data is there you may be able to find a way of dumping the data to file and then importing with wee_import or the like.

Finally, remember that the 1080 uses a circular buffer, once it is full the oldest data is overwritten. It appears the 1080 has 4080 buffer locations, how long that covers in time depends on the interval. The default 30 minutes will give you about 85 days of coverage, something like 5 minutes will give you around 14 days. So I wouldn't leave it for a few months to sort out!

Gary

pligg...@gmail.com

unread,

Jun 27, 2021, 5:19:19 AM6/27/21

to weewx-user

Gary,

thank you for your answer!

I tried the first option you wrote, I stopped WeeWX, did a backup of database, deleted the records after the outage and then restarted WeeWX.

This didn't backfill my database.

I used DB browser for sqlite and just marked all those lines after the outage and deleted the content of the cells, and then saved the database.

The dateTime column didn't get erased but that maybe is ok? Or must that also be erased?

Here are the log file after I started WeeWX, there are some errors, like " ValueError: Inconsistent units (1 vs vs 1) when querying for hourRain"

Do you have any idea what could be wrong?

Jun 27 10:44:39 raspberrypi weewx[8704] INFO __main__: Initializing weewx version 4.0.0

Jun 27 10:44:39 raspberrypi weewx[8704] INFO __main__: Using Python 2.7.13 (default, Sep 26 2018, 18:42:22) #012[GCC 6.3.0 20170516]

Jun 27 10:44:39 raspberrypi weewx[8704] INFO __main__: Platform Linux-4.14.98-v7+-armv7l-with-debian-9.9

Jun 27 10:44:39 raspberrypi weewx[8704] INFO __main__: Locale is 'sv_SE.UTF-8'

Jun 27 10:44:39 raspberrypi weewx[8704] INFO __main__: PID file is /var/run/weewx.pid

Jun 27 10:44:39 raspberrypi weewx[8692]: Starting weewx weather system: weewx.

Jun 27 10:44:39 raspberrypi systemd[1]: Started LSB: weewx weather system.

Jun 27 10:44:39 raspberrypi weewx[8708] INFO __main__: Using configuration file /etc/weewx/weewx.conf

Jun 27 10:44:39 raspberrypi weewx[8708] INFO weewx.engine: Loading station type FineOffsetUSB (weewx.drivers.fousb)

Jun 27 10:44:39 raspberrypi weewx[8708] INFO weewx.drivers.fousb: driver version is 1.20

Jun 27 10:44:39 raspberrypi weewx[8708] INFO weewx.drivers.fousb: polling mode is PERIODIC

Jun 27 10:44:39 raspberrypi weewx[8708] INFO weewx.drivers.fousb: polling interval is 50

Jun 27 10:44:39 raspberrypi weewx[8708] INFO weewx.drivers.fousb: found station on USB bus= device=

Jun 27 10:44:39 raspberrypi weewxd: pond: using /home/pliggen/log.txt

Jun 27 10:44:39 raspberrypi weewxd: pond: using /home/pliggen/log2.txt

Jun 27 10:44:39 raspberrypi weewxd: pond: using /home/pliggen/dht11_humid.txt

Jun 27 10:44:39 raspberrypi weewxd: pond: using /home/pliggen/log3.txt

Jun 27 10:44:39 raspberrypi weewx[8708] INFO weewx.engine: StdConvert target unit is 0x1

Jun 27 10:44:39 raspberrypi weewx[8708] INFO weewx.wxservices: The following values will be calculated: pressure=prefer_hardware, barometer=prefer_hardware, altimeter=prefer_hardware, windchill=prefer_hardware, heatindex=prefer_hardware, dewpoint=prefer_hardware, inDewpoint=prefer_hardware, rainRate=prefer_hardware, maxSolarRad=prefer_hardware, cloudbase=prefer_hardware, humidex=prefer_hardware, appTemp=prefer_hardware, ET=prefer_hardware, windrun=prefer_hardware

Jun 27 10:44:39 raspberrypi weewx[8708] INFO weewx.wxservices: The following algorithms will be used for calculations: altimeter=aaASOS, maxSolarRad=RS

Jun 27 10:44:39 raspberrypi weewx[8708] INFO weewx.engine: Archive will use data binding wx_binding

Jun 27 10:44:39 raspberrypi weewx[8708] INFO weewx.engine: Record generation will be attempted in 'software'

Jun 27 10:44:39 raspberrypi weewx[8708] INFO weewx.engine: Using archive interval of 600 seconds (software record generation)

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: Zambretti: forecast version 3.3.0

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: Zambretti: last forecast issued 2020-05-04 22:40:00 CEST (1588624800), requested 2020-05-04 22:40:00 CEST (1588624800)

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: Zambretti: interval=600 max_age=604800 winddir_period=1800 pressure_period=10800 hemisphere=NORTH lower_pressure=950.0 upper_pressure=1050.0

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: NWS: forecast version 3.3.0

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: NWS: location ID (lid) is not specified

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: NWS: forecast office ID (foid) is not specified

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: NWS: forecast will not be run

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: WU: forecast version 3.3.0

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: WU: API key (api_key) is not specified

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: WU: forecast will not be run

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: OWM: forecast version 3.3.0

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: OWM: last forecast issued 2020-05-04 19:50:58 CEST (1588614658), requested 2020-05-04 19:50:58 CEST (1588614658)

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: OWM: interval=10800 max_age=604800 api_key=XXXXXXXXXXXXXXXXXXXXXXXXXXXX2250 location=57.961878399999996,13.750314399999999 fc=5day3hour

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: UKMO: forecast version 3.3.0

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: UKMO: interval=10800 max_age=604800 api_key=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX5b92 location=2337

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: Aeris: forecast version 3.3.0

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: Aeris: client identifier (client_id) is not specified

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: Aeris: client secret (client_secret) is not specified

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: Aeris: forecast will not be run

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: WWO: forecast version 3.3.0

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: WWO: API key (api_key) is not specified

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: WWO: forecast will not be run

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: DS: forecast version 3.3.0

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: DS: API key (api_key) is not specified

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: DS: forecast will not be run

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: XTide: forecast version 3.3.0

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: XTide: location (location) has not been specified

Jun 27 10:44:40 raspberrypi weewxd: forecast: MainThread: XTide: forecast will not be run

Jun 27 10:44:40 raspberrypi weewx[8708] INFO weewx.restx: StationRegistry: Station will be registered.

Jun 27 10:44:40 raspberrypi weewx[8708] INFO weewx.restx: Wunderground-PWS: Data for station XXXXXXXXX will be posted

Jun 27 10:44:40 raspberrypi weewx[8708] INFO weewx.restx: PWSweather: Posting not enabled.

Jun 27 10:44:40 raspberrypi weewx[8708] INFO weewx.restx: CWOP: Posting not enabled.

Jun 27 10:44:40 raspberrypi weewx[8708] INFO weewx.restx: WOW: Data for station XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX will be posted

Jun 27 10:44:40 raspberrypi weewx[8708] INFO weewx.restx: AWEKAS: Posting not enabled.

Jun 27 10:44:40 raspberrypi weewxd: restx: WeatherCloud: service version is 0.11

Jun 27 10:44:40 raspberrypi weewxd: restx: WeatherCloud: Data will be uploaded for id=XXXXXXXXXXXXXX

Jun 27 10:44:40 raspberrypi weewxd: restx: OWM: service version is 0.7

Jun 27 10:44:40 raspberrypi weewxd: restx: OWM: Data will not be posted: Missing option 'appid'

Jun 27 10:44:40 raspberrypi weewxd: windy: version is 0.6

Jun 27 10:44:40 raspberrypi weewxd: windy: Data will be uploaded to https://stations.windy.com/pws/update

Jun 27 10:44:40 raspberrypi weewx[8708] INFO user.mqtt: service version is 0.23

Jun 27 10:44:40 raspberrypi weewx[8708] INFO user.mqtt: binding to [u'archive', u'loop']

Jun 27 10:44:40 raspberrypi weewx[8708] INFO user.mqtt: topic is weather

Jun 27 10:44:40 raspberrypi weewx[8708] INFO user.mqtt: desired unit system is METRIC

Jun 27 10:44:40 raspberrypi weewx[8708] INFO user.mqtt: data will be uploaded to mqtt://192.168.1.199:1883

Jun 27 10:44:40 raspberrypi weewx[8708] INFO user.alarm_multi: Alarm set for expression 0: "extraTemp2 < 57"

Jun 27 10:44:40 raspberrypi weewx[8708] INFO user.alarm_multi: Alarm set for expression 1: "extraTemp2 > 75"

Jun 27 10:44:40 raspberrypi weewx[8708] INFO user.alarm_multi: Alarm set for expression 2: "extraTemp3 < 37"

Jun 27 10:44:40 raspberrypi weewx[8708] INFO user.alarm_multi: Alarm set for expression 3: "extraTemp3 > 75"

Jun 27 10:44:40 raspberrypi weewx[8708] INFO user.alarm_multi: Alarm set for expression 4: "extraTemp1 < 33"

Jun 27 10:44:40 raspberrypi weewx[8708] INFO user.alarm_multi: Alarm set for expression 5: "extraTemp1 > 65"

Jun 27 10:44:40 raspberrypi weewx[8708] INFO user.alarm_multi: Alarm set for expression 6: "rxCheckPercent == 0"

Jun 27 10:44:40 raspberrypi weewx[8708] INFO __main__: Starting up weewx version 4.0.0

Jun 27 10:44:40 raspberrypi weewx[8708] INFO weewx.engine: Using binding 'wx_binding' to database 'weewx.sdb'

Jun 27 10:44:40 raspberrypi weewx[8708] INFO weewx.manager: Starting backfill of daily summaries

Jun 27 10:44:41 raspberrypi weewx[8708] ERROR weewx.drivers.fousb: unrecognised magic number 1104

Jun 27 10:44:41 raspberrypi weewx[8708] INFO weewx.drivers.fousb: synchronising to the weather station (quality=0)

Jun 27 10:44:59 raspberrypi weewx[8708] INFO weewx.drivers.fousb: unstable read: blocks differ for ptr 0x000000

Jun 27 10:45:00 raspberrypi weewx[8708] INFO weewx.drivers.fousb: unstable read: blocks differ for ptr 0x00aa80

Jun 27 10:45:00 raspberrypi weewxd: pond: found value of 55.2866

Jun 27 10:45:00 raspberrypi weewxd: pond: found value of 70.925

Jun 27 10:45:00 raspberrypi weewxd: pond: found value of 99.9000

Jun 27 10:45:00 raspberrypi weewxd: pond: found value of 67.325

Jun 27 10:45:00 raspberrypi weewx[8708] WARNING weewx.qc: 2021-06-27 10:44:59 CEST (1624783499) Archive value 'extraTemp2' 159.665 outside limits (32.0, 95.0)

Jun 27 10:45:00 raspberrypi weewx[8708] WARNING weewx.qc: 2021-06-27 10:44:59 CEST (1624783499) Archive value 'extraTemp3' 153.185 outside limits (32.0, 95.0)

Jun 27 10:45:00 raspberrypi weewx[8708] WARNING weewx.qc: 2021-06-27 10:44:59 CEST (1624783499) Archive value 'extraTemp1' 131.51588 outside limits (23.0, 68.0)

Jun 27 10:45:00 raspberrypi weewx[8708] INFO weewx.manager: Added record 2021-06-27 10:44:59 CEST (1624783499) to database 'weewx.sdb'

Jun 27 10:45:00 raspberrypi weewx[8708] INFO weewx.manager: Added record 2021-06-27 10:44:59 CEST (1624783499) to daily summary in 'weewx.sdb'

Jun 27 10:45:00 raspberrypi weewxd: forecast: MainThread: Zambretti: starting thread

Jun 27 10:45:00 raspberrypi weewxd: forecast: ZambrettiThread: Zambretti: generating forecast for 2021-06-27 09:00:00 CEST (1624777200)

Jun 27 10:45:00 raspberrypi weewxd: forecast: ZambrettiThread: Zambretti: using winddir from 2021-06-27 08:30:00 CEST (1624775400) to 2021-06-27 09:00:00 CEST (1624777200)

Jun 27 10:45:00 raspberrypi weewxd: forecast: ZambrettiThread: Zambretti: using pressure from 2021-06-27 06:00:00 CEST (1624766400) to 2021-06-27 09:00:00 CEST (1624777200)

Jun 27 10:45:00 raspberrypi weewxd: forecast: MainThread: OWM: starting thread

Jun 27 10:45:00 raspberrypi weewxd: forecast: MainThread: UKMO: starting thread

Jun 27 10:45:00 raspberrypi weewxd: forecast: OWMThread: OWM: download forecast from 'http://api.openweathermap.org/data/2.5/forecast?APPID=XXXXXXXXXXXXXXXXXXXXXXXXXXXX2250&lat=57.961878399999996&lon=13.750314399999999'

Jun 27 10:45:00 raspberrypi weewxd: forecast: UKMOThread: UKMO: download forecast from 'http://datapoint.metoffice.gov.uk/public/data/val/wxfcs/all/json/2337?res=3hourly&key=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX5b92'

Jun 27 10:45:00 raspberrypi weewx[8708] ERROR weewx.restx: Windy: Unexpected exception of type <type 'exceptions.ValueError'>

Jun 27 10:45:00 raspberrypi weewx[8708] ERROR weewx.restx: WOW: Unexpected exception of type <type 'exceptions.ValueError'>

Jun 27 10:45:00 raspberrypi weewx[8708] ERROR weewx.restx: WeatherCloud: Unexpected exception of type <type 'exceptions.ValueError'>

Jun 27 10:45:00 raspberrypi weewx[8708] ERROR weewx.restx: Wunderground-PWS: Unexpected exception of type <type 'exceptions.ValueError'>

Jun 27 10:45:00 raspberrypi weewx[8708] ERROR weewx.restx: MQTT: Unexpected exception of type <type 'exceptions.ValueError'>

Jun 27 10:45:00 raspberrypi weewx[8708] ERROR weewx.restx: *** Traceback (most recent call last):

Jun 27 10:45:00 raspberrypi weewx[8708] INFO user.alarm_multi: Alarm expression "extraTemp2 < 57" evaluated True at 2021-06-27 10:44:59 CEST (1624783499)

Jun 27 10:45:00 raspberrypi weewx[8708] ERROR weewx.restx: *** File "/usr/share/weewx/weewx/restx.py", line 378, in run_loop

Jun 27 10:45:00 raspberrypi weewx[8708] ERROR weewx.restx: *** self.process_record(_record, dbmanager)

Jun 27 10:45:00 raspberrypi weewx[8708] ERROR weewx.restx: *** File "/usr/share/weewx/weewx/restx.py", line 422, in process_record

Jun 27 10:45:00 raspberrypi weewx[8708] ERROR weewx.restx: *** _full_record = self.get_record(record, dbmanager)

Jun 27 10:45:00 raspberrypi weewx[8708] ERROR weewx.restx: *** File "/usr/share/weewx/weewx/restx.py", line 300, in get_record

Jun 27 10:45:00 raspberrypi weewx[8708] ERROR weewx.restx: *** % (_result[1], _result[2], record['usUnits']))

Jun 27 10:45:00 raspberrypi weewx[8708] ERROR weewx.restx: *** ValueError: Inconsistent units (1 vs vs 1) when querying for hourRain

Jun 27 10:45:00 raspberrypi weewx[8708] CRITICAL weewx.restx: Windy: Thread terminating. Reason: Inconsistent units (1 vs vs 1) when querying for hourRain