Missing data

david_h

unread,

Apr 7, 2021, 6:45:53 AM4/7/21

to weewx-user

I had a problem due to Daylight Saving Time, which I have resolved in the sense that my station is running fine now and recording data etc. My weather website is www.ashteadweather.com. However, it has left me with 3 days of no data in my NOAA reports for 29 March to 31 March 2021 inclusive, and hence incorrect "Record" data as this period coincided with the warmest day of the year so far in my location. I have a Davis Vantage Pro2 with IP Datalogger. Everything is fine from 1 April onwards, which is when I managed to correct the DST issue. Does anyone have any ideas as to how I can look for the missing data? Will it be in my datalogger, or is it lost forever? Thanks. David.

gjr80

unread,

Apr 7, 2021, 7:07:46 AM4/7/21

to weewx-user

Hi depends on the archive interval you are using, the IP logger can store 2560 archive records. So at a five minute archive interval that is 12800 minutes = 213 hours 20 minutes = 8 days 21 hours 20 minutes. It's 7 April today so if you are using five minutes you may have lost 29 March and a bit of 30 March. If you are using > five minutes you are fine, less than five minutes, sorry, too bad they are gone.

Either way I would immediately stop WeeWX and then using wee_device I would dump the logger memory using wee_device --dump. This will dump all archive records in the logger memory to the WeeWX database, you will likely see many duplicate primary key errors in the log, they are just records that have been dumped but are already in your archive and can be ignored. Anything that is in the logger and not in your archive will be saved. Once --dump is finished rebuild your daily summaries and delete your March 2021 and April 2021 NOAA reports and they should be regenerated on the first report cycle when you restart WeeWX.

Gary

gjr80

unread,

Apr 7, 2021, 7:13:27 AM4/7/21

to weewx-user

Just a quick change to my last, there should be no need to rebuild the daily summaries after the --dump completes.

Gary

david_h

unread,

Apr 7, 2021, 7:15:29 AM4/7/21

to weewx-user

Gary - many thanks. I think I did the wee_device dump previously, but will retry and follow your other steps. Do I rebuild daily summaries using wee_database --rebuild-daily --from=YYYY-mm-dd?

Thanks

David.

david_h

unread,

Apr 7, 2021, 7:16:21 AM4/7/21

to weewx-user

OK. Ignore my question about daily summaries, given your follow-up email.

david_h

unread,

Apr 7, 2021, 7:35:11 AM4/7/21

to weewx-user

Tried this and am now getting errors in my log file. Not sure if it is related to the missing data or not. Log file extract is below. Any suggestions as to how to fix this would be much appreciated.

12:33:24 raspberrypi weewxd: restx: MQTT: network encryption/authentication will be attempted

Apr 7 12:33:24 raspberrypi weewx[8459] DEBUG weewx.engine: Finished loading service user.mqtt.MQTT

Apr 7 12:33:24 raspberrypi weewx[8459] DEBUG weewx.engine: Loading service weewx.engine.StdPrint

Apr 7 12:33:24 raspberrypi weewx[8459] DEBUG weewx.engine: Finished loading service weewx.engine.StdPrint

Apr 7 12:33:24 raspberrypi weewx[8459] DEBUG weewx.engine: Loading service weewx.engine.StdReport

Apr 7 12:33:24 raspberrypi weewx[8459] DEBUG weewx.engine: Finished loading service weewx.engine.StdReport

Apr 7 12:33:24 raspberrypi weewx[8459] INFO __main__: Starting up weewx version 4.4.0

Apr 7 12:33:24 raspberrypi weewx[8459] DEBUG weewx.manager: Daily summary version is 4.0

Apr 7 12:33:24 raspberrypi weewx[8459] DEBUG weewx.drivers.vantage: Gentle wake up of console successful

Apr 7 12:33:25 raspberrypi weewx[8459] INFO weewx.engine: Clock error is 1.83 seconds (positive is fast)

Apr 7 12:33:25 raspberrypi weewx[8459] INFO weewx.engine: Using binding 'wx_binding' to database 'weewx.sdb'

Apr 7 12:33:25 raspberrypi weewx[8459] INFO weewx.manager: Starting backfill of daily summaries

Apr 7 12:33:25 raspberrypi weewx[8459] INFO weewx.manager: Daily summaries up to date

Apr 7 12:33:25 raspberrypi weewx[8459] DEBUG weewx.drivers.vantage: Getting archive packets since 2021-04-07 12:30:00 BST (1617795000)

Apr 7 12:33:25 raspberrypi weewx[8459] DEBUG weewx.drivers.vantage: Gentle wake up of console successful

Apr 7 12:33:26 raspberrypi weewx[4516] ERROR weewx.drivers.vantage: ip-read error: timed out

Apr 7 12:33:26 raspberrypi weewx[4516] ERROR weewx.drivers.vantage: LOOP try #2; error: timed out

Apr 7 12:33:27 raspberrypi weewx[8459] DEBUG weewx.drivers.vantage: Retrieving 0 page(s); starting index= 0

Apr 7 12:33:27 raspberrypi weewx[8459] INFO weewx.engine: Starting main packet loop.

Apr 7 12:33:27 raspberrypi weewx[8459] DEBUG weewx.drivers.vantage: Gentle wake up of console successful

Apr 7 12:33:28 raspberrypi weewx[8459] DEBUG weewx.drivers.vantage: Requesting 200 LOOP packets.

Apr 7 12:33:28 raspberrypi weewx[8459] DEBUG weewx.drivers.vantage: Gentle wake up of console successful

Apr 7 12:33:30 raspberrypi weewx[4516] ERROR weewx.drivers.vantage: ip-read error: timed out

Apr 7 12:33:30 raspberrypi weewx[4516] ERROR weewx.drivers.vantage: LOOP try #3; error: timed out

Apr 7 12:33:34 raspberrypi weewx[4516] ERROR weewx.drivers.vantage: ip-read error: timed out

Apr 7 12:33:34 raspberrypi weewx[4516] ERROR weewx.drivers.vantage: LOOP try #4; error: timed out

Apr 7 12:33:34 raspberrypi weewx[4516] ERROR weewx.drivers.vantage: LOOP max tries (4) exceeded.

Apr 7 12:33:34 raspberrypi weewx[4516] INFO weewx.engine: Main loop exiting. Shutting engine down.

Apr 7 12:33:34 raspberrypi weewx[4516] DEBUG weewx.restx: Shut down MQTT thread.

Apr 7 12:33:34 raspberrypi weewx[4516] DEBUG weewx.restx: Shut down WOW thread.

Apr 7 12:33:34 raspberrypi weewx[4516] DEBUG weewx.restx: Shut down PWSWeather thread.

Apr 7 12:33:34 raspberrypi weewx[4516] DEBUG weewx.restx: Shut down Wunderground-RF thread.

Apr 7 12:33:34 raspberrypi weewx[4516] DEBUG weewx.restx: Shut down StationRegistry thread.

Apr 7 12:33:34 raspberrypi weewx[4516] CRITICAL __main__: Caught WeeWxIOError: Max tries exceeded while getting LOOP data.

Apr 7 12:33:34 raspberrypi weewx[4516] CRITICAL __main__: **** Waiting 60 seconds then retrying...

gjr80

unread,

Apr 7, 2021, 7:38:55 AM4/7/21

to weewx-user

Did you shut down WeeWX before running wee_device? It appears you have two processes running trying to access the logger/console.

Gary

david_h

unread,

Apr 7, 2021, 8:20:55 AM4/7/21

to weewx-user

When I originally ran wee_device maybe not, no.

gjr80

unread,

Apr 7, 2021, 8:25:28 AM4/7/21

to weewx-user

You need to shut down WeeWX before running wee_device.

Gary

david_h

unread,

Apr 7, 2021, 8:28:45 AM4/7/21

to weewx-user

OK. Thanks. I stopped weewx, rebooted my Pi, then restarted weewx, which seems to have removed all the ip-read error stuff. Thanks for your help.

Reply all

Reply to author

Forward