WeeWX reports not updating: Database corrupt?

[David Lewison]

Hello everyone. First off, please pardon my ignorance of Linux. I was able to follow the wonderful instructions for setting up WeeWX on a RasPi running Debian Linux, but aside from what's written in the tutorials, my debugging skills are very primitive. 

That said, I'm running WeeWX v3.7.1 on a RasPi model B using Debian pre-packaged install. Weather station is a Davis Vantage Pro with USB data logger. It has been running pretty much flawlessly for 3 years, generating reports every 10 minutes and uploading to a webpage. Occasionally, it'll stop updating, but restarting the RasPi always fixes it. The other day, we had a violent storm that knocked out power for 4 hours. When everything came back up, WeeWX wouldn't update no matter what I did. 

What I've done: 

- Checked WeeWX live connection to Vantage. Data is reporting properly every 2 seconds.

- Checked Vantage datalogger using Davis Weatherlink software. The logger has been recording data properly since the power came back on.

- Ran WeeWX reports manually. The image & text files update with a new timestamp, but no new data. The WeeWX server uptime on the HTML file also updates. Seems like the reports are generating correctly...just with no new data.

- Checked the weewx.sdb database file. The file's timestamp hasn't been updated since the power went out. This is where I think the problem is. For some reason, WeeWX is not adding new data to the database. Perhaps a corrupt database? 

If that's the issue, I'm not sure what to do about repairing the database. I'd hate to lose 3 years worth of data by doing a clean re-install. And I'm reluctant to wipe the logger's memory, since I'd lose 2 days worth of data...and it's looking like the logger is working properly anyway. 

Attached is a log file for a few archive cycles today. Thanks in advance for any help!!

[Greg Troxel]

David Lewison <dave.l...@gmail.com> writes:

> That said, I'm running WeeWX v3.7.1 on a RasPi model B using Debian
> pre-packaged install. Weather station is a Davis Vantage Pro with USB data
> logger. It has been running pretty much flawlessly for 3 years, generating
> reports every 10 minutes and uploading to a webpage. Occasionally, it'll
> stop updating, but restarting the RasPi always fixes it. The other day, we
> had a violent storm that knocked out power for 4 hours. When everything
> came back up, WeeWX wouldn't update no matter what I did.

My advice:

tread lightly until you get advice from others more clueful than me

immediately, copy the archive file to someplace else in the
filesystem, then to a different computer, and ideally to an offsite
computer. Hopefully you have been doing backups anyway.

check that time sync is ok on your machine (logs don't suggest this is wrong)

make sure you aren't running multiple copies of weewx (I see no hints
of this, but am a fan of doing the easy checks anyway)

read the following:
http://www.weewx.com/docs/utilities.htm#wee_device_utility
http://www.weewx.com/docs/hardware.htm#vantage_notes
https://github.com/weewx/weewx/wiki/Troubleshooting-the-Davis-Vantage-station

I am guessing from your logs -- but wait for someone else to confirm
-- that you are having a corruption issue and that you should be using
--dump.

re-read the hint #1 above about making backups and really do it

do an sqlite integrity check (with weewx stopped)
sqlite3 weewx.sdb 'pragma integrity_check;'

[Tom Keffer]

You have all the classic symptoms of corrupted memory in your logger. See the wiki article WeeWX generates HTML pages, but it does not update them

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/7c29793a-88ef-435a-a80c-b88f493864fbn%40googlegroups.com.

[vince]

Agree with Greg's suggestions for sure 'especially' the thought to get a copy of your db 'off' of that SD card onto another computer immediately just in case....

Always always always have a backup of your weewx db if your old data is important to you.  There are many threads in the weewx-user group archives for how to do this, but FWIW the script I cooked up back in 2013 or so (wow) might be worth taking a look at -  (link) - I just run it from cron daily and try to remember to grab it from another computer weekly or so.

[David Lewison]

Oh yes, I definitely backed up the SDB and CONF files, along with the skins directory, as suggested in the tutorials on how to back up. I FTP them to another server for safe-keeping every few weeks. I probably should have mentioned that, but didn't think it was relevant to the issue I was having. 

But on a happy note, it looks like the wee_device --dump did the trick! I guess the station's logger got corrupted somehow. I thought it was fine because I could retrieve all the data from it via the Davis software. But when I ran the dump command, it polled the logger and updated the SDB with all the missing data. I then ran the reports and all looked normal. I probably didn't need to run the clear-memory command afterwards, but the tutorial suggested doing it, so I did. No harm in clearing the data, now that it's been ingested into the database. Hope there weren't issues with duplicate records, but it looks normal from what I can tell. 

THANKS EVERYONE FOR YOUR HELP! :D