NOAA Reports Gone Wild

[Chris Martino]

Howdy!

First time poster, so please be gentle if this has been asked before. My search results turned up nada, so here I am!

Sometime within the past few weeks my NOAA Reports have gone back in time to December 31, 1969, as seen on my weewx site: https://bridgewaterweather.com/reports

The Dec. 31, 1969 date contains 1 entry with some bogus data for "AVG WIND SPEED", "HIGH", "TIME" and "DOM DIR".

Is it possible to clear this data so my reports don't go back 51 years?

Thanks!

Chris

[Tom Keffer]

Most likely, your database includes records with timestamps of zero, or near zero. In turn, this is usually caused by not having a real-time clock: the computer dumbly sets the clock to zero unix epoch time, which is 1 January 1970, on startup. 

You need to fix two things:

1. Make sure your computer is synchronized with a definitive time source before starting weewx. This is best done by using a real-time clock, then running NTP, or another time synchronization utility. See the Wiki guide on Running WeeWX on a Raspberry Pi. Even if you do not have an RPi, the principles are the same.

2. Get rid of the bogus archive records. First, you'll have to find your database. If you used the package installer, it will be in /var/lib/weewx/weewx.sdb. If you used the setup.py method, it will be in /home/weewx/archive/weewx.sdb. Let's assume the former.

# First install the utility sqlite3 if you don't already have it

sudo apt-get install sqlite3

# Back up the database:

cd /var/lib/weewx

sudo cp weewx.sdb weewx.sdb.backup

# Use the utility sqlite3 to look at the database

sqlite3 weewx.sdb

# This will tell you how many records you have before 1 January 2000

sqlite> count(dateTime) from archive where dateTime < 946684800;

# Delete them

sqlite> delete from archive where dateTime < 946684800;

sqlite> .quit

# Now you need to drop the daily summaries. Use the utility wee_database to do this:

sudo wee_database --drop-daily

The daily summaries will automatically be rebuilt the next time your run weewxd.

See if that helps.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/0f79940a-68a5-4968-a2c1-ffdd21a9a0d5n%40googlegroups.com.

[Chris Martino]

Thanks Tom, but unfortunately that doesn't appear to have worked.

The count operation failed completely, but that was just informational, so no worries there.

# sqlite3 weewx.sdb
SQLite version 3.7.17 2013-05-20 00:56:22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"

sqlite> count(dateTime) from archive where dateTime < 946684800;

Error: near "count": syntax error

The delete operation didn't give any errors, neither did wee_database --drop-daily, but after starting weewx (I had stopped it prior to running the commands you sent) the records are still going back to Dec 31, 1969. I've waited about 30 minutes before sending this in case it takes some time to regenerate. 

Any other ideas on how to resolve?

Thanks,

Chris

[John Kline]

While you’re waiting for Tom:

The “count operation” just needs the word “select” in front of it.  It was a typo.  If you want, you could try again with select prepended.

Did you delete the NOAA report files?  When you say the records are still there, do you mean in the NOAA reports that you may or may not have deleted?  Since you deleted the old records, I suspect that the NOAA reports will no longer have these old dates when they are regenerated (after you delete the existing ones).

On Aug 10, 2020, at 12:25 PM, Chris Martino <ch...@console.org> wrote:

Thanks Tom, but unfortunately that doesn't appear to have worked.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/88362906-a8fa-443f-8606-0d99da55da73n%40googlegroups.com.

[Chris Martino]

That was the ticket, John! Once I deleted the already existing NOAA reports and they were regenerated things were back to normal. Thanks!