Weewx generating reports two days head

[Mike Dame]

Last night I rebooted my raspberry running that is running "weewx v3.8.0".  And the report are dated two days ahead, so today is the 25th of Dec, reported that the temps are from 27th of Dec. I have double checked the system clock is correct on the raspberry and the time on the "WMR200a" is set correct, within minute of the raspberry.  notice the date stamp on the database check below.

The only error that I can see in the messages log is "wmr200: MainThread: W Clock time unsynchronized", I did not really get any hits that would give me a hint if this is the error or not.

pi@raspberrypi:~ $ sudo  wee_database --check
Using configuration file /etc/weewx/weewx.conf
Using database binding 'wx_binding', which is bound to database 'archive_sqlite'
Checking daily summary tables version...
Daily summary tables are at version 1.0
Recommend running --update to recalculate interval weightings.
Daily summary tables version check completed in 0.02 seconds.
Preparing Null String Check, this may take awhile...
Checking record: 739537; Timestamp: 2017-12-27 04:31:37 CST (1514370697)
No null strings found.
Completed Null String Check in 70.77 seconds.
pi@raspberrypi:~ $ date
Mon 25 Dec 20:42:46 CST 2017
pi@raspberrypi:~ $

Anybody got some ideas?

Regards,
Mike

[gjr80]

Hi,

What does the log say? Stop weeWX, start weeWX and let it run for a couple of archive periods. Post the log from startup covering the couple of archive periods.

Gary

[Mike Dame]

I have attached two cycles, the log looks normal to me (but not an expert).
Without doing a sql query to look at the database, it weekly temp graph looks like it is adding data on the correct date, but the current graph is plotting data that was dated wrong....

Mike

[Andrew Milner]

When you rebooted your raspberry did you ensure that the rpi had he correct time before starting weewx??

If you didn't then you could have got some rogue data into the database

If you have got rogue data in the database then you need to delete that data before you will get any sense

Your weekly graph is also not correct, as it is showing data for 27th December also

If your rpi does not have a real time clock installed, but is using the fake clock then you need to ideally install an RTC and ensure tha the RTC and the console are both set for the correct time and timezones before weewx is run - otherwise you have a risk of all sorts of time related issues.

I note tht your console is not synched to the rpi time anyway - but is 95 seconds adrift apparently

[Thomas Keffer]

Somehow the clock on your weather station or your RPi got a couple days ahead of reality, perhaps when you rebooted. Do you have a real-time clock? To figure out what happened you would have to go through the historical RPi system logs from back when you rebooted.

In any case, no matter what the cause, the fix is the same: delete the "in the future" database records and rebuild the daily summaries. Here's the steps (NOT TESTED):

1. Stop weewx

2. Make a backup copy of your database.

sudo cp /var/lib/weewx/weewx.sdb /var/tmp/weewx.sdb.backup

3. Figure out what the last good record is in your database. Let's say it was 0100 on 25-Dec-2017 CST (UTC-6). That would be 1514181600 unix epoch time. This website can help.

4. Make sure the tool sqlite3 is on your system

apt-get install sqlite3

5. Use it to delete the future records

sudo sqlite /var/lib/weewx/weewx.sdb

sqlite> DELETE FROM archive WHERE dateTime > 1514181600;

sqlite> .quit

6. Rebuild the daily summaries

sudo wee_database --rebuild-daily --from 2017-12-25

7. Restart weewx.

Let us know how it goes.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mike Dame]

Got an error with trying to rebuild database.

pi@raspberrypi:~ $ sudo wee_database --rebuild-daily --from 2017-12-25

Using configuration file /etc/weewx/weewx.conf
Using database binding 'wx_binding', which is bound to database 'archive_sqlite'

Daily summaries from 2017-12-25 through the end will be rebuilt.
Proceed (y/n)? y
Rebuilding daily summaries in database 'weewx.sdb' ...
Traceback (most recent call last):
  File "/usr/bin/wee_database", line 793, in <module>
    main()
  File "/usr/bin/wee_database", line 153, in main
    rebuildDaily(config_dict, db_binding, options)
  File "/usr/bin/wee_database", line 262, in rebuildDaily
    trans_days=20)
  File "/usr/share/weewx/weewx/manager.py", line 1428, in backfill_day_summary
    timestamp_to_string(lastRecord)))
weewx.ViolatedPrecondition: lastUpdate(2017-12-26 08:55:00 CST (1514300100)) > lastRecord(2017-12-25 00:00:00 CST (1514181600))

On Monday, December 25, 2017 at 8:53:02 PM UTC-6, Mike Dame wrote:

[gjr80]

Hi,

Try dropping the daily summaries before rebuilding:

sudo wee_database --drop-daily
sudo wee_database --rebuild-daily

Since 3.7.0 (I think) you can usually, but not always, rebuild without first dropping. I think this is one of those exceptions.

Gary

[Mike Dame]

So it will not restart, I copied back the backup database. I tried with and without the sql drop, now the database will not start... :(

pi@raspberrypi:~ $ sudo cp -p /var/tmp/weewx.sdb.backup /var/lib/weewx/weewx.sdb

pi@raspberrypi:~ $ sudo sqlite3 /var/lib/weewx/weewx.sdb
SQLite version 3.8.7.1 2014-10-29 13:59:56
Enter ".help" for usage hints.
sqlite>

sqlite> DELETE FROM archive WHERE dateTime > 1514181600;
sqlite> .quit

pi@raspberrypi:~ $ sudo wee_database --drop-daily

Using configuration file /etc/weewx/weewx.conf
Using database binding 'wx_binding', which is bound to database 'archive_sqlite'

Proceeding will delete all your daily summaries from database 'weewx.sdb'
Are you sure you want to proceed (y/n)? y
Dropping daily summary tables from 'weewx.sdb' ...
Daily summary tables dropped from database 'weewx.sdb' in 2.58 seconds

pi@raspberrypi:~ $ sudo wee_database --rebuild-daily

Using configuration file /etc/weewx/weewx.conf
Using database binding 'wx_binding', which is bound to database 'archive_sqlite'

All daily summaries will be rebuilt.

Proceed (y/n)? y
Rebuilding daily summaries in database 'weewx.sdb' ...

Records processed: 738000; Last date: 2017-12-21 11:30:00 CST (1513877400)
Processed 738994 records to rebuild 2585 day summaries in 698.76 seconds     
Rebuild of daily summaries in database 'weewx.sdb' complete

pi@raspberrypi:~ $ sudo systemctl start weewx;tail -f /var/log/messages
Dec 26 21:04:06 raspberrypi wee_database[18640]: manager: Processed 738994 records to backfill 2585 day summaries in 698.48 seconds
Dec 26 21:04:06 raspberrypi wee_database[18640]: Rebuild of daily summaries in database 'weewx.sdb' complete
Dec 26 21:13:36 raspberrypi wee_database[18824]: manager: Dropping daily summary tables from 'weewx.sdb' ...
Dec 26 21:13:38 raspberrypi wee_database[18824]: manager: Dropped daily summary tables from database 'weewx.sdb'
Dec 26 21:13:45 raspberrypi wee_database[18832]: All daily summaries will be rebuilt.
Dec 26 21:13:47 raspberrypi wee_database[18832]: manager: Created daily summary tables
Dec 26 21:13:47 raspberrypi wee_database[18832]: Rebuilding daily summaries in database 'weewx.sdb' ...
Dec 26 21:13:47 raspberrypi wee_database[18832]: manager: Starting backfill of daily summaries
Dec 26 21:25:21 raspberrypi wee_database[18832]: manager: Processed 739683 records to backfill 2588 day summaries in 694.00 seconds
Dec 26 21:25:22 raspberrypi wee_database[18832]: Rebuild of daily summaries in database 'weewx.sdb' complete
^C
pi@raspberrypi:~ $ ps -ef | grep weewx
pi       18967  1332  0 21:26 pts/0    00:00:00 grep --color=auto weewx
pi@raspberrypi:~ $

pi@raspberrypi:~ $ sudo systemctl -l status weewx
● weewx.service - LSB: weewx weather system
   Loaded: loaded (/etc/init.d/weewx)
   Active: active (exited) since Tue 2017-12-26 09:11:01 CST; 11h ago
  Process: 12968 ExecStop=/etc/init.d/weewx stop (code=exited, status=0/SUCCESS)
  Process: 13199 ExecStart=/etc/init.d/weewx start (code=exited, status=0/SUCCESS)

Dec 26 09:11:02 raspberrypi weewx[13213]: ****    File "/usr/share/weewx/weewx/engine.py", line 604, in setup_database
Dec 26 09:11:02 raspberrypi weewx[13213]: ****      _nrecs, _ndays = dbmanager.backfill_day_summary() # @UnusedVariable
Dec 26 09:11:02 raspberrypi weewx[13213]: ****    File "/usr/share/weewx/weewx/manager.py", line 1428, in backfill_day_summary
Dec 26 09:11:02 raspberrypi weewx[13213]: ****      timestamp_to_string(lastRecord)))
Dec 26 09:11:02 raspberrypi weewx[13213]: ****  ViolatedPrecondition: lastUpdate(2017-12-26 08:55:00 CST (1514300100)) > lastRecord(2017-12-25 00:00:00 CST (1514181600))
Dec 26 09:11:02 raspberrypi weewx[13213]: ****  Exiting.
Dec 26 20:44:30 raspberrypi systemd[1]: Started LSB: weewx weather system.
Dec 26 20:45:47 raspberrypi systemd[1]: Started LSB: weewx weather system.
Dec 26 20:47:02 raspberrypi systemd[1]: Started LSB: weewx weather system.
Dec 26 21:05:26 raspberrypi systemd[1]: Started LSB: weewx weather system.
pi@raspberrypi:~ $

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/RezDiRbbCks/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+unsubscribe@googlegroups.com.

[Andrew Milner]

I suspect you did not delete the first entry for 25th December

the archive delete should, i think, have been where dateTime>= 1514181600; , assuming your timezone is GMT-6

Try again, after restoring the database from backup, but this time

a) check you are using the correct delete timestamp

b) after deleting check the timestamp of the last record in the database and make sure it is not for > 25 dec

then drop daily

and rebuild daily

then restart weewx

Unfortunately you terminated the log with ctrl/c before the tail had chance to display anything from the startup process.  Try sand do one thing at a time - start weewx, then start the tail in the log as two commands.

My unix is not good enough to say if the log snippet from the status command relates to an earlier failure, or a current failure.  the times of the log entries implies that the fsilure relates to 0900 in the morning and there were several successful restarts at later times - with no errors

To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.

[gjr80]

I'm not necessarily seeing anything wrong here. I would take it from the top again and this time post the entire log, covering the wee_database operations and weeWX startup and any errors. Better to post an full and complete log rather than disjointed excerpts, far easier to read what is happening. systemctl status is great for looking at the state of the weeWX daemon but not much chop for looking at logs, it is like reading them through a drinking straw.

Gary

[Mike Dame]

Tried a delete from archive after midnight 12/25 that did not work after drop and rebuild still did not start. So per  request, I have attached the message logs from the 22nd till 26th.

I tried turning on debug in the config file, still did not log anything or start....

Anyway to start in foreground to see if there is any clue why it is not starting up....

On Tue, Dec 26, 2017 at 10:54 PM, gjr80 <gjrod...@gmail.com> wrote:

I'm not necessarily seeing anything wrong here. I would take it from the top again and this time post the entire log, covering the wee_database operations and weeWX startup and any errors. Better to post an full and complete log rather than disjointed excerpts, far easier to read what is happening. systemctl status is great for looking at the state of the weeWX daemon but not much chop for looking at logs, it is like reading them through a drinking straw.

Gary

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/RezDiRbbCks/unsubscribe.

To unsubscribe from this group and all its topics, send an email to weewx-user+unsubscribe@googlegroups.com.

[Mike Dame]

I finally was able to take a little time to dig through some of the scripts, and after deleting pid file, and  issuing  a "systemctl daemon-reload", and then weewx service finally restarted. It took a little while to dump and record the missed records.

I think I am good now.

Thanks for everyone's assistance.