Porting data from WVIEW

183 views
Skip to first unread message

Richard Rosa

unread,
Jul 30, 2019, 5:04:43 PM7/30/19
to weewx-user
Greetings All,
 
 
  My installation of WVIEW stopped working yesterday (on SUSE 15.1), so I figured it was time to move to something that is better supported.

The WEEWX installation went fairly smoothly (RPM install). There was a bit of confusion as to how the program was started.
The manual indicated that the command was in /etc/rc.d (/etc/init.d), the same as WVIEW.
However, the program actually runs as a SYSTEMD service on the current SUSE level.

The problem I have is porting 10+ years of WVIEW logs.

Following the manual's instructions, I copied archive databases from wview/archive -> weewx.sdb
There appear to be two different WVIEW archive files: wview-hilow.sdb and wview-history.sdb.
After copying either file, and restarting the service, WEEWX grabbed recent data from my Vantage Pro.
After running wee_reports, I had HTML files for the past few weeks of data, but nothing else.
No reports for prior months or years.
This procedure was repeated with both files, with the same results.


Am I porting these files correctly? If so, how do I get reports for data prior to what is stored in the Davis?
Which one of these files is the correct file to copy?

Thanx.

Richard Rosa

vince

unread,
Jul 30, 2019, 5:15:44 PM7/30/19
to weewx-user
On Tuesday, July 30, 2019 at 2:04:43 PM UTC-7, Richard Rosa wrote:
The problem I have is porting 10+ years of WVIEW logs
Following the manual's instructions, I copied archive databases from wview/archive -> weewx.sdb

you'd have to point us at which instructions you followed perhaps - hopefully it was http://weewx.com/docs/usersguide.htm#wview_compatibility

There appear to be two different WVIEW archive files: wview-hilow.sdb and wview-history.sdb.
After copying either file, and restarting the service, WEEWX grabbed recent data from my Vantage Pro.

yes it will always try to catch up to what's in your VP2 datalogger....

I'm pretty sure it's "not" the hilow database, but the wview docs say the archive is in wview-archive.sdb typically (http://www.wviewweather.com/release-notes/wview-User-Manual.html#Advanced-sqlite)


After running wee_reports, I had HTML files for the past few weeks of data, but nothing else.
No reports for prior months or years.

You want to check your syslog - 10 years of data is a lot, it might take a long time for weewx to build the summary tables and NOAA files (once) on your initial bootup.   It's possible you have some confusing data in there perhaps.

Definitely set debug=1 in weewx.conf to get more verbose logging and watch your syslogs for info letting you know it looks ok (or not).

Richard Rosa

unread,
Jul 30, 2019, 5:53:25 PM7/30/19
to weewx-user

Other than NOT changing to MYSQL, those were the instructions I followed. Using sqlitebrowser, it appears that the records from WVIEW are in this file, but not being a DB expert, I'm not sure if they are in the right place.

I DID set debug on, and follow the log messages (journalctl -f). The wee_reports command generates a handful of messages within a few seconds, then 'Gentle wake up of console'. SeasonsReport seems to be the only thing generated. SmartphoneReport, MobileReport, StandardReport, FTP and RSYNC are not generated. Donno if these should be enabled or not (they were disabled by default in the install). No other messages, no status indication.

I don't see any other messages in the log that indicate something is running. CPU & Disk usage seem pretty quiet after the last 'wake up' message.



Richard Rosa

Thomas Keffer

unread,
Jul 30, 2019, 7:03:13 PM7/30/19
to weewx-user
wee_reports is for generating reports only. It is a standalone program, mostly used for testing purposes.

You want weewxd: the main weewx program.

Take a look in the section Running weewx in the manual. If you still have questions, come on back, but be sure to include the system log.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/efb3984a-2be2-412d-8273-489a2c1bc4ef%40googlegroups.com.

Richard Rosa

unread,
Jul 30, 2019, 7:49:58 PM7/30/19
to weewx-user
The main program IS running. How do I tell it to generate reports? When the program started, no reports were generated until the 'wee_reports' command was issued.



System Log (journalctl):

 
Jul 30 19:39:54 rubberducky.rbm systemd[1]: Started weewx weather system.
Jul 30 19:39:54 rubberducky.rbm weewx[11021]: engine: Initializing weewx version 3.9.2
Jul 30 19:39:54 rubberducky.rbm weewx[11021]: engine: Using Python 2.7.14 (default, Oct 12 2017, 15:50:02) [GCC]
Jul 30 19:39:54 rubberducky.rbm weewx[11021]: engine: Platform Linux-4.12.14-lp151.28.10-default-x86_64-with-glibc2.2.5
Jul 30 19:39:54 rubberducky.rbm weewx[11021]: engine: Locale is 'en_US.UTF-8'
Jul 30 19:39:54 rubberducky.rbm weewx[11021]: engine: pid file is /var/run/weewx.pid
Jul 30 19:39:54 rubberducky.rbm systemd[1]: weewx.service: Supervising process 11033 which is not our child. We'll most likely not notice when it exits.
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Using configuration file /etc/weewx/weewx.conf
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Debug is 1
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Initializing engine
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Loading station type Vantage (weewx.drivers.vantage)
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: vantage: Driver version is 3.1.1
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: vantage: Opened up serial port /dev/vpro; baud 19200; timeout 4.00
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: vantage: Gentle wake up of console successful
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: vantage: Hardware type is 16
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: vantage: ISS ID is 1
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: vantage: Hardware name: Vantage Pro2
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Loading service weewx.engine.StdTimeSynch
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Finished loading service weewx.engine.StdTimeSynch
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Loading service weewx.engine.StdConvert
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: StdConvert target unit is 0x1
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Finished loading service weewx.engine.StdConvert
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Loading service weewx.engine.StdCalibrate
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Finished loading service weewx.engine.StdCalibrate
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Loading service weewx.engine.StdQC
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Finished loading service weewx.engine.StdQC
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Loading service weewx.wxservices.StdWXCalculate
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: wxcalculate: The following values will be calculated: barometer=prefer_hardware, windchill=prefer_hardware, dewpoint=prefer_hardware, appTemp=prefer_hardware, rainRate=prefer_hardware, windrun=prefer_hardware, heatindex=prefer_hardware, maxSolarRad=prefer_hardware, humidex=prefer_hardware, pressure=prefer_hardware, inDewpoint=prefer_hardware, ET=prefer_hardware, altimeter=prefer_hardware, cloudbase=prefer_hardware
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: wxcalculate: The following algorithms will be used for calculations: altimeter=aaNOAA, maxSolarRad=RS
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Finished loading service weewx.wxservices.StdWXCalculate
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Loading service weewx.engine.StdArchive
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Archive will use data binding wx_binding
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Record generation will be attempted in 'hardware'
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Using archive interval of 1800 seconds (specified by hardware)
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Use LOOP data in hi/low calculations: 1
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: manager: Daily summary version is 2.0
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Using binding 'wx_binding' to database 'weewx.sdb'
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: manager: Starting backfill of daily summaries
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Finished loading service weewx.engine.StdArchive
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Loading service weewx.restx.StdStationRegistry
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: restx: StationRegistry: Registration not requested.
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Finished loading service weewx.restx.StdStationRegistry
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Loading service weewx.restx.StdWunderground
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: restx: Wunderground: Posting not enabled.
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Finished loading service weewx.restx.StdWunderground
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Loading service weewx.restx.StdPWSweather
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: restx: PWSweather: Posting not enabled.
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Finished loading service weewx.restx.StdPWSweather
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Loading service weewx.restx.StdCWOP
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: restx: CWOP: Posting not enabled.
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Finished loading service weewx.restx.StdCWOP
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Loading service weewx.restx.StdWOW
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: restx: WOW: Posting not enabled.
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Finished loading service weewx.restx.StdWOW
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Loading service weewx.restx.StdAWEKAS
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: restx: AWEKAS: Posting not enabled.
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Finished loading service weewx.restx.StdAWEKAS
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Loading service weewx.engine.StdPrint
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Finished loading service weewx.engine.StdPrint
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Loading service weewx.engine.StdReport
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Finished loading service weewx.engine.StdReport
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Starting up weewx version 3.9.2
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: vantage: Gentle wake up of console successful
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Clock error is -0.86 seconds (positive is fast)
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: vantage: Getting archive packets since 2019-07-30 19:30:00 EDT (1564529400)
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: vantage: Gentle wake up of console successful
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: vantage: Retrieving 0 page(s); starting index= 0
Jul 30 19:39:54 rubberducky.rbm weewx[11033]: engine: Starting main packet loop.
Jul 30 19:39:58 rubberducky.rbm weewx[11033]: vantage: Retry  #0 failed
Jul 30 19:39:58 rubberducky.rbm weewx[11033]: vantage: Gentle wake up of console successful
Jul 30 19:39:58 rubberducky.rbm weewx[11033]: vantage: Requesting 200 LOOP packets.
Jul 30 19:39:58 rubberducky.rbm weewx[11033]: vantage: Gentle wake up of console successful
To unsubscribe from this group and stop receiving emails from it, send an email to weewx...@googlegroups.com.

Thomas Keffer

unread,
Jul 30, 2019, 7:59:46 PM7/30/19
to weewx-user
Your VP2 has an archive interval of one hour, so it will take that long to generate an archive record, and kick off a reporting cycle. 

If you want something more often, change the interval to, say, 5 minutes. See the section Archive interval in the User's Guide.

-tk

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/7bc2a16d-ecbe-4749-91a8-7d7376f615bd%40googlegroups.com.

Richard Rosa

unread,
Jul 30, 2019, 8:23:58 PM7/30/19
to weewx-user
Is there any downside (or possible loss of data) with changing the interval? I vaguely remember something in the WVIEW setup about changing the interval would make for data incompatibility issues. I don't want to lose the info currently stored in the device (or in the WVIEW archives).

Thanx.

vince

unread,
Jul 30, 2019, 8:50:39 PM7/30/19
to weewx-user
On Tuesday, July 30, 2019 at 5:23:58 PM UTC-7, Richard Rosa wrote:
Is there any downside (or possible loss of data) with changing the interval? I vaguely remember something in the WVIEW setup about changing the interval would make for data incompatibility issues. I don't want to lose the info currently stored in the device (or in the WVIEW archives).


Nope.

As always, work off a 'copy' of your database.  Given you have a VP2 you have a week of datalogger storage giving you time to get it right without missing any logged records :-)

Graham Eddy

unread,
Jul 30, 2019, 10:45:56 PM7/30/19
to weewx...@googlegroups.com
i did this about 6 months ago with 12 years of wview data, “this” being copying wview db into weewx space and changing vp2’s interval from 30 mins to 5 mins. the old data on 30 mins archive interval freely mixes in reports with newer data on 5 min interval

the only problem i struck was some not-quite-right data in the wview db that weewx objected to, probably caused by my patching a few data entries years ago with strings not numbers, which was simple to fix
____________
Graham Eddy

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/7bc274f1-d5e0-4462-86c8-d88eba49777c%40googlegroups.com.

Richard Rosa

unread,
Jul 31, 2019, 5:29:49 AM7/31/19
to weewx-user
To be on the safe side, I decided NOT to change the interval (at least for now).

I started WEEWX using the weewxd command (rather than the SYSTEMD service) and let it run for about 2 hours.
At the end of the two hours I have reports for the past month and a half of data, but no more.

My suspect is that the WVIEW database file does NOT have the same structure as the WEEWX, even though the manual says otherwise. A quick peek using the sqlitebrowser on each .sdb file seems to validate this.

As I said, I am not very knowledgeable about database manipulation.  I think there is some magic that needs to be performed on the WVIEW database file, but I do not know the correct incantation...


Here is the 'schema' of the wview-history.sdb file:

qlite3 wview-history.sdb
SQLite version 3.28.0 2019-04-16 19:49:53
Enter ".help" for usage hints.
sqlite> .schema
CREATE TABLE dayHistory ( date INTEGER ,barometer REAL ,pressure REAL ,altimeter REAL ,inTemp REAL ,outTemp REAL ,inHumidity REAL ,outHumidity REAL ,windSpeed REAL ,windDir REAL ,windGust REAL ,windGustDir REAL ,rainRate REAL ,rain REAL ,dewpoint REAL ,windchill REAL ,heatindex REAL ,rxCheckPercent REAL ,ET REAL ,radiation REAL ,UV REAL ,extraTemp1 REAL ,extraTemp2 REAL ,extraTemp3 REAL ,soilTemp1 REAL ,soilTemp2 REAL ,soilTemp3 REAL ,soilTemp4 REAL ,leafTemp1 REAL ,leafTemp2 REAL ,extraHumid1 REAL ,extraHumid2 REAL ,soilMoist1 REAL ,soilMoist2 REAL ,soilMoist3 REAL ,soilMoist4 REAL ,leafWet1 REAL ,leafWet2 REAL ,txBatteryStatus REAL ,consBatteryVoltage REAL ,hail REAL ,hailRate REAL ,heatingTemp REAL ,heatingVoltage REAL ,supplyVoltage REAL ,referenceVoltage REAL ,windBatteryStatus REAL ,rainBatteryStatus REAL ,outTempBatteryStatus REAL ,inTempBatteryStatus REAL ,PRIMARY KEY (date) );


Thanx

Richard Rosa

gjr80

unread,
Jul 31, 2019, 6:40:23 AM7/31/19
to weewx-user
If just copying and renaming the archive database then you likely need to build/rebuild the daily summaries using wee_database (http://weewx.com/docs/utilities.htm#wee_database utility). Once done don't forget to delete all NOAA format reports to force generation of the older reports.


Gary

Richard Rosa

unread,
Jul 31, 2019, 9:36:56 AM7/31/19
to weewx-user
Ran wee_database --rebuild-daily. Deleted ALL generated reports. Started the service. Waited one hour for reports to appear. Still only a few weeks of info (what is stored in the Davis).

The data from WVIEW is NOT being processed..

Richard Rosa.

gjr80

unread,
Jul 31, 2019, 10:00:52 AM7/31/19
to weewx-user
Ok, a couple of more things to do/try. First run wee_debug and post the output here. Just make sure you check the wee_debug output for any sensitive info such as user names, passwords, keys etc before posting; wee_debug should obfuscate such info but it’s not perfect. The wee_debug report should tell us the oldest and most recent records in the archive.

The other thing is to do a wee_database —drop-daily and then rebuild the daily summaries (and delete the reports to force regeneration or run wee_ reports).

Gary

vince

unread,
Jul 31, 2019, 10:35:14 AM7/31/19
to weewx-user
On Wednesday, July 31, 2019 at 2:29:49 AM UTC-7, Richard Rosa wrote:
sqlite3 wview-history.sdb
SQLite version 3.28.0 2019-04-16 19:49:53
Enter ".help" for usage hints.
sqlite> .schema
CREATE TABLE
dayHistory ( date INTEGER ,barometer REAL ,pressure REAL ,altimeter REAL ,inTemp REAL ,outTemp REAL ,inHumidity REAL ,outHumidity REAL ,windSpeed REAL ,windDir REAL ,windGust REAL ,windGustDir REAL ,rainRate REAL ,rain REAL ,dewpoint REAL ,windchill REAL ,heatindex REAL ,rxCheckPercent REAL ,ET REAL ,radiation REAL ,UV REAL ,extraTemp1 REAL ,extraTemp2 REAL ,extraTemp3 REAL ,soilTemp1 REAL ,soilTemp2 REAL ,soilTemp3 REAL ,soilTemp4 REAL ,leafTemp1 REAL ,leafTemp2 REAL ,extraHumid1 REAL ,extraHumid2 REAL ,soilMoist1 REAL ,soilMoist2 REAL ,soilMoist3 REAL ,soilMoist4 REAL ,leafWet1 REAL ,leafWet2 REAL ,txBatteryStatus REAL ,consBatteryVoltage REAL ,hail REAL ,hailRate REAL ,heatingTemp REAL ,heatingVoltage REAL ,supplyVoltage REAL ,referenceVoltage REAL ,windBatteryStatus REAL ,rainBatteryStatus REAL ,outTempBatteryStatus REAL ,inTempBatteryStatus REAL ,PRIMARY KEY (date) );



The weewx schema is at https://github.com/weewx/weewx/blob/master/bin/schemas/wview.py if you want to do a visual comparison.

You'd have to check the schema for both your wview databases I think.   It's been so many years I don't remember any more what Mark's software evolved to.

My thought is you could get there with a dump/hand-edit/restore of your old data as the worst case, but I'd have to defer to somebody who has run wview in the last few years who has done the process you're trying to accomplish.  I just can't remember any more whether I had any touch labor when I switched to weewx a long time ago.


Richard Rosa

unread,
Jul 31, 2019, 10:35:27 AM7/31/19
to weewx-user
Debug file attached.
Ran :
  wee_database --drop-daily
  wee_database --rebuild-daily


Again, removed all generated files.
Waited for reports to be generated. Same issue. No data except what is stored in the Davis Vantage Pro.

I'm pretty sure the issue is that the database 'schema' used by the version of WVIEW that was running,
is DIFFERENT than the current definition of WEEWX. The imported data appears in the file, but it appears to have
it's own structure (see my previous post with the WVIEW schema display). WEEWX doesn't appear to know that these records are present. There is probably some SQL commands that will fix this, but I have no idea what they would be.

If you want, I can attach the two .sdb files that I have tried. The wview-history.sdb file is fairly small (>1mb). The
wview-hilo.sdb file is about 50mb. Donno what the size limit is on file postings here.

Thanx


Richard Rosa
weewx.debug

gjr80

unread,
Jul 31, 2019, 10:53:47 AM7/31/19
to weewx-user
Ok, so wee_ debug says there is only 54 odd days of data in your archive, expect these records are the history downloaded from the logger. So we are not going to get any reports using your historical wview data from that database.

Not convinced this a schema issue.

Just to step back a bit, when you copied the wview database did you make a copy and rename it weewx.sdb? Was it saved to /var/lib/weewx? If not that is what you need to do. If you did do this then stop WeeWX if it is running, move /var/lib/weewx/weewx.sdb aside and make a fresh copy of your wview history database, rename it weewx.sdb and save it to /var/lib/weewx. Run wee_debug again, what does it show under ‘Archive info’?

Gary

Richard Rosa

unread,
Jul 31, 2019, 11:32:04 AM7/31/19
to weewx-user
Almost the exact procedure followed.
 Not that it should matter, /var/lib/weewx is symlinked to a different location (to make backups a bit easier). A Config file change also sets SQLITE_ROOT to that location as well.


Retrying again (with wee_debug before start)
Commands issued (as root):

 systemctl stop weewx
 cd /data/weewx/lib
 mv weewx.sdb weewx-backup.sdb
 cp wview-history.sdb weewx.sdb
cd /data/weewx/html
rm -R *       ### clear out generated reports
wee_debug --info --output

 results in these messages:

wee_debug output will be written to /var/tmp/weewx.debug

Traceback (most recent call last):
  File "/usr/bin/wee_debug", line 420, in <module>
    main()
  File "/usr/bin/wee_debug", line 160, in main
    options.verbosity)
  File "/usr/bin/wee_debug", line 232, in generateDebugInfo
    manager_info_dict = getManagerInfo(config_dict, db_binding_wx)
  File "/usr/bin/wee_debug", line 365, in getManagerInfo
    with weewx.manager.open_manager_with_config(config_dict, db_binding_wx) as dbmanager_wx:
  File "/usr/share/weewx/weewx/manager.py", line 1032, in open_manager_with_config
    return open_manager(manager_dict, initialize)
  File "/usr/share/weewx/weewx/manager.py", line 1024, in open_manager
    manager_dict['table_name'])
  File "/usr/share/weewx/weewx/manager.py", line 105, in open
    dbmanager = cls(connection, table_name)
  File "/usr/share/weewx/weewx/manager.py", line 1157, in __init__
    super(DaySummaryManager, self).__init__(connection, table_name, schema)
  File "/usr/share/weewx/weewx/manager.py", line 73, in __init__
    self.sqlkeys = self.connection.columnsOf(self.table_name)
  File "/usr/share/weewx/weedb/sqlite.py", line 171, in columnsOf
    raise weedb.ProgrammingError("No such table %s" % table)
weedb.ProgrammingError: No such table archive



Richard Rosa

gjr80

unread,
Jul 31, 2019, 11:54:03 AM7/31/19
to weewx-user
Hmmm, seems your wview database is indeed different. You might need some input from Tom or another wview user. I was under the impression the wview database contained a table named ‘archive’ that is identical in structure to the archive table in the WeeWX database. Many wview users have changed to WeeWX and this is the first time I have heard of a wview database that cannot be used directly by WeeWX. I note the wview documentation states the wview uses wview-archive.sdb whereas you have wview-history.sdb.

Gsry

Richard Rosa

unread,
Jul 31, 2019, 12:40:53 PM7/31/19
to weewx-user
Looking through some old backups, it seems that there WAS a wview-archive file from about five years ago. It would appear that there was a format change (at least in the version I was running) around the beginning of 2015. The two .sdb files that I have that were updated recently (the last time I was able to run the program) are 'wview-hilow.sdb' (a 50mb file) and 'wview-history.sdb' (a 1mb file).

I should also mention that there is a WVIEW  ../img/Archive directory that contains a .txt file for every day that was logged. Not sure if that info can be used or not.

Hopefully, someone knows how to import or convert this data.

Thanx

Richard Rosa

vince

unread,
Jul 31, 2019, 12:46:39 PM7/31/19
to weewx-user
On Wednesday, July 31, 2019 at 8:54:03 AM UTC-7, gjr80 wrote:
Hmmm, seems your wview database is indeed different. You might need some input from Tom or another wview user. I was under the impression the wview database contained a table named ‘archive’ that is identical in structure to the archive table in the WeeWX database. Many wview users have changed to WeeWX and this is the first time I have heard of a wview database that cannot be used directly by WeeWX. I note the wview documentation states the wview uses wview-archive.sdb whereas you have wview-history.sdb.



yup - that's kinda what I was saying.   Converted his data to something that is weewx-compatible to me is a sqlite3 dump, perhaps a little light editing, and a sqlite3 restore (notionally)

Thomas Keffer

unread,
Jul 31, 2019, 2:34:17 PM7/31/19
to weewx...@googlegroups.com
I believe you copied the wrong file. You want wview-archive.sdb.

I'm on an iPhone right now, so it's hard for me to give any more details.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/09653a3b-be25-4eb3-81ce-e58a2be40f29%40googlegroups.com.
--
-tk

Richard Rosa

unread,
Jul 31, 2019, 2:46:02 PM7/31/19
to weewx-user

There is NO current wview-archive.sdb. I only have one from an old backup from 5 years ago. Recent archive database files are wview-hilow.sdb and wview-history.sdb

Richard Rosa




On Wednesday, July 31, 2019 at 2:34:17 PM UTC-4, Thomas Keffer wrote:
I believe you copied the wrong file. You want wview-archive.sdb.

I'm on an iPhone right now, so it's hard for me to give any more details.
On Wed, Jul 31, 2019 at 12:46 PM vince <vince...@gmail.com> wrote:
On Wednesday, July 31, 2019 at 8:54:03 AM UTC-7, gjr80 wrote:
Hmmm, seems your wview database is indeed different. You might need some input from Tom or another wview user. I was under the impression the wview database contained a table named ‘archive’ that is identical in structure to the archive table in the WeeWX database. Many wview users have changed to WeeWX and this is the first time I have heard of a wview database that cannot be used directly by WeeWX. I note the wview documentation states the wview uses wview-archive.sdb whereas you have wview-history.sdb.



yup - that's kinda what I was saying.   Converted his data to something that is weewx-compatible to me is a sqlite3 dump, perhaps a little light editing, and a sqlite3 restore (notionally)

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx...@googlegroups.com.
--
-tk

Thomas Keffer

unread,
Jul 31, 2019, 3:28:45 PM7/31/19
to weewx...@googlegroups.com
Take a look at chapter 7 of the wview users guide (). I'm pretty sure it's called wview-archive.sdb. 

Or, ask Graham Eddy.



To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/f576aea1-5fe1-411a-8481-12da60cc9331%40googlegroups.com.
--
-tk

Richard Rosa

unread,
Jul 31, 2019, 6:31:01 PM7/31/19
to weewx-user
It looks like I have an alternative:

  Thankfully, WVIEW saved a bunch of .txt files for every logged day (almost 4k). Although I know very little about SQL databases, I am pretty decent at converting text file formats to CSV files.

I picked the OLDEST text file, created a script to convert it to a CSV and ran it through the wee_import utility.
SUCCESS! The reports now show data for all months/years since the date of the first log in that file (2008),
All the information between that file and the first record extracted from the Davis is blank, but that SHOULD be fixed
once all the other text files are imported.

It may take a day or so to get all the info into the program, but it looks like my prior year's data can be salvaged.

Thanx to all for your assistance.

Richard Rosa

vince

unread,
Jul 31, 2019, 6:45:27 PM7/31/19
to weewx-user
On Wednesday, July 31, 2019 at 3:31:01 PM UTC-7, Richard Rosa wrote:
It looks like I have an alternative:

  Thankfully, WVIEW saved a bunch of .txt files for every logged day (almost 4k). Although I know very little about SQL databases, I am pretty decent at converting text file formats to CSV files.



Still think a sqlite dump, a little editing, and a sqlite restore is the right move here, but of course do what you think is right for you.

 

Andrew Milner

unread,
Jul 31, 2019, 9:52:03 PM7/31/19
to weewx-user
I don't think you have an alternative, you have something different.

From wview manual:
Day history Table (wview-history.sdb)

Overview

This table is generated internally by wview to avoid having to compute the daily summary records used for the yearly charts every time wview starts. This was not a time concern with the flat WLK files but became more costly when we started storing data in SQLite3. The external use or utility of this table is limited at best.

Regeneration

To regenerate the NOAA database:

----------------------------------

I think you need to double check with wview, but the manual CLEARLY says the wview archive is wview-archive.sdb and that wview-history.sdb is an internal database used by wview for the yearly reports.

Graham Eddy

unread,
Jul 31, 2019, 11:25:42 PM7/31/19
to weewx...@googlegroups.com
It is wview-archive.sdb. The other dbs are generated from wview-archive.sdb if absent. wview cannot run without wview-archive.sdb. If absent from backups, the backups were not done correctly
(from phone -> forgive brevity)
GE


Graham Eddy

unread,
Aug 1, 2019, 12:40:19 AM8/1/19
to weewx...@googlegroups.com
The NOAA files contain data only of interest to US govt so miss extra sensors etc
_____________
Graham Eddy
via phone: forgive brevity

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

Richard Rosa

unread,
Aug 1, 2019, 8:19:49 AM8/1/19
to weewx-user
What the manual says and what is available on my system don't quite jive.
The ONLY wview-archive.sdb that I can find was last updated 5 years ago.

It has been running for the past 5 years WITHOUT generating this file.
Unless the file has been rendered invisible, it does NOT exist on my current system.

I would LOVE to try to extract the data from the existing SDB file, but as I indicated,
I do not know ANYTHING about SQL databases (or how to manipulate them).
If you can suggest the correct commands or operations, I will be more than happy to give it a shot.

BTW, I finished my script last night. Took about 10 seconds to create the CSV file, and about 10 minutes to import everything.

So far, it looks like most of my info got imported. As expected, some information (such as inside temp/console voltage/etc) is not there, but I can live with that.


Thanx.


Richard Rosa

Andrew Milner

unread,
Aug 1, 2019, 9:02:42 AM8/1/19
to weewx-user
the simplest thing is to install sqlite3 


and give a command such as
sqlite3 fullpathtosdbfile

then
select * from archive order by dateTime desc limit 10;

should give you the last 10 entries
Reply all
Reply to author
Forward
0 new messages