vproRunState: retrying archive record...

[xotimir]

Hi,

I've built radlib-2.8.2 and wview-5.5.1 and all seems well except that
I don't get archive data. I do get hi low data, etc.

I've looked around the archives and found the post about adding a
recently dated dummy entry in the archive table and I did that
successfully. That got rid of the first error showing up in the
system log, 'getNewestDateTime: radsqlitedirectGetRow failed!'. But I
still don't see new rows showing up in the archive database.

Then there's the error which is the subject of this message and that
continues.

Not sure how to attach things in google groups, so I'll append my
notes to the end of this message. There are a couple of things that I
ran into mentioned in the notes that may or may not be bugs to fix.
If there's a bug tracker I can file a couple of bugs.

I'm also wondering if mysql is neccessary to display data in the
charts, eg "last 24 hrs" displays. I doubt it, since sqlite has an
archive table, but I'd like to know for sure.

Thanks,
Derek


NOTES:

RADLIB:

The README says not to enable more than one database option at a time
but it
only mentions mysql and postgres, not sqlite. I'll take their word
anyway
for now:

./configure --enable-sqlite
make
sudo make install

VWIEW:

Build wview. Need to add mysqlclient library by editing the configure
script, here's the diff:

--- configure-orig 2009-07-14 21:31:21.000000000 -0400
+++ configure 2009-06-14 13:26:27.000000000 -0400
@@ -6527,7 +6527,7 @@
$as_echo_n "(cached) " >&6
else
ac_check_lib_save_LIBS=$LIBS
-LIBS="-lsqlite3 -lmysqlclient $LIBS"
+LIBS="-lsqlite3 $LIBS"
cat >conftest.$ac_ext <<_ACEOF
/* confdefs.h. */
_ACEOF
@@ -6591,7 +6591,7 @@
#define HAVE_LIBSQLITE3 1
_ACEOF

- LIBS="-lsqlite3 -lmysqlclient $LIBS"
+ LIBS="-lsqlite3 $LIBS"

fi

patch -p0 < ../wview-configure.patch
sudo rm -rf /usr/local/etc/wview
sudo rm -rf /usr/local/var/wview
sudo rm -rf /var/www/wview
sudo rm -rf /var/www/wviewmgmt/
HTTP_DOC_ROOT=/var/www ./configure --enable-station-vpro
HTTP_DOC_ROOT=/var/www make
sudo HTTP_DOC_ROOT=/var/www make install-env

[ It says this at the end of the last step:

Don't forget to run "wviewhtmlconfig" to configure the template
directory
Copying wviewmgmt to /var/www/wviewmgmt
For wview start/stop control via wviewmgmt, add your http user to /etc/
sudoers
(with no password required for a few commands):
"sudo visudo"
(add the following line to the bottom of the file):
"www-data ALL = NOPASSWD: /etc/init.d/wview, /etc/init.d/networking, /
usr/local/bin/wviewPlug-updater"
(save and exit the file):
Only these three commands are enabled as root for the http user.
Then point your browser to: http://[your_server_url_or_IP]/wviewmgmt/login.php
to manage wview via your web browser.
DISCLAIMER: web control via sudo may be a security threat, enable at
your own risk. ]

/usr/local/bin/vpinstall # No need to sudo
Serial port or host:port the VP console is connected to (i.e. /dev/
ttyS0, mss1:3001)
/dev/ttyUSB0
Desired archive interval (minutes) (5,10,15,30,60)
5
Station elevation (feet) ((-2000) - (15000))
36
Station latitude (tenths of degrees) ((-900) - (900), (-) => South)
42.42 {!!!!!!!!!! wait! see below}
Station longitude (tenths of degrees) ((-1799) - (1800), (-) =>
West)
-71.16 {!!!!!!!!!!!!!!}
Rain season start (month) (1 - 12)
1
Year-to-Date Rain (inches) (0.00 - 200.00)
0.00
Year-to-Date ET (inches) (0.00 - 200.00)
0.00

/usr/local/bin/wviewconfig
[take defaults except as noted below, do not screw this up!]
PARAMETER: STATION_TYPE
VantagePro
PARAMETER: STATION_HOST
anushka
PARAMETER: STATION_ELEVATION
36
PARAMETER: STATION_LATITUDE
42.42
PARAMETER: STATION_LONGITUDE
-71.16
PARAMETER: HTMLGEN_STATION_NAME
anushka
PARAMETER: HTMLGEN_STATION_CITY
Arlington
HTMLGEN_STATION_STATE
Massachusetts
PARAMETER: HTMLGEN_ARCHIVE_BROWSER_FILES_TO_KEEP
10000
PARAMETER: HTMLGEN_LOCAL_RADAR_URL
http://arlmawx.com/CLOSEUP/street/kbox_br1_0.png
PARAMETER: HTMLGEN_LOCAL_FORECAST_URL
http://www.wunderground.com/cgi-bin/findweather/getForecast?query=Arlington,
MA&wuSelect=WEATHER
PARAMETER: EMAIL_ADDRESS
dda...@ddavies.net
PARAMETER: ENABLE_EMAIL_ALERTS
yes
PARAMETER: SEND_TEST_EMAIL
yes
[Default admin password is <deleted>]


sudo /usr/local/bin/wviewhtmlconfig

sudo cp ./examples/Debian/wview /etc/init.d/wview
sudo chmod +x /etc/init.d/wview
tail -f /var/log/messages
sudo /etc/init.d/wview start


cd /usr/local/var/wview/archive
sudo sqlite3 wview-archive.sdb
sqlite> insert into archive values ('1247709600', 1, 5, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0);




Note: There is a bug in vpinstall (long/lat) where there's an implied
decimal point in the input. The confirmation display gets it
wrong!). At the end you can see the real values used:


ddavies@anushka:/usr/local/var/wview/archive$ vpinstall
################################################################################

--> Installation Configuration for Vantage Pro(2) Console

--> To reset historical highs and lows, use the Vantage Pro Console

--> Note: values in parenthesis are used if you hit enter at a
prompt...

################################################################################


-------------------------------------------------------------
--NOTE-- ** Serial and USB data loggers are both considered serial -
--NOTE-- examples of this are: /dev/ttyS0, /dev/ttyUSB0, /dev/
cuaa0, etc.
--NOTE-- ** If your Vantage Pro is connected to a terminal or serial
server,
--NOTE-- you should enter the host and port in the format host:port
-
--NOTE-- examples of this are: 10.10.10.10:3001, xyplex1:2102, etc.
Serial port or host:port the VP console is connected to (i.e. /dev/
ttyS0, mss1:3001)
(/dev/ttyS0): /dev/ttyUSB0

-------------------------------------------------------------
Desired archive interval (minutes) (5,10,15,30,60)
(5):

-------------------------------------------------------------
Station elevation (feet) ((-2000) - (15000))
(0): 36

-------------------------------------------------------------
Station latitude (tenths of degrees) ((-900) - (900), (-) => South)
(0): 424

-------------------------------------------------------------
Station longitude (tenths of degrees) ((-1799) - (1800), (-) => West)
(0): -712

-------------------------------------------------------------
Rain season start (month) (1 - 12)
(1):

-------------------------------------------------------------
Year-to-Date Rain (inches) (0.00 - 200.00)
(0.00):

-------------------------------------------------------------
Year-to-Date ET (inches) (0.00 - 200.00)
(0.00):


-------------------------------------------------------------
Let's review your answers:

VP Console Device: /dev/ttyUSB0
Archive Interval: 5 (minutes)
Station Elevation: 36 (feet)
Station Latitude: 424 (tenths of a degree)
Station Longitude: -712 (tenths of a degree)
Rain Season Start Month: 1
Year-to-Date Rain: 0
Year-to-Date ET: 0
-------------------------------------------------------------

WARNING: All archive records will be erased on the VP console!
Do you want to proceed with these values? (y/n)
(n): y

Configuring with new values (takes some time, be patient) ...

elevation set to 36 feet
rain season start month set to 1
rain year-to-date set to 0.00
ET year-to-date set to 0.00
archive memory cleared and interval set to 5 minutes
latitude set to 42.4, longitude set to -71.2

Please wait - allowing VP console to digest new settings .....


Firmware Version: Dec 30 2008
Station Location: 42.4 N, 71.2 W, 36 feet
Archive Interval: 5 minutes
Rain Season Start Month: 1
Wind direction calibration: 0 degrees
Listening on channel: 1 for ISS
Listening on channel: 2 for no sensor
Listening on channel: 3 for no sensor
Listening on channel: 4 for no sensor
Listening on channel: 5 for no sensor
Listening on channel: 6 for no sensor
Listening on channel: 7 for no sensor
Listening on channel: 8 for no sensor
Retransmit channel: off
Rain collector size: 0.01 in
Wind cup size: large
RX Check Stats: 0 0 0 0 0 recvd:missed:resyncs:good-pkts-in-a-
row:CRC errors


################################################################################
It is advisable to wait 5 minutes before starting wview for the
first time so the Vantage Pro can become consistent with the new
settings...

In the meantime, be sure you have deleted any archive files in
/var/wview/archive if you have changed the archive interval...

Vantage Pro configuration complete!

################################################################################
ddavies@anushka:/usr/local/var/wview/archive$



End of NOTES

[xotimir]

Sorry, a couple other thing I should've mentioned:

* The only other worrisome thing I see in the log is:

WV_EXPORT: /usr/local/etc/wview/export/archive_marker not found

I'm going to look into that next.

* Also, from my notes I patch configure to add the mysql client
library. That is actually a left over from when I was trying to get
sqlite and mysql at the same time. I /think/ that should be possible,
but in some doc I've seen stuff that makes me wonder (eg, what I
mention in my nots about it being one or the other of mysql/
postgressql w/o a mention of how sqlite fits into that). I've since
decided to try to just get it working with sqlite to start.

[xotimir]

Archiving is finally working. The only thing I can think of is that
maybe I didn't wait 5 mins after runing vpinstall. I don't think I
ever skimped there, but I have no other explanation. And what I saw
in the logs previously is different than now [see below]. I think if
there's no archive record in the console to download when first
starting wviewd then you are forever hosed as far as reading archive
records after that.

If it helps anyone (it would've helped me) I can say a few things:

1) You do not need mysql to get data in the we graphs. Just sqlite.

2) There is a bug in vpinstall with lat/long (see my first post
"NOTES").

3) This is shakier, but I'm somewhat confident: I think it's ok, when
starting wviewd the first time or after a totally clean re-install to
see
the getNewestDateTime failure. This is from the first time startup of
a clean re-install:

Jul 19 23:33:20 anushka wviewd[20344]: <1248060800496> : Vantage Pro
on /dev/ttyUSB0 opened ...
Jul 19 23:33:20 anushka wviewd[20344]: <1248060800497> : stationInit:
VP rxcheck is DISABLED
Jul 19 23:33:20 anushka wviewd[20344]: <1248060800497> :
getNewestDateTime: radsqlitedirectGetRow fa\
iled!
Jul 19 23:33:20 anushka wviewd[20344]: <1248060800497> : stationInit:
no archive records found in da\
tabase!
Jul 19 23:33:20 anushka wviewd[20344]: <1248060800534> :
getNewestDateTime: radsqlitedirectGetRow fa\
iled!
Jul 19 23:33:20 anushka wviewd[20344]: <1248060800534> : station
archive interval: 5 minutes
Jul 19 23:33:20 anushka wviewd[20344]: <1248060800553> : station rain
ticks/inch: 100
Jul 19 23:33:20 anushka wviewd[20344]: <1248060800564> : running...
Jul 19 23:33:20 anushka wviewd[20344]: <1248060800742> : storing
record for 2009-07-19 23:15
Jul 19 23:33:20 anushka wviewd[20344]: <1248060800839> : storing
record for 2009-07-19 23:20

It's those sweet, sweet "storing record for"'s that show it's working.

Also:

0.:/usr/local/var/wview/archive>cd /usr/local/var/wview/archive/
0.:/usr/local/var/wview/archive>ls
./ readme.txt wview-archive.sql wview-history.sdb
../ wview-archive.sdb wview-hilow.sdb wview-noaa.sdb
0.:/usr/local/var/wview/archive>sqlite3 wview-archive.sdb
SQLite version 3.6.10
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> select * from archive;
1248059700|1|5|30.143|30.143|30.134142|86.699997|68.900002|37.0|73.0|
0.0||0.0||0.0|0.0|59.877304|68.\
900002|68.900002|0.0||||||||||||||||||||0.0|0.0|0.0||||||||||
1248060000|1|5|30.143999|30.143999|30.135139|86.900002|68.800003|37.0|
74.0|0.0||0.0||0.0|0.0|60.1637\
46|68.800003|68.800003|0.0||||||||||||||||||||0.0|0.0|0.0||||||||||
1248060300|1|5|30.142|30.142|30.133141|86.900002|68.599998|38.0|74.0|
0.0||0.0||0.0|0.0|59.971111|68.\
599998|68.599998|0.0||||||||||||||||||||0.0|0.0|0.0||||||||||
1248060600|1|5|30.143|30.143|30.134142|86.900002|68.5|37.0|74.0|0.0||
0.0||0.0|0.0|59.874802|68.5|68.\
5|0.0||||||||||||||||||||0.0|0.0|0.0||||||||||
1248060900|1|5|30.145|30.106518|30.136806|87.099998|68.400002|37.0|
75.0|0.0||0.0||0.0|0.0|60.155933|\
68.400002|68.400002|0.0||||||||||||||||||||0.0|0.0|821.0||||||||||
1248061200|1|5|30.150999|30.112503|30.142797|87.300003|68.300003|37.0|
75.0|0.0||0.0||0.0|0.0|60.0594\
56|68.300003|68.300003|0.0||||||||||||||||||||0.0|0.0|821.0||||||||||
sqlite> .quit
0.:/usr/local/var/wview/archive>

One other thing: if you type mysql instead of sqlite3 then it tells
you the archive is encrypted! I had a few hours of fun with that :)

I'm now the proud owner of my own hand drawn vpro state transition
diagram! I got to crawl all through the code and, in all seriousness,
it's really nice!

Thanks,
Derek