Odd rainrate issue

[Ernest Jillson]

Weewx 4.5.1

Belchertown 1.3b1

Davis VP2

 

I don't think this is a true weewx or Belchertown issue since my rainrate is set to prefer hardware, but I'm getting some new behavior over the last few months that I hadn't seen before.

 

Every once in a while, I will get a rainrate of 655.35, even if it didn't rain. That becomes the max for the day/month/year, of course. But when I check the weewx.sdb, there are no rainrates of 655.35. Nothing even close.  I grep for 655.35 in the syslog and come up empty.

 

I can stop weewx, rebuild the daily for that day, then rm everything from the /var/www/html folder, restart weewx and it's fixed. But where is it coming from? Any ideas?

 

Thanks in advance,

 

Ernie

[gjr80]

Hi,

This is a real 'it depends' type answer. A few things to consider before coming up with a way ahead.

prefer_hardware. All prefer_hardware does is look for an observation (in this case rainRate) from the driver and if it is present it is used. Otherwise WeeWX attempts to calculate rainRate. The vantage driver emits rainRate in both loop packets and archive records so WeeWX should be using rainRate as it comes from the driver. The 'it depends' part here is that if you are using software record generation (record_generation = software under [StdArchive] in weewx.conf) then WeeWX will be calculating the archive record rainRate value (this is the value stored in your archive table in your database and used in reports) as the average of the accumulated loop packet rainRate values (this is slightly different to the approach used within the Davis console, and hence emitted by the vantage driver, where archive record rainRate is actually the highest rainRate value seen during the archive period). The upshot is you could be seeing a value direct from the console or a value calculated by WeeWX based on loop packet values from the console. In either case you should not be seeing inordinately large and erroneous rainRate values.

655.35. You may or may not have 655.35 in your archive table but depending on units you will have an equivalent value, perhaps in other units, in your database. The fact that you can clear the daily summaries and force regeneration of the output to fix the problem proves this. What units is the 655.35 value being reported in? inches/hour, mm/hour or cm/hour? The units used in a report do not necessarily match the units used in the database. To cut a long story short a default WeeWX install will see the database store rainRate data in inches/hour (rain is stored in inches). You can force WeeWX to use one of two Metric variations where rainRate is stored in mm/hour or cm/hour (rain is stored as mm or cm respectively) but you would need to make specific changes to weewx.conf to do this, most folks don't. Have a look at the target_unit setting under [StdConvert] in weewx.conf. If it is set to US your database will store rainRate in inches/hour, if METRICWX it will be in mm/hour and if METRIC it will be in cm/hour. For example, if your 655.35 value is mm/hour and you have a database using inches/hour then you need to be looking for a value around 25.8 not 655.35.

Also, where are you looking for the value? In the archive table or in the rainRate daily summary table (table name archive_day_rainRate)? The daily summary tables (archive_day_rainRate in this case) are based on data from the archive table so you should not find anything in there that is not in the archive table, but I guess it is possible there could be a disconnect, I can't think of a likely mechanism though.

How are you looking for the value? Assuming you are using a SQLite database and have the sqlite3 package installed (if not install using sudo apt install sqlite3 on Debian based systems) then something like the following will show you the highest rainRate value in your archive table along with the timestamp and date-time it occurred and the unit system used by your database:

$ echo "SELECT dateTime,datetime(dateTime,'unixepoch','localtime'),MAX(rainRate),usUnits FROM archive;" | sqlite3 /var/lib/weewx/weewx.sdb

Note the above command assumes your database is /var/lib/weewx/weewx.sdb (the default for a WeeWX package install). If you did a setup.py install it will likely be /home/weewx/archive/weewx.sdb and you should change the above command accordingly. Note also the usUnits value will tell you what units rainRate is stored in in your database; 1=US=inches/hour, 17=METRICWX=mm/hour and 16=METRIC=cm/hour.

To look at the data in the rainRate daily summary table:

$ echo "SELECT dateTime,datetime(dateTime,'unixepoch','localtime'),MAX(max) FROM archive_day_rainRate;" | sqlite3 /var/lib/weewx/weewx.sdb

You should find the same max rainRate value as found in the archive table but the timestamps/date-times will be different. This is because in the daily summary tables each row holds aggregate data for each day (ie one record per day), whereas the archive table holds each record (ie many records per day).

I would not expect you to find any rainRate data in the log, WeeWX does not record observation data in the log nor does the vantage driver (other drivers or addons may).

The above might help you find the bad data but it isn't going to change anything. Once you identify what bad data is coming in the obvious choice for keeping it from polluting your database is to use the StdQC service to set some min/max values. This can be quite effective for wildly wrong spurious values (eg winds of 300km/hr) but not much use if you are getting a plausible, but wrong value (eg on a warm day of 29C you suddenly get a temperature that is 10C lower than the last). The usefulness of StdQC should be clear once you know what units you are using, but I am guessing a value of 655.35/hour is obviously high irrespective of the units. One thing to remember about StdQC MinMax settings is that they are in the units used in your database unless you explicitly include units.

Gary

[Ernest Jillson]

Thanks for the detailed response!

 

I'll try to elaborate on everything here.

1) Under StdArchive: record_generation = hardware

2) Units are in "inches/hour"

3) First sql query from archive is the same as the second from archive_day_rainRate. 9.44, which while high, is certainly plausible in Florida.

 

I don't know where it's getting that value. It was never written to the database. Same value each time when it happens. But I think your idea of using the StdQC service is a good one.

 

We may just have to leave this one a mystery and move on. Thanks so much for your time!

 

Ernie

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/f583faad-5dde-43be-90c2-acbfaf81ae92n%40googlegroups.com.

[gjr80]

OK, so it's not being calculated by WeeWX (record_generation = hardware) and since you are in Florida I'm guessing the inches/hour you quote refers to both what is being displayed by WeeWX and what is in the database. I don't much like mysteries, well unsolved ones but since it is dead easy to fix when it occurs I would tend to hold off putting in the minmax limit and when it next strikes drop us a line here and while the bad data is showing we can do a bit more digging to see where it is coming from (or at least where it is stored). It will be there somewhere and if it is persistent it will be being stored somewhere. As far as I know Belchertown does nothing with rainRate so it must be coming from WeeWX or the driver.

While I think of it, if not already enabled you could try enabling the Seasons skin (the WeeWX default skin), no need to publish it to a web server you could just look locally at the generated HTML on your WeeWX machine. When the anomaly next appears does Seasons show the same high value as Belchertown?

Gary

[Karen K]

655.35 is a very special value for computers. It is 2^16-1, that is the biggest value to store in a 16 bit large memory. Additionally, if signed values and unsigned values are mixed up, the value could be in reality -1. And additionally again, -1 is sometimes used to mark an erroneous value. So 655.35 is a very suspicious value.

[Ernest Jillson]

I do have the seasons skin installed and it did show up there as well.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/9c359423-3623-43f9-aba4-b6d3f1c31c15n%40googlegroups.com.