Realtime Gauge-Data Extension 0.2.5 'gauge-data.txt' Issues

[tempus]

rtgd-0.2.5 still communicates "cloudbaseunit": "m". See the attached gauge-data.txt file.

Bob

[gjr80]

Well I am running 0.2.5 here and I have 'ft'. Its either an error in the config or rtgd.py is not the current version. What is in your [RealtimeGaugeData] stanza in weewx.conf and when did you download 0.2.5?

Gary

[tempus]

I fixed the problem by adding group_altitude = foot to the weewx.conf options for 'Rtgd'. That isn't included under [[Groups]] at the top of the rtgd.py file, or if it is, I didn't see it.  gauge-data.txt now contains "cloudbaseunit": "ft".

Bob

[tempus]

If air isn't currently moving it obviously doesn't have a current direction of movement, so to report a current direction of movement would be wrong.  It is extremely rare for outdoor air to not be moving, but not so rare for it to be moving at speeds that are too slow to be measured by ordinary weather station sensors.  There are different degrees of "wrong ways" to report wind direction in those "not-so-rare" cases.

Wind that is varying between "too slow to measure" and "barely fast enough to measure" is more apt to continue moving generally in the same direction it was moving at a last measurement than in any other direction, so reporting the last known direction during speed measurement gaps, though wrong in the sense that speed wasn't measured, is a reasonable guess and far "less wrong" than reporting some arbitrary direction, such as North.  Furthermore, wind speed and direction are measured by different sensors.  A direction sensor may still be reporting accurately at speeds too low for a speed sensor to measure, so is it reasonable to assume direction sensor data is wrong merely because a speed sensor wasn't sensitive enough to measure the speed?

The SteelSeries wind-gauge has a data-field labeled "Latest".  The meaning of "latest" is different than "current", such as in "What is the latest information about the suspect's heading?"  To arbitrarily report a suspect is moving from North to South when his current heading is unknown would be misleading and absolutely wrong.  However, reporting the latest information known about his movement, such as "last seen heading East," wouldn't be wrong, even though he might now be moving in some other direction.

The same is true for the wind gauge.  Reporting the wind as coming from North when the direction isn't known is misleading and absolutely wrong, because there is no reasonable basis to believe it is true.  Reporting the latest known direction is "far less wrong," because it is the best guess of current direction and the data display is labeled "Latest" (information).

Bob

[Devonian]

Gary, my gauge-data is also "cloudbaseunit": "m" with 0.2.5 - downloaded a short while ago (file attached)

No other changes and usual stop/start of weewx.

I think the gauges showing zero wind speed and direction of North is misleading too and if it is at all possible, the direction gauge should at least point to the 'last known position' rather than North - pretty much what my weather vane does when it's 'dead calm'.

It's not a show stopper, just misleading ;-)

Nigel.

[gjr80]

Happy to make latest wind direction display last non-None direction.

The cloud base units are based on the realtime gauge data group_altitude config option in weewx.conf:

[RealtimeGaugeData]
[[Groups]]
group_altitude =

Available options are meter and foot. If omitted the default is meter. This affects the cloud base units used in the gauge-data.txt file. Irrespective of the units used in the file, the user is still free to select ft or m on the SteelSeries Weather Gauges page to display (and retain if cookies are enabled) cloudbase in feet or metres.

Gary

[tempus]

Thanks Gary. That will be a big improvement.

Bob

[tempus]

Gary, how much trouble would it be to include "glatest", so an enhanced wind-speed gauge (patterned after the two-pointer wind-direction gauge) could have two pointers with one showing the latest gust-speed and the other the "wlatest" latest wind-speed?

Bob

[gjr80]

Calculating and adding fields is easy; defining and then using them is the trick.

What exactly do you mean by glatest? Given wlatest is the latest windspeed from the station I presume glatest is the latest gust speed from the station? But gust implies a maximum wind speed value over some period of time, gauge-data.txt field wgust already provides this over a sliding 10 minute window using the maximum value of the loop wind speeds seen in the last 10 minutes. The SteelSeries Weather Gauges already displays wgust as the value associated with the 'upper edge' of the light red 'wedge' on the wind speed gauge.

Creating a dual pointer wind speed gauge is beyond rtgd, that will require some SteelSeries Weather Gauges code changes. That will be one to be taken up with the SteelSeries Weather Gauges author.

Gary

[gjr80]

v0.2.6 now uses the last non-None wind direction value for 'Latest' and 'Average' wind directions. Also added a scroller_text config option that allows a fixed string to be displayed in lieu of the forecast (or the 'forecast not available' string). All available config options are now detailed in the comments at the front of rtgd.py.

Gary

[tempus]

I am wanting glatest to be the peak gust over the moving rtgd 'min_interval'. I realize wgust already provides peak gust over a 10-minute moving window, but in contrast, glatest would provide near-real-time gust information if min_interval is made short.

I realize SteelSeries Weather Gauges enhancement is beyond the scope of rtgd.  However, standard installations would ignore glatest and remain compatible and I can provide a modified gauges.js file with a near-real-time gust pointer to weewx users who want it (or ideally to the SS Weather Gauges author if he is willing to incorporate it for weewx users).  Unrelated to the gust issue, I have fixed gauges.js coding mistakes that should be submitted to him anyway.

Bob

[tempus]

Terrific!  v0.2.6 is installed and working as advertised.  Thanks much.

Bob

[gjr80]

On Thursday, 23 February 2017 17:55:12 UTC+10, tempus wrote:

I am wanting glatest to be the peak gust over the moving rtgd 'min_interval'. I realize wgust already provides peak gust over a 10-minute moving window, but in contrast, glatest would provide near-real-time gust information if min_interval is made short.

I really am having a hard time coming to grips with this. I understand your definition of glatest; but glatest would provide near-real-time gust information if min_interval is made short? I don't see that a 10 min gust calculated now or a 30 second gust calculated now are any more or less 'near-real time'. One uses a smaller window than the other but they are both just as near real time as each other. If decreasing min_interval makes glatest 'more near real-time' does reducing it to 0 may it as real time as it gets? On one hand you seem be advocating pumping out wind data as fast as you can (which implies min_interval == 0) but on the other hand you want max(windSpeed) over min_interval. On a Vantage, and I suspect most all of the other stations, glatest=wlatest for min_interval <= loop period.

Nothwithstanding, I will look at doing adding it.All of the data is being cached so it should be easy enough to do.

I realize SteelSeries Weather Gauges enhancement is beyond the scope of rtgd.  However, standard installations would ignore glatest and remain compatible and I can provide a modified gauges.js file with a near-real-time gust pointer to weewx users who want it (or ideally to the SS Weather Gauges author if he is willing to incorporate it for weewx users).  Unrelated to the gust issue, I have fixed gauges.js coding mistakes that should be submitted to him anyway.

No problems, just wanted to make it clear I wasn't touching gauges.js.

Gary

[Devonian]

I noticed 0.2.6 late last night and installed it.

Works fine, but no chance to see a 'no wind' situation as it's blowing a hooligan outside with gust predicted to 70MPH later...

Nigel.

[Bill Morrow]

Gary, I tried rtgd (0.2.6) last night for the first time, and have an issue. My driver yields packets which don't always contain the same data. I have a station which provides indoor conditions and a separate one that provides outdoor conditions. If the indoor station reports first, I get an error because outTemp is not in the packet. See the log extract below.

In your opinion, should I handle this in my driver, is there something I can do in weewx, or should rtgdthread handle this? The indoor station reports every 2-3 seconds, To conserve power, the outdoor about every 30 seconds.

Feb 22 16:20:46 walrus weewx[29153]: wxMesh: key: TIME value: 1487794846
Feb 22 16:20:46 walrus weewx[29153]: wxMesh: key: INTE value: 22.60
Feb 22 16:20:46 walrus weewx[29153]: wxMesh: key: INHU value: 32.59
Feb 22 16:20:46 walrus weewx[29153]: rtgd: queued loop packet: {'barometer': None, 'windchill': None, 'dewpoint': None, 'humidex': None, 'maxSolarRad': None, 'pressure': None, 'altimeter': None, 'usUnits': 17, 'appTemp': None, 'rainRate': 0.0, 'heatindex': None, 'dateTime': 1487794846.0, 'inHumidity': 32.59, 'inTemp': 22.600000000000005, 'cloudbase': None, 'inDewpoint': 41.58132605144329}
Feb 22 16:20:46 walrus weewx[29153]: rtgdthread: received packet: {'barometer': None, 'windchill': None, 'dewpoint': None, 'humidex': None, 'maxSolarRad': None, 'pressure': None, 'altimeter': None, 'usUnits': 17, 'appTemp': None, 'rainRate': 0.0, 'heatindex': None, 'dateTime': 1487794846.0, 'inHumidity': 32.59, 'inTemp': 22.600000000000005, 'cloudbase': None, 'inDewpoint': 41.58132605144329}
Feb 22 16:20:46 walrus weewx[29153]: rtgdthread: Unexpected exception of type <type 'exceptions.KeyError'>
Feb 22 16:20:46 walrus weewx[29153]: *** Traceback (most recent call last):
Feb 22 16:20:46 walrus weewx[29153]: ***   File "/usr/share/weewx/user/rtgd.py", line 773, in run
Feb 22 16:20:46 walrus weewx[29153]: ***     self.process_packet(_package['payload'])
Feb 22 16:20:46 walrus weewx[29153]: ***   File "/usr/share/weewx/user/rtgd.py", line 790, in process_packet
Feb 22 16:20:46 walrus weewx[29153]: ***     self.buffer.setLowsAndHighs(packet)
Feb 22 16:20:46 walrus weewx[29153]: ***   File "/usr/share/weewx/user/rtgd.py", line 1614, in setLowsAndHighs
Feb 22 16:20:46 walrus weewx[29153]: ***     outTemp = packet_d['outTemp']
Feb 22 16:20:46 walrus weewx[29153]: *** KeyError: 'outTemp'

[gjr80]

Yes, its on the to-do list.

Gary

[tempus]

I have the opposite problem here.  Why is it that there are gale-force winds when I want to go fishing and winds practically stop when I want to test wind-speed software?

Bob

[tempus]

You can see it working here:

http://www.lablibrary.com/ss/

Bob

[tempus]

With a 10-minute gust window, a high-velocity-gust 10-minutes ago still determines the indicated gust level, even though gust velocities since that time may have been much lower.  The gust indication in that case is 10-minute-old information.  I am wanting gust information that is much closer to real time.  A min_interval <= loop-period with min_interval set to 3-seconds may be too short to provide useful data, because of the high-frequency filtering-effect of cup-rotational-inertia.  If so, some small multiple of that interval can be used instead.  What I am wanting is a gust-window that is much shorter than 10-minutes, but of course long enough to differentiate between very recent gust velocities and the periodic latest-point-in-time wlatest velocity samples.

Bob

[tempus]

Gary, after carefully watching data from an anemometer and thinking about this issue more, I have decided that a 10-minute moving gust-window is more reasonable than I had thought. So unless you are already underway with what I described below, let's forget about it.

Bob

[gjr80]

Agreed.

Gary

[gjr80]

I've published v0.2.7 which now caches loop packets, shoudl be a much better experience for those with partial loop packet stations.

Gary

On Thursday, 23 February 2017 23:40:09 UTC+10, Bill Morrow wrote:

[Bill Morrow]

It works for me, thank you Gary. 

It has been essentially 100% humidity here for about a day, and my humidity sensor is sending data that weewx rejects. I think that's the reason I am getting this error:

Feb 26 10:26:08 weewx[22818]: rtgdthread: **** Traceback (most recent call last):
Feb 26 10:26:08 weewx[22818]: rtgdthread: ****   File "/usr/share/weewx/user/rtgd.py", line 851, in process_packet
Feb 26 10:26:08 weewx[22818]: rtgdthread: ****     data = self.calculate(cached_packet)
Feb 26 10:26:08 weewx[22818]: rtgdthread: ****   File "/usr/share/weewx/user/rtgd.py", line 989, in calculate
Feb 26 10:26:08 weewx[22818]: rtgdthread: ****     humTL = min(i for i in [self.buffer.humL_loop[0], self.day_stats['outHumidity'].min] if i is not None)
Feb 26 10:26:08 weewx[22818]: rtgdthread: **** ValueError: min() arg is an empty sequence

I don't consider this a problem with rtgdthread. I should not be throwing it bad data.

[gjr80]

Could you give me some more details Bill. There is a bug in the code if that error is being thrown, irrespective of what is going on with outHumidity. That trace is telling me there is no outHumidty information for the entire day (to 10:26am) which seems odd. When you say weeWX is rejecting your data is that due to a QC setting or is your driver just not emitting outHumidity? Just want to understand what is going on so I can harden up the code.

Gary

[Bill Morrow]

Weewx was rejecting readings around minus 24% (yes, -24%) because they were outside my QC settings in StdQC of 0% to 100%. It's a problem with the humidity sensor when the humidity is extreme.

It did this for about a day, while we had a lot of fog and rain.

I widened out the QC limits

[StdQC]
    [[MinMax]]
...
        outHumidity = -30, 100

and it stopped erroring on outHumidity. There were a couple on dewpointTL after that:

Feb 26 10:31:18 weewx[23716]: rtgdthread: **** Traceback (most recent call last):
Feb 26 10:31:18 weewx[23716]: rtgdthread: ****   File "/usr/share/weewx/user/rtgd.py", line 851, in process_packet
Feb 26 10:31:18 weewx[23716]: rtgdthread: ****     data = self.calculate(cached_packet)
Feb 26 10:31:18 weewx[23716]: rtgdthread: ****   File "/usr/share/weewx/user/rtgd.py", line 1022, in calculate
Feb 26 10:31:18 weewx[23716]: rtgdthread: ****     dewpointTL = min(i for i in [dewpointL_loop, dewpointTL] if i is not None)
Feb 26 10:31:18 weewx[23716]: rtgdthread: **** ValueError: min() arg is an empty sequence

[gjr80]

Published v0.2.8 which should gracefully handle missing historical data when calculating day max/mins.

Gary