Crazy windSpeed from BloomSky when STORM's battery goes flat

[Rick M0LEP]

I have weewx getting data from my BloomSky SKY-2/STORM, and most of the
time it seems to work pretty well. However, the last few days have been
decidedly not sunny, and the STORM's solar panel has not managed to put
sufficient charge into its battery to lastthe night. When the STORM's
battery gives out it stops sending data, and I start seeing the
following in my logs:

Nov 25 19:25:49 host weewx[7451]: engine: 2019-11-25 19:25:49 UTC
(1574709949) LOOP value 'windSpeed' 22367.1815775 outside limits (0.0,
120.0)
Nov 25 19:28:20 host weewx[7451]: engine: 2019-11-25 19:28:20 UTC
(1574710100) LOOP value 'windSpeed' 22367.1815775 outside limits (0.0,
120.0)
Nov 25 19:30:51 host weewx[7451]: engine: 2019-11-25 19:30:51 UTC
(1574710251) LOOP value 'windSpeed' 22367.1815775 outside limits (0.0,
120.0)

I'm not sure where that 22367.1815775 value is coming from, but it's way
outside the limits, but it gets put into the database anyway, and I keep
having to clean the over-high values out and re-build the historical
tables.

Can I do anything to stop out-of-range values from going into the
database in the first place?

--
73, Rick, M0LEP

[vince]

On Monday, November 25, 2019 at 12:13:02 PM UTC-8, Rick Hewett wrote:

I'm not sure where that 22367.1815775 value is coming from, but it's way
outside the limits, but it gets put into the database anyway, and I keep
having to clean the over-high values out and re-build the historical
tables.

Can I do anything to stop out-of-range values from going into the
database in the first place?

Those messages say that the [[StdQC]] min/max values in your weewx.conf 'did' stop them from going into the database, or it should have prevented it.

Which version of weewx and did you verify with a db query that they're indeed not getting blocked from getting into the db ?

[Rick M0LEP]

On Mon 25 Nov vince wrote:
> Those messages say that the [[StdQC]] min/max values in your
> weewx.conf 'did' stop them from going into the database, or it should
> have prevented it.
>
> Which version of weewx and did you verify with a db query that they're
> indeed not getting blocked from getting into the db ?

Ah! The 22367.1815775411 isn't going into windSpeed, but it is going
into windGust, and because those are both plotted on the same graph...

....so somewhere I need to add a limit for windGust as well.

--
73, Rick, M0LEP

[Rick M0LEP]

On Mon 25 Nov Rick M0LEP wrote:
> Ah! The 22367.1815775411 isn't going into windSpeed, but it is going
> into windGust, and because those are both plotted on the same graph...
>
> ....so somewhere I need to add a limit for windGust as well.

....and there's a similar (but less noticeable) issue with rain and
rainRate.

I'm running 3.9.1.

--
73, Rick, M0LEP

[gjr80]

Judging by your first post the issue is hardware (and not driver) related. In that case the best you can do is apply some QC minmax limits via StdQC. windGust is taken as the max windSpeed value seen so you will need to minmax both. Likewise for any other obs that are wildly incorrect.

Of course if you are seeing sensible values on the Bloomsky app when WeeWX is showing nonsense values that is another issue and will require further investigation.

Gary

[vince]

I told you where......look in your weewx.conf for StdQC, add a line there for windGust, and stop/restart weewx.

If you have junk in your db, you'll have to clear that up.  There were a couple threads that Tom answered within the last few days with pointers to how to do that.

 

[Rick M0LEP]

On Mon 25 Nov gjr80 wrote:
> Judging by your first post the issue is hardware (and not driver) related.
> In that case the best you can do is apply some QC minmax limits via StdQC

> <http://weewx.com/docs/usersguide.htm#StdQC>. windGust is taken as the max

> windSpeed value seen so you will need to minmax both. Likewise for any
> other obs that are wildly incorrect.
>
> Of course if you are seeing sensible values on the Bloomsky app when WeeWX
> is showing nonsense values that is another issue and will require further
> investigation.

When the STORM's battery is flat it stops sending anything to BloomSky,
and the API call doesn't return any of the STORM fields. Here's a dump
I've just taken (slightly edited for identity etc.) showing all the
fields I'm getting back while the STORM is MIA:

city_name ***
outdoor {'data_timestamp': '2019-11-26T10:29:59+00:00', 'pressure':
28.82, 'image_timestamp': '2019-11-26T10:29:59+00:00', 'temperature':
53.69, 'luminance': 9999, 'is_night': False, 'uv_index': 9999,
'humidity': 97, 'voltage': 2523, 'device_type': 'SKY2', 'is_raining':
False, 'image_url': 'http://s3-us-west-1.amazonaws.com/bskyimgs/***.jpg'}
full_address ***, England, GB
device_name ***
utc_offset 0
device_id ***
is_dst False
longitude -2.***
is_searchable True
latitude 51.***
registered_timestamp 2019-05-16T09:17:10+00:00
altitude 129.0
indoor {'temperature': None, 'humidity': None}
favorites_count None
video_urls ['http://s3.amazonaws.com/bskytimelapses/***.mp4', ***]
followers_count 0
street_name ***

--
73, Rick, M0LEP

[Rick M0LEP]

On Tue 26 Nov vince wrote:
> I told you where......look in your weewx.conf for StdQC, add a line
> there for windGust, and stop/restart weewx.

Thanks, I'll try adding limits for windGust, rain and rainRate.

> If you have junk in your db, you'll have to clear that up. There were
> a couple threads that Tom answered within the last few days with
> pointers to how to do that.

Yeah, done that more than once. Thanks.

--
73, Rick, M0LEP

[Rick M0LEP]

Not sure, now, whether that quick dump is digging deep enough to find
the STORM's data...

[gjr80]

The bloomsky driver should return data as per the BloomSky API response, so if the API response has nonsense data then the driver will return nonsense data. If the API response includes no data then the driver should return None or nothing. It is quite possible there are issues with the driver as I do not recall testing the driver against a non-responding Storm (I don't have a Storm). I will have a look at the driver to see if it can be improved. The driver can be run standalone to return the raw Bloomsky API response, something like:

$ PYTHONPATH=/home/weewx/bin python /home/weewx/bin/user/bloomsky.py --get-json-data

or

$ PYTHONPATH=/usr/share/weewx python /usr/share/weewx/user/bloomsky.py --get-json-data

Is that how you obtained the data above? If not the output from the --get-json-data when the STORM is misbehaving would help.

Gary

[Rick M0LEP]

On Wed 27 Nov gjr80 wrote:
> Is that how you obtained the data above? If not the output from the
> --get-json-data when the STORM is misbehaving would help.

No, I was using a script that uses BloomSkyAPIClient, and I've a
feeling that may be somewhat older than the present BloomSky driver in
weewx.

> $ PYTHONPATH=/usr/share/weewx python /usr/share/weewx/user/bloomsky.py
> --get-json-data

When I try this I get an error thus:

Traceback (most recent call last):
File "/usr/share/weewx/user/bloomsky.py", line 827, in <module>
main()
File "/usr/share/weewx/user/bloomsky.py", line 798, in main
get_json_data(opts.api_key)
File "/usr/share/weewx/user/bloomsky.py", line 823, in get_json_data
raw_data = api_client.sd.get_data()
File "/usr/share/weewx/user/bloomsky.py", line 708, in get_data
params, headers)
File "/usr/share/weewx/user/bloomsky.py", line 720, in get_request
obfuscated['Authorization'] = ''.join(("....",
obfuscated['Authorization'][-4:]))
TypeError: 'NoneType' object has no attribute '__getitem__'

The python command is ponting at python2.7

I tried using python3 but that didn't like the weewx modules.

--
73, Rick, M0LEP

[gjr80]

Whoops, the development version of the BloomSky driver picks up the api key from weewx.conf, the current release version does not. Try adding —api-key=YOUR_API_KEY to the command line you are using.

Gary

[Rick M0LEP]

On Wed 27 Nov gjr80 wrote:

> Whoops, the development version of the BloomSky driver picks up the
> api key from weewx.conf, the current release version does not. Try

> adding --api-key=YOUR_API_KEY to the command line you are using.

I'd just figured that out from burrowing into the code. Of course, with
the sun up (and my having given the STORM a 100% charge overnight via
USB) the STORM is now reporting nicely, so this is roughly what it
looks like when it's working:

[
{
"ALT": 129.0,
"BoundedPoint": "",
"CityName": "***",
"DST": 0,
"Data": {
"DeviceType": "SKY2",
"Humidity": 99,
"ImageTS": 1574847464,
"ImageURL": "http://s3-us-west-1.amazonaws.com/bskyimgs/****vmZionpk=.jpg",
"Luminance": 9999,
"Night": false,
"Pressure": 967,
"Rain": false,
"TS": 1574847464,
"Temperature": 7.8,
"UVIndex": 9999,
"Voltage": 2610
},
"DeviceID": "***",
"DeviceName": "Hillside",
"FullAddress": "***, ***, England, GB",
"LAT": 51.***,
"LON": -2.***,
"NumOfFollowers": 0,
"Point": {},
"PreviewImageList": [
"http://s3-us-west-1.amazonaws.com/bskyimgs/****rmJakoJU=.jpg",
"http://s3-us-west-1.amazonaws.com/bskyimgs/****sl5KnoJ4=.jpg",
"http://s3-us-west-1.amazonaws.com/bskyimgs/****tlZiooJo=.jpg",
"http://s3-us-west-1.amazonaws.com/bskyimgs/****tnpWnmpU=.jpg",
"http://s3-us-west-1.amazonaws.com/bskyimgs/****unZGsnp4=.jpg"
],
"RegisterTime": 1557998230,
"Searchable": true,
"Storm": {
"24hRain": 1.6,
"RainDaily": 0.0,
"RainRate": 0.0,
"SustainedWindSpeed": 0.62,
"UVIndex": "1",
"WindDirection": "SE",
"WindGust": 1.31
},
"StreetName": "***",
"UTC": 0,
"VideoList": [
"http://s3.amazonaws.com/bskytimelapses/***2_0_2019-11-22.mp4",
"http://s3.amazonaws.com/bskytimelapses/***2_0_2019-11-23.mp4",
"http://s3.amazonaws.com/bskytimelapses/***2_0_2019-11-24.mp4",
"http://s3.amazonaws.com/bskytimelapses/***2_0_2019-11-25.mp4",
"http://s3.amazonaws.com/bskytimelapses/***2_0_2019-11-26.mp4"
],
"VideoList_C": [
"http://s3.amazonaws.com/bskytimelapses/***2_0_2019-11-22_C.mp4",
"http://s3.amazonaws.com/bskytimelapses/***2_0_2019-11-23_C.mp4",
"http://s3.amazonaws.com/bskytimelapses/***2_0_2019-11-24_C.mp4",
"http://s3.amazonaws.com/bskytimelapses/***2_0_2019-11-25_C.mp4",
"http://s3.amazonaws.com/bskytimelapses/***2_0_2019-11-26_C.mp4"
]
}
]

I'll try to grab the data next time the STORM's battery goes flat.

--
73, Rick, M0LEP

[gjr80]

Good, thanks. That will give me some time to juggle drivers; I have three versions at various states of completeness, the current release, a multi-id version and a WeeWX 4.0/python2/3 version. Need some time to massage them together and get a WeeWX 3 version and a WeeWX 4 version.

Gary

[Rick M0LEP]

On Wed 27 Nov gjr80 wrote:

> Good, thanks. That will give me some time to juggle drivers;

Sounds like fun... ;)

I figure, unless we have some good sunny mornings, the STORM's battery
will be flat again inside a week. It took a bit over four dull grey days
to drop from full to shut-down this week.

The STORM's solar panel coped OK during the brighter months, but now
we've got to the time of the year when, even if the Sun were to shine
all day, the panel on mine would only get two or three hours of direct
sunshine, and that mostly at a low angle. There's a forest on the
hillside to the south of me, so there's not much chance of direct
sunlight in the afternoon. I may have to try pointing the panel East
rather than South (the geometry of the mounting doesn't allow for
anything in between) and setting it so the panel's nearer vertical to
try to catch any morning sun better.

--
73, Rick, M0LEP