BloomSky traceback after IndexError

[Rick M0LEP]

Occasionally I see the BloomSky data collector fail. When it does so I
get a traceback like this in user.log:

Jun 23 03:50:06 duma weewx[19415]:
bloomsky: bloomsky-client: Exception:
Traceback (most recent call last):
File "/usr/share/weewx/user/bloomsky.py", line 674, in run
self.client.collect_data()
File "/usr/share/weewx/user/bloomsky.py", line 486, in collect_data
data = ApiClient.extract_data(raw_data)
File "/usr/share/weewx/user/bloomsky.py", line 530, in extract_data
if item in data[0]:
IndexError: list index out of range

Once this has happened weewx needs to be stopped and re-started before
data collection resumes.

Using bloomsky-1.0.0.tar.gz from github with weewx v3.9.1

--
73, Rick, M0LEP

[gjr80]

Hi,

It would appear that at times the driver is getting a response from the bloomsky API but that response is empty. I have modified the driver to catch the exception, wait for the retry_wait period (10 sec by default) and then try again. If it continues to fail after 3 attempts that download is skipped. The failures will be logged. this won't fix the problem but the driver will now tolerate the error.

Could you replace your current bloomsky driver with the new version. To do this:

1. on your WeeWX machine move aside the current bloomsky driver:

$ sudo my /usr/share/weewx/user/bloomsky.py /usr/share/weewx/user/bloomsky_orig.py

2. download the revised bloomsky driver to your WeeWX machine:

$ wget -P /usr/share/weewx/user https://raw.githubusercontent.com/gjr80/weewx-bloomsky/master/bin/user/bloomsky.py

3. restart WeeWX

4. monitor the log for correct operation

I am not sure how often you were seeing the IndexError, but you might want to browse your logs every now and then, if the error occurs you will see lines similar to:

Jun 23 23:20:34 stretch14 weewx[1001]: bloomsky: bloomsky-client: Invalid data received on attempt 1 of 3: list index out of range

Would be good to get some feedback, I have not had this issue raised by anyone previously plus I don't use the driver myself other than for testing. hopefully not a sign that the Bloomsky API is becoming flaky.

Gary

[Rick M0LEP]

On Sun 23 Jun gjr80 wrote:
> It would appear that at times the driver is getting a response from
> the bloomsky API but that response is empty. I have modified the
> driver to catch the exception, wait for the retry_wait period (10 sec
> by default) and then try again. If it continues to fail after 3
> attempts that download is skipped. The failures will be logged. this
> won't fix the problem but the driver will now tolerate the error.

Thanks. Code replaced as instructed. :)

> I am not sure how often you were seeing the IndexError,

Not frequently, but often enough for me to notice cause and effect.
There were a couple of instances on June 20th and one more early this
morning. I'll keep an eye on the logs...

> Would be good to get some feedback, I have not had this issue raised
> by anyone previously plus I don't use the driver myself other than for
> testing. hopefully not a sign that the Bloomsky API is becoming flaky.

I've had a few issues with my SKY2 showing as offline on the BloomSky
site even though I know my broadband connection's been up solidly.
There's a possibility that's because my SKY2's near the limit for my
WiFi, and may be dropping off it at times. It's possible the empty
responses come back as the SKY2's dropping off.

--
73, Rick, M0LEP

[gjr80]

Thanks for the background. I might do some testing with my BloomSky and take it offline, I must admit I did not do that test when I wrote the driver. Unfortunately, that won't be possible for a few months. Should not matter though as the updated driver you have now should catch the error, do keep an eye on your logs and let me know if the driver causes WeeWX to fail again.

Gary

[Rick M0LEP]

On Mon 24 Jun gjr80 wrote:
> Thanks for the background. I might do some testing with my BloomSky
> and take it offline, I must admit I did not do that test when I wrote
> the driver. Unfortunately, that won't be possible for a few months.
> Should not matter though as the updated driver you have now should
> catch the error, do keep an eye on your logs and let me know if the
> driver causes WeeWX to fail again.

The following traceback in the logs this morning.

Jun 25 05:58:40 duma weewx[10623]:

bloomsky: bloomsky-client: Exception:
Traceback (most recent call last):

File "/usr/share/weewx/user/bloomsky.py", line 684, in run
self.client.collect_data()
File "/usr/share/weewx/user/bloomsky.py", line 485, in collect_data
raw_data = self.sd.get_data()
File "/usr/share/weewx/user/bloomsky.py", line 708, in get_data
params, headers)
File "/usr/share/weewx/user/bloomsky.py", line 751, in get_request
"Failed to get BloomSky API data")
TypeError: logerr() takes exactly 1 argument (2 given)

--
73, Rick, M0LEP

[gjr80]

Hmm, my apologies, not very good form on my part. I have just pushed a commit to fix this problem, you just need to re-install bloomsky.py as follows:

1. on your WeeWX machine delete the current bloomsky driver:

$ sudo rm /usr/share/weewx/user/bloomsky.py

2. download the revised bloomsky driver to your WeeWX machine:

$ wget -P /usr/share/weewx/user https://raw.githubusercontent.com/gjr80/weewx-bloomsky/master/bin/user/bloomsky.py

3. restart WeeWX

Gary

[Rick M0LEP]

On Tue 25 Jun gjr80 wrote:
> Hmm, my apologies, not very good form on my part. I have just pushed a
> commit to fix this problem, you just need to re-install bloomsky.py as
> follows:

Thanks. Fix made. This time the exception was co-incident with a brief
break in broadband connection.

--
73, Rick, M0LEP

[Rick M0LEP]

On Tue 25 Jun Rick M0LEP wrote:
> Thanks. Fix made. This time the exception was co-incident with a brief
> break in broadband connection.

Another day, another break in Net connectivity, another exception:

Jun 28 05:58:42 duma weewx[27097]:
bloomsky: bloomsky-client: **** <urlopen error [Errno -3] Temporary
failure in name resolution>

bloomsky: bloomsky-client: Exception:
Traceback (most recent call last):
File "/usr/share/weewx/user/bloomsky.py", line 684, in run
self.client.collect_data()
File "/usr/share/weewx/user/bloomsky.py", line 485, in collect_data
raw_data = self.sd.get_data()
File "/usr/share/weewx/user/bloomsky.py", line 708, in get_data
params, headers)

File "/usr/share/weewx/user/bloomsky.py", line 753, in get_request
resp_json = json.loads(resp)
UnboundLocalError: local variable 'resp' referenced before assignment

Not yet encountered a repeat of the issue that started this thread, but
I'm keeping a watch for it and I'll report what the new code does when
it happens...

--
73, Rick, M0LEP