weewx-influx crashing

[Tom Corbett]

Hi All,

I'm new to weewx and  python in general so have been plugging away for a couple of weeks getting everything setup and working.

My hardware is an ecowitt WH9200 and I'm using weewx 4.2 on FreeBSD which I setup using the py-setup method. I am capturing the data using the weewx-interceptor in listen mode. This all seems to be flowing fine into the sqlite DB.

I then have the weewx-influx extension sending data to an InfluxDB every minute. This will work fine for several hours; ~ 11 hours being the current record, but will crash with the following in the logs: My take on that is that for some reason a string is being sent to Influx rather than an integer or boolean, however I could be seeing the symptom rather than the cause.

To investigate I checked the record in the sqlite DB immediately after the last one in the InfluxDB, ie the one that caused the crash, and could see nothing that looked different to all the previous one. IE, I couldn't see any strings in the record.

Any help would be appreciated in tracking this down. I'm a bit perplexed as to why it runs happily for hours then has a coniption.

Thanks,

Tom

Here's the relevent section of weewx.conf, nothing too interesting AFAIK.

[[Influx]]
        database = weewx_hades_test
        host = 192.168.178.25
        unit_system = METRICWX

Logs:

Jan 14 10:54:45 28spots kernel: Jan 14 10:54:45 28spots weewx[964] ERROR weewx.restx: Influx: Unexpected exception of type <class 'TypeError'>

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: *** Traceback (most recent call last):

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***   File "/usr/home/weewx/bin/weewx/restx.py", line 475, in post_with_retries

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***     _response = self.post_request(request, data)

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***   File "/usr/home/weewx/bin/user/influx.py", line 498, in post_request

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***     return super(InfluxThread, self).post_request(request, payload)

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***   File "/usr/home/weewx/bin/weewx/restx.py", line 537, in post_request

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***     _response = urllib.request.urlopen(request, data=data_bytes, timeout=self.timeout)

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***   File "/usr/local/lib/python3.7/urllib/request.py", line 222, in urlopen

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***     return opener.open(url, data, timeout)

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***   File "/usr/local/lib/python3.7/urllib/request.py", line 531, in open

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***     response = meth(req, response)

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***   File "/usr/local/lib/python3.7/urllib/request.py", line 641, in http_response

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***     'http', request, response, code, msg, hdrs)

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***   File "/usr/local/lib/python3.7/urllib/request.py", line 569, in error

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***     return self._call_chain(*args)

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***   File "/usr/local/lib/python3.7/urllib/request.py", line 503, in _call_chain

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***     result = func(*args)

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***   File "/usr/local/lib/python3.7/urllib/request.py", line 649, in http_error_default

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***     raise HTTPError(req.full_url, code, msg, hdrs, fp)

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: *** urllib.error.HTTPError: HTTP Error 500: Internal Server Error

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: *** During handling of the above exception, another exception occurred:

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: *** Traceback (most recent call last):

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***   File "/usr/home/weewx/bin/weewx/restx.py", line 381, in run_loop

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***     self.process_record(_record, dbmanager)

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***   File "/usr/home/weewx/bin/weewx/restx.py", line 446, in process_record

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***     self.post_with_retries(_request, data)

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***   File "/usr/home/weewx/bin/weewx/restx.py", line 496, in post_with_retries

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***     self.handle_exception(e, _count + 1)

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***   File "/usr/home/weewx/bin/user/influx.py", line 486, in handle_exception

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: ***     if payload and payload.find("error") >=0:

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] ERROR weewx.restx: *** TypeError: argument should be integer or bytes-like object, not 'str'

Jan 14 10:54:47 28spots kernel: Jan 14 10:54:47 28spots weewx[964] CRITICAL weewx.restx: Influx: Thread terminating. Reason: argument should be integer or bytes-like object, not 'str'

[Tom Keffer]

Normally, I don't deal with the influx driver, but Matthew is swamped so I took a look.

I believe the problem is that the driver was not completely ported to Python 3. The HTTP response needs to be converted from a byte array to a string before performing the find() operation.

Try this version of influx.py.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/d556f993-de2b-4016-b7f8-b396ab32b327n%40googlegroups.com.

[Tom Corbett]

Thanks very much. I have replaced the existing influx.py with your version and restarted weewx. It is putting data into the InfluxDb, so fingers crossed it keeps doing so.

Cheers,

Tom

[Tom Corbett]

This seems to have fixed my issues. Has been diligently putting data in every minute for over a week now. What's the protocol for adding a pull request to Matthew's github so that other Python 3 users don't have this issue? It was your change, but happy to do the pull request if it helps others.

Tom

[Tom Keffer]

Already done!

https://github.com/matthewwall/weewx-influx/pull/27

Thanks for testing it.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/0a55d6f7-e173-4538-910a-338bc08caeaan%40googlegroups.com.

[JJ]

I have been getting a similar error with my attempted weewx influx connection never working with a Raspberry Pi. I get a syslog error message about thread terminating because "POST data should be bytes, an iterable of bytes, or a file object. It cannot be of type str". I am using the Simulator with weewx and I am currently trying the influxdb 2.0 forked code: https://github.com/david-lutz/weewx-influx2

Is it possible that the influx2 code does not have this update that was made to the influx1 code?

[Tom Keffer]

It looks like it does. But, why not use the original code? There have been several modifications since the david-lutz fork.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/79a5c5a2-352f-4778-b5b6-4741f9404ad5n%40googlegroups.com.

[JJ]

Thanks, though that makes the error I am getting even more puzzling. By original code, do you mean this version (https://github.com/matthewwall/weewx-influx)? If so, does the original code support token-based authentication and connect with influxdb 2.0 databases? I thought I would need need to use the david lutz version to send data to an influx 2.0 database with a token, org and bucket: https://github.com/matthewwall/weewx-influx/issues/28

[Tom Keffer]

I don't know. You would have to ask the extension author (Matthew Wall).

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/e253809c-2864-418f-854f-95b058514107n%40googlegroups.com.