Syslog use in thread possibly cause thread exit?

[James]

Hi,

I'm trying to diagnose a problem.  I constructed a water tank monitoring system for our 5 rain water tanks that communicate wireless back to a PC running weewx.  The interface to the PC is an FTDI USB serial port.  I followed the instructions to add my water level inputs to the weewx system and everything was running just fine for months.  The custom service runs a thread to open the serial port and capture raw data via select and readline, that is then passed to the service in a data structure with locking.  The data is filtered and scaled when the new archive record is added.  The problem is that the thread sometimes exits.  I had some syslog messages coming from the thread, and as I added more syslog messages to try to debug the problem, the thread exit problem seemed to get more frequent.  I switched to using a file to log messages from the thread, using logger instead of syslog.  Now, perhaps coincidentally, the thread has been running happily for a few days when previously it had been crashing often more than once a day.

I just have to wonder about the thread safety of the syslog facility?  I'm not very familiar with python, I must admit, so I could be doing something else stupid, but I thought I'd ask the question.

--

JS

[gjr80]

Hi,

Can’t see why syslog is not thread safe, before v4 the WeeWX report engine ran in a separate thread and all logging was done through syslog and I don’t recall there ever being a syslog/thread issue. Likewise a number of WeeWX v3 and earlier 3rd party drivers/services use threading and syslog without issue.

Gary

[James]

Hi Gary,

Thanks.  I've found the cause of the thread exit.  Sometimes a garbled string gets received by the radio comms and it is a problem to parse.

WLST: parsing ST<EA><CA><EA>%bwater_level 1 0x0 water_level 2 0x0135
Traceback (most recent call last):
  File "/usr/share/weewx/user/water_level.py", line 191, in run
    for p in re.split("[,]",line.decode("utf-8")):
  File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xea in position 2: invalid continuation byte
WLST: Exit inner while.

(WLST stands for Water Level Server Thread).

I'm not yet sure why my code didn't catch this exception when I was using syslog yet, but at least I now have a stable system. 

It is lovely to watch the water volume in our tanks increasing when a shower of rain comes over.

Cheers,

James.