[RFC] Support for the Oregon Scientific WMR89

[Ray Kinsella]

Hi folks,

I could not see a formal procedure to submit a patch for weewx.

So I though I would add it here for comments. 

This is a request for comments, for driver support for the WMR89a Weather 

Station from Oregon Scientific. The actual sensor data is captured an Arduino 

sketch, and then relayed to WeeWX over a USB Serial connection. 

This driver does *not* connect directly to the WMR89 basestation, so any stats

calculated by the basestation are not collected.

WRM89 sensors supported

 * Oregon-THGR810 Temp-Hygro

 * Oregon-WGR800 Anemometer

This is the initial RFC, circuit layout and Rainsensor support to follow. 

Ray K

[mwall]

ray,

nice work!

you might want to do this as a standalone driver instead of try to merge the functionality into the wmr9x8 driver.

it probably means quite a bit of code duplication, at least for the first implementation.  but there is already a lot of duplication between the wmr drivers (other than wmr300 - it is a different beast altogether).

by any chance have you been able to communicate with the wmr89 via usb?  i cannot even get the oregon scientific software to run, let alone sniff whatever it does on the usb.

m

[Ray Kinsella]

Thanks M!

On Tuesday, January 2, 2018 at 1:17:46 AM UTC, mwall wrote:

you might want to do this as a standalone driver instead of try to merge the functionality into the wmr9x8 driver.

Presume this is reduce the risk of messing up anyone using the wm918 and wm918 after upstreaming?

I cleaned up the original wmr9x8.py driver a bit,

* removing most of the obvious duplicated code.

* making the serial handling code more robust

* aggregating all the model specific magic numbers in model_settings etc.

 

My initial feeling was the there was enough similarity with the wm89, wm918 etc to justify trying to reuse?

it probably means quite a bit of code duplication, at least for the first implementation.  but there is already a lot of duplication between the wmr drivers (other than wmr300 - it is a different beast altogether).

 

by any chance have you been able to communicate with the wmr89 via usb?  i cannot even get the oregon scientific software to run, let alone sniff whatever it does on the usb.

Nope, its a complete horror show.

I tried playing with the windows software a bit, sniffing the USB etc, it was a struggle - I made very little progress with it.

I figured out it was ultimately going to be counter productive, as the 'Main Unit' - the monitoring base station was silently dropping samples pretty consistently. At times the Wind Speed on the LCD wouldn't update on the base station for minutes at a time, meanwhile I was seeing a rock solid constant data stream through WeeWx with Arduino + RXB6. The antenna in the base station mustn't be up to much, but from the LCD you have no way of knowing. 

 

m

[mwall]

ray,

'marunio' over at wxforum figured out that the baud rate on the wmr89 is non-standard.  that is why no one has been able to make any sense of the comms.

now that we know the baud rate (128000) it should be pretty easy to make a weewx driver for the wmr89.  it looks like the communications protocol is similar to those of other wmr stations.

if you could do some usb captures at the 128000 baud rate, that would help to cover the corner cases.

m

[andr3id]

Hi!

Sorry for just posting here, I'm new to this group...

Anyway, with the information found on wxforum, I've managed to modify a linux kernel driver to make the WMR89 show up as a USB serial interface in Linux.

I've also been able to receive some data from the WMR89 using pyserial-asyncio, but since I'm new to the weather data stuff, I still have to decode it and there are hints on the wxforum. This project however seems to focus on that and visualize the data so it catched my interest.

The point is that adding support for WMR89 to this project is doable.

[mwall]

On Wednesday, January 17, 2018 at 5:32:07 PM UTC-5, andr3id wrote:

Anyway, with the information found on wxforum, I've managed to modify a linux kernel driver to make the WMR89 show up as a USB serial interface in Linux.

welcome!  could you post the instructions for what you did to modify the kernel driver?  based on conversations with marunio, i wrote a wmr89 driver that *should* properly decode data.  unfortunately, all of the hardware i have at hand has serial drivers that refuse to permit the non-standard baud.

my work thus far is here:

https://github.com/matthewwall/weewx-wmr89

this should be an easy one to add to the weewx core.

m 

[andr3id]

On Wednesday, January 17, 2018 at 11:55:23 PM UTC+1, mwall wrote:

On Wednesday, January 17, 2018 at 5:32:07 PM UTC-5, andr3id wrote:

Anyway, with the information found on wxforum, I've managed to modify a linux kernel driver to make the WMR89 show up as a USB serial interface in Linux.

welcome!  could you post the instructions for what you did to modify the kernel driver?  based on conversations with marunio, i wrote a wmr89 driver that *should* properly decode data.  unfortunately, all of the hardware i have at hand has serial drivers that refuse to permit the non-standard baud.

I will try to sum up what I've did in a future post. It was just a quick hack to make it work and not really a great solution.

Here are some hints if you want to try:

- You'll need all the tools to be able to build the kernel, the linux headers and the source code for the running kernel.

- The driver that WMR89 uses is https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/tree/drivers/usb/serial/cp210x.c?h=v4.14.14. That is just an example to find it in the path.

- For simplicity, I've just made a copy and pretended that it was a new kernel module and I've used information online for how to build the driver and load it to the running kernel.

This was the messy process and it's not fail proof for future kernel updates, but my focus was just to make it work. The goal was to have the weather station show up as /dev/ttyUSB0 for example. I'm not a kernel developer so I have to figure out how this process should be handled in a better way. There are instructions online on how to build an own kernel module but unfortunately not all of them are great.

- The driver has to be patched by adding usb_device_id for WMR89, as stated in the wxforum. Basically add a line with { USB_DEVICE(<vendor id>, <product id>) }, to the device id table in the driver file. I was also mentioned that maybe the non standard baud rate 128000 that is used, should be mapped to some other standard value. I haven't done that since it wasn't needed for pyserial, but maybe it is needed for other serial communication libraries.

- I had to load the usbserial module before loading the built module for WMR89. Use dmesg to look for error messages.

[andr3id]

Oh, I'b a bit behind... I've probably should have read your work before posting :)

If this works then the patch is probably not needed for having WMR89 show up as /dev/ttyUSB0. 

sudo sh -c 'echo 0fde ca0a > /sys/bus/usb-serial/drivers/cp210x/new_id

But maybe the hints are useful to map a standard baud rate, for example 50 as mentioned in the wxforum, to 128000.

[Ray Kinsella]

Hey all,

This is great news ... really great!

It should be very possible in that case to reuse the driver I have been developing for the WMR89.

The arduino sketch I was developing was just dumping the sensor data to the wire anyway. 

If that is all the base station is doing, it should be possible to reuse as is. 

Going to send a tidied up v2 next week, that should be perfect for this purpose. 

BTW I don't think the idea of using the Arduino is completely dead as in my experience the base station is pretty lossey.

But decoding the base station is definitely the the first priority!

Ray K

[andr3id]

Matthew, I now understand what you mean, because I also have the problem with baud rate on a raspberry pi running raspbian with kernel 4.1.19+.

What is working on another computer with ubuntu, doesn't work on the RPi. The baud rate 128000 can not be set. I will investigate a bit and see if I can make it work. I'm not sure if it works due to a newer kernel or if it's something else.

I've tried the weewx-wmr89 driver and it worked on my computer, but the received data wasn't the same as on the weather station display. I will investigate this when I have more time. Maybe I just have the weewx configuration messed up.

[andr3id]

I did a bit of quick research and found out a couple of things.

The cp210x driver seems to not be able to handle TIOCGSERIAL and TIOCSSERIAL ioctl wich happens to be used by all serial communication programs that I have on RPi, to set the non standard baud rate. I have an older version of pyserial on the RPi and it also uses the same ioctl to set the baud rate. pyserial on my VM with ubuntu that I've used to communicate with WMR89 uses other ioctl that seems to be supported by the driver.

This is ofcourse a bit of guessing so I'll try to make it work somehow, either by trying with a newer version of pyserial on the RPi or trying to fix the driver. I've found this patched driver which might work https://github.com/GBert/misc/blob/master/cp2102-dcc/src/cp210x.c but it would be nicer if it worked with a new pyserial and not mess around with the device driver.

[andr3id]

It worked with a newer version of Python and a newer pyserial. I've build python 3.6.4 on the RPi and then installed pyserial through the built-in pip tool. To test the weewx driver I have to fix a newer Python 2 on RPi and then I'll spend some time to verify the data.

[mwall]

On Friday, January 26, 2018 at 4:34:07 PM UTC-5, andr3id wrote:

It worked with a newer version of Python and a newer pyserial. I've build python 3.6.4 on the RPi and then installed pyserial through the built-in pip tool. To test the weewx driver I have to fix a newer Python 2 on RPi and then I'll spend some time to verify the data.

ray, thanks for grinding through this!

weewx currently targets only python 2.  more specifically, weewx is fully supported on 2.6 and 2.7, with conditional support for 2.5.

it probably is not a problem if we require users to have python 2.7 in order to use this oregon scientific hardware, but python 3 is out for now.

do you know yet whether this is a python issue versus a pyserial issue versus the cp21x kernel driver?

m

[mwall]

oops!  andr3id not ray!

[andr3id]

mwall, as I've stated before, the cp210x driver can not handle the ioctl requests TIOCGSERIAL and TIOCSSERIAL which many serial communication clients use and the fallback in the tty driver seems to not allow the baud rate 128000 to be set. The baud rate can however be set with other ioctl requests but the pyserial module that works with the older versions of Python, for example 2.7.3, uses the ioctl requests that are not supported by the cp210x driver.

Python is not a problem here but the version of the pyserial that works may have a minimum version of python required which creates a dependency. The cp210x kernel driver is problematic in the way that it doesn't handle the ioctl request above. The actual problem I would say is the 128000 baud rate which require us to make work arounds and have a matching combination of serial communication client and kernel driver.

I still have to try the modified kernel driver that I've linked to in a previous post. From the weewx perspective is better to have a kernel driver that works with the weather station and not require a specific version och pyserial. Ofcourse, just requiring a minimum version of pyserial is the least effort to make it work.

Now I want to spend some time using your weewx driver and get familiar with the weather data and verify that it matches the WMR89 display.

[RodA]

Hi mwall,

I tried to use your driver but get the following error. is there an easy fix for this?

Thanks.

Apr 04 16:44:06 weatherstation weewx[6069]:     ****    File "/usr/share/weewx/weewx/engine.py", line 871, in main
Apr 04 16:44:06 weatherstation weewx[6069]:     ****      engine.run()
Apr 04 16:44:06 weatherstation weewx[6069]:     ****    File "/usr/share/weewx/weewx/engine.py", line 187, in run
Apr 04 16:44:06 weatherstation weewx[6069]:     ****      for packet in self.console.genLoopPackets():
Apr 04 16:44:06 weatherstation weewx[6069]:     ****    File "/usr/share/weewx/user/wmr89mwall.py", line 106, in genLoopPackets
Apr 04 16:44:06 weatherstation weewx[6069]:     ****      for pkt in self.station.get_data():
Apr 04 16:44:06 weatherstation weewx[6069]:     ****    File "/usr/share/weewx/user/wmr89mwall.py", line 222, in get_data
Apr 04 16:44:06 weatherstation weewx[6069]:     ****      (ord(x), _fmt(a[i])))
Apr 04 16:44:06 weatherstation weewx[6069]:     ****  TypeError: ord() expected a character, but string of length 2 found
Apr 04 16:44:06 weatherstation weewx[6069]:     ****  Exiting.

[Kimmo Linna]

Hi Andr3id,

Did you manage to solve the problem? Or do you have instruction how to do a quick fix? I am getting the broken pipe error with my version.

Best regards,

Kimmo

[Kimmo Linna]

I just installed the latest version of pyserial by downloading the package from web page and now the driver of cp210x works nicely in Raspberry Pi.

I still have some issues with WMR89 driver aka wmr89.py but I think those will be sorted out quite nicely in GitHub.

Best regards,

Kimmo