Trying to decide between multiple weewx instances or expanding the weewx database

[Patrick Mendiuk]

Hi,

I have been running some Weatherflow Sky and Air sensors along with a number of MQTT sensors feeding into one instance of weewx.  I recently added a Tempest sensor and a Vantage 2 Pro ISS.  I am trying to decide if I should setup two additional weewx instances or expand the original database.  I would like to generate comparative data plots and possibly expose the data via CWOP.  Is it possible to generate multiple CWOP output streams from one weewx instance?

Thanks,

Patrick

[vince]

On Wednesday, March 18, 2020 at 11:17:23 AM UTC-7, Patrick Mendiuk wrote:

I have been running some Weatherflow Sky and Air sensors along with a number of MQTT sensors feeding into one instance of weewx.  I recently added a Tempest sensor and a Vantage 2 Pro ISS.  I am trying to decide if I should setup two additional weewx instances or expand the original database.  I would like to generate comparative data plots and possibly expose the data via CWOP.  Is it possible to generate multiple CWOP output streams from one weewx instance?

I have lots of posts in the WF Field Tester forums there...

WF has asked that folks do not (yet) publicize their Tempest sites to WU or the web, as it's still in the beta period

Re: weewx specifically:

• you can use the extended v4 schema and an updated weewx driver for WF to do it in one weewx instance probably.  Contact the WF driver author for a pointer to his v4-compatible driver.  Basically you'd just define a custom sensor map in weewx.conf, I think.  Untested.
• you could alternately do weewx-multi and do multiple instances I guess, each with the right driver
  
• but that said, I run my Air/Sky WF on a separate pi3 and a separate weewx instance mainly because my Davis is the reference platform for feeding WU/CWOP/PWS and uploading to my internet-facing Lightsail VM on the web.
  

For comparisons, I do it a little differently:

• my VP2 runs the mqtt extension, feeds mqtt on a pi3plus, which feeds a docker container on an i3 nuc running telegraf, feeding another container running influxdb.   Actually not as complicated as it sounds.  I didn't want to mess with my weewx too much, so I just added mqtt to publish the data and did mqtt=>telegraf=>influxdb like most IoT things do.
• my WF gear, Air/two-Sky/Tempest get into the same influxdb via a UDP listener that I wrote (details on the WF Forums) that listens for the UDP and publishes directly to influxdb in one step.   That runs as a standalone process on the i3 nuc currently.
• I have a lot of ds18b20 sensor arduinos also feeding influxdb via some arduino code I wrote that feeds influxdb directly too
  
• I have a separate docker container on the i3 nuc running grafana for displays, so I can trivially overlay data from the WF/Davis/Arduino gear all on one display.  Works wonderfully.  Totally hands-off.

[Patrick Mendiuk]

Thanks Vince.  

I wasn't planning on exposing the Tempest data right now, we're still getting acquainted with each other and I found something else to trip over...

I previously used a Davis Envoy to listen to my older Vantage 2 Pro ISS.  It's been a while but I recall that I used to be able to get data every 2 seconds from the Envoy with an ancient version of weatherflow.  I downloaded the current non subscription version and it seems like I'm only getting the 1 min archive data and no loop data at all.  I then tried to use an RPI and a RTL SDR with the rtldavis driver but I must be doing something wrong with the install of weewx or the rtldavis driver.  I downloaded, installed and configured the driver but weewx can't load the driver with the default path in weewx.conf.  That directory does not exist after the driver install.  The readme talks about compiling the go sources but I didn't the exported directories mentioned in step 5 of the readme nor did I see anything source files.  

Here's what I did:

2) download the driver

pi@raspberrypi:~ $ wget -O weewx-rtldavis-master.zip https://github.com/lheijst/weewx-rtldavis/archive/master.zip
--2020-03-18 11:20:36--  https://github.com/lheijst/weewx-rtldavis/archive/master.zip
Resolving github.com (github.com)... 192.30.255.113
Connecting to github.com (github.com)|192.30.255.113|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/lheijst/weewx-rtldavis/zip/master [following]
--2020-03-18 11:20:36--  https://codeload.github.com/lheijst/weewx-rtldavis/zip/master
Resolving codeload.github.com (codeload.github.com)... 192.30.255.120
Connecting to codeload.github.com (codeload.github.com)|192.30.255.120|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/zip]
Saving to: ‘weewx-rtldavis-master.zip’


2020-03-18 11:20:37 (2.65 MB/s) - ‘weewx-rtldavis-master.zip’ saved [31153]



weewx-rtldavis-master.z     [ <=>                           ]  30.42K  --.-KB/s    in 0.01s

3) install the driver

pi@raspberrypi:~ $ sudo wee_extension --install weewx-rtldavis-master.zip
Request to install 'weewx-rtldavis-master.zip'
Extracting from zip archive weewx-rtldavis-master.zip
Saving installer file to /usr/share/weewx/user/installer/rtldavis
Finished installing extension 'weewx-rtldavis-master.zip'

4) configure the driver

pi@raspberrypi:~ $ sudo wee_config --reconfigure --driver=user.rtldavis --no-prompt
Using configuration file /etc/weewx/weewx.conf
Saved backup to /etc/weewx/weewx.conf.20200318213936
Saved configuration to /etc/weewx/weewx.conf

When I try to run weewx here is what shows up in syslog:

Mar 18 22:51:29 raspberrypi weewx[12904]: rtldavis: using anemometer_channel 0
Mar 18 22:51:29 raspberrypi weewx[12904]: rtldavis: using leaf_soil_channel 0
Mar 18 22:51:29 raspberrypi weewx[12904]: rtldavis: using temp_hum_1_channel 0
Mar 18 22:51:29 raspberrypi weewx[12904]: rtldavis: using temp_hum_2_channel 0
Mar 18 22:51:29 raspberrypi weewx[12904]: rtldavis: using transmitters 1
Mar 18 22:51:29 raspberrypi weewx[12904]: rtldavis: log_humidity_raw False
Mar 18 22:51:29 raspberrypi weewx[12904]: rtldavis: startup process '/home/pi/work/bin/rtldavis [options] -tf US -tr 1'
Mar 18 22:51:29 raspberrypi weewx[12904]: engine: Import of driver failed: failed to start process: [Errno 2] No such file or directory (<class 'weewx.WeeWxIOError'>)
Mar 18 22:51:29 raspberrypi weewx[12904]: engine: Unable to load driver: failed to start process: [Errno 2] No such file or directory
Mar 18 22:51:29 raspberrypi weewx[12904]:     ****  Exiting...

Thanks,

Patrick

[rich T]

Need to compile the "go" source files found here https://github.com/lheijst/rtldavis.

[Patrick Mendiuk]

Hi Rich,

Thanks for the correct github repo.  There are two github repos that have very similar names.  The first repo (https://github.com/lheijst/weewx-rtldavis) installation instructions are missing a few things.

https://github.com/lheijst/weewx-rtldavis

https://github.com/lheijst/rtldavis

After I installed using the repo you suggested I got the driver compiled and was able to run it stand alone.  It soon it found the ISS transmitter and I thought things were good.  I rebooted the RPi and ran it again and it doesn't seem to be doing anything.

pi@raspberrypi:~/work/src/github.com/lheijst/rtldavis $ $GOPATH/bin/rtldavis -tf US
18:54:36.723394 rtldavis.go VERSION=0.12
18:54:36.724355 tr=1 fc=0 ppm=0 gain=0 ex=0 maxmissed=51 actChan=[0] maxChan=1
18:54:36.725198 BitRate: 19200
18:54:36.725323 SymbolLength: 14
18:54:36.725403 SampleRate: 268800
18:54:36.725485 Preamble: 1100101110001001
18:54:36.725561 PreambleSymbols: 16
18:54:36.725638 PreambleLength: 224
18:54:36.725713 PacketSymbols: 80
18:54:36.725786 PacketLength: 1120
18:54:36.725860 BlockSize: 512
18:54:36.725934 BufferLength: 2048
Detached kernel driver
Found Rafael Micro R820T tuner
18:54:37.234946 Hop: {ChannelIdx:0 ChannelFreq:902419338 FreqError:0}
Exact sample rate is: 268800.001367 Hz
18:54:37.425818 GetTunerGain: 0 Db
18:54:37.425898 SetFreqCorrection 0 ppm Successful
Allocating 1 zero-copy buffers
18:54:37.430642 Init channels: wait max 135 seconds for a message of each transmitter
18:56:53.244105 Init channels: wait max 135 seconds for a message of each transmitter
18:56:53.244325 Hop: {ChannelIdx:0 ChannelFreq:902419338 FreqError:0}

I moved the rtl-sdr dongle to a PC and ran SDRsharp with no issue.  Is there any additional debugging that can be done from the command line?

Thanks,

Patrick

[rich T]

Patrick

I'm running the RTLDavis driver for almost a year now on Raspbian GNU/Linux 9 (stretch) with no issues. 

Mar 19 22:50:43 DavisSDR weewx[1846]: rtldavis: info: 22:50:42.676463 rtldavis.go VERSION=0.12
Mar 19 22:50:43 DavisSDR weewx[1846]: rtldavis: info: 22:50:42.676948 tr=1 fc=0 ppm=0 gain=0 ex=0 maxmissed=51 actChan=[0] maxChan=1
Mar 19 22:50:43 DavisSDR weewx[1846]: rtldavis: info: 22:50:42.677518 BitRate: 19200
Mar 19 22:50:43 DavisSDR weewx[1846]: rtldavis: info: 22:50:42.677604 SymbolLength: 14
Mar 19 22:50:43 DavisSDR weewx[1846]: rtldavis: info: 22:50:42.677623 SampleRate: 268800
Mar 19 22:50:43 DavisSDR weewx[1846]: rtldavis: info: 22:50:42.677637 Preamble: 1100101110001001
Mar 19 22:50:43 DavisSDR weewx[1846]: rtldavis: info: 22:50:42.677652 PreambleSymbols: 16
Mar 19 22:50:43 DavisSDR weewx[1846]: rtldavis: info: 22:50:42.677667 PreambleLength: 224
Mar 19 22:50:43 DavisSDR weewx[1846]: rtldavis: info: 22:50:42.677681 PacketSymbols: 80
Mar 19 22:50:43 DavisSDR weewx[1846]: rtldavis: info: 22:50:42.677698 PacketLength: 1120
Mar 19 22:50:43 DavisSDR weewx[1846]: rtldavis: info: 22:50:42.677712 BlockSize: 512
Mar 19 22:50:43 DavisSDR weewx[1846]: rtldavis: info: 22:50:42.677729 BufferLength: 2048
Mar 19 22:50:43 DavisSDR weewx[1846]: rtldavis: info: Found Rafael Micro R820T tuner
Mar 19 22:50:43 DavisSDR weewx[1846]: rtldavis: chan: 22:50:43.144108 Hop: {ChannelIdx:0 ChannelFreq:902419338 FreqError:0}
Mar 19 22:50:43 DavisSDR weewx[1846]: rtldavis: info: Exact sample rate is: 268800.001367 Hz
Mar 19 22:50:43 DavisSDR weewx[1846]: manager: Daily summary version is 2.0
Mar 19 22:50:43 DavisSDR weewx[1846]: rtldavis: info: 22:50:43.344512 GetTunerGain: 0 Db
Mar 19 22:50:43 DavisSDR weewx[1846]: rtldavis: info: 22:50:43.344576 SetFreqCorrection 0 ppm Successful
Mar 19 22:50:43 DavisSDR weewx[1846]: rtldavis: info: 22:50:43.348503 Init channels: wait max 135 seconds for a message of each transmitter
Mar 19 22:52:29 DavisSDR weewx[1846]: rtldavis: info: 22:52:29.125524 TRANSMITTER 0 SEEN
Mar 19 22:52:29 DavisSDR weewx[1846]: rtldavis: chan: 22:52:29.126139 Hop: {ChannelIdx:19 ChannelFreq:911952597 FreqError:11}
Mar 19 22:52:31 DavisSDR weewx[1846]: rtldavis: data: 22:52:31.689255 E00550770100B3FD 2 0 0 0 0 msg.ID=0
Mar 19 22:52:31 DavisSDR weewx[1846]: bme280: BME280 data compensated_reading(id=c1169d86-82f3-4d65-b2d3-db82fd7e2c6f, timestamp=2020-03-19 22:52:31.757738, temp=25.082 °C, pressure=960.93 hPa, humidity=0.00 % rH)
Mar 19 22:52:31 DavisSDR weewx[1846]: bme280: {'txBatteryStatus': 0, 'rain': 0.0, 'dateTime': 1584672752, 'windDir': 115.7905138339921, 'windSpeed': 2.6822333333333335, 'inHumidity': 0.0, 'inTemp': 25.081832114519784, 'usUnits': 17}
Mar 19 22:52:31 DavisSDR weewx[1846]: rtldavis: chan: 22:52:31.689505 Hop: {ChannelIdx:41 ChannelFreq:922991108 FreqError:-28}
Mar 19 22:52:34 DavisSDR weewx[1846]: rtldavis: data: 22:52:34.252057 500550FF7100D2F3 3 0 0 0 0 msg.ID=0
Mar 19 22:52:34 DavisSDR weewx[1846]: bme280: BME280 data compensated_reading(id=11faef2a-0efa-49aa-933a-d42f22f789f1, timestamp=2020-03-19 22:52:34.321828, temp=25.062 °C, pressure=961.00 hPa, humidity=0.00 % rH)
Mar 19 22:52:34 DavisSDR weewx[1846]: bme280: {'txBatteryStatus': 0, 'dateTime': 1584672754, 'windDir': 115.7905138339921, 'windSpeed': 2.6822333333333335, 'inHumidity': 0.0, 'inTemp': 25.0622148035618, 'rainRate': 0, 'usUnits': 17}
Mar 19 22:52:34 DavisSDR weewx[1846]: rtldavis: chan: 22:52:34.252289 Hop: {ChannelIdx:25 ChannelFreq:914963100 FreqError:-14}
Mar 19 22:52:36 DavisSDR weewx[1846]: rtldavis: data: 22:52:36.813679 9006500800058F1E 4 0 0 0 0 msg.ID=0
Mar 19 22:52:36 DavisSDR weewx[1846]: bme280: BME280 data compensated_reading(id=3b6d93ef-4865-4612-8097-c777cc37a450, timestamp=2020-03-19 22:52:36.873941, temp=25.082 °C, pressure=960.93 hPa, humidity=0.00 % rH)
Mar 19 22:52:36 DavisSDR weewx[1846]: bme280: {'txBatteryStatus': 0, 'dateTime': 1584672757, 'windDir': 115.7905138339921, 'windSpeed': 3.1292722222222222, 'inHumidity': 0.0, 'inTemp': 25.081832114519784, 'usUnits': 17}
Mar 19 22:52:36 DavisSDR weewx[1846]: rtldavis: chan: 22:52:36.813839 Hop: {ChannelIdx:8 ChannelFreq:906433342 FreqError:-81}
Mar 19 22:52:39 DavisSDR weewx[1846]: rtldavis: data: 22:52:39.375179 8005501BB800A67A 5 0 0 0 0 msg.ID=0
Mar 19 22:52:39 DavisSDR weewx[1846]: bme280: BME280 data compensated_reading(id=df88b40c-8446-4816-8e58-f74365d6f438, timestamp=2020-03-19 22:52:39.434277, temp=25.077 °C, pressure=960.89 hPa, humidity=0.00 % rH)
Mar 19 22:52:39 DavisSDR weewx[1846]: bme280: {'txBatteryStatus': 0, 'pressure': 960.8916787263882, 'dateTime': 1584672759, 'windDir': 115.7905138339921, 'outTemp': 6.833333333333332, 'windSpeed': 2.6822333333333335, 'inHumidity': 0.0, 'inTemp': 25.07692778634373, 'usUnits': 17}
Mar 19 22:52:39 DavisSDR weewx[1846]: rtldavis: chan: 22:52:39.375340 Hop: {ChannelIdx:47 ChannelFreq:926001611 FreqError:17}

Yesterday I installed the driver on RPI running Raspbian GNU/Linux 10 (buster) and I'm seem to be having the same issues you are having.

pi@raspberrypi:~ $ $GOPATH/bin/rtldavis -tr 1 -tf US
22:24:33.805278 rtldavis.go VERSION=0.12
22:24:33.806647 tr=1 fc=0 ppm=0 gain=0 ex=0 maxmissed=51 actChan=[0] maxChan=1
22:24:33.807837 BitRate: 19200
22:24:33.807900 SymbolLength: 14
22:24:33.808431 SampleRate: 268800
22:24:33.808640 Preamble: 1100101110001001
22:24:33.808683 PreambleSymbols: 16
22:24:33.809030 PreambleLength: 224
22:24:33.809271 PacketSymbols: 80
22:24:33.809494 PacketLength: 1120
22:24:33.809703 BlockSize: 512
22:24:33.809896 BufferLength: 2048

Found Rafael Micro R820T tuner

22:24:34.231572 Hop: {ChannelIdx:0 ChannelFreq:902419338 FreqError:0}

Exact sample rate is: 268800.001367 Hz

22:24:34.408971 GetTunerGain: 0 Db
22:24:34.409069 SetFreqCorrection 0 ppm Successful
Allocating 1 zero-copy buffers
22:24:34.415381 Init channels: wait max 135 seconds for a message of each transmitter
22:26:50.228986 Init channels: wait max 135 seconds for a message of each transmitter

At first, I was thinking it was location/antenna issue, but the only difference is the RPI OSes (Stretch vs Buster).I'm assuming you are running Buster?

[rich T]

Patrick

 I started a topic over at the weewx-development group.  https://groups.google.com/forum/#!topic/weewx-development/L5Lt8x87I_s

   Rich