how to diagnose a davis envoy that lost connection to iss?

[matthew wall]

what could cause a davis envoy to lose wireless connection to an iss that is about 6 feet away from it?  how do i figure out why a davis envoy is no longer receiving data from the iss?

i have a davis envoy connected to a low-power intel computer via a davis usb data logger. the envoy was paired to an iss about 6 feet away via a wireless connection (and a wireless connection to a soil monitoring station, but that is a separate problem).  weewx has been collecting data from the envoy for almost 3 months now.  there were interruptions early on due to power (and heat?) issues, but data have been flowing continuously for the past 20+ days.

until 00:00 on 19jul2022.  at 00:05 the envoy reported lost of connection to the iss - the rxCheckPercent field no longer showed up in the loop or rec packets from the envoy. for a day prior to the fail, outside humidity was 95-100% for over 24 hours.  this is a salty environment - the station is about 10-15 feet above mean high tide on a pier that sticks out a few hundred feet from a rocky coastline.  there were moderate (20-30 mph) winds before the fail, but nothing unusual, and no significant swells or waves.

the envoy seems to be healthy - i can talk to it using wee_device (output for '--info' and '--current' for that is attached).  one unusual thing is that i see readings for 'outHumidity' and 'outTemperature' that are obviously wrong, but no fields for any of the wind, uv, rain, or solarradiation that would normally come from the iss.

this is at a site that is a few hours away from me, so before i get on a boat and/or send someone else to diagnose, i'd like to be sure i have probed it electronically as much as i can.

what else can i probe?  and if i have to go there, what can i do physically to figure out what is happening?

versions:

os: Debian GNU/Linux 11
python: 3.9.2
weewx: 4.7.0
weewx-influx: 0.16
influx: 1.8.10-1
grafana: 8.4.5

envoy: 3.88 Nov 12 2019 (purchased jan 2022)
iss: ? (purchased jan 2022)
soil station: ? (purchased jan 2022)

m

[Tom Keffer]

Many Davis stations show dropped reception at precisely midnight. It usually only stays that way for only a single archive record, but I recall one instance on my station where it fell into the 50% range precisely at midnight, stayed there for 3 days, then went back up to 99%, again precisely at midnight. 

I've asked Davis about this, and they claim that my station locked into a far away neighbor's ISS for a few days, then went back to mine, but I'm dubious. Precisely at midnight? 

Some users have had similar experiences that they've traced back to bad batteries or supercapacitors. 

One other note: rxCheckPercent is calculated from the number of wind samples. So, it's quite possible that it can drop due to something in the anemometer sampling, while leaving everything else, including outside temperature, alone.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/ebc08611-aee2-4032-b825-75e5785ddc4bn%40googlegroups.com.

[vince]

Is that thing actually underwater ?  OutTemp of 61.8 and humidity=100 ?

Some wild hardware guesses:

 - did it 'lose connection' or are the readings just whacko ?

 - does part of a Davis ISS suite fail high (ie, outHumidity=100) and others fail low (zeroes or no data) when something shorts or loses its ground in the ISS ?

 - did you get hit by a wave (or bird or gust) and things are just unplugged or loose or wet ?

Software stuff:

 - how are you getting tide data into your db ?

 - I see failures to upload to influxdb starting shortly before midnight but not at 00:00:00 precisely ?   Might it be an influxdb or disk issue ? 

 - are you still seeing failure to upload to influxdb errors ? 

 - what do the influxdb logs look like ?  A loss of sensor data shouldn't cause failure to upload to influxdb (should it?)

 - what's your influxdb setup look like ?  Are you using telegraf at all ?

Wilder guess:

- do you really have a soil sensor on channel 2 ?

- can you remotely change ISS+Envoy channel to something non-default ?

But I'm leaning toward the ISS either being dripping wet internally or swimming....

[Greg Troxel]

I would try listening with rtldavis to see if you can tell if the ISS is
transmitting.

Do you have a console also? Is that displaying?

Did you recently install any equipment near the envoy, that would create
RF interference?

(I agree that 0000 is suspicious.)

[vince]

Agree with Greg on rtldavis.  That would be a fine test.

Unrelated, but the instructions for that are 'very' cryptic and incorrect for a modern debian-11 pi....it was quite the battle figuring out the correct incantation to get it running on a pi4 with current raspi os.

* if you get device busy, add  'blacklist dvb_usb_rtl28xxu' to /etc/modprobe.d/blacklist-rt8xxxu.conf and reboot

* use -DINSTALL_UDEV_RULES=OFF so it doesn't add a conflicting set of udev rules since you're hand-editing a file that is correct already

* running weewx interactively blows up because it tries to parse data that isn't there yet.  There's no try/except around line 1407 in rtldavis.py - just give up on that one and start weewx normally via systemctl with debug=1 at least initially

* the options in weewx.conf are cryptic again.  Delete the word '[options] that the driver puts there.  Set the transmitter_frequency to US.  Start weewx.  It'll work just like running rtldavis did once you blacklisted the driver above.

[matthew wall]

On Thursday, July 21, 2022 at 12:46:24 PM UTC-4 tke...@gmail.com wrote:

> Many Davis stations show dropped reception at precisely midnight. It usually only 

> stays that way for only a single archive record, but I recall one instance on my station 

> where it fell into the 50% range precisely at midnight, stayed there for 3 days, then 

> went back up to 99%, again precisely at midnight. 

i suppose there might be another davis in the area, but this envoy has not yet reconnected to the iss that it is supposed to use.  it is a brand new system (5 months old!), so kinda doubtful that it is a battery/supercap issue.  good to know about the rxCheckPercent - i thought that was a direct measure of rf integrity, but if it is based on anemometer samples then anemometer problems could make you think you have rf problems.

there has been no new hardware installed by us, but there is tons of boat traffic all around the station, so rf interference is definitely possible.

On Thursday, July 21, 2022 at 1:18:47 PM UTC-4 vince wrote:

> Is that thing actually underwater ?  OutTemp of 61.8 and humidity=100 ?

ha! water temperature here rarely gets over 55F, and that would only be in a very shallow bay with little flow.

> Some wild hardware guesses:

 > - did it 'lose connection' or are the readings just whacko ?

 > -  does part of a Davis ISS suite fail high (ie, outHumidity=100) and others fail low (zeroes or no data) 

>     when something shorts or loses its ground in the ISS ?

>  - did you get hit by a wave (or bird or gust) and things are just unplugged or loose or wet ?

definitely no wave or critter damage.  we got pictures of the station late yesterday, and the iss is pristine.  don't know what values the iss reports when sensors fail.  years ago (before davis changed to a better humidity sensor) i had a davis station on which all of the iss sensors pegged at an arbitrary value when the humidity sensor died.  had to send that one in to davis for repair - they did a full overhaul for $100, plus shipping.  i think that station was 7 years old at the time. it is still going strong now.  i just hope that this new station is not having a similar kind of failure.

> Software stuff:

> - how are you getting tide data into your db ?

a separate weewx instance using the weewx-maxbotix driver.  maxbotix is an ultrasonic sensor.  we sample at 1Hz for loop data (sensor can do 6Hz) and use 5 minute average for archive.  this is a prototype station that will hopefully be used to guide the builds for community-supported tide-only stations.  currently there are volunteers going to sites at high tide and measure sticks.  think cocoras, but for tide height.  slogging through the mud at 02:00 to catch a high tide measurement is only fun the first two or three times.  not twice a day.  fwiw, hohonu makes a tide station that uses the same maxbotix sensors.

> - I see failures to upload to influxdb starting shortly before midnight but not at 00:00:00 precisely ?   Might it be an influxdb or disk issue ? 

> - are you still seeing failure to upload to influxdb errors ? 

> - what do the influxdb logs look like ?  A loss of sensor data shouldn't cause failure to upload to influxdb (should it?)

> - what's your influxdb setup look like ?  Are you using telegraf at all ?

it turns out that this is a bug in the influx uploader.  it improperly handles empty/None/string values. i think i fixed it now in the weewx-influx uploader. not using telegraf.  influx2 has explicit support for None, but for various reasons i cannot (yet) move to influx2.  love the fluxql features in influx2, understand why they put lots of grafana-like capabilities into a default influx2 install, not crazy about all the influx2/grafana overlap.

> Wilder guess:

> - do you really have a soil sensor on channel 2 ?

> - can you remotely change ISS+Envoy channel to something non-default ?

yes, there is a soil station on channel 2.  currently it has one temperature probe on it - that measures water temperature in the tidal zone.  plan is to add a few more at different depths, but the one sensor is not working yet.  the installer might have connected the sensor to the wrong terminal.

my takeaways:

- use a spare davis vantage console to see if it will see the iss

- try a wired connection between the envoy and iss

- try unsetting then resetting the iss binding in the envoy

- try changing the iss channel identifier, then bind the envoy to that

- verify that the water temperature sensor is connected to the correct terminals in the soil station

- get an sdr dongle to the site and do some rf scanning

thank you tom, greg, and vince!  we now have some things to try before sending the iss back to davis.

[Tom Keffer]

I would add one more to your list: change the battery.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/9658cf7d-c78c-4980-b389-ce8429e5b453n%40googlegroups.com.

[Dan Hinckley]

On 22 Jul, 2022, at 08:47, matthew wall <mwall...@gmail.com> wrote:

a separate weewx instance using the weewx-maxbotix driver.  maxbotix is an ultrasonic sensor.  we sample at 1Hz for loop data (sensor can do 6Hz) and use 5 minute average for archive.  this is a prototype station that will hopefully be used to guide the builds for community-supported tide-only stations.  currently there are volunteers going to sites at high tide and measure sticks.  think cocoras, but for tide height.  slogging through the mud at 02:00 to catch a high tide measurement is only fun the first two or three times.  not twice a day.  fwiw, hohonu makes a tide station that uses the same maxbotix sensors.

Weather + tide station.

For the DIY’er interested in weather + tides, there is this project which also uses the Maxbotix ultrasound units.

[matthew wall]

not sure what caused the problem, but we figured out how to fix it!  we had a spare vantage console, and took that near the iss.  the console showed everything working just fine.  so that pointed to the envoy as the problem.  we considered a hard reset, but that would require removing batteries and unplug/replug the power supply, which would have required pulling everything out of the waterproof enclosures.

so we tried a remote configuration.  first we changed channel 1 to a temperature station instead of iss:

/opt/weewx/bin/wee_device /etc/weewx/envoy.conf --set-transmitter-type=1,1,1,1

then we changed it back to the iss:

/opt/weewx/bin/wee_device /etc/weewx/envoy.conf --set-transmitter-type=1,0

and voila!  the data started coming in properly again!

i would have guessed that this was due to static discharge or a lightning strike, but the fact that it happened on the 00:00 cycle makes that rather unlikely.  so still no idea what caused it, but at least we know how to fix it!

also, thank you dbh for the hackster link to mawrey's work!

m