weewx crashed - issue with gw1000.py when Ecowitt WH45 5-in-1 combo sensor gets activated (5= T&H&CO2+PM2.5+PM10)

[Rainer Lang]

Ever since integrating the new WH45 sensor, weewx went nuts:

(see below): no more reports written, no more archive records stored.

Had to take off the sensor and reset the GW1000 - then weewx went back to normal.

I think it's an "API mismatch" - btw - Meteobridge also produces weird sensor output, but doesn't crash.

Probably the WH45 with its 5 sensors  (5= T&H&CO2+PM2.5+PM10) is not yet properly implemented
as real life examples were missing. I'm holding one of the first sold WH45 since today.

Need more info, logs, .... - let me know.

regrads

Rainer

Dec 31 15:49:50 raspberrypi systemd[1]: Starting LSB: weewx weather system...
Dec 31 15:49:50 raspberrypi weewx[5528] INFO __main__: Initializing weewx version 4.2.0
Dec 31 15:49:50 raspberrypi weewx[5528] INFO __main__: Using Python 2.7.16 (default, Oct 10 2019, 22:02:15) #012[GCC 8.3.0]
Dec 31 15:49:50 raspberrypi weewx[5528] INFO __main__: Platform Linux-5.4.79-v7l+-armv7l-with-debian-10.7
Dec 31 15:49:50 raspberrypi weewx[5528] INFO __main__: Locale is 'en_US.UTF-8'
Dec 31 15:49:50 raspberrypi weewx[5528] INFO __main__: PID file is /var/run/weewx.pid
Dec 31 15:49:50 raspberrypi weewx[5532] INFO __main__: Using configuration file /etc/weewx/weewx.conf
Dec 31 15:49:50 raspberrypi weewx[5532] INFO __main__: Debug is 0
Dec 31 15:49:50 raspberrypi weewx[5516]: Starting weewx weather system: weewx.
Dec 31 15:49:50 raspberrypi systemd[1]: Started LSB: weewx weather system.
Dec 31 15:49:50 raspberrypi weewx[5532] INFO weewx.engine: Loading station type GW1000 (user.gw1000)
Dec 31 15:49:50 raspberrypi weewx[5532] INFO user.gw1000: field map is {'UV': 'uvi', 'dateTime': 'datetime', 'dayRain': 'rainday', 'daymaxwind': 'daymaxwind', 'dewpoint': 'dewpoint', 'extraHumid1': 'humid1', 'extraHumid2': 'humid2', 'extraHumid3': 'humid3', 'extraHumid4': 'humid4', 'extraHumid5': 'humid5', 'extraHumid6': 'humid6', 'extraHumid7': 'humid7', 'extraHumid8': 'humid8', 'extraTemp1': 'temp1', 'extraTemp2': 'temp2', 'extraTemp3': 'temp3', 'extraTemp4': 'temp4', 'extraTemp5': 'temp5', 'extraTemp6': 'temp6', 'extraTemp7': 'temp7', 'extraTemp8': 'temp8', 'heatindex': 'heatindex', 'hourRain': 'rainhour', 'inHumidity': 'inhumid', 'inTemp': 'intemp', 'leak1': 'leak1', 'leak2': 'leak2', 'leak3': 'leak3', 'leak4': 'leak4', 'lightning_distance': 'lightningdist', 'lightning_last_det_time': 'lightningdettime', 'lightning_strike_count': 'lightning_strike_count', 'luminosity': 'light', 'monthRain': 'rainmonth', 'outHumidity': 'outhumid', 'outTemp': 'outtemp', 'pm2_5': 'pm251', 'pm2_51_24hav': 'pm251_24hav', 'pm2_52': 'pm252', 'pm2_52_24hav': 'pm252_24hav', 'pm2_53': 'pm253', 'pm2_53_24hav': 'pm253_24hav', 'pm2_54': 'pm254', 'pm2_54_24hav': 'pm254_24hav', 'pressure': 'absbarometer', 'rain': 'rain', 'rainEvent': 'rainevent', 'rainRate': 'rainrate', 'relbarometer': 'relbarometer', 'soilMoist1': 'soilmoist1', 'soilMoist2': 'soilmoist2', 'soilMoist3': 'soilmoist3', 'soilMoist4': 'soilmoist4', 'soilMoist5': 'soilmoist5', 'soilMoist6': 'soilmoist6', 'soilMoist7': 'soilmoist7', 'soilMoist8': 'soilmoist8', 'soilMoist9': 'soilmoist9', 'soilMoist10': 'soilmoist10', 'soilMoist11': 'soilmoist11', 'soilMoist12': 'soilmoist12', 'soilMoist13': 'soilmoist13', 'soilMoist14': 'soilmoist14', 'soilMoist15': 'soilmoist15', 'soilMoist16': 'soilmoist16', 'soilTemp1': 'soiltemp1', 'soilTemp2': 'soiltemp2', 'soilTemp3': 'soiltemp3', 'soilTemp4': 'soiltemp4', 'soilTemp5': 'soiltemp5', 'soilTemp6': 'soiltemp6', 'soilTemp7': 'soiltemp7', 'soilTemp8': 'soiltemp8', 'soilTemp9': 'soiltemp9', 'soilTemp10': 'soiltemp10', 'soilTemp11': 'soiltemp11', 'soilTemp12': 'soiltemp12', 'soilTemp13': 'soiltemp13', 'soilTemp14': 'soiltemp14', 'soilTemp15': 'soiltemp15', 'soilTemp16': 'soiltemp16', 'totalRain': 'raintotals', 'uvradiation': 'uv', 'weekRain': 'rainweek', 'wh25_batt': 'wh25_batt', 'wh26_batt': 'wh26_batt', 'wh31_ch1_batt': 'wh31_ch1_batt', 'wh31_ch2_batt': 'wh31_ch2_batt', 'wh31_ch3_batt': 'wh31_ch3_batt', 'wh31_ch4_batt': 'wh31_ch4_batt', 'wh31_ch5_batt': 'wh31_ch5_batt', 'wh31_ch6_batt': 'wh31_ch6_batt', 'wh31_ch7_batt': 'wh31_ch7_batt', 'wh31_ch8_batt': 'wh31_ch8_batt', 'wh40_batt': 'wh40_batt', 'wh41_ch1_batt': 'wh41_ch1_batt', 'wh41_ch2_batt': 'wh41_ch2_batt', 'wh41_ch3_batt': 'wh41_ch3_batt', 'wh41_ch4_batt': 'wh41_ch4_batt', 'wh51_ch1_batt': 'wh51_ch1_batt', 'wh51_ch2_batt': 'wh51_ch2_batt', 'wh51_ch3_batt': 'wh51_ch3_batt', 'wh51_ch4_batt': 'wh51_ch4_batt', 'wh51_ch5_batt': 'wh51_ch5_batt', 'wh51_ch6_batt': 'wh51_ch6_batt', 'wh51_ch7_batt': 'wh51_ch7_batt', 'wh51_ch8_batt': 'wh51_ch8_batt', 'wh51_ch9_batt': 'wh51_ch9_batt', 'wh51_ch10_batt': 'wh51_ch10_batt', 'wh51_ch11_batt': 'wh51_ch11_batt', 'wh51_ch12_batt': 'wh51_ch12_batt', 'wh51_ch13_batt': 'wh51_ch13_batt', 'wh51_ch14_batt': 'wh51_ch14_batt', 'wh51_ch15_batt': 'wh51_ch15_batt', 'wh51_ch16_batt': 'wh51_ch16_batt', 'wh55_ch1_batt': 'wh55_ch1_batt', 'wh55_ch2_batt': 'wh55_ch2_batt', 'wh55_ch3_batt': 'wh55_ch3_batt', 'wh55_ch4_batt': 'wh55_ch4_batt', 'wh57_batt': 'wh57_batt', 'wh65_batt': 'wh65_batt', 'wh68_batt': 'wh68_batt', 'windDir': 'winddir', 'windGust': 'gustspeed', 'windSpeed': 'windspeed', 'windchill': 'windchill', 'ws80_batt': 'ws80_batt', 'yearRain': 'rainyear'}
Dec 31 15:49:50 raspberrypi weewx[5532] INFO user.gw1000: driver version is 0.1.0b12
Dec 31 15:49:50 raspberrypi weewx[5532] INFO user.gw1000: GW1000 address is 192.168.1.27:45000
Dec 31 15:49:50 raspberrypi weewx[5532] INFO user.gw1000: poll interval is 15 seconds
Dec 31 15:49:50 raspberrypi weewx[5532] INFO weewx.engine: StdConvert target unit is 0x1
Dec 31 15:49:50 raspberrypi weewx[5532] INFO weewx.engine: Archive will use data binding wx_binding
Dec 31 15:49:50 raspberrypi weewx[5532] INFO weewx.engine: Record generation will be attempted in 'software'
Dec 31 15:49:50 raspberrypi weewx[5532] INFO weewx.engine: Using archive interval of 300 seconds (software record generation)
Dec 31 15:49:50 raspberrypi weewx[5532] INFO weewx.restx: StationRegistry: Registration not requested.
Dec 31 15:49:50 raspberrypi weewx[5532] INFO weewx.restx: Wunderground: Posting not enabled.
Dec 31 15:49:50 raspberrypi weewx[5532] INFO weewx.restx: PWSweather: Posting not enabled.
Dec 31 15:49:50 raspberrypi weewx[5532] INFO weewx.restx: CWOP: Posting not enabled.
Dec 31 15:49:50 raspberrypi weewx[5532] INFO weewx.restx: WOW: Posting not enabled.
Dec 31 15:49:50 raspberrypi weewx[5532] INFO weewx.restx: AWEKAS: Posting not enabled.
Dec 31 15:49:50 raspberrypi weewx[5532] INFO __main__: Starting up weewx version 4.2.0
Dec 31 15:49:50 raspberrypi weewx[5532] INFO weewx.engine: Using binding 'wx_binding' to database 'weewx.sdb'
Dec 31 15:49:50 raspberrypi weewx[5532] INFO weewx.manager: Starting backfill of daily summaries
Dec 31 15:49:50 raspberrypi weewx[5532] INFO weewx.engine: Starting main packet loop.
Dec 31 15:49:50 raspberrypi weewx[5532] CRITICAL user.gw1000:     ****  Traceback (most recent call last):
Dec 31 15:49:50 raspberrypi weewx[5532] CRITICAL user.gw1000:     ****    File "/usr/share/weewx/user/gw1000.py", line 2150, in run
Dec 31 15:49:50 raspberrypi weewx[5532] CRITICAL user.gw1000:     ****      self.client.collect_sensor_data()
Dec 31 15:49:50 raspberrypi weewx[5532] CRITICAL user.gw1000:     ****    File "/usr/share/weewx/user/gw1000.py", line 1909, in collect_sensor_data
Dec 31 15:49:50 raspberrypi weewx[5532] CRITICAL user.gw1000:     ****      queue_data = self.get_live_sensor_data()
Dec 31 15:49:50 raspberrypi weewx[5532] CRITICAL user.gw1000:     ****    File "/usr/share/weewx/user/gw1000.py", line 1945, in get_live_sensor_data
Dec 31 15:49:50 raspberrypi weewx[5532] CRITICAL user.gw1000:     ****      parsed_data = self.parser.parse(raw_data, _timestamp)
Dec 31 15:49:50 raspberrypi weewx[5532] CRITICAL user.gw1000:     ****    File "/usr/share/weewx/user/gw1000.py", line 2951, in parse
Dec 31 15:49:50 raspberrypi weewx[5532] CRITICAL user.gw1000:     ****      decode_str, field_size, field = self.response_struct[resp[index:index + 1]]
Dec 31 15:49:50 raspberrypi weewx[5532] CRITICAL user.gw1000:     ****  KeyError: 'p'

[galfert]

Hopefully Gary will pick this up and add the WH45 compatibility to the GW100 API driver.

[Rainer Lang]

He's already been working on it - will check/test the suggestions tomorrow respectively today later in the day - now too early (or late) here.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/1607c22b-ba52-458f-be41-47a4818843a9n%40googlegroups.com.

[gjr80]

WH45 support was coded some time ago though of course not tested with an actual device. Have had a number of other features I was working on to add at the same time which delayed release of the WH45 compatible version of the driver. Rainer now has a copy of a compatible version of the driver to test and once he confirms it works I will shortly afterwards release the WH45 compatible version.

As an aside, and on a technical note, this did highlight that I need to change the approach the driver uses to dealing with sensor data from the GW1000. Ideally adding an new sensor type should not cause a driver that does not know about that new sensor type to crash; rather the driver should continue to operate but just not provide data for the (to it) unknown sensor. This will be something to work on for the next version of the driver, fortunately Ecowitt is not releasing new sensors that often.

Gary

[galfert]

Thank you for the quick work Gary!

Please note that even though the raw data coming from this new sensor is for the PM10 to come first and then the PM2.5 that is not the order that we would typically refer to the sensor as ...it is called the PM2.5, PM10, CO2 sensor in that order. Looking at the GW1000 API documentation it baffles me why the engineers decided to send PM10 data first. Also on the GW1000 mobile app (WS View) and on Ecowitt.net the order is also PM2.5 first and then PM10. I fee therefore that WeeWX should keep the order as PM2.5 first and then PM10.

[Paul Ward]

Gary I have a WH45 on order so subject to shipping delays I’m happy to test the code too in couple of weeks when I get it, if you don’t release beforehand.

[gjr80]

On Saturday, 2 January 2021 at 06:52:36 UTC+10 galfert wrote:

I fee therefore that WeeWX should keep the order as PM2.5 first and then PM10.

 

I am a bit lost, what do you mean by this? Where?

Gary

[Rainer Lang]

As for me, the code of the new driver (0.2.1b1) works fine as far as I can tell.

All data arrives and can be addressed, and the application doesn't crash anymore.
It can be archived (some assignment needed as not all fields are part of the extended database schema),
and it can be used in the reports.

And the order in the field map is:

    .......................

        'pm2_5': 'pm251',  # =WH41/43 #1
       'pm2_52': 'pm252', # =WH41/43 #2
        'pm2_53': 'pm253', # =WH41/43 #3
        'pm2_54': 'pm254', # = WH41/43 #4
        'pm2_55': 'pm255', # = WH45 PM2.5
        'pm10': 'pm10',      # = WH45 PM10
        'co2': 'co2',            # =WH45 CO2

    ...........................     

Temperature and Humidity are sorted into the extraTemperature 'section' as #17

Here I don't really understand George's request regarding sequence/order either.

"
Please note that even though the raw data coming from this new sensor is for the PM10 to come first and then the PM2.5 that is not the order that we would typically refer to the sensor as ...it is called the PM2.5, PM10, CO2 sensor in that order. Looking at the GW1000 API documentation it baffles me why the engineers decided to send PM10 data first. Also on the GW1000 mobile app (WS View) and on Ecowitt.net the order is also PM2.5 first and then PM10. I fee therefore that WeeWX should keep the order as PM2.5 first and then PM10."

@George: maybe you explain more in detail what exactly you mean, where some sequence (order) is not maintained and why this is important 😎.

At least in the field map (in my terms/language: the weewx GW1000 driver interface record declaration [but we may have different wording here, depending on our
programming language history]) exactly this sequence (order) is maintained. Where else is this order missing ?

At the end of the day, e.g. in the WSView app, it's a matter of presentation. And which sequence/order you choose for showing this data in the reports (e.g. Seasons skin)
is up to you. There is no preset report for the WH45 data (yet), afaik, but my knowledge may not be complete.

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/afcd4401-6b46-421f-a25c-e74aa235e870n%40googlegroups.com.

[galfert]

I realize that this WH45 data is not yet part of any skin.  I was only trying to preempt its default inclusion in a skin. In that case my comments were that the PM2.5 be first and the PM10 data second. and the CO2 data third.

The reason I state this is because in the raw data from the GW1000 API ....something you can't see unless you are in the know with the GW1000 API as I and Gary are (Ecowitt NDA regarding GW1000 API documentation). This note only will make sense to Gary. The data as is comes off the GW1000 sends PM10 before PM2.5.

[galfert]

The data comes off the GW1000 API in this order:

  1　temp_co2

  2　humi_co2 

  3　pm10_co2

  4　pm10_24h_co2

  5　pm25_co2

  6　pm25_24h_co2

  7　co2 

  8　co2_24h

  9　co2_batt　

I'm just trying to impart some input if it were to ever be included in any default skin or presentation. This order is not good.

* Reference: Official Ecowitt GW1000 API documentation. Which has limited access by Ecowitt for now. I have it as I'm like an ambassador / liaison for Ecowitt and the various software developers. Bound to by NDA.

[vince]

Just a guess but perhaps it's in alphabetical/numerical order, given the vendor being not in a native English speaking country.

It's been a lot of years since I had to tear apart binary data in the very-large-flying-things industry.  Looking a bit more at the driver under the hood...wow.  Glad it's you guys :-)

[gjr80]

OK, understand. Nothing to concern the GW1000 driver.

I suspect the vast majority of cases where WH45 obs are included in a skin will be user modified/developed skins and experience shows the user will format/order as the user sees fit. I suspect the default skins that ship with WeeWX will continue to be fairly generic in nature.

Gary