WH57(lighning detection) and GW1000 send daily summary on every records

[Johann Destombes]

Hello everyone,

I had my first lightning detection this afternoon! And also my first problem...

The GW1000 show 5 lightnings, this number is ok on the WS View apps and on ecowitt.net but it's now more than 1000 on weewx database....

When I use the command to directly watch the datas send by the GW1000:

PYTHONPATH=/usr/share/weewx python /usr/share/weewx/user/interceptor.py --device=ecowitt-client --port 8000 --debug

raw data: PASSKEY=XXXX&stationtype=GW1000A_V1.6.6&dateutc=2021-04-10+20:55:22&tempinf=73.4&humidityin=49&baromrelin=29.533&baromabsin=29.533&rainratein=0.000&eventrainin=0.165&hourlyrainin=0.000&dailyrainin=0.067&weeklyrainin=0.165&monthlyrainin=0.165&yearlyrainin=0.165&totalrainin=0.165&lightning_time=1618082478&lightning_num=5&lightning=14&wh57batt=5&freq=868M&model=GW1000_Pro

raw packet: {'humidity_in': 49.0, 'temperature_in': 73.4, 'rain_total': 0.165, 'lightning_num': 5.0, 'rain': 0.0, 'lightning': 14.0, 'pressure': 29.533, 'wh57_battery': 5.0, 'dateTime': 1618088122, 'lightning_time': 1618082478.0, 'rain_rate': 0.0, 'usUnits': 1}

mapped packet: {'rain': 0.0, 'dateTime': 1618088122, 'pressure': 29.533, 'inHumidity': 49.0, 'inTemp': 73.4, 'rainRate': 0.0, 'usUnits': 1}

The "mapped packet" do not show any lightning but 5 lightnings strike are record on my database on lightning_strike_count field on every archive interval.

When I installed my GW1000 I followed this tip :
https://github.com/matthewwall/weewx-interceptor/issues/69

Is anyone have more success with lightning dectection on GW1000 and how can I fix this bad situation?

Thanks

[Graham Eddy]

post your gw1000 config especially map. suspect you are accumulating an already-accumulated count

[Johann Destombes]

Yes I'm agree. This is the driver part of my weewx.conf:

###############################
[Interceptor]
    driver = user.interceptor
    device_type = ecowitt-client
    port = 8000

    [[sensor_map_extensions]]
        lightning_strike_count = lightning_num
        lightning_distance = lightning
 ###############################      
       
I followed this link: https://github.com/matthewwall/weewx-interceptor/issues/69 to modifie interceptor.py.

So this is the ecowitt-client part of my interceptor.py:

Row 2306 to 2459.

...
class EcowittClient(Consumer):
    """Use the ecowitt protocol (not WU protocol) to capture data"""

    def __init__(self, **stn_dict):
        super(EcowittClient, self).__init__(
            EcowittClient.Parser(), handler=EcowittClient.Handler, **stn_dict)

    class Handler(Consumer.Handler):

        def get_response(self):
            return '{"errcode":"0","errmsg":"ok","UTC_offset":"-18000"}'

    class Parser(Consumer.Parser):

        # map labels to observation names
        LABEL_MAP = {
            'baromabsin': 'pressure',
            'humidity': 'humidity_out',
            'humidityin': 'humidity_in',
            'tempf': 'temperature_out',
            'tempinf': 'temperature_in',
            'temp1f': 'temperature_1',
            'temp2f': 'temperature_2',
            'temp3f': 'temperature_3',
            'temp4f': 'temperature_4',
            'temp5f': 'temperature_5',
            'temp6f': 'temperature_6',
            'temp7f': 'temperature_7',
            'temp8f': 'temperature_8',
            'humidity1': 'humidity_1',
            'humidity2': 'humidity_2',
            'humidity3': 'humidity_3',
            'humidity4': 'humidity_4',
            'humidity5': 'humidity_5',
            'humidity6': 'humidity_6',
            'humidity7': 'humidity_7',
            'humidity8': 'humidity_8',
            'batt1': 'battery_1',
            'batt2': 'battery_2',
            'batt3': 'battery_3',
            'batt4': 'battery_4',
            'batt5': 'battery_5',
            'batt6': 'battery_6',
            'batt7': 'battery_7',
            'batt8': 'battery_8',
            'soilmoisture1': 'soil_moisture_1',
            'soilmoisture2': 'soil_moisture_2',
            'soilmoisture3': 'soil_moisture_3',
            'soilmoisture4': 'soil_moisture_4',
            'soilmoisture5': 'soil_moisture_5',
            'soilmoisture6': 'soil_moisture_6',
            'soilmoisture7': 'soil_moisture_7',
            'soilmoisture8': 'soil_moisture_8',
            'soilbatt1': 'soil_battery_1',
            'soilbatt2': 'soil_battery_2',
            'soilbatt3': 'soil_battery_3',
            'soilbatt4': 'soil_battery_4',
            'soilbatt5': 'soil_battery_5',
            'soilbatt6': 'soil_battery_6',
            'soilbatt7': 'soil_battery_7',
            'soilbatt8': 'soil_battery_8',
            'windspeedmph': 'wind_speed',
            'windgustmph': 'wind_gust',
            'winddir': 'wind_dir',
            'solarradiation': 'solar_radiation',
            'uv': 'uv',
            'totalrainin': 'rain_total',
            'rainratein': 'rain_rate',
            'wh25batt': 'wh25_battery',
            'wh26batt': 'wh26_battery',
            'wh40batt': 'wh40_battery',
            'wh65batt': 'wh65_battery',
             'pm25_ch1': 'pm2_5',
            'pm25batt1': 'pm25_battery',
            'lightning': 'lightning',
            'lightning_time': 'lightning_time',
            'lightning_num': 'lightning_num',
            'wh57batt': 'wh57_battery',
            'leak_ch1': 'leak_1',
            'leak_ch2': 'leak_2',
            'leak_ch3': 'leak_3',
            'leak_ch4': 'leak_4',
            'leakbatt1': 'leak_battery_1',
            'leakbatt2': 'leak_battery_2',
            'leakbatt3': 'leak_battery_3',
            'leakbatt4': 'leak_battery_4',
        }

        IGNORED_LABELS = [
            'PASSKEY', 'dateutc', 'stationtype', 'model', 'freq', 'baromrelin',
            'maxdailygust', 'eventrainin', 'hourlyrainin', 'dailyrainin',
            'weeklyrainin', 'monthlyrainin', 'yearlyrainin',
            'pm25_avg_24h_ch1', 'winddir_avg10m', 'windspdmph_avg10m',
        ]

        def __init__(self):
            self._last_rain = None
            self._rain_mapping_confirmed = False

        def parse(self, s):
            pkt = dict()
            try:
                data = _cgi_to_dict(s)
                pkt['dateTime'] = self.decode_datetime(
                    data.pop('dateutc', int(time.time() + 0.5)))
                pkt['usUnits'] = weewx.US

                # some devices (e.g., HP2551_V1.5.7) emit something that looks
                # a lot like ecowitt protocol, but not quite.  one thing that
                # they get wrong is the rain - that have no totalrainin.  so
                # for those devices, substitute a different cumulative rain
                # measurement.  do this only once, and do not be fooled by
                # partial packets.
                if not self._rain_mapping_confirmed:
                    if 'totalrainin' not in data and 'yearlyrainin' in data:
                        self.LABEL_MAP.pop('totalrainin')
                        self.LABEL_MAP['yearlyrainin'] = 'rain_total'
                        self._rain_mapping_confirmed = True
                        loginf("using 'yearlyrainin' for rain_total")
                    elif 'totalrainin' in data:
                        self._rain_mapping_confirmed = True
                        loginf("using 'totalrainin' for rain_total")

                # get all of the other parameters
                for n in data:
                    if n in self.LABEL_MAP:
                        #pkt[self.LABEL_MAP[n]] = self.decode_float(data[n])
                        # Oliver, 18.06.20 further processing requires a num so set 0 to prevent float-conversion error
                        pkt[self.LABEL_MAP[n]] = self.decode_float(data[n]) if data[n] != '' else 0
                    elif n in self.IGNORED_LABELS:
                        val = data[n]
                        if n == 'PASSKEY':
                            val = 'X' * len(data[n])
                        logdbg("ignored parameter %s=%s" % (n, val))
                    else:
                        loginf("unrecognized parameter %s=%s" % (n, data[n]))

                # get the rain this period from total
                if 'rain_total' in pkt:
                    newtot = pkt['rain_total']
                    pkt['rain'] = self._delta_rain(newtot, self._last_rain)
                    self._last_rain = newtot

            except ValueError as e:
                logerr("parse failed for %s: %s" % (s, e))
            return pkt

        @staticmethod
        def decode_float(x):
            # these stations send a value of -9999 to indicate no value, so
            # convert that to a proper None.
            x = Consumer.Parser.decode_float(x)
            return None if x == -9999 else x
...

Thanks

[Rainer Lang]

we recommend NOT to use the intereptor driver for a GW1000/WH2650 console but to use the meanwhile existing

GW1000 API driver instead - otherwise you tend to run into multiple issues (and unnecessarily use the

console's "Customized"server option which exists only once) due to the many extra sensors being released.

The GW1000 API driver covers all existing FineOffset/Ecowitt (clone) sensors and is very stable.

Gary did an excellent job here.

https://github.com/gjr80/weewx-gw1000

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/e6761a59-bb0f-40c1-b2a5-9d22898fa869n%40googlegroups.com.

[Johann Destombes]

Thanks, I just install gw1000 driver and it's works great, I just need a storm to see if everything is well record.

When I read the GW1000driver wiki I understand that the driver record somewhere more extended data like wh57 battery level or  timestamp of last lightning strike .

In my case Weewx record archive data in a mysql database but I was not able to find where these data are stored. Are they somewhere in the archive database or are they just loop data ?

[Rainer Lang]

While the GW1000 API driver provides you all the fields you can see in the field map (--> wiki, https://github.com/gjr80/weewx-gw1000/wiki), weewx archives/stores/writes to the database only the fields which are contained in its database schema.

If you started with weewx 4.x.x that's most likely the wview_extended schema (what you have you can see in your weewx.conf

in [DataBindings]     [[wx_binding]].

It resides (under Debian/Raspbian) at /usr/share/weewx/schemas as wview_extended.py

If your schema is still the old wview schema because you created your database with weewx V3.x.x or earlier, you will have to change that (see wiki [https://github.com/weewx/weewx/wiki] and customization guide [http://www.weewx.com/docs/customizing.htm]).

If the driver provides fields for processing which are not in the database schema, they will not be archived.

There a several ways how to change that:
- repurposing existing but not used fields
- extending you database schema ... (also wview_extended can be further extended 😉)

There is a good, brief description of the options to be found at

https://www.wxforum.net/index.php?topic=34857.msg355511#msg355511

I personally went for the re-purposing method (even though I acknowledge that extending the DB schema is more elegant)
to accommodate my WH41, WH51 and WH45 data in the database. But the same applies for the battery status data etc.

e

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/9662ecd3-1b91-46b3-825a-2fcd84d0e632n%40googlegroups.com.

[gjr80]

On Tuesday, 13 April 2021 at 18:14:35 UTC+10 lang....@googlemail.com wrote:

There is a good, brief description of the options to be found at

https://www.wxforum.net/index.php?topic=34857.msg355511#msg355511

Please keep in mind that my post linked above was made over two years ago when WeeWX v.3.8.1 was the current release; v4.5.1 is the current release and things have changed. In particular the process for adding (and removing) columns (observations) from an existing archive/database has been greatly simplified. I suggest that if you want to add to or change the database you refer to the Customizing the database section of the Customization Guide - it contains the most up to date information.

Gary

[Johann Destombes]

Thanks very much! Indeed it's well documented and very simple with wee_database --add-column=XXXX.

Now I can play and display all my new data.