WH57 and gateway 1100

[bell...@gmail.com]

I finally had some time to experiment with my WH57. Here is what I ended up doing.
I have the driver polling the 1100 every 20 seconds and my archive interval is 5 minutes.

‘Out of the box’ the number of strikes in an archive interval is persisted and the distance from the last strike, even if no strikes occurred in the interval. Like many others, I only wanted the distance if a strike occurred. In the [StdCalibrate][[Corrections]] I added the following.
lightning_distance = lightning_distance if lightning_strike_count and lightning_strike_count > 0 else None
This checks that the loop packet field lightning_strike_count has a value and it is greater than 0. Checking that the field has a value is important because the first loop packet after starting WeeWX sets it to None.

Persisting the last strike in an archive interval is nice, but I also wanted to persist the closest. I also figured if I was capturing the last strike, I might as well capture the first strike. I also decided to capture the time of the first and last strike. (Note, without writing some code I could not figure out a way to capture the time of the min (closest) strike.

I also decided that the lightning_distance field would persist the average distance to the strikes in the archive period. I did this for a couple of reasons. First, for more consistent naming conventions. Second, it would easily work with the Seasons skin. Note, I have no intention of using the additional first, last, and min values in the Seasons skin. I will use my WeeWX-JAS skin to display these.

The first thing to do is get the data into these additional fields. I used the [StdCalibrate][[Corrections]] section. I ended up with the following.
lightning_last_distance = lightning_distance if lightning_strike_count and lightning_strike_count > 0 else None
lightning_last_det_time = lightning_last_det_time if lightning_strike_count and lightning_strike_count > 0 else None

lightning_first_distance = lightning_distance if lightning_strike_count and lightning_strike_count > 0 else None
lightning_first_det_time = lightning_last_det_time if lightning_strike_count and lightning_strike_count > 0 else None

lightning_min_distance = lightning_distance if lightning_strike_count and lightning_strike_count > 0 else None

lightning_distance = lightning_distance if lightning_strike_count and lightning_strike_count > 0 else None

With these ‘corrections’, WeeWX now accumulates the lightning data into multiple fields. In the loop packet, all of the distance fields have the same value. Same with the time fields. Using the accumulator function, the first, last, and min values can be extracted and put into the archive record.

My [Accumulator] section looks like the following.
    # Additional lightning data, note lightning_last_det_time is below in the GW1000 section
    [[lightning_last_distance]]
        extractor = last
    [[lightning_first_distance]]
        extractor = first
    [[lightning_first_det_time]]
        extractor = first
    [[lightning_min_distance]]
        extractor = min
    # 'override' the setting that GW1000 driver has (I decided to set it here and delete from the GW1000 section)
    [[lightning_distance]]
        extractor = avg

Next, I used weectl database to add the new fields to the database.

Finally I updated the bin/user/extensions.py with the units for the new fields.
import weewx.units
weewx.units.obs_group_dict['lightning_last_distance'] = 'group_distance'
weewx.units.obs_group_dict['lightning_last_det_time'] = 'group_time'
weewx.units.obs_group_dict['lightning_first_distance'] = 'group_distance'
weewx.units.obs_group_dict['lightning_first_det_time'] = 'group_time'
weewx.units.obs_group_dict['lightning_min_distance'] = 'group_distance'

Known limitations.
If there is more than one strike is in a loop packet interval (20 seconds), the loop packet will have the number of strikes and the packet will have the distance to and time of the last strike.

Now to get some real data to experiment with displaying the data…
rich

[michael.k...@gmx.at]

Hi Rich,

that sounds to me as if your approach is as close as you can get, given that the Ecowitt devices only let you poll data in a given Interval. Also, the WH57 is more a "toy" than a sensor, missing out the vast majority of lightning strikes, and mine is set to the maximum sensitivity (but not indoor), and it often misses lightnings that are within less than 3km. Anyway, as I am running a couple of Ecowitt stations in parallel, I'll configure one of them the way you described here and compare the one to the other. Last night would have delivered good data:

[bell...@gmail.com]

Got some data, seems to be doing what I wanted/expected. I was confused by the lightning_first_det_time of 1722466487. One would have thought this should be in the previous archive record. Then I realized that this is probably because of the polling time of 20 seconds. The last loop packet for the archive record of 1722466500 was most likely created prior to 1722466487.

I have thought about increasing the polling frequency, but at this time I don’t think it is worth it to me.
+------------+------------------------+--------------------------+--------------------------+-------------------------+-------------------------+------------------------+
|  dateTime  | lightning_strike_count | lightning_first_det_time | lightning_first_distance | lightning_last_det_time | lightning_last_distance | lightning_min_distance |
+------------+------------------------+--------------------------+--------------------------+-------------------------+-------------------------+------------------------+
| 1722462300 | 1.0                    | 1722462266.0             | 10.563310264             | 1722462266.0            | 10.563310264            | 10.563310264           |
| 1722465000 | 1.0                    | 1722464831.0             | 14.912908608             | 1722464831.0            | 14.912908608            | 14.912908608           |
| 1722466500 | 1.0                    | 1722466382.0             | 12.42742384              | 1722466382.0            | 12.42742384             | 12.42742384            |
| 1722466800 | 2.0                    | 1722466487.0             | 10.563310264             | 1722466622.0            | 14.912908608            | 10.563310264           |
| 1722472800 | 1.0                    | 1722472693.0             | 6.21371192               | 1722472693.0            | 6.21371192              | 6.21371192             |
| 1722485100 | 1.0                    | 1722484859.0             | 6.21371192               | 1722484859.0            | 6.21371192              | 6.21371192             |
| 1722485700 | 1.0                    | 1722485491.0             | 6.21371192               | 1722485491.0            | 6.21371192              | 6.21371192             |
+------------+------------------------+--------------------------+--------------------------+-------------------------+-------------------------+------------------------+
rich

[bell...@gmail.com]

I thought about increasing the polling frequency. But I’d want to do some performance benchmarking first. Things like how long does it take to send the request and process it - to give me an idea of a reasonable minimum value. I’d also want to check on the load on the system. And ultimately this would just decrease the possibility of multiple strikes in a loop packet or the first strike “belonging” to the previous archive record.
I also have played around with SDR. I think I could use rtl_433 and MQTTSubscribe to get the data. But my other sensors are frequency 433 and my ecowitt is 915, so I’d have to run another rtl_433 instance. I may still do it, it could be a good MTTSubscribe wiki page.
At this time neither idea is worth the effort to me and I’ve already got too many “irons in the fire”…..
Looking forward to what you learn.
rich

[michael.k...@gmx.at]

I've set my polling rate to 10s, because the WH90 provides data in about that frequency. Works without any hiccups in my setup, but I use GW2000 hardware.

I could again rant about the fact, that the ecowitt don't emit events for such data as lightning strikes, like weatherflow devices do. But summing everything up, it wouldn't make that big of a difference, because of the already mentioned data quality on the sensor's side in the first place.

If one really cares about lightning events in a given radius around the station, participating https://www.blitzortung.org/de/live_lightning_maps.php would be the way to go and then write an extension that extracts the relevant data for the individual site in realtime.

[michael.k...@gmx.at]

I've set up my test installation with the settings above: https://kainzbauer.net/weather/Test/Rif/en/index.html

You can compare the lightning chart with the "vanilla" installation here: https://kainzbauer.net/weather/Rif/en/index.html

[michael.k...@gmx.at]

I've set up the new test service after 20:00 CEST(18:00 UTC), so comparing only makes sense for data after the change. But more thunderstorms are predicted for the coming hours. :)

[bell...@gmail.com]

Thanks for the real world experience on a 10 second polling interval. I’ll probably give that a go. 

Also, thanks for the URL. Looks like an interesting place to explore.

rich

[michael.k...@gmx.at]

What a day, what a coincidence. One Thunderstorm after another:
Vanilla (left) vs. the above config adaptions (right):

The more lightnings in an archive_interval, the more difference (obviously!) A storm with some 300+ lightnings detected in one hour would be interesting. Thunderstorm season seems coming to an end at my place, anyway, I'm still convinced it's as close as you can get for the given hardware.

[michael.k...@gmx.at]

Something like this could solve your rtl_433 instance "problem": https://www.rtl-sdr.com/rtl_433-ported-to-esp32-microcontrollers-with-cc1101-or-sx127x-transceiver-chips/ and scan for the WH57's (or other) transmissions. Every single lightning detected by the WH57 could then be converted to a MQTT message and be received by mqttsubcribe. I'm thinking about buying something like this https://www.lilygo.cc/products/lora3?_pos=1&_sid=0081a9887&_ss=r but I don't see I have enough spare time to play with it.

Letting this little device do the 433/868/915 to MQTT conversion as a standalone device, could make things a lot easier.

[bell...@gmail.com]

Interesting. Thanks for the information. Yeah, at this time I don’t have enough time either. Might be a winter project….

I’m currently displaying the closest strike. I have found I miss that I can’t display the date/time of it. So the first thing I need/want to do is figure out how to capture the time of the closest strike. I am thinking that I will need to write a small service to perform this. But even this will need to wait a bit.

[bell...@gmail.com]

I saw this conversation, https://groups.google.com/g/weewx-development/c/TXH8pgRs51c/m/jJrnulyiAQAJ, and thought I should circle back and update this thread on where I am at.

1. I have decreased the polling time to 10 seconds and all is behaving well.
2. I really wanted to capture the time of the closest strike. So, I wrote a small service that captures min, max, first, and last observation values and times in the archive period. It’s still very much beta code and is in my ‘extras’ repositoryhttps://github.com/bellrichm/WeeWX-Extras/blob/master/bin/user/observationtime.py; so has limited documentation and needs to be manually installed.
3. I am experimenting with my extra rtl-sdr dongle and ‘capturing’ strikes ‘real time’. Of course we are past thunderstorm season now… But it is interesting what else I came ‘capturing’, water meter, gas meter, electric meter…..
4. I am think of updating MQTTSubscribe to deal with dependent fields. This would eliminate the need for the ‘corrections’ configuration, which would make configuring the order of services in the [Engine] section easier.

rich