How to properly record Lightning Data

[Earl Baugh]

Got a few questions about lightning data.  

(to start with, I'm utilizing my own sensors and posting data from each into an MQTT channel, and using the Weewx MQTT reader to take that data and convert it into loop records and then let the rest of the system take care of it from there...)

I see in the weewx db schema that there are:

 'lightning_distance',        'REAL'
 'lightning_disturber_count', 'REAL'
 'lightning_energy',          'REAL'
 'lightning_noise_count',     'REAL'
 'lightning_strike_count',    'REAL'

When I receive an interrupt from the Lightning sensor,  I can post a record with lightning_distance and lightning_energy.  That seems fairly clear.

But how do I post a noise or disturber record?   That isn't clear.  Given I just see a count, I'm not sure what kind of data would be needed to cause that to increment.

I'm assuming weewx updates the strike count based on the new lightning record.  But that's a guess...  is that guess accurate? 

Thanks in advance for any pointers...

Earl

[Earl Baugh]

Are there any references to how these are supposed to be added?

Was there any specific release notes on this?

I did find one forum post when they were added to the distribution, but no details on their use...

Earl

[vince]

On Friday, November 6, 2020 at 10:53:12 AM UTC-8, Earl Baugh wrote:

I see in the weewx db schema that there are:

 'lightning_distance',        'REAL'
 'lightning_disturber_count', 'REAL'
 'lightning_energy',          'REAL'
 'lightning_noise_count',     'REAL'
 'lightning_strike_count',    'REAL'

[...]

 

But how do I post a noise or disturber record?   That isn't clear.  Given I just see a count, I'm not sure what kind of data would be needed to cause that to increment.

Some gear has noise+disturber info available.  Some do not.

All those elements are available if you choose to use them.

But they're not mandatory.

You can use them, or ignore them, or use the available fiels in the db for other uses (count every time your nest doorbell triggers and call it a disturber event :-)

 

I'm assuming weewx updates the strike count based on the new lightning record.  But that's a guess...  is that guess accurate? 

Not in the least.   Your extension/service/driver would have to monitor some sensor and choose to save it in those elements in the db. 

It's all up to the software developer who writes the extension or service or driver or however you chose to read the sensor's available measurements and get them into the db.   After that weewx can handle the graphing and db querying etc. kind of heavy lifting for you, but you kinda have to do the low-level stuff to get the sensor info into the db.

[Earl Baugh]

On Nov 7, 2020, at 4:47 PM, vince <vince...@gmail.com> wrote:



Well in this case, I’m the SW developer :-) ( it’s my day job as well ).  I’m just trying to understand the thoughts behind the “count” fields.  I have access to disturber and noise events. 

Would it make sense to post an event that has a value of 1 for count every time?   By the name, I was thinking that this was a running total, or number that occurred since last report.   

I’ve added a number of extra fields to my DB already (different pollution gases, uSieverts — have a Geiger counter feeding data :-)  ) So I’m extending already, but trying to understand existing fields expected purpose. Most skins, etc. have very useful behavior already, so trying to leverage as much as possible.    Aside from battery slots ( won’t ever use those, have everything on live power) I’m slowly trying to see what I can fill in. That’s been part of the fun. 

Earl 

[vince]

On Saturday, November 7, 2020 at 2:00:31 PM UTC-8, Earl Baugh wrote:

Would it make sense to post an event that has a value of 1 for count every time?   By the name, I was thinking that this was a running total, or number that occurred since last report.   

Can't really speculate what you're trying to accomplish. I could see counting strikes and being able to graph strikes vs. time such as "detected 12 strikes in the past 5 minutes". 

I never saw value in the disturber stuff when I had WeatherFlow gear that had a lightning sensor.  I always thought of that as just 'noise the sensor picked up', but I might be wrong there.

I guess my suggestion is don't sweat "expected purpose" too much.  You can make the purpose anything you want.    The type of the db element kinda tells you enough to figure out what typical usage might be.  Some are counting things.  Some are values.

[Tom Keffer]

Generally, the way WeeWX handles extensive types (such as rain, ET, energy, etc.), is that it's the amount of "stuff" seen during the observation period. So, in this case, for LOOP packets, it's the number of strikes detected since the last LOOP packet. For archive records, it's the number of strikes detected during the archive period. 

The latter will be calculated automatically by the accumulators. See the wiki article Accumulators for how accumulators work. In particular, note that the default type of extractor for lightning_strike_count is 'sum'. That means, the value in the record will be filled in with the sum of all values seen in the LOOP packets over the archive period, in other words, the total number of lightning strikes.

Hope that helps.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/A39AB5D9-146C-4E44-8F99-97F72976B5CE%40baugh.org.

[Earl Baugh]

Tom & Vince,

All very helpful info!  It aligns with what I was thinking so far.  This wiki page on accumulators was very informative.

Still got a couple questions -- I see Lightning Strikes as an accumulator.  But nothing for Noise_Count and Disturber_Count.

I'd expect those were accumulators but I don't see anything that would feed into them.

I could, I guess, add new records for Noise and Disturber and then accumulate them into the _Count fields... it seems like

there is a gap here in what I see.

Thoughts?

And how might I give inputs on some data that I'd like to suggest be added to the "main-line" rather than things just in

my copy.   Other other feature suggestions?

Earl

You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/O6A9fEFcnOc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEDLCa5SRJQp5L4r7QuCeUFW-CXYjvZoEakcb8TWU%2B0Eow%40mail.gmail.com.

[Tom Keffer]

If a new type appears in the data stream, a new "scalar" accumulator will automatically be created for it. When it comes time to extract a datum for a new record, the average will be used by default. If you don't like that, then you can specify a different "extractor."

You can suggest some changes here (or, better, in weewx-development), or submit a PR on GitHub.

-tk

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAAucK2Uzj9PfPfEpqZk6U-1m8h1hVEUoWGAk4nBYxaY0qa1BYg%40mail.gmail.com.

[Earl Baugh]

So, how would a "Lighting_Noise" value be saved in the DB?

Anywhere?   Or would it just go into the "Lightning_Nose_Count"? 

(that's where I'm confused)

Earl

P.S. Thanks for the info where to suggest changes.    I'll submit an Enhancement Request for the other data fields that I'm now capturing.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEDKERxOSwAHFYqNSq_%3DFTotZRSEOVN%2B3MiGgFqbscdFSQ%40mail.gmail.com.

[Tom Keffer]

There's no magic here. If the record has a type 'lightning_noise_count', and the database schema has a matching entry, then it will be put in the database. But, the names have to match precisely, including case. 

The wview_extended schema has 'lightning_noise_count'. A record with "Lightning_Count" is not going to cut it.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAAucK2V5TN6ekjFH5hK6fnPUEUHtF3RNESuvXQzA1qR%3D-fkcyw%40mail.gmail.com.

[Earl Baugh]

Ok, so that's where I'm confused.  (yes I know the packets have to match exactly, have already had some earlier bugs because of that)

The DB schema has :

 'lightning_distance',        'REAL'
 'lightning_disturber_count', 'REAL'
 'lightning_energy',          'REAL'
 'lightning_noise_count',     'REAL'
 'lightning_strike_count',    'REAL'

So, when a strike occurs, I should generate  a record (as an example) with :

{"dateTime":1605212295, "lightning_distance":3.5, "lightning_energy: "46.3", "lightning_strike_count": "1.0"}

I was assuming I didn't supply lightning_strike_count and a lightning_distance record would be counted,

more like this:

{"dateTime":1605212295, "lightning_distance":3.5, "lightning_energy: "46.3"}

I guess there are a number of use cases here.  If "lightning_strike_count" was ever, say 2.0, then should "lightning_distance" be the average distance?  Or is the system just trying to be flexible for all users (and you just wouldn't provide lightning_distance in this case)

I was leaning towards the 2nd case, so I was confused as to what would be summed up for noise and disturber.

But if the first is the use, then I need to adjust my code already to provide the count..

Earl

p.s. I've not looked at the API enough to know how the accumulated value is retrieved... so that's another place that may provide clarity for me... maybe my assumption that "count" would be the value over a time window (not that it would be "accumulated" across a number of count records...) is incorrect, that also would be clarifying.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEDqsy-6KpabjBee_%3DgLCRPnS_maCVwP84JbJmzAokkb3A%40mail.gmail.com.

[Tom Keffer]

Frankly, there is no formal semantics for what any of the lightning quantities mean. They have appeared in various sensors that we have seen through the years (more precisely, that Matthew has seen), so we included them in the extended schema. But, their definition varies from sensor to sensor (much like 'radiation' or 'luminance'). 

So, it's up to you to supply meaning. Maybe your definition will eventually catch on! :-)

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAAucK2W-mqJu2_aXKiwKN2Em1mjDaXBN0XrjpymuPoAnukRCRA%40mail.gmail.com.

[tarob...@gmail.com]

Not sure if this would help you Earl but I've set up to capture lightning as follows :

weewx.conf 

[SDR]

    [[sensor_map]]

        lightning_distance = distance.xxx.AcuriteLightningPacket

        strikes_total = strikes_total.xxx.AcuriteLightningPacket

        lightning_disturber_count = rfi.xxx.AcuriteLightningPacket

        lightning_noise_count = active.xxx.AcuriteLightningPacket

        lightning_energy = exception.xxx.AcuriteLightningPacket

    [[deltas]]

        rain = rain_total

        lightning_strike_count = strikes_total

[Accumulator]

    [[lightning_strike_count]]

        extractor = sum

    [[lightning_distance]]

        extractor = min

    [[lightning_noise_count]]

        extractor = sum

I don't exactly know what rfi, active, and exception are in the data packets but decided to capture them anyways using existing DB schema titles.

I use the Seasons Skin and added lightning data to Current Conditions as "Strikes Today" (total strikes for the current day with the timestamp of the last strike) and "Closest Strike" (closest strike in miles with timestamp of when it occurred). I also have a bar chart that works very similar to rain accumulation (hourly total for day, etc).