Example of how to add second data source that gets data via loop record

[Earl Baugh]

Hello,

I'm new to weewx and would like to add some additional data being collected by some gas sensors that I have installed at my house (they help track gases that some other data sites track as part of pollution tracking)

I read the section on Adding a Second Datasource, but that doesn't quite match my use case.    I'm feeding all data into the system via the WeeWX-MQTTSubscribe add on.  I have an outside program that is pulling data from the sensors periodically and then posting it to MQTT, and the WeeWX-MQTTSubscribe ingests the data turning in into LOOP records (at least that's my current level of understanding... I'm only on day 3 here with weewx...)

This works fine for the data that maps to existing fields.  However, I also gather data on C2H5OH, C3H8, C4H10, CH4 and H2.... (the quality of which I am still assessing, but it's at least being measured by the sensors)

Looking at the Adding a Second Datasorce example comes close, but it's not clear to me how to get the data from the LOOP record into the archive record (the new_archive_record function).   

Is there an example I've just missed in my readings?  Or could someone point me towards an example of an add on that adds this?

Thanks in advance!

Earl

[Rich Bell]

Earl,

I’ll take a first crack at this, and the WeeWX experts can clarify/help as needed. WeeWX really does all the heavily lifting. Whatever name/values are in the loop data that MQTTSubscribe generates will be accumulated into an archive record. You should see this happening if you run WeeWX from the command line.

But, if the name is not known to WeeWX, it will not be added to the database. Note, I’ve never done this, but I’d start by reading this http://weewx.com/docs/customizing.htm#add_archive_type

rich

[Greg Troxel]

Earl Baugh <ea...@baugh.org> writes:

> I read the section on Adding a Second Datasource, but that doesn't quite
> match my use case. I'm feeding all data into the system via the

> WeeWX-MQTTSubscribe <https://github.com/bellrichm/WeeWX-MQTTSubscribe> add

> on. I have an outside program that is pulling data from the sensors
> periodically and then posting it to MQTT, and the WeeWX-MQTTSubscribe
> ingests the data turning in into LOOP records (at least that's my current
> level of understanding... I'm only on day 3 here with weewx...)
>
> This works fine for the data that maps to existing fields. However, I also
> gather data on C2H5OH, C3H8, C4H10, CH4 and H2.... (the quality of which I
> am still assessing, but it's at least being measured by the sensors)

Basically you need to extend your database with new fields, and also map
the topics to the fields.

http://www.weewx.com/docs/customizing.htm#add_archive_type

But I would advise reading the entire customization guide through end to end.


Also, I suspect that people would be interested in knowing more about
your sensors (and perhaps getting some), if you have a link or brief
description and pointers.

[Rich Bell]

Woops, my bad... I think I remember there might bevanother step to tell WeeWX to put it in the archive record.I’ll do a bit more research

rich

[Rich Bell]

This is what I was thinking needs to be done to accumulate loop data into archive data

https://github.com/weewx/weewx/wiki/Accumulators

rich

[Earl Baugh]

Yes, I've read

1) http://www.weewx.com/docs/customizing.htm#Adding_2nd_source

and

2)http://www.weewx.com/docs/customizing.htm#add_archive_type

From the combined two sections, I can see how to add the columns.

However, it wasn't clear what f I need to write code like listed in Adding_2nd_source.

That section shows a function : download_total_power()

which magically gets the reading of the current total power ever used.

That's what I wasn't sure about.  

From what I'm reading between the lines here, it seems that I'm hearing that if I can generated a LOOP record for say H2.

If I just add columns (as per the add_archive_type) it will get "automagically" mapped?

Currently I output records like this : {"dateTime":1593627602,"extraTemp1":78.818}

For a new gas record I'd output a record like this {"dateTime":1593627602,"h2":.0002} (for whatever value I have)

That plus what's in the add_archive_type link is all that's required?

Earl

[Rich Bell]

I’m pretty sure you also need this 

https://github.com/weewx/weewx/wiki/Accumulators

But, I will shut up and let the experts answer.

[Earl Baugh]

Sensor wise, I have three WioNodes (https://www.seeedstudio.com/Wio-Node.html).

Connected to them I have:

On #1 -- a Multichannel gas sensor ( https://www.seeedstudio.com/Grove-Multichannel-Gas-Sensor.html)

and a dust sensor ( https://www.seeedstudio.com/Grove-Dust-Sensor-PPD42NS.html)

On #2 -- A Temp/Humidity/Barometer  Sensor( https://www.seeedstudio.com/Grove-BME280-Environmental-Sensor-Temperature-Humidity-Barometer.html

On #3 --  A Temp/Humidty sensor ( https://wiki.seeedstudio.com/Grove-TempAndHumi_Sensor-SHT31/)

I found a nice 3d case design for the WioNode and a slightly modified one for the sensors.

( think this is the case : https://www.tinkercad.com/things/dRPiUfLRj3w-wio-node-case)

What's nice about these is that it's just a simple process to set up:

1) Put in case

2) Plug in cables

3) onboard to wifi

4) select sensors in app

5) Click button and firmware is built and deployed (assuming you use supported sensors)

Then data can be accessed via REST call (which you can actually do via the app... it gives you the full REST call and a button for getting current values)

Easy Peazy.

Some sensors don't play together (the WioNode has two different configurations on the two plugs... so it just depends on which sensors and which interface...)  You can add a "grove hub" to a side if you wish, but given the cost, I just went with an extra WioNode for what I needed.  It also gives me the option of adding some other sensors (like the AQI sensor..)

Earl

[Earl Baugh]

Ok, so I go it solved, so figured I'd list solution here.

1) Created a file called "additionalGasSensors.py" and placed it in /usr/share/weewx/user (at least that's where it goes on my machine)

import schemas.wview_extended

schema_with_additionalGasSensors = {
    'table': schemas.wview_extended.table + [('c2h5oh', 'REAL')] + [('c3h8', 'REAL')] + [('c4h10', 'REAL')] + [('ch4', 'REAL')] + [('h2', 'REAL')],
    'day_summaries' : schemas.wview_extended.day_summaries + [('c2h5oh', 'SCALAR')] + [('c3h8', 'SCALAR')] + [('c4h10', 'SCALAR')] + [('ch4', 'SCALAR')] + [('h2', 'SCALAR')]
}

2) Followed the instructions on http://www.weewx.com/docs/customizing.htm#add_archive_type (didn't add any other code....just what I listed above)

Then "automagically"  WeeWX-MQTTSubscribe imports the Loop records it receives (an example :: {"dateTime":1593633630,"h2":0.08} )

And the H2 column (in this example) gets data... archiving works, etc.  

Thanks for the pointers... I wouldn't have thought that was all that I needed to do... but that's it... very easy.

Earl