weewx with multiple stations supported by a driver

[Till Maas]

Hi,

I am currently developing a driver for Tinkerforge sensors. Their
architecture is similar to this:

PC <-- USB -- Tinkerforge Master Brick +- Bricklet -- Weather Station
| `- Sensors
|
`- Bricklet2 -- ...

So a PC supports probably several Master Bricks, each master Brick
several Outdoor Weather Bricklets (they contain the wireless receiver)
or other sensors and each Outdoor Weather Bricklet supports 255 wireless
sensors and 255 wireless weather stations. The weather stations provide
wind speed, direction, gust, humidity, temperature and rain values. The
wireless sensors temperate and humidity.

Since I only have one weather station at the moment, I need to support
only one. But since the hardware supports more, I wonder how to do this
with weewx. It seems that the database schema does not have an ID for
different weather stations. So I guess each weewx instance/config file
is meant to accommodate only one weewx station. I guess this logical
weewx station could still consist of one Tinkerforge weather station and
multiple extra sensors because of fields like
"extraHumid1".."extraHumid8" in the schema. Are there some best
practices how to map those values in a driver from different sensors?

What are your comments about this?

Thanks
Till

[mwall]

ah, the "where should we do the aggregation?" problem

you should consider where you want the aggregation to happen.  there are at least three options:

- when the data are collected

- when the data are stored

- when the data are reported

baseline for weewx databases is one-per-driver, but as you noted you can feed more data into a database by adding one or more services to augment the data feed from the driver

some people just get everything into MQTT as soon as they can, then chuck it all into influx (or other timeseries database).  then you can throw grafana at it and go wild.

you might find these weewx wiki pages helpful:

https://github.com/weewx/weewx/wiki/intake-strategies

https://github.com/weewx/weewx/wiki/dashboards

[Greg Troxel]

Till Maas <opens...@till.name> writes:

> So a PC supports probably several Master Bricks, each master Brick
> several Outdoor Weather Bricklets (they contain the wireless receiver)
> or other sensors and each Outdoor Weather Bricklet supports 255 wireless
> sensors and 255 wireless weather stations. The weather stations provide
> wind speed, direction, gust, humidity, temperature and rain values. The
> wireless sensors temperate and humidity.

I think the big question from the weewx viewpoint is "what is a weather
station".

I think the answer at the high level is "a collection of sensors that
are more or less in one place", and this is what is reflected in a
single database. In this high level view, sensors that are not in the
same package or from the same vendor can still be aggregated.

From the driver viewpoint, a weather station is a thing you can talk to
that provides some number of sensors. It may or may not have multiple
pieces and wireless -- the point is that there is a common protocol to
talk to it. And, from the weewx viewpoint, there is "driver" which is
the main thing, and "service" which are extra things. I would suggest
that the driver/service distinction is arbitrary and becoming less
important. The world used to be "buy a weather station (collection of
sensors with an interface)" and "hook it up to weewx to manage it".
Now, gluing together a system of a bunch of sensors that did not
necesssily come together is more common.

You are using "weather station" to mean "a thing that has some number of
sensors that sends data to a bricklet", which is not quite the same as
the weewx view.

As an extreme example, I have a Davis Vantage Pro2 and ingest that into
weewx, entirely normally. I have a friend with a Pro2, some 10s of km
away, and he runs weewx (also normally) and publishes data to mqtt. I
have a home assistant instance that shows both of these (no weewx
involved, other than that both weewx instances publish).

Now, I could use a mqtt service to put the other station's
temps/etc. into my weewx database so that I would have "two weather
stations" in my weewx instance. But this really seems contrary to the
weewx way of managing a single logical "weather station" which may
include some extra sensors, perhaps more temps, perhaps things like
PM2.5 not included in the base system.

> Since I only have one weather station at the moment, I need to support
> only one. But since the hardware supports more, I wonder how to do this
> with weewx. It seems that the database schema does not have an ID for
> different weather stations. So I guess each weewx instance/config file
> is meant to accommodate only one weewx station. I guess this logical
> weewx station could still consist of one Tinkerforge weather station and
> multiple extra sensors because of fields like
> "extraHumid1".."extraHumid8" in the schema. Are there some best
> practices how to map those values in a driver from different sensors?

I'm not sure about best practice - it really depends on what those
sensors are measuring and what they mean.

Let me ask you what you are trying to do with "multiple weather
stations" in the tinkerforge sense. Do you want to report them as
separate logical things? Average them?

Why does it matter that tinkerforge calls things that also do rain/wind
"weather station" while they call things that only do
temperature/humidify "sensor"? In what fundamental sense are they
really different?


You could write a driver for tinkerforge stuff with a notion that all of
the tinkerforge sensors (what they call sensors and what they call
stations both) are available behind a single interface.

But, I think the real problem is that weewx's database schema, while of
course flexible, is (inherited and) designed around the notion of
storing an analyzing data from a traditional weather station, meaning
one that captures the traditional in/out temp/humidity, pressure, rain,
wind and perhaps also extra things like uv and a few more temp sensors.
It sounds like you have a much grander vision of what you are
measuring. It might help to articulate what you see as an example
system setup that goes beyond what's normal now.

Hope this helps and wasn't too rambling.

[Till Maas]

Hi Greg,

On Wed, May 06, 2020 at 01:01:24PM -0400, Greg Troxel wrote:
> Till Maas <opens...@till.name> writes:
>
> > So a PC supports probably several Master Bricks, each master Brick
> > several Outdoor Weather Bricklets (they contain the wireless receiver)
> > or other sensors and each Outdoor Weather Bricklet supports 255 wireless
> > sensors and 255 wireless weather stations. The weather stations provide
> > wind speed, direction, gust, humidity, temperature and rain values. The
> > wireless sensors temperate and humidity.
>
> I think the big question from the weewx viewpoint is "what is a weather
> station".

yes. If I understand correctly after also looking at Matthews link:

A weewx weather station consists of one driver and possibly multiple
services. It supports multiple temperature/humidity sensors but only one
rain/wind sensor.

> > Since I only have one weather station at the moment, I need to support
> > only one. But since the hardware supports more, I wonder how to do this
> > with weewx. It seems that the database schema does not have an ID for
> > different weather stations. So I guess each weewx instance/config file
> > is meant to accommodate only one weewx station. I guess this logical
> > weewx station could still consist of one Tinkerforge weather station and
> > multiple extra sensors because of fields like
> > "extraHumid1".."extraHumid8" in the schema. Are there some best
> > practices how to map those values in a driver from different sensors?
>
> I'm not sure about best practice - it really depends on what those
> sensors are measuring and what they mean.
>
> Let me ask you what you are trying to do with "multiple weather
> stations" in the tinkerforge sense. Do you want to report them as
> separate logical things? Average them?

I don't know. I don't have a use case for myself, therefore I was
wondering if it makes sense to support more than one of them. Since
there is no way to store most of it's data in weewx (rain & wind), it
does not seem to make sense. Not sure, if someone might care about wind
speed/direction in different height or points in an area.

> Why does it matter that tinkerforge calls things that also do rain/wind
> "weather station" while they call things that only do
> temperature/humidify "sensor"? In what fundamental sense are they
> really different?

Since weewx supports several temperature/humidity sensors, it seems it
makes sense to support several of those Tinkerforge sensors in one
driver instance but only one Tinkerforge weather station.

> You could write a driver for tinkerforge stuff with a notion that all of
> the tinkerforge sensors (what they call sensors and what they call
> stations both) are available behind a single interface.
>
> But, I think the real problem is that weewx's database schema, while of
> course flexible, is (inherited and) designed around the notion of
> storing an analyzing data from a traditional weather station, meaning
> one that captures the traditional in/out temp/humidity, pressure, rain,
> wind and perhaps also extra things like uv and a few more temp sensors.
> It sounds like you have a much grander vision of what you are
> measuring. It might help to articulate what you see as an example
> system setup that goes beyond what's normal now.

I don't have a vision for myself because I borrowed only one
Tinkerforge weather station and I will probably also buy only one but
maybe some extra sensors, too.

> Hope this helps and wasn't too rambling.

Thank you, it helped me to get some clarity.

Till

[Bill Burton]

Hello Till,

Right. In any case, WeeWX only supports rain and wind from one station. However, as temperature and humidity and a few other data elements can vary quite a bit at a single location, WeeWX supports multiple sensors in those cases.

For WeeWX, what could make sense is to initially write the driver to support the weather station's built-in sensors. There seems to be a well-defined Python API for the Outdoor Weather Bricklet at https://www.tinkerforge.com/en/doc/Software/Bricklets/OutdoorWeather_Bricklet_Python.html.

Then, supporting the HumidityV2 Bricklet would provide both temperature and humidity. The configuration in weewx.conf could be added in a way to support many temperature and humidity Bricklets. For example:

[TinkerForge]
    driver = user.tinkerforge
    host = localhost
    port = 4223
    bricklets = OutdoorWeather, IndoorTempHumidity, AtticTempHumidity, GreenhouseTempHumidity

    [[OutdoorWeather]]
        uid = XYZ  # assuming each weather station has a unique UID
        bricklet = OutdoorWeather
        ...

    [[IndoorTempHumidity]]
        uid = ABC
        bricklet = HumidityV2
        humidity = inHumidity
        temperature = inTemp

    [[AtticTempHumidity]]
        uid = DEF
        bricklet = HumidityV2
        humidity = extraHumid1
        temperature = extraTemp1


    [[GreenhouseTempHumidity]]
        uid = GHI
        bricklet = HumidityV2
        humidity = extraHumid2
        temperature = extraTemp2

The above is one theoretical way to map Bricklets within a driver configuration. Other Bricklet sensors could also be mapped the same way provided they can be mapped to an associated existing WeeWX data element. If you want to support Bricklet sensors that are not weather related, that's entirely possible but requires extending the database or using a second database for those data elements.

In theory, it should be possible to bridge the brickd daemon with WeeWX using MQTT - https://www.tinkerforge.com/en/doc/Software/API_Bindings_MQTT.html#api-bindings-mqtt and then use the WeewX MQTT subscribe driver to load the data via MQTT. However, brickd doesn't support MQTT natively so it requires running the tinkerforge_mqtt script as a daemon to relay the data from brickd to the MQTT broker.

> Why does it matter that tinkerforge calls things that also do rain/wind
> "weather station" while they call things that only do
> temperature/humidify "sensor"?  In what fundamental sense are they
> really different?

Since weewx supports several temperature/humidity sensors, it seems it
makes sense to support several of those Tinkerforge sensors in one
driver instance but only one Tinkerforge weather station.

Exactly.

Hope this helps,

-Bill

[Greg Troxel]

Till Maas <opens...@till.name> writes:

> A weewx weather station consists of one driver and possibly multiple
> services. It supports multiple temperature/humidity sensors but only one
> rain/wind sensor.

I don't think that's quite right. "weather station" there means "pile
of stuff that weewx gets data from". The driver/service distinction is
an arbitrary software thing and I think we shouldn't give it too much
architectural weight.

As for multiple temperature and only one rain, I don't think that's true
either. There is a database with a schema, and *by default* there are
single columns for rain, wind, inTemp, outTemp, and so on. There are
also extraTemp1 and so on by default.

The reason the schema has the above defaults is as I understand it
partly becuse wview did it this way, but I think it's really because a
typical high-end weather station (e.g. Davis) supports extra temp
sensors.

If you simply extend your database to have extraRain1, extraRain2, and
have whatever driver/service get that data and put it in the database,
you should be able to do this. There's nothing magic about rain vs temp
- it's that historically having extra temp sensors was normal and having
extra rain sensors was not.

So you could have an extraRain1 graph also in your skin.

I added inDewpoint to my database, because I wanted to record and graph
that. Granted that's extra vs a second of the same kind of sensor, but
weewx doesn't care.

The real issue is managing the data logically in terms of having lots
of sensors and deciding what the right output is.

>> Let me ask you what you are trying to do with "multiple weather
>> stations" in the tinkerforge sense. Do you want to report them as
>> separate logical things? Average them?
>
> I don't know. I don't have a use case for myself, therefore I was
> wondering if it makes sense to support more than one of them. Since
> there is no way to store most of it's data in weewx (rain & wind), it
> does not seem to make sense. Not sure, if someone might care about wind
> speed/direction in different height or points in an area.

I think without a plan you can't explain what is sensible.

Your notion of wind at varying heights is a good idea. Of course, most
people don't manage to even have the first wind sensor at 10m like they
should.

Overall, I think that we are arriving at a world of more flexible sensor
configuration and it's time to take a step back and look at the current
schema and perhaps make changing the schema to add sensor columns more
of a normal thing rather than a scary thing.

>> Why does it matter that tinkerforge calls things that also do rain/wind
>> "weather station" while they call things that only do
>> temperature/humidify "sensor"? In what fundamental sense are they
>> really different?
>
> Since weewx supports several temperature/humidity sensors, it seems it
> makes sense to support several of those Tinkerforge sensors in one
> driver instance but only one Tinkerforge weather station.

I think they aren't really different; you were just misled by extraTemp1
being prepopulated in the standard schema, becuase others before you
wanted that, while nobody before has talked about on the list wanting
multiple rain sensors.


Pivoting weewx's db to have unit numbers for sensors, or location codes,
is a huge amount of work and disruption and I don't see that happening.
Right now we have extraTemp1 which is kind of a denormalized table that
might instead look like

time | sensor number | temp

or

time | sensor type | sensor number | sensor value


but I think the standard view is that the db columns represent fields in
packets from Davis and similar stations, which itself have that kind of
denormalized schema.


Overall I'd encourage code to be written so that there can be both a
driver and a service, making most of it a library essentially, and to be
able to handle multiple bricks, with some internal interface that looks
like the second example above, and then a way to map that to weewx
database fields.

[Graham Eddy]

true. just a collection of data sources (and sinks) now, we have abstracted away from the hardware, term “driver” is just historical leftover.

the packet is just a train with data getting on and off along the way

[Vince Skahan]

On Friday, May 8, 2020 at 6:09:47 AM UTC-7, Greg Troxel wrote:

Till Maas <opens...@till.name> writes:

> A weewx weather station consists of one driver and possibly multiple
> services. It supports multiple temperature/humidity sensors but only one
> rain/wind sensor.

I don't think that's quite right.  "weather station" there means "pile
of stuff that weewx gets data from".  The driver/service distinction is
an arbitrary software thing and I think we shouldn't give it too much
architectural weight.

Terminology is hard and English is an inexact language.

Just reading between the historical lines a little, my interpretation has always been that a driver 'interfaces' weewx with a weather station console or equivalent, while a service 'extends' weewx to do something it doesn't do in the core software.

So starting back with its Davis-centric lineage, the Vantage 'driver' is what talked serial/usb to the Vantage console where the loop packets come from.  Fast forward to 2018 and consider the WeatherFlow stations that emit loop packet equivalents via UDP broadcast.   The WeatherFlowUDP 'driver' grabs that raw data off the network just like as if it had been physically connected to the weewx system.  Same definition applies.

For myself, I wrote a 'service' that looks up data from a raspi on my network via http GET and adds that data as a new element in my weewx db.   For historical reasons, I used the available extraTempNNN fields for that.  So I 'extended' weewx to do something it didn't do in its core.

As people got more creative, folks wanted to use code that was typically an extension (example - MQTT subscribing and saving to the db) as a 'driver' (making that how weewx gets its data in all case, using weewx as a general purpose db and graphing engine so to speak).  Some such extensions can be run either way....as an extension (integrate multiple sources of data) and as a service (be 'the' way weewx gets its data).    It's actually pretty flexible that way.

The db schema is where it gets rough, as there are infinite needs as the community gets more creative.  What do you call the element for 'how much battery life is left on your UPS' versus 'how much power is coming out of your 4th solar panel' and the like.

Weewx v4 'does' have the very extended schema as the default, so there are 'many' new elements pre-defined for folks to pick from.   Certainly the intent to cover almost all people's needs was there, even if we can infinitely argue over the precise field names in the v4-default bigger schema.

If you want infinite flexibility in database element names, just go with influxdb and get it over with...but with great flexibility comes great confusion sometimes.   At some point you'll need to write your skins and cheetah is not so flexible.   You're going to eventually need a mapping of your database element names somewhere in the actual code that runs.

example here:

• I have multiple weewx stations feeding influxdb
• a VP2 feeding it via MQTT
• a WeatherFlow SmartWeather station feeding it via some python code I wrote  (one 'Air' and two 'Sky' sensor suites)
  
• a WeatherFlow Tempest station feeding it via the same python code I wrote
  
• I have multiple arduino temperature sensor rigs feeding influxdb as well
• each write to influxdb individually over the network
• the dashboard is grafana which picks the right data elements from influxdb and displays it
  

So using temperature as an example:

• vp2 temperature is in influxdb topic vp2/loop and field outTemp
  
• wf data is in influxdb topic sensors/AR-00013349/wf/obs_air and field temperature
  
• tempest data is in topic sensors/ST-00006021/wf/obs_st and field temperature
• the inside arduino is in topic esp/obs/4535161 field degF
• the second arduino is in topic esp/obs/<itsHardwareID> field degF
• the third arduino is in topic esp/obs/<itsHardwareID> field degF
• the fourth arduino is in topic esp/obs/<itsHardwareID> field degF
• and so on

So the map of field name to what the heck it 'is' still has to exist.  It's in my grafana dashboard config.  And it's a pain to keep track of.

I at least settled on a naming convention that works enough for my needs:

• vp2/loop for the vantage, matching weewx's MQTT extension terminology
  
• sensors/<id>/wf/obs_air for WeatherFlow Air, matching its terminology
• sensors/<id>/wf/obs_sky for WeatherFlow Sky, matching its terminology
• sensors/<id>/wf/obs_st for WeatherFlow Tempest, matching its terminology
• esp/obs/<id>/degF for arduino temperature sensors

Using the hardware <id> lets me support multiple instances of the same hardware, as long as I can keep track of 'which room has arduino 1345631 in it' so the dashboard is user friendly enough. Keeping track of that mapping is a royal pain.  Figuring out what database and which element to store whatever in was a pain too at the time.

It's a hard problem to make things infinitely configurable and flexible, yet not require some kind of map 'someplace' to either get data into the database(s), or get and display data out of them. If you have a better way, please let us know.

And give Matthew's links in his reply a read.  It does a nice job explaining some possible options.  I picked/cobbled-together a variant that works for me well enough for my needs, but everybody's needs differ.

[Greg Troxel]

Vince Skahan <vince...@gmail.com> writes:

> Just reading between the historical lines a little, my interpretation has
> always been that a driver 'interfaces' weewx with a weather station console
> or equivalent, while a service 'extends' weewx to do something it doesn't
> do in the core software.

That's true, but I see it a bit more abstractly, that there must be one
driver which is the main thing, and services can be extra.

Do you think there are some schema columns which can be written only by
drivers, and other columns which are the domain of services? Or is it
simply a notion of main vs not main?

> So starting back with its Davis-centric lineage, the Vantage 'driver' is
> what talked serial/usb to the Vantage console where the loop packets come
> from. Fast forward to 2018 and consider the WeatherFlow stations that emit
> loop packet equivalents via UDP broadcast. The WeatherFlowUDP 'driver'
> grabs that raw data off the network just like as if it had been physically
> connected to the weewx system. Same definition applies.

sure

> For myself, I wrote a 'service' that looks up data from a raspi on my
> network via http GET and adds that data as a new element in my weewx db.
> For historical reasons, I used the available extraTempNNN fields for that.
> So I 'extended' weewx to do something it didn't do in its core.

You did, but getting extraTemp from your stuff instead of extra davis
sensors is more or less the same concept.

> As people got more creative, folks wanted to use code that was typically an
> extension (example - MQTT subscribing and saving to the db) as a 'driver'
> (making that how weewx gets its data in all case, using weewx as a general
> purpose db and graphing engine so to speak). Some such extensions can be
> run either way....as an extension (integrate multiple sources of data) and
> as a service (be 'the' way weewx gets its data). It's actually pretty
> flexible that way.
>
> The db schema is where it gets rough, as there are infinite needs as the
> community gets more creative. What do you call the element for 'how much
> battery life is left on your UPS' versus 'how much power is coming out of
> your 4th solar panel' and the like.

completely agreed

> Weewx v4 'does' have the very extended schema as the default, so there are
> 'many' new elements pre-defined for folks to pick from. Certainly the
> intent to cover almost all people's needs was there, even if we can
> infinitely argue over the precise field names in the v4-default bigger
> schema.
>
> If you want infinite flexibility in database element names, just go with
> influxdb and get it over with...but with great flexibility comes great
> confusion sometimes. At some point you'll need to write your skins and
> cheetah is not so flexible. You're going to eventually need a mapping of
> your database element names somewhere in the actual code that runs.

when you talk about influxdb, do you mean still using weewx to

get data from station
put into influxdb

get from influxdb and make graphs, as in skins

?

> - I have multiple weewx stations feeding influxdb
> - a VP2 feeding it via MQTT
> - a WeatherFlow SmartWeather station feeding it via some python code

> I wrote (one 'Air' and two 'Sky' sensor suites)

> - a WeatherFlow Tempest station feeding it via the same python code I
> wrote
> - I have multiple arduino temperature sensor rigs feeding influxdb as
> well
> - each write to influxdb individually over the network
> - the dashboard is grafana which picks the right data elements from
> influxdb and displays it

So here you are using weewx as a driver, and influxdb/grafana as your
main weather storage/display software?

[Vince Skahan]

On Friday, May 8, 2020 at 11:39:14 AM UTC-7, Greg Troxel wrote:

Do you think there are some schema columns which can be written only by
drivers, and other columns which are the domain of services?  Or is it
simply a notion of main vs not main?

No and no. The database doesn't know or care how the fields got populated.

• Some fields are populated by telegraf subscribing to MQTT topic that the weewx MQTT extension publishes
  
• Some fields are written by arduino doing a HTTP POST over the network
• Some fields are written by python influxdb calls from a standlone program I wrote
  

>
> If you want infinite flexibility in database element names, just go with
> influxdb and get it over with...but with great flexibility comes great
> confusion sometimes.   At some point you'll need to write your skins and
> cheetah is not so flexible.   You're going to eventually need a mapping of
> your database element names somewhere in the actual code that runs.

when you talk about influxdb, do you mean still using weewx to

  get data from station
  put into influxdb

  get from influxdb and make graphs, as in skins

?

I use influxdb to do data aggregation and comparison locally on the LAN, comparing a lot of sensors from multiple vendors.

In doing that, weewx does the talking to the Davis console hardware since I have a serial datalogger. If I had a tcp/ip enabled datalogger or equivalent, I wouldn't need weewx for that for most things, although weewx's automatic catchup feature to read the datalogger menu is super slick and very useful if we have power issues or the like.

Influxdb is just a time-series database.  It requires something else to do graphs+dashboards.  I use 'grafana' for that.  Other options exist.

>    - I have multiple weewx stations feeding influxdb
>       - a VP2 feeding it via MQTT
>       - a WeatherFlow SmartWeather station feeding it via some python code
>       I wrote  (one 'Air' and two 'Sky' sensor suites)
>       - a WeatherFlow Tempest station feeding it via the same python code I
>       wrote
>       - I have multiple arduino temperature sensor rigs feeding influxdb as
>    well
>       - each write to influxdb individually over the network
>    - the dashboard is grafana which picks the right data elements from
>    influxdb and displays it

So here you are using weewx as a driver, and influxdb/grafana as your
main weather storage/display software?

Not 'as a driver' perhaps.  More like 'using the appropriate hardware to talk to the VP2 console for me'.

I use weewx to connect to the VP2 console in order to get the data the hardware emits in LOOP packets.  I then jump through some hoops to get that data into influxdb so I can cook up an integrated dashboard.

I later do comparisons of VP2 vs. WF SmartWeather vs. WF Tempest vs. Arduino vs. PurpleAir hardware in one integrate dashboard.  To do that I need to get all the data into one place.  Influxdb is great for that, at the cost of setting it up and cooking up an integrated dashboard myself.

Examples:

• I use weewx's v4 extended schema to get me maxSolarRad graphed nicely in grafana, from data in influxdb.  I overlay the WeatherFlow Tempest solar sensor readings.  I overlay the WeatherFlow SmartWeather station solar sensor readings.   I make them available to the WF vendor as part of being a beta tester for tuning their sensors.  They use this to do things like make sure we don't exceed the maxSolarRad here in the usually grey Pacific Northwest.  You can't have a UV of 12 here :-)
  
  
• Similarly I overlay 5 sources of outside temperature for comparison and to help the WF vendor tune their gear.  I'm a beta tester for a couple of their products.
  
  
• Similarly I overlay 3 sources of outside pressure.  One thing I found is some gear corrects pressure for altitude.  Some gear doesn't.  Depends on the vendor.

(that kind of stuff)

If I tried to all this via weewx only, it is technically possible, but I'd pull my few remaining hairs out battling mapping stuff to available fields in the weewx v4 extended schema.   I'd then need to battle writing a custom cheetah skin to display it.  And I mean battle.  Cheetah is kinda brutal if you try to do to much in it.

Taking the influxdb + grafana path, I still need to write a custom dashboard, but it's much much simpler to handle.  Not free, but way less stressful.

But.....a local LAN-only grafana dashboard does 'not' get me the same functionality as weewx uploading to a static website on Internet via rsync.   Different tools for different jobs.

Hope this helps explain.

[Greg Troxel]

Vince Skahan <vince...@gmail.com> writes:

>> when you talk about influxdb, do you mean still using weewx to
>>
>> get data from station
>> put into influxdb
>>
>> get from influxdb and make graphs, as in skins
>>
>> ?
>
> I use influxdb to do data aggregation and comparison locally on the LAN,
> comparing a lot of sensors from multiple vendors.
>
> In doing that, weewx does the talking to the Davis console hardware since I
> have a serial datalogger. If I had a tcp/ip enabled datalogger or
> equivalent, I wouldn't need weewx for that for most things, although
> weewx's automatic catchup feature to read the datalogger menu is super
> slick and very useful if we have power issues or the like.
>
> Influxdb is just a time-series database. It requires something else to do
> graphs+dashboards. I use 'grafana' for that. Other options exist.

Thanks. That all makes complete sense. What I was getting at is that
weewx has as I see it 4 different things (being a bit hand wavy):

1) talk to hardware to get data
2) put that data in a weewx database
3) send data to CWOP etc.
4) generate web pages and publish them

and it sounds like you are using weewx to do (1), and maybe a bit of (2)
is happening, and you have (3) publish data to influxdb, either like
people send to CWOP, or like people publish loop packets to mqtt.

Hence you are really only using (1) and some sort of [put into influxdb]
part of weewx, and not using the rest. Which all makes sense to me, for
what you are doing.

> Not 'as a driver' perhaps. More like 'using the appropriate hardware to
> talk to the VP2 console for me'.

That's what I meant.

> I use weewx to connect to the VP2 console in order to get the data the
> hardware emits in LOOP packets. I then jump through some hoops to get that
> data into influxdb so I can cook up an integrated dashboard.

Would be nice if that were standard, included code.

> Examples:

That's very cool stuff!

> If I tried to all this via weewx only, it is technically possible, but I'd
> pull my few remaining hairs out battling mapping stuff to available fields
> in the weewx v4 extended schema. I'd then need to battle writing a custom
> cheetah skin to display it. And I mean battle. Cheetah is kinda brutal if
> you try to do to much in it.

Makes sense to me, even though I've done very little in skins.

> Hope this helps explain.

Yes - thanks for taking the time to explain. I hope others appreciated
it too.

[Vince Skahan]

On Friday, May 8, 2020 at 5:34:18 PM UTC-7, Greg Troxel wrote:

Vince Skahan <vince...@gmail.com> writes:

Thanks.  That all makes complete sense.  What I was getting at is that
weewx has as I see it 4 different things (being a bit hand wavy):

  1) talk to hardware to get data
  2) put that data in a weewx database
  3) send data to CWOP etc.
  4) generate web pages and publish them

and it sounds like you are using weewx to do (1), and maybe a bit of (2)
is happening, and you have (3) publish data to influxdb, either like
people send to CWOP, or like people publish loop packets to mqtt.

I do 1,2,3,4 in my weewx setup for the VP2 'plus' publishing to MQTT via that extension.

I do 1,2,4 in my weewx setup for the WeatherFlow Smart Weather station

In both cases I use the core 'rsync' uploader to also upload my Internet website.

 

Hence you are really only using (1) and some sort of [put into influxdb]
part of weewx, and not using the rest.  Which all makes sense to me, for
what you are doing.

For the WF Tempest I don't even use weewx quite yet, but I'll do 1,2,4 there too once we're allowed to have public data visible with that currently-beta gear.

 

> I use weewx to connect to the VP2 console in order to get the data the
> hardware emits in LOOP packets.  I then jump through some hoops to get that
> data into influxdb so I can cook up an integrated dashboard.

Would be nice if that were standard, included code.

Many things are nice, and Tom always likes to see a PR :-)

I don't sweat the influxdb part much as my convoluted way is good enough for me.  If it bothered me then I'd add the influxdb extension, which 'does' exist and 'would' simplify things a lot.  I just never had a reason to spend the time switching to it here.  As I said, the MQTT => influxdb (via telegraf) thing was something I was interested in as a learning experience, as was using grafana as a dashboard rather than going to the whole TICK stack (which I looked at, but decided to not go all the way thattaway)

Here's a couple comparison charts from grafana

First one is solar radiation.  The green dots are maxSolarRad as calculated by weewx. The other lines are three different WeatherFlow sensor suites.   As you can tell, they're still tuned a little high in value.   The bounciness is clouds etc. but you can even tell when each device went into the shade as the sun went down this evening.   It is a little funny seeing a drop in solar when the pole on my basketball hoop blocks the sun for a few minutes every afternoon.

Second graph is temperature comparison.  Green dots that are infrequent are the VP2 from weewx.  The others are the WeatherFlow sensors that emit data much more frequently, and the blue line is my arduino with a DS18b20 sensor on it, located inside.  Don't sweat minor differences in the outside gear, it all makes sense based on where they're located in the yard.  The air is in a fake Stephenson screen (actually, a birdhouse with lots of holes drilled in it) so it shows very smooth curves.  The Tempest is out in the air next to the VP2 so it will show bouncier fractions of a degree temperatures since it was gusty today with wind coming off the very cold water in Puget Sound a mile or so away, but if you check at 17:00 when they both went out of direct sunlight, they was basically identical to small fractions of a degF.  Good enough :-)

You 'can' do similar integrated dashboards in weewx if you map the data correctly and cook up custom skins.   I've seen lots of examples using only weewx.