Load data into weeWx database from multiple scripts

[Markus A]

Hi all,

I have running multiple, separate scripts and each script is collecting data from a different source (lightsensor, weather-house, etc).

Other people are sometimes using a fileparse driver to get the data into weeWX.

I don't want to go this way, as the scripts are running at slightly different times and need different durations for one loop. So the synchronization may be tricky.

What I would more prefer, do load the data directly from each script into the weeWX database.

Is this generally possibly and if yes, how to do this the easiest way (with yield _packet ?) ?

Or I am running in various issues?

Any other suggestions?

Many Thanks

Markus

[Markus A]

One addition (don't know how to modify the original post):

Currently all data is stored to an influxDB, but in future I want to use also weeWX

[Tom Keffer]

What do you mean by "scripts"? Shell scripts?

Why not just run multiple instances of weewx?

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/a6482d76-688b-4a3f-a11f-f43ba75fcf46%40googlegroups.com.

[Markus A]

Actually this "scripts" are C-programs, shell (bash) or python scripts. Those programs/scripts are triggered by cron or running continously (C-program for wind measurements).

[Tom Keffer]

OK.

Inside a WeeWX database is actually two databases: the "archive" database, and the "daily summaries" database. You can't just add new data in the former (using SQL INSERT statements) without updating the latter. They have to be kept in sync.

It would be easier to write custom drivers for each of your sources and let WeeWX do the syncing. It's also possible to wrap your existing C driver in Python, although I'm not an expert on how to do this.

-tk

On Wed, May 13, 2020 at 3:26 PM Markus A <marku...@gmail.com> wrote:

Actually this "scripts" are C-programs, shell (bash) or python scripts. Those programs/scripts are triggered by cron or running continously (C-program for wind measurements).

--

You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/1a24ed5d-4fbf-4fa8-9f4a-d528746b74e1%40googlegroups.com.

[Graham Eddy]

it is better to think of the database as a service sink near the end of the conveyor belt - your job is to insert stuff near the start of the conveyor belt. have your external apps talk to data services (create them, if no-one else has created suitable adapters yet) to inject your values

besides which, the interface between database and conveyor belt is well-defined and maintained but interface between database and external world is not

--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/a6482d76-688b-4a3f-a11f-f43ba75fcf46%40googlegroups.com.

[Markus A]

Thanks guys for your input.

Probably I will try to write my outputs all to RAM files and will parse them with one of the available fileparse drivers.