Loxone + InfluxDB + Grafana is nice :)

[Rob_in]

Hi all,

Just wanted to thank Andrew for bringing up Grafana in the DIY 1-Wire stuff (CO2, UDP interface) thread.

Had a quick play around with it this arvo and very quickly one can get some really nice graphs. FWIW, I wrote a Perl script to read the raw Loxone stats files (which I archive off every hour) and pump into an InfluxDB instance. It's impressively fast - around 30 stats for the last 3 months imported in under 10 seconds on my 6 year old laptop! From there, connecting Grafana is super easy. If one had a dedicated PC lying around (I did this on a laptop so can't do this right now) it might be best to get a load of Virtual Outputs from Loxone speaking directly to InfluxDB and turn of Loxone stats altogether. Well... depending on your point of view ;) One advantage of course would be if you can update InfluxDB quickly a nice 'real time' Dashboard could be created.

I already like Grafana a lot. For example, easy to create this stacked graph of daily power consumption with outside temperature overlaid (big impact on heating of course):

Overview of our heat pump - notice when I accidentally turned DHW off! ;)

Other stuff like temperatures, etc. even easier - can anyone spot when we actually moved in? ;)

If anyone is interested, shout out and I'll try and tidy the code and put it in GitHub or something.

Cheers,

Robin

[Václav Novák]

EXCELENT JOB!!!!

it is what is missing for my house. ... but I don´t  have knowleadges for this implementation.

Vaclav

Dne sobota 24. března 2018 20:51:52 UTC+1 Rob_in napsal(a):

[Aleš Berka]

Hi Robin,
interesting indeed. Please do go ahead, clean up your sources and publish.
It might be great together with on-line InfluxDB feeder https://github.com/dusanmsk/loxone-grafana pushing all made by MSK from Czech loxone forum Vodníci.

On topic with graphs, I hope it won't be taken as hijacking your thread, I wanted to mention also https://github.com/eik3/loxgraph (live demo here). 

It is much more simple than Grafana, but doesn't require anything but Loxone Miniserver on the other hand. You just deploy the ZIP content to Loxone Miniserver via FTP and access the web application via http://<LOXONE_IP>/loxgraph
I have recently started to continue with Eik's (original author) development, added multiple graph synchronization (showing the same timeframe) and soon to come displaying multiple values in 1 graph.

It will be published https://github.com/aleqcz/loxgraph when it's ready.

Cheers,
Aleš

[Andrew B]

That’s a very cool approach! Certainly a big step up from the in-built functionality.

[Tico]

Great work with the loxgraph! Can't wait for multiple values in 1 graph. :)

[Rob_in]

On Tuesday, 27 March 2018 18:41:24 UTC+2, Aleš Berka wrote:

interesting indeed. Please do go ahead, clean up your sources and publish.

I'll try ;)

 

It might be great together with on-line InfluxDB feeder https://github.com/dusanmsk/loxone-grafana pushing all made by MSK from Czech loxone forum Vodníci.

Ah-ha! Yes... this is very useful. Although they do seem to be going at it in a bit of a roundabout way. I followed their lead to https://github.com/alladdin/node-lox-ws-api and will investigate this for real time updates.

FWIW, I see the scripts I wrote as just a once off import and using the WS API to get every change after that import would be ideal. Then in Grafana one could build a real time dashboard which is what I would really like to see.

 

On topic with graphs, I hope it won't be taken as hijacking your thread, I wanted to mention also https://github.com/eik3/loxgraph (live demo here). 

That is very cool! Nice half way solution for those not wanting to go all out with Influx/Grafana.

Robin 

[Jan De Bock]

I see a reference to UDP but could not find the doc on this;

So in case you are interested:

• I wrote a very small python script listening to a UDP Port, splitting the input message and inserting this into InfuxDB (I split only in 4 pieces: Time, Category, Room, Value)
• Added a logger with those UDP characteristics  (IP address and port) in loxone
• Now when I want new statistics I just add the logger to the item I want in influxDB
• Then you can play with the very easy tool Grafana

if there is interest I can add some screenshots.

As Andrew B, I am looking into TimescaleDB because of the richness of their SQL, but still need to start with this.

[Rob_in]

Quick update:

I stole this Loxone API test... https://github.com/alladdin/node-lox-ws-api-testing ... and added literally 10 lines of code to it (and a bit of config in a file) and voila! Every time it sees a value update from Loxone simply throws that into InfluxDB.

With basically real time data going into Influx one can build a real time dashboard in Grafana too. Very nice.

FWIW, I now have InfluxDB, Grafana and this API updater thingo running in Docker containers on a Raspberry Pi (3B). 75% of the RAM is still free and the Pi's CPU hardly moves. OK, queries aren't instantaneous like they are when it's running on my laptop, but still, it really is quite ridiculous what you can do with something so cheap.

Will try and find the time to put something in Github soon. Ping back here if you are interested and I forget ;)

Cheers,

Robin

[Aleš Berka]

Definitely interested, Robin. Looking forward to it.

Thanks in advance!
Aleš

[Rob_in]

YMMV ;)

https://github.com/raintonr/loxone-stats-influx

Please report back how you go.

Cheers,

Robin

[Maxim Van Gelder]

Hey!

I like this a lot! I just got Grafana up and running for graphing power usage through Smappee.

So now that I have that up and running I'd like some decent stats from Loxone to come in. 

Stupid questions first, to get the miniserver to generate the required info I checked the Statistics box on the temperature sensors and averaged them over 5m.

How do I track usage of the digital outputs like when is DO activated, and how long?

Also, how do I get the data from Loxone into my InfluxDB? I noticed a dockerfile and some Perl scripts.

Just run the loxone_stats_influx_import.pl once for the initial import and then start the docker container?

[Andrew Brownsword]

I usually add a Virtual State and feed it from the source I want to log.

There was an earlier post by someone who had figured out how to get the miniserver to post directly to influxdb...

--
You received this message because you are subscribed to a topic in the Google Groups "Loxone English" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/loxone-english/ijOUU8FHMKA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to loxone-englis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/loxone-english/1f16fef7-9229-4bff-9f20-dbade16aea7d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rob_in]

On Sunday, 8 April 2018 23:21:03 UTC+2, Maxim Van Gelder wrote:

How do I track usage of the digital outputs like when is DO activated, and how long?

Just turn on statistics. There's no 'average over 5 minutes' or anything like that for digital stats. It just writes 1s and 0s. You can graph these to see when your stuff is active or not. If there is no 'statistics' option on the object you want to graph connect it to a Virtual Output and generate stats for that.

 

Also, how do I get the data from Loxone into my InfluxDB? I noticed a dockerfile and some Perl scripts. 

Just run the loxone_stats_influx_import.pl once for the initial import and then start the docker container?

I updated the https://github.com/raintonr/loxone-stats-influx readme file. Should be a little clearer now.

Cheers,

Robin 

[Aleš Berka]

Robin meant Virtual State, not Virtual Output (I also tend to mix these
up) - that's generic output which supports statistics.

A.

[Rob_in]

On Monday, 9 April 2018 16:16:43 UTC+2, Aleš Berka wrote:

Robin meant Virtual State, not Virtual Output (I also tend to mix these  up) - that's generic output which supports statistics.

No. I meant Virtual Output :P

I have commented on the other thread as to why: https://groups.google.com/d/msg/loxone-english/5p-rEdIdPQc/CVwxoMb9CAAJ

Robin 

[Filip Vanierschot]

Hi Rob_in,

just wanted to mention I'm installing loxone for my new home. I have some basic use cases with loxone. Not sure about what to monitor. Having loxone tree touch switches, temp sensor incorporated. I am an IT cyber security specialist and very much interested in certain things to monitor through the loxone touch switch panels. influxdb/grafana is something I ran into a couple of months ago.

thinking of temp sensor (loxone touch tree) measurement. heating/cooling is not connected to my loxone miniserver.

[Filip Vanierschot]

any other (security related) measurement?

[David]

Add some temperature sensors (one wire) to the hot water pipes feeding showers and baths. You can then turn on fan / towel rail / etc. when someone takes a shower / bath.

Easy to do when building - hard later on!

Cheers David

[Rob_in]

Hi,

That might be an OK approximation but I'm not sure measuring temperature is a great way to determine flow. Especially when the flow stops as the pipe will keep warm for some time afterwards I assume.

If you are doing plumbing from scratch I'd be more inclined to install flow meters where you need. I'm curious as to how well those cheap hall effect meters on Aliexpress work. In theory there really isn't much to go wrong with them.

Robin

[Andrew Brownsword]

Flow meters can get gummed up over time if there is anything in the water supply, including left over solder or shavings from the plumbing.  Plus they have a mechanical part and plumbing fittings.  Replacing them means doing plumbing.  Yes they are somewhat more precise (although there is still surprising variation in how often you get a pulse), but for purposes like turning on a towel heater using the temperature is fine — you probably want to run the heater for a while afterwards anyhow.  A water meter is useful if you want to run a preheat process on the water, for example.

By the way, after struggling to make influxdb stable I switched to TimescaleDB which is a PostgreSQL extension.  It has been stable and fast, and having full sql is really nice.  A grafana update is in beta now that’ll make some big timescaledb integration improvements.

--

You received this message because you are subscribed to a topic in the Google Groups "Loxone English" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/loxone-english/ijOUU8FHMKA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to loxone-englis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/loxone-english/e359fe7b-174c-45ab-a9c3-bdb9c07bdf5f%40googlegroups.com.

[Sven Van den brande]

Any guide to install all this stuff?

Would love to integrate this in my new home. :)

On Saturday, March 24, 2018 at 8:51:52 PM UTC+1, Rob_in wrote:

[Rob_in]

See above... https://github.com/raintonr/loxone-stats-influx

It's up to you how you want to import/organise/graph your stuff though.

Robin

[Andras Gaal]

Any recommendations how to visualize gas consumption? I am looking for a way to show daily consumption as the resolution of my measurement isn't great (0.1m3 / impulse). Which Loxone output should I send to influx? An analog memory that updates at 23:59:59?

[Jan De Bock]

Thanks all for sharing the info here.

Let me further document how I work with Loxone, Influx and grafana (and python)

1) I have setup influx (very easy to do, just follow instructions of influx db)

2) created a python script which (see in attach)

      a) listens to an UDP port

      b) decompose the message into structured fields (timestamp, data point, attributes)

      c) insert the data into influx DB (the config parameters need to match the ones you used in setup:port, ip address, db name, user, password)

3) created a Logger in Loxone which is very simple as per below, you can use a local address, as you ran the python script locally. The ip address needs to be the one frmom the system where you run the python script, the port number shold be the one you listen on in your python script.

4) for each field you want to monitor in Loxone you add now the UDP logger as per below

You need to name the fields carefully: first word will become the category in infux, all the rest of the field name will become the "room" in influx

Important: the logger is taking the description as name in case it is present, otherwise it takes the field name.  So if you want to keep you current field names, you can use the description to change the name according to the logic attribute room.

The time interval is somethign you need to play with. The higher you take it the less impact on your system performance, but the less accurate data in influx. I have not seen any issues with performance till now.

5) play with grafana to visualize your statistics. Is a relatively easy tool to use. Some examples below:

Visualizing sunboiler, buffer, outside temperature and a few others, together with weather condition, outdoor lux of sunlight, solar power (coming from wunderground) and gas consumption.

For a given room, visualize actual temp, target temp, outside temp and if heating is on or off.

or a partial overview of our house:

or the daily consumption of gas as Andras was looking for, and water and electricity

Hope this helps.

I am interested to move to postgress time series database as well, because it give me a few more possibiities, but for now, influx is doing already many good things. I stopped all stats on loxone, so reducing risk of SD card issues, as it is barely used now.

On Saturday, March 24, 2018 at 8:51:52 PM UTC+1, Rob_in wrote:

[Tom Bernaerd]

Jan,

Thanks for sharing the script, will certainly play with this in the future.

A bit offtopic, but since you are probably from the same region: are you monitoring gas and electricity consumption with pulses on the utility company meters, or do you have your own hardware to do this?

Regards,
Tom

--
You received this message because you are subscribed to the Google Groups "Loxone English" group.
To unsubscribe from this group and stop receiving emails from it, send an email to loxone-englis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/loxone-english/5a1ec4f5-1249-4610-9ee0-d3f60cc446e3%40googlegroups.com.

[Andrew Brownsword]

On Nov 22, 2018, at 12:33 AM, 'Jan De Bock' via Loxone English <loxone-...@googlegroups.com> wrote:

I am interested to move to postgress time series database as well

https://www.timescale.com

I have been running this since January (they’re now at 1.0.0, and the recent Grafana versions have added support for it), and am *very* happy with it.  I had a rough ride with InfluxDB running on a 32bit ARM, but no problems since I switched.  If you are running InfluxDB on a non-x86 system, I suggest you make sure to test restarting the host running your database… my database had serious issues actually persisting data to storage, and it only became evident when you forced a restart.

[Jan De Bock]

HI Andrew,

ok, tx. I will go to timescale maybe during Xmas break ;-)

But I do not have problems with influx; I am running this on Synology NAS.

Best regards,

Jan 

--
You received this message because you are subscribed to a topic in the Google Groups "Loxone English" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/loxone-english/ijOUU8FHMKA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to loxone-englis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/loxone-english/A9EB9DF4-468A-4734-A5BA-020BFEDEA27E%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Met vriendelijke groeten,

Jan.

[Andrew Brownsword]

Good to hear.  At work we have an Influx instance running and haven’t seen the same issue.  They may have also fixed it since I last used the product.

I do rather like the TimeScaleDB though, and am not interested in going back to Influx.  Having options is good though.    :)

[Andras Gaal]

I am using the built-in impulse output on the gas meter (Flogistion device - 0.1m3/impulse - acceptable for daily usage monitoring, not really for current, instantaneous consumption monitoring) and a dedicated electricity meter wired in serial (my provider does not allow access to the built-in impulse output of the "official" meter) also with impulse output (800 impulse / kwh). My water meter (also in serial to the "official" meter) also supports impulse out.

[David Wallis]

just having a play with this and getting the following error:

2018-12-22T19:03:39.143Z INFO : Update event value: uuid=121af5a4-0245-1c47-ffff1a9eccc1508b, evt=232.25806451612897

2018-12-22T19:03:39.149Z ERROR : Error saving data to InfluxDB! Error: A 400 Bad Request error occurred: {"error":"unable to parse 'Oil Tank Level,room=Office,uuid=121af5a4-0245-1c47-ffff1a9eccc1508b,src=ws value=232.25806451612897': invalid field format"}

    at new RequestError (/home/davidw/loxone-stats-influx/loxone-ws-influx/node_modules/influx/lib/src/pool.js:52:28)

    at IncomingMessage.<anonymous> (/home/davidw/loxone-stats-influx/loxone-ws-influx/node_modules/influx/lib/src/pool.js:62:53)

    at emitNone (events.js:111:20)

    at IncomingMessage.emit (events.js:208:7)

    at endReadableNT (_stream_readable.js:1064:12)

    at _combinedTickCallback (internal/process/next_tick.js:138:11)

    at process._tickCallback (internal/process/next_tick.js:180:9)

what am I doing wrong?

[Robin Rainton]

I imagine your 'Oil Tank Error' should be one word (no spaces).

--
You received this message because you are subscribed to a topic in the Google Groups "Loxone English" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/loxone-english/ijOUU8FHMKA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to loxone-englis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/loxone-english/6840ae2a-eea6-4a58-bab2-c4629e426bad%40googlegroups.com.

[Andy Hummel]

Hey Jan,

Sorry to bother you, but I tried your python script, but currently I get an error message if I´d like to run the script.

root@RaspberryHummel:/opt# sudo python UDP.py

Traceback (most recent call last):

  File "UDP.py", line 5, in <module>

    from influxdb import InfluxDBClient

ImportError: No module named influxdb

Sorry, but since I'm new to try this it is maybe a stupid question.

Any Ideas

Thanx

Andy

[David Wallis]

sudo apt-get install libfile-slurp-perl -y

sudo apt-get install libfile-touch-perl -y

sudo apt-get install libjson-perl -y

sudo apt-get install libhttp-tiny-perl -y

 

From: Andy Hummel
Sent: 26 December 2018 00:22
To: Loxone English
Subject: Re: Loxone + InfluxDB + Grafana is nice :)

 

Hey Jan,

 

Sorry to bother you, but I tried your python script, but currently I get an error message if I´d like to run the script.

 

root@RaspberryHummel:/opt# sudo python UDP.py

Traceback (most recent call last):

  File "UDP.py", line 5, in <module>

    from influxdb import InfluxDBClient

ImportError: No module named influxdb

Sorry, but since I'm new to try this it is maybe a stupid question.

 

Any Ideas

 

Thanx

Andy

 

Am Donnerstag, 22. November 2018 09:33:51 UTC+1 schrieb Jan De Bock:

Thanks all for sharing the info here.

Let me further document how I work with Loxone, Influx and grafana (and python)

 

1) I have setup influx (very easy to do, just follow instructions of influx db)

2) created a python script which (see in attach)

      a) listens to an UDP port

      b) decompose the message into structured fields (timestamp, data point, attributes)

      c) insert the data into influx DB (the config parameters need to match the ones you used in setup:port, ip address, db name, user, password)

 

3) created a Logger in Loxone which is very simple as per below, you can use a local address, as you ran the python script locally. The ip address needs to be the one frmom the system where you run the python script, the port number shold be the one you listen on in your python script.

 

 

 

4) for each field you want to monitor in Loxone you add now the UDP logger as per below

You need to name the fields carefully: first word will become the category in infux, all the rest of the field name will become the "room" in influx

Important: the logger is taking the description as name in case it is present, otherwise it takes the field name.  So if you want to keep you current field names, you can use the description to change the name according to the logic attribute room.

The time interval is somethign you need to play with. The higher you take it the less impact on your system performance, but the less accurate data in influx. I have not seen any issues with performance till now.

 

 

 

 

5) play with grafana to visualize your statistics. Is a relatively easy tool to use. Some examples below:

 

Visualizing sunboiler, buffer, outside temperature and a few others, together with weather condition, outdoor lux of sunlight, solar power (coming from wunderground) and gas consumption.

 

 

For a given room, visualize actual temp, target temp, outside temp and if heating is on or off.

 

or a partial overview of our house:

 

or the daily consumption of gas as Andras was looking for, and water and electricity

 

 

Hope this helps.

I am interested to move to postgress time series database as well, because it give me a few more possibiities, but for now, influx is doing already many good things. I stopped all stats on loxone, so reducing risk of SD card issues, as it is barely used now.

 

On Saturday, March 24, 2018 at 8:51:52 PM UTC+1, Rob_in wrote:

Hi all,

 

Just wanted to thank Andrew for bringing up Grafana in the DIY 1-Wire stuff (CO2, UDP interface) thread.

 

Had a quick play around with it this arvo and very quickly one can get some really nice graphs. FWIW, I wrote a Perl script to read the raw Loxone stats files (which I archive off every hour) and pump into an InfluxDB instance. It's impressively fast - around 30 stats for the last 3 months imported in under 10 seconds on my 6 year old laptop! From there, connecting Grafana is super easy. If one had a dedicated PC lying around (I did this on a laptop so can't do this right now) it might be best to get a load of Virtual Outputs from Loxone speaking directly to InfluxDB and turn of Loxone stats altogether. Well... depending on your point of view ;) One advantage of course would be if you can update InfluxDB quickly a nice 'real time' Dashboard could be created.

 

I already like Grafana a lot. For example, easy to create this stacked graph of daily power consumption with outside temperature overlaid (big impact on heating of course):

 

 

Overview of our heat pump - notice when I accidentally turned DHW off! ;)

 

 

Other stuff like temperatures, etc. even easier - can anyone spot when we actually moved in? ;)

 

 

If anyone is interested, shout out and I'll try and tidy the code and put it in GitHub or something.

 

Cheers,

 

Robin

 

--

You received this message because you are subscribed to the Google Groups "Loxone English" group.

To unsubscribe from this group and stop receiving emails from it, send an email to loxone-englis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/loxone-english/8f84c9bb-8052-4599-99c2-b6788d6dee74%40googlegroups.com.

[Dave K]

Hi guys,

Please could someone explain how to run / setup this solution to load data from Loxone into Grafana? (not the UDP way)

I have already InfluxDB with Grafana running and working.

What else do i need to do and how? dont understand where this script should be running or if i need to add it somehow to NodeRed., confused :(

Thanks

[Robin Rainton]

Look at the README file. What's not clear?

--
You received this message because you are subscribed to a topic in the Google Groups "Loxone English" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/loxone-english/ijOUU8FHMKA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to loxone-englis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/loxone-english/b6d9f90e-d516-4ae8-884f-a53ba85cb178%40googlegroups.com.

[Dave K]

Hello,

Let me try to ask in more detail.

1. Create .loxone_stats_influx in your home dir and link/copy into the config folder for the Node.js app.

1. 1 Homedir of what?

1.2 Node.js app where / how do i run it?

2. 

2.1 Run loxone_stats_influx_import.pl - how ? locally on Windows machine or Linux?

2.2 Setup loxone-ws-influx Node.js app and run continuously. - do i need some docker to run it?




3. Im no familiar with this so maybe a silly question - does this solution have something to do with NodeRed or its completely out of picture?

4. Does it work in a way that "manual" copy stats from Loxone wont be needed and it will log stats locally on Loxone and into InfluxDB / Grafana as well?







Thanks


[Robin Rainton]

Hi,

This is nothing to do with NodeRed. These are Linux scripts. I don't meant to be rude, but if you don't know what Node.js or your home directory is maybe this isn't for you. It certainly isn't a polished product but something I flung together and shared in case others found it useful.

Moreover, this group isn't the place to discuss problems/support requests for this. It's Github project would be.

Cheers,

Robin

To view this discussion on the web visit https://groups.google.com/d/msgid/loxone-english/010fe3cf-5de1-4468-a076-d30039c6b4f5%40googlegroups.com.

[Dave K]

Just found that within "Nodered" you can run various scripts in "function" node so this might be the way how to run it without extra linux server in place.

Anyone here has this working?

In parallel im working also with this option https://github.com/codmpm/node-red-contrib-loxone which can be used take directly live data as they are generated into InfluxDB directly.

Thanks

[David Wallis]

You dont need node red.. you are over complicating things..

If you have a box running grafana / influx then it can run this node js script and subscribe to the events and insert them - I got it running pretty easily - I did document the steps. If I get chance I'll spin up another vm and do it again and get you step by step instructions - but you'll need to work out how to edit a yaml file yourself :)

Cheers

David

[Dave K]

Thanks for helping out David - currently im also on German Loxone forum working out the NodeRed way and i got it working - only disadvantage that i have to create for each Category a separate node (node8).

What im trying to archive (node7) that i can load the whole "stats stream" into InfluxDB, then ill sort it out in Grafana. Getting really close now :D

Im not a programmer, but rather Admin, so for scripting i always need to google a lot or ask friend who does that for living ;)

[David Wallis]

your still creating a massive overhead of manual work there that you just don't need - as far as I can tell :)

[denver]

Am I right so far? Ive got as far as this but not sure where to go from here, any pointers would be helpful.

[Rob_in]

Yes. Then just create your config file and 'npm start'.

[denver]

OK, got this far but thrown up an error, would be grateful for more advise please where I've gone wrong, thank you.

[denver]

Sorted it, I think. Deleted the node_modules folder and files and installed again. Pi now saying

2019-01-17T07:30:07.480Z INFO : Loxone authorized

[denver]

Took me all day to work out why some measurements were not coming through despite being shown in Loxone statistics, and it was because I never set 'Use' as well as 'Statistics' in the properties. I didn't think 'Use' would make a difference but it did.

On Monday, 9 April 2018 14:56:28 UTC+1, Rob_in wrote:

On Sunday, 8 April 2018 23:21:03 UTC+2, Maxim Van Gelder wrote:

How do I track usage of the digital outputs like when is DO activated, and how long?

Just turn on statistics. There's no 'average over 5 minutes' or anything like that for digital stats. It just writes 1s and 0s. You can graph these to see when your stuff is active or not. If there is no 'statistics' option on the object you want to graph connect it to a Virtual Output and generate stats for that.

 

Also, how do I get the data from Loxone into my InfluxDB? I noticed a dockerfile and some Perl scripts. 

Just run the loxone_stats_influx_import.pl once for the initial import and then start the docker container?

I updated the https://github.com/raintonr/loxone-stats-influx readme file. Should be a little clearer now.

Cheers,

Robin 

[denver]

Ok, I have every thing running well now and would like to start the logger when the pi boots or reboots.

Im have trouble writing a systemd service file, can anyone help on this?

My code is:

[Unit]

Description=Loxone log

After=network.target

[Service]

ExecStartPre=/home/pi/loxone-stats-influx/loxone-ws-influx/

ExecStart=/home/pi/n/bin npm start

#Restart=always

User=root

Group=root

[Install]

#WantedBy=multi-user.target

and the errors are:

pi@raspberrypi:~ $ sudo systemctl daemon-reload

pi@raspberrypi:~ $ sudo systemctl start loxone_log

pi@raspberrypi:~ $ sudo systemctl status loxone_log

● loxone_log.service - Loxone log

   Loaded: loaded (/etc/systemd/system/loxone_log.service; static; vendor preset: enabled)

   Active: failed (Result: exit-code) since Tue 2019-01-22 13:32:04 GMT; 2s ago

  Process: 4406 ExecStart=/home/pi/n/bin npm start (code=exited, status=203/EXEC)

 Main PID: 4406 (code=exited, status=203/EXEC)

Jan 22 13:32:04 raspberrypi systemd[1]: Started Loxone log.

Jan 22 13:32:04 raspberrypi systemd[1]: loxone_log.service: Main process exited, code=exited, status=203/EXEC

Jan 22 13:32:04 raspberrypi systemd[1]: loxone_log.service: Unit entered failed state.

Jan 22 13:32:04 raspberrypi systemd[1]: loxone_log.service: Failed with result 'exit-code'.

To run the logger once the pi starts, I would change directory to the following before typing the 'npm start' as in:

pi@raspberrypi:~/loxone-stats-influx/loxone-ws-influx $ npm start

Was wondering if anyone has managed to create a auto start for the logger.

[Robin Rainton]

If you run it in a Docker container it's not only isolated, but you can set your container to start automatically.

--

You received this message because you are subscribed to a topic in the Google Groups "Loxone English" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/loxone-english/ijOUU8FHMKA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to loxone-englis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/loxone-english/6435b304-83db-48ac-9470-20c90387a07c%40googlegroups.com.

[denver]

Ok, ill look into that, not used Docker containers before, heard about them, so guess this will be the right time to start learning and implementing them.

[Tom B]

Hi,

I tried the method described by Jan and it works as advertised. 

My InfluxDB and Grafana are set up on a DigitalOcean VPS since I want to test it first without fiddling with hardware. I'm planning to push a lot of loxone data to Influx and see how the cheap DO box handles the load. If it does well, I might just leave it running there and collect statistics for two other Loxone houses as well. Stats as a Service :-)

The python script is running on a RaspPi in the local LAN. 

If someone wants to replicate this setup, be sure to use Basic Auth in InfluxDB, as it uses no authentication by default and the VPS is wide open to the entire internet. 

Jan, how are you running the python script? I have it running in a Screen at the moment, so I have no way to know if it is still running. Ideally, I'd like to run it as a service somehow and have it auto run at boot. 

Any pointers on this? My experience with python scripting is very limited, but I'd like to learn more and maybe adapt the script a bit to fit more use cases and have some basic error handling. 

Thanks for your work so far on this.

Tom

On Thursday, November 22, 2018 at 9:33:51 AM UTC+1, Jan De Bock wrote:

Thanks all for sharing the info here.

Let me further document how I work with Loxone, Influx and grafana (and python)

1) I have setup influx (very easy to do, just follow instructions of influx db)

2) created a python script which (see in attach)

      a) listens to an UDP port

      b) decompose the message into structured fields (timestamp, data point, attributes)

      c) insert the data into influx DB (the config parameters need to match the ones you used in setup:port, ip address, db name, user, password)

3) created a Logger in Loxone which is very simple as per below, you can use a local address, as you ran the python script locally. The ip address needs to be the one frmom the system where you run the python script, the port number shold be the one you listen on in your python script.

4) for each field you want to monitor in Loxone you add now the UDP logger as per below

You need to name the fields carefully: first word will become the category in infux, all the rest of the field name will become the "room" in influx

Important: the logger is taking the description as name in case it is present, otherwise it takes the field name.  So if you want to keep you current field names, you can use the description to change the name according to the logic attribute room.

The time interval is somethign you need to play with. The higher you take it the less impact on your system performance, but the less accurate data in influx. I have not seen any issues with performance till now.

5) play with grafana to visualize your statistics. Is a relatively easy tool to use. Some examples below:

Visualizing sunboiler, buffer, outside temperature and a few others, together with weather condition, outdoor lux of sunlight, solar power (coming from wunderground) and gas consumption.

For a given room, visualize actual temp, target temp, outside temp and if heating is on or off.

or a partial overview of our house:

or the daily consumption of gas as Andras was looking for, and water and electricity

Hope this helps.

I am interested to move to postgress time series database as well, because it give me a few more possibiities, but for now, influx is doing already many good things. I stopped all stats on loxone, so reducing risk of SD card issues, as it is barely used now.

On Saturday, March 24, 2018 at 8:51:52 PM UTC+1, Rob_in wrote:

Hi all,

Just wanted to thank Andrew for bringing up Grafana in the DIY 1-Wire stuff (CO2, UDP interface) thread.

Had a quick play around with it this arvo and very quickly one can get some really nice graphs. FWIW, I wrote a Perl script to read the raw Loxone stats files (which I archive off every hour) and pump into an InfluxDB instance. It's impressively fast - around 30 stats for the last 3 months imported in under 10 seconds on my 6 year old laptop! From there, connecting Grafana is super easy. If one had a dedicated PC lying around (I did this on a laptop so can't do this right now) it might be best to get a load of Virtual Outputs from Loxone speaking directly to InfluxDB and turn of Loxone stats altogether. Well... depending on your point of view ;) One advantage of course would be if you can update InfluxDB quickly a nice 'real time' Dashboard could be created.

I already like Grafana a lot. For example, easy to create this stacked graph of daily power consumption with outside temperature overlaid (big impact on heating of course):

Overview of our heat pump - notice when I accidentally turned DHW off! ;)

Other stuff like temperatures, etc. even easier - can anyone spot when we actually moved in? ;)

[Jan De Bock]

Hi Tom,

Basically I am running the python script in the background, so can switch off the screens (I run it on synology);

To check connection is still there I monitor the number of incoming temperature changes in  between -10 min from now and -20 min from now

If this is 0 or gives a data error, I am raising an alert. This is all done in Grafana.

hope this helps,

Jan

[denver]

I didn't want to pursue a container as thats all new to me.

I've managed to get it running when the pi starts with the systemd system file in the end:

[Unit]
Description=Loxone log
After=network.target

[Service]

WorkingDirectory=/home/pi/loxone-stats-influx/loxone-ws-influx
ExecStart=/home/pi/n/bin/node loxone-ws-influx.js
Restart=always
Environment=NODE_ENV=production
User=pi
Group=pi

[Install]
#WantedBy=multi-user.target

On Tuesday, 22 January 2019 17:31:54 UTC, Robin Rainton wrote:

[Andrew B]

Yeah, docker shouldn’t really be necessary on a small system like a RPi running a couple of programs. It is more useful in environments where you may not control the OS, or have programs with lots of potentially conflicting dependencies. I find on my ARM systems it just adds yet another layer of complexity without bringing much benefit.

Glad to see the time-series db + Grafana movement is alive and well.

[denver]

Thats next on my list, looking into TimeseriesDB in place of influxDB

[Andrew B]

You mean TimescaleDB (an extension to PostgreSQL)? I really like it, and have over a year’s worth of data in it now. Being built upon PostgreSQL brings quite a few nice bonuses as there are a lot of tools, and SQL is very powerful. Thus far I have encountered no issues with timescale, unlike my troublesome encounter with InfluxDB (which was over a year ago, so I imagine it has improved by leaps and bounds). The Grafana support for Timescale has come along quite well too. I write into Timescale from my C++ application and haven’t done any investigation into how one could insert directly from the Loxone (it is likely to be possible, I just haven’t given it a moment’s thought).

I had used the term “time series DB” in the generic sense, trying to be inclusive of all technologies available.

[Tom Bernaerd]

I have now switched off all Loxone statistics and I'm using the UDP logger combined with the python script from Jan to upload data to Influx.

Influxdb and grafana are both running on a Digitalocean VPS, the python script runs locally on a raspberry pi. 

The only issue I noticed is that the script sometimes crashes when the VPS is not responsive for a split second right when it's trying to submit data. Grafana can notify me when this happens (no data for x amount of time) but when I can't get to it fast enough, I still have holes in my data. 

Rather than fixing that in the python script, I went with this solution to auto restart the script when it crashes :-)

https://www.alexkras.com/how-to-restart-python-script-after-exception-and-run-it-forever/ 

Seems to work well so far.  

On Fri, Jan 25, 2019 at 2:00 AM Andrew B <andrew.e....@gmail.com> wrote:

You mean TimescaleDB (an extension to PostgreSQL)?  I really like it, and have over a year’s worth of data in it now.  Being built upon PostgreSQL brings quite a few nice bonuses as there are a lot of tools, and SQL is very powerful.  Thus far I have encountered no issues with timescale, unlike my troublesome encounter with InfluxDB (which was over a year ago, so I imagine it has improved by leaps and bounds).  The Grafana support for Timescale has come along quite well too.  I write into Timescale from my C++ application and haven’t done any investigation into how one could insert directly from the Loxone (it is likely to be possible, I just haven’t given it a moment’s thought).

I had used the term “time series DB” in the generic sense, trying to be inclusive of all technologies available.

--

You received this message because you are subscribed to the Google Groups "Loxone English" group.

To unsubscribe from this group and stop receiving emails from it, send an email to loxone-englis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/loxone-english/9d70398b-a198-4c5a-80c3-3827440eade4%40googlegroups.com.

[Saïd ELHAMAOUI]

did anyone use this with mysql database instead of influx

[Rob_in]

On Monday, 22 February 2021 at 23:55:04 UTC+1 saidelh...@gmail.com wrote:

did anyone use this with mysql database instead of influx

No. MySQL is not a time series database engine.

[Jan De Bock]

But you can use timescale db,  a time series database based on postgresql and working pretty good. You can use all sql commands + some extensions for time series. 

--
You received this message because you are subscribed to a topic in the Google Groups "Loxone English" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/loxone-english/ijOUU8FHMKA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to loxone-englis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/loxone-english/42d56754-2ad4-4b0c-9be6-0a0e4c814b86n%40googlegroups.com.