Dropped KNX Packets?

[Rob_in]

Hi all,

I have noticed some odd behaviour lately with our KNX switches. Everything used to work flawlessly but recently they appear to be randomly dropping packets. Or the Miniserver is either not receiving or processing them.

I find it hard to believe this is really happening because KNX is such a mature technology but it does appear to be happening on multiple switches.

Absolutely nothing has changed in the hardware for months now.

I do tweak our config from time to time though. Recently I have added some relatively complex temperature calculations and a few local HTTP Virtual Inputs that are polled regularly.

Is it possible that the Miniserver could actually be dropping KNX packets because it's busy doing other things? I would have imagined this is highly unlikely but you never know.

Has anyone noticed degradation of KNX performance over time or anything similar?

I did reboot the Miniserver the other day when it appeared to be stuck and not responding to any of our KNX switches at all and thought things were good after. But tonight I've noticed a few random failures to respond again so sure there's something odd going on.

Confused :(

Robin

[Rob_in]

As a follow up...

The next time I saw this behaving very badly logged into Loxone Config and took a look. Sure enough, I see CPU spikes 90-100%. Ouch!

Reboot the Miniserver and now these spikes are more like 45-50% and things work as normal again.

I would immediately point the finger at a memory leak or some other failure to free up resources because this only happens after a certain period of time.

That said, it would also appear that the Miniserver is really not very efficient at handling even small HTTP Virtual inputs. I will have a look at see if instead of polling I can get UDP packets generated and sent to the Miniserver. I didn't want to rely on this before as UDP packets can get lost, but perhaps 'pinging' the Miniserver with 10 UDP packets every few seconds is more efficient than asking it to make a HTTP request and parse the result every few seconds. Any thoughts on that?

Cheers,

Robin

[Rob_in]

And more...

I made the polling frequency of the HTTP Virtual inputs much longer and still saw these CPU spikes regularly (every 10 seconds or so)... which happens to be polling time for our temperature/humidity sensors.

I guess, therefore, that it's the newer temperature calculations that are the issue and they are being triggered on every temperature update.

... or not? Whatever is causing the memory/resource leak is the problem as things work fine directly after a reboot.

FWIW, I do know that polling for temperatures every 10 seconds is overkill, but... these same sensors measure humidity and we have a nice mechanism for detecting rapid humidity spikes in the ensuite which shows someone is in the shower and puts the ventilation system on boost. Guess this could work just as well with 30s or 60s polling. Might play with that too.

Robin

[Arnaud]

Hello, Robin,

I have two remarks: 

Loxone informed me that it you should'nt  be in last version of the firmware to work on KNX currently, in your place I would try to downgrader to check. 

Two, why don't you use presence detection in the bathroom instead of this high-impact moisture calculation for the miniser? 

It is moreover to specify in Loxone config, a too frequent interrogation can destabilize the miniser.

If I were you, I'd try to get around the problem and get back to a configuration that Loxone could accept, an eventually support...

[Rob_in]

On Monday, 18 June 2018 08:44:32 UTC+2, Arnaud wrote:

Loxone informed me that it you should'nt  be in last version of the firmware to work on KNX currently, in your place I would try to downgrader to check. 

Wow! Really? What exactly did they say? Is there a ticket or something to track this issue?

 

Two, why don't you use presence detection in the bathroom instead of this high-impact moisture calculation for the miniser? 

Because I don't want the ventilation to be put on boost when someone is in the bathroom, only when someone is taking a shower.

TBH, it's not that polling that is the issue, but the waterfall effect of the temperature being updated at the same time. I calculate the weighted average temperature for the house for example (Loxone should do this themselves as they know the floor area of each room but they don't). Then I calculate predicted temperature based on the outside weather forecast and known heat losses. This prevents our passive cooling system (ventilation bypass) going too far and perhaps causing a heating demand.

I should probably put analogue memory blocks in front of all these calculations so they only happen once every few minutes no matter how often the inputs change - I think that could help. Although that in itself will add complexity.

Thank for the input,

Robin

[BartVB]

Hello Robin,

It would certainly make sense to only update the average temperatures every couple of minutes (or even less). Or even split up these calculations to reduce the load. I.e. fetch forcasts only once every 30 minutes, calculate the weighted temperature 2 or 3 times in between.

But that would only reduce your problem (if you split those tasks up), not solve it. The problem is that KNX packets are being dropped when the Miniserver CPU is busy. That's really bad if you ask me, it should at least buffer the KNX input.

Sounds like a firmware problem on the Loxone side, also very curious what they said exactly. I'm still in the planning stages and if Loxone can't reliable deal with KNX then there is no way that I'm going to stay with Loxone.

Kind regards,

Bart

[Arnaud]

"Wow! Really? What exactly did they say? Is there a ticket or something to track this issue?"

i invite you to call them, i've done it for a project to put Loxone as KNX IP Gateway and they told me to downgrade the miniserver in order to work with KNX. it was approximatively 10 days ago.

[Rob_in]

On Monday, 18 June 2018 09:59:20 UTC+2, BartVB wrote:

It would certainly make sense to only update the average temperatures every couple of minutes (or even less). Or even split up these calculations to reduce the load. I.e. fetch forcasts only once every 30 minutes, calculate the weighted temperature 2 or 3 times in between.

This is true. It's just I assumed these calculations wouldn't be a problem for the Miniserver but seems they are.

I modified the inputs to put analogue memory blocks inline and triggered these with a timer. Thus, I can trigger when these calculations happen rather than them always happening when any of the sensor inputs changes.

As a by-product instead of having a couple of the calculations both happen at the same time (one for temperature and one for humidity for example) I can trigger each a few seconds apart, further spreading CPU load. This works really well.

 

But that would only reduce your problem (if you split those tasks up), not solve it. The problem is that KNX packets are being dropped when the Miniserver CPU is busy. That's really bad if you ask me, it should at least buffer the KNX input.

Exactly. 

Sounds like a firmware problem on the Loxone side, also very curious what they said exactly. I'm still in the planning stages and if Loxone can't reliable deal with KNX then there is no way that I'm going to stay with Loxone.

To be fair, the KNX devices in our system have been working perfectly for months so I wouldn't panic about this.

Robin

[Simon Still]

On Sunday, 17 June 2018 23:43:49 UTC+1, Rob_in wrote:

And more...

I made the polling frequency of the HTTP Virtual inputs much longer and still saw these CPU spikes regularly (every 10 seconds or so)... which happens to be polling time for our temperature/humidity sensors.

these same sensors measure humidity and we have a nice mechanism for detecting rapid humidity spikes in the ensuite which shows someone is in the shower and puts the ventilation system on boost. Guess this could work just as well with 30s or 60s polling. Might play with that too.

What are the sensors you're using?  Also, why the need to detect rapid spikes - I'm just using the humidity level from a Loxone analogue Temp/humidity sensor to trigger our ventilation boost.  It boosts when humidity is above X and turns off again when it drops below Y.  The only tweak i have needed is a switch that raises the lower boundary when ambient humidity is very high (outside the UK, in a very humid climate, I can see you might need something more complex) 

[Rob_in]

On Tuesday, 19 June 2018 08:15:08 UTC+2, Arnaud wrote:

i invite you to call them, i've done it for a project to put Loxone as KNX IP Gateway and they told me to downgrade the miniserver in order to work with KNX. it was approximatively 10 days ago.

Well, I wrote to support and they said to look here for known issues:

https://www.loxone.com/enen/support/changelog-config/

Currently it says...

KNX IP Gateway: Communication with the ETS not working if “NAT Mode” is activated for the connection.
Workaround:
disable “Nat Mode” in the ETS Software
or use an external EIB Gateway to connect via ETS

 

... which will be absolutely nothing to do with packets on the physical KNX bus.

So whatever we saw, they don't seem to know about it.

Robin

[Rob_in]

On Wednesday, 20 June 2018 11:25:15 UTC+2, Simon Still wrote:

What are the sensors you're using?  Also, why the need to detect rapid spikes - I'm just using the humidity level from a Loxone analogue Temp/humidity sensor to trigger our entilation boost.  It boosts when humidity is above X and turns off again when it drops below Y.  The only tweak i have needed is a switch that raises the lower boundary when ambient humidity is very high (outside the UK, in a very humid climate, I can see you might need something more complex) 

We have a 1-Wire bus with temperature/humidity sensors in each room.

I started out with a scale block that turned up the ventilation as the humidity rises but found this was a bit too gentle although it works well in general conditions.

What I basically wanted was to turn the ventilation on very high for a short period of time when someone takes a shower, rather than waiting for the humidity (and thus the ventilation) to creep up slowly.

This can be achieved by looking for that quick spike in humidity that turning on a hot shower causes. Works like a charm and clears the steam much quicker than the scale block alone.

This is now the only calculation being performed on that sensor update. I've modified the behaviour of the others to occur far less frequently, and that seems to have caused a nice reduction in Miniserver CPU usage. I will assume things will be good from now on then, but will report back if the KNX line gets stuck again.

Cheers,

Robin

[Andrew B]

What are you using to implement your calculations —formula block, chains of blocks, or picoC?

Are you looking at rate of change or 1-2 rooms out of line with all the others?

[Rob_in]

Hi,

For the humidity calculation it's just a chain of a few analogue memory blocks. That's simple stuff and obviously not too taxing for it.

I'm just looking for a change over the last minute in that room, not comparing with any of the other rooms. It does work well.

The more taxing calculations were a long chain of add/multiply blocks that works out predicted temperature inside based on forecasts outside. Things are very changeable here! For example, yesterday it was 28C and the house did get pretty warm inside. Overnight the temperature fell so we had the ventilation pulling in cold night air. I wake up this morning and we're in thick mist and it's rather cool outside. Clearly knowing what's happening is very important because you don't want to over-cool the house and then be cold the next day.

Robin

On Thursday, 21 June 2018 02:47:20 UTC+2, Andrew B wrote:

What are you using to implement your calculations —formula block, chains of blocks, or picoC?

Are you looking at rate of change or 1-2 rooms out of line with all the others? ccincidence

[Arnaud]

I agree with you that their answer doesn't seem to concern you.

But here, they clearly told me to downgrader to work with KNX.

If I were you, I would test the downgrade, it would allow you to have certainties. at worst it doesn't change anything...

We often have the problem in computing: the configuration seems good but poses a problem and the proposed solution does not seem relevant...

[Simon Still]

On Wednesday, 20 June 2018 10:42:12 UTC+1, Rob_in wrote:

I started out with a scale block that turned up the ventilation as the humidity rises but found this was a bit too gentle although it works well in general conditions.

What I basically wanted was to turn the ventilation on very high for a short period of time when someone takes a shower, rather than waiting for the humidity (and thus the ventilation) to creep up slowly.

This can be achieved by looking for that quick spike in humidity that turning on a hot shower causes. Works like a charm and clears the steam much quicker than the scale block alone.

Sounds like your ventilation might be a bit more advanced than my MVHR which just has low/normal/boost levels.  I've found a shower or bath raises the humidity level very quickly - boost comes on within 30 seconds at most.  

[David]

I detect shower being taken - and hence turn on ventilation, by using a one wire temperature sensor on the hot water pipe to the shower.  Again look for change  - it is very responsive.

Cheers David

[Rob_in]

On Thursday, 21 June 2018 11:19:39 UTC+2, David wrote:

I detect shower being taken - and hence turn on ventilation, by using a one wire temperature sensor on the hot water pipe to the shower.  Again look for change  - it is very responsive.

Geeze... that would require a bit too much forward thinking! ;)

Nice solution though!

Robin 

[Rob_in]

On Thursday, 21 June 2018 10:43:38 UTC+2, Simon Still wrote:

Sounds like your ventilation might be a bit more advanced than my MVHR which just has low/normal/boost levels.  I've found a shower or bath raises the humidity level very quickly - boost comes on within 30 seconds at most.  

I doubt it, ours is a Unelvent Domeo 210 which is  very simple.

It only has standard/absence/boost/bypass controls. However, through tweaking of it's various registers one can get a bit better control. I basically coded some stuff where the required flow was input and it worked out which of these modes and to use and then tweaked the flow set for that mode.

As noted in a previous thread, I've read elsewhere that adjusting it to run in 'German' mode one can actually turn the whole thing on and off but never got that to work :(

Robin

[Andras Gaal]

You could use some very simple device (pi or some arduino board) to do the 1-wire polling and send it to Loxone periodically (longer intervals) and when the value changes more than by a fix value (eg: 1%). This obviously won't solve the cascade high cpu effect but could take over some simple calculations (average temp, delta detect) and reduce the load (and therefore the spikes) on the miniserver.