MQTT Client Bug

[Adrian Brown]

Hi Guys,

Have notice that the new MQTT Client is very unstable. Lots and lots of disconnects and reconnects. This behaviour is not broker related. Happens with Mosquitto, RabbitMQ or ActiveMQ.

Welcome to Node-RED

===================

 

7 Dec 23:22:21 - [info] Node-RED version: v0.12.2

7 Dec 23:22:21 - [info] Node.js  version: v0.10.40

7 Dec 23:22:21 - [info] Loading palette nodes

7 Dec 23:22:25 - [warn] ------------------------------------------

7 Dec 23:22:25 - [warn] [rpi-gpio] Info : Ignoring Raspberry Pi specific node

7 Dec 23:22:25 - [warn] ------------------------------------------

7 Dec 23:22:25 - [info] Settings file  : /usr/lib/node_modules/node-red/settings.js

7 Dec 23:22:25 - [info] User directory : /home/activemq/.node-red

7 Dec 23:22:25 - [info] Flows file : /home/activemq/.node-red/flows_iot-dev-activemq1.json

7 Dec 23:22:25 - [info] Server now running at https://127.0.0.1:1880/

7 Dec 23:22:25 - [info] Starting flows

7 Dec 23:22:25 - [info] Started flows

7 Dec 23:22:25 - [info] [mqtt-broker:20dfcbb3.df2034] Connected to broker: Cospac_Consumer1@mqtt://localhost:1883

Running on Centos 7

ActiveMQ console output - occasionally it will complain about a malformed packet QOS and Retain Settings

INFO | jolokia-agent: No access restrictor found at classpath:/jolokia-access.xml, access to all MBeans is allowed

 WARN | Stealing link for clientId Cospac_Publisher1 From Connection Transport Connection to: tcp://134.178.15.45:56807

 WARN | Stealing link for clientId Cospac_Publisher1 From Connection Transport Connection to: tcp://134.178.15.45:56808

 WARN | Stealing link for clientId Cospac_Publisher1 From Connection Transport Connection to: tcp://134.178.15.45:56809

 WARN | Stealing link for clientId Cospac_Publisher1 From Connection Transport Connection to: tcp://134.178.15.45:56810

 WARN | Stealing link for clientId Cospac_Publisher1 From Connection Transport Connection to: tcp://134.178.15.45:56811

 WARN | Stealing link for clientId Cospac_Publisher1 From Connection Transport Connection to: tcp://134.178.15.45:56812

 WARN | Stealing link for clientId Cospac_Publisher1 From Connection Transport Connection to: tcp://134.178.15.45:56813

 WARN | Stealing link for clientId Cospac_Publisher1 From Connection Transport Connection to: tcp://134.178.15.45:56814

 WARN | Stealing link for clientId Cospac_Publisher1 From Connection Transport Connection to: tcp://134.178.15.45:56815

 WARN | Stealing link for clientId Cospac_Publisher1 From Connection Transport Connection to: tcp://134.178.15.45:56817

 WARN | Stealing link for clientId Cospac_Publisher1 From Connection Transport Connection to: tcp://134.178.15.45:56818

 WARN | Stealing link for clientId Cospac_Publisher1 From Connection Transport Connection to: tcp://134.178.15.45:56819

 WARN | Stealing link for clientId Cospac_Publisher1 From Connection Transport Connection to: tcp://134.178.15.45:56820

Attached Flows

[{"id":"29e4fb8b.d61b04","type":"mqtt-broker","z":"b70d3b9e.48f2c8","broker":"104.236.171.177","port":"1883","clientid":"Cospac_Consumer","usetls":false,"verifyservercert":true,"compatmode":false,"keepalive":"15","cleansession":true,"willTopic":"","willQos":"0","willRetain":"false","willPayload":"","birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":""},{"id":"734eaad0.8cb154","type":"mongodb","z":"b70d3b9e.48f2c8","hostname":"localhost","port":"27017","db":"cosppac","name":"localhost"},{"id":"11e030f1.ee1fcf","type":"function","z":"b70d3b9e.48f2c8","name":"parse","func":"context.global.original_msg = context.global.original_msg || {};\n\nvar _ = context.global.lodash,\n    data = JSON.parse(msg.payload), \n    allowed = [\n        200891, \n        200892,\n        200859\n    ];\n\nif (_.includes(allowed, data.stn_num)) {\n    \n    context.global.original_msg[data.stn_num] = data;\n    \n    return _.assign({}, msg, {\n        payload: { \n            stn_num: data.stn_num \n        },\n        limit: 0,\n        skip:0\n    });\n}\n","outputs":1,"noerr":0,"x":315,"y":228,"wires":[["df90334.f206fd"]]},{"id":"df90334.f206fd","type":"mongodb in","z":"b70d3b9e.48f2c8","mongodb":"734eaad0.8cb154","name":"get","collection":"items","operation":"find","x":453,"y":227,"wires":[["1be9abf0.e41654","444a892d.bbb578"]]},{"id":"1be9abf0.e41654","type":"function","z":"b70d3b9e.48f2c8","name":"update","func":"var _ = context.global.lodash,\n  doc = msg.payload.pop(),\n  data = context.global.original_msg[doc.stn_num];\n\nfunction updateDoc(doc, data) {\n  \n  doc.name = data.name;\n  doc.last_tx = data.x_tm;\n  doc.wla = data.wla;\n  doc.wla_2 = data.wla_2;\n  doc.wla_3 = data.wla_3;\n  doc.wla_4 = data.wla_4;\n  doc.sea_lvl_pred = data.sea_lvl_pred;\n  doc.residual_1 = data.residual_1;\n  doc.residual_2 = data.residual_2;\n  doc.residual_3 = data.residual_3;\n  doc.residual_4 = data.residual_4;\n  doc.t = data.t;\n  doc.wt = data.wt;\n  doc.qfe = data.qfe;\n  doc.da = data.da;\n  doc.ds = data.ds;\n  doc.sa = data.sa;\n  doc.sx = data.sx;\n  doc.da10 = data.da10;\n  doc.sa10 = data.sa10;\n  doc.sx10 = data.sx10;\n  doc.bv = data.bv;\n  doc.ti = data.ti;\n  doc.tat1 = data.tat1;\n  doc.tat2 = data.tat2;\n  \n}\n\nfunction chartSeries() {\n\n  return [{\n    name: \"sea_lvl_pred_3h\"\n  }, {\n    name: \"wla\"\n  }, {\n    name: \"wla_2\"\n  }, {\n    name: \"wla_3\"\n  }, {\n    name: \"wla_4\"\n  },  {\n    name: \"residual_1\"\n  }, {\n    name: \"residual_2\"\n  }, {\n    name: \"residual_3\"\n  }, {\n    name: \"residual_4\"\n  },{\n    name: \"t\"\n  }, {\n    name: \"tx\"\n  }, {\n    name: \"tn\"\n  }, {\n    name: \"wt\"\n  }, {\n    name: \"qfe\"\n  }, {\n    name: \"da\"\n  }, {\n    name: \"da10\"\n  }, {\n    name: \"sa\"\n  }, {\n    name: \"sa10\"\n  }, {\n    name: \"sx\"\n  }, {\n    name: \"sx10\"\n  }, {\n    name: \"bv\"\n  },{\n    name: \"ti\"\n  },{\n    name: \"tat1\"\n  },{\n    name: \"tat2\"\n  }];\n}\n\n\nfunction padNulls(data, value, currTm) {\n\n  if (data.length) {\n\n    var last = _.last(data),\n      mins = (currTm - last[0]) / (1000 * 60);\n    \n    //only need one null \n    if (mins > 1) {\n      data.push([last[0] + 60000, null]);\n    }\n    //for (var i = 0; i < mins - 1; ++i) {//}\n  }\n}\n\nfunction appendData(series, data, tm, len) {\n\n  padNulls(series.data, data[series.name], tm);\n\n  series.data.push([tm, data[series.name]]);\n\n  if (series.data.length > len) {\n    series.data = _.takeRight(series.data, len);\n  }\n}\n\n\nif (doc && data) {\n\n  //if it's a new message\n  if (!doc.last_tx || doc.last_tx && doc.last_tx !== data.x_tm) {\n\n    updateDoc(doc, data);\n    \n    doc.series = doc.series || chartSeries();\n\n    doc.series.forEach(function(s) {\n\n        s.data = s.data || [];\n\n        if (s.name === \"sea_lvl_pred_3h\") {\n          appendData(s, data, data.x_tm_3h, 3060); //3060\n\n        } else {\n          appendData(s, data, data.x_tm, 2880);\n        }\n        \n    });\n\n  }\n}\n\nreturn _.assign({}, msg, {\n      query: { \n          stn_num: data.stn_num \n      },\n      payload: doc,\n      skip: 0,\n      limit: 0\n    });\n","outputs":1,"noerr":0,"x":602,"y":122,"wires":[["b7f6873e.480978","967c66d4.698398"]]},{"id":"b7f6873e.480978","type":"mongodb out","z":"b70d3b9e.48f2c8","mongodb":"734eaad0.8cb154","name":"set","collection":"items","payonly":false,"upsert":false,"multi":false,"operation":"update","x":796,"y":123,"wires":[]},{"id":"5761b96b.a89e48","type":"comment","z":"b70d3b9e.48f2c8","name":"consume sea level stations","info":"","x":144,"y":55,"wires":[]},{"id":"444a892d.bbb578","type":"debug","z":"b70d3b9e.48f2c8","name":"","active":true,"console":"false","complete":"false","x":633,"y":352,"wires":[]},{"id":"6cbdfeb6.9342","type":"debug","z":"b70d3b9e.48f2c8","name":"","active":false,"console":"false","complete":"false","x":100,"y":295,"wires":[]},{"id":"967c66d4.698398","type":"debug","z":"b70d3b9e.48f2c8","name":"","active":true,"console":"false","complete":"false","x":768,"y":204,"wires":[]},{"id":"af32fca3.50cd","type":"debug","z":"b70d3b9e.48f2c8","name":"","active":false,"console":"false","complete":"false","x":98,"y":170,"wires":[]},{"id":"1ce288dc.e31d77","type":"mqtt in","z":"b70d3b9e.48f2c8","name":"","topic":"asos/station/200891","broker":"29e4fb8b.d61b04","x":104,"y":107,"wires":[["af32fca3.50cd","11e030f1.ee1fcf"]]},{"id":"73d79b9c.8c2864","type":"mqtt in","z":"b70d3b9e.48f2c8","name":"","topic":"asos/station/200892","broker":"29e4fb8b.d61b04","x":107,"y":230,"wires":[["6cbdfeb6.9342","11e030f1.ee1fcf"]]},{"id":"9be820e1.6417e","type":"mqtt in","z":"b70d3b9e.48f2c8","name":"","topic":"asos/station/200859","broker":"29e4fb8b.d61b04","x":103,"y":353,"wires":[["8d8e6411.727198","11e030f1.ee1fcf"]]},{"id":"8d8e6411.727198","type":"debug","z":"b70d3b9e.48f2c8","name":"","active":false,"console":"false","complete":"false","x":94,"y":423,"wires":[]}]

 

[{"id":"7ec7cd8f.813834","type":"mqtt-broker","z":"94ab46e1.6b54b8","broker":"104.236.171.177","port":"1883","clientid":"Cospac_Publisher","usetls":false,"verifyservercert":true,"compatmode":false,"keepalive":"15","cleansession":true,"willTopic":"","willQos":"0","willRetain":"false","willPayload":"","birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":""},{"id":"cd9fb843.326048","type":"inject","z":"94ab46e1.6b54b8","name":"Trigger ASOS Query","topic":"","payload":"","payloadType":"date","repeat":"60","crontab":"","once":false,"x":131,"y":135,"wires":[["ea518850.15ae78"]]},{"id":"ea518850.15ae78","type":"http request","z":"94ab46e1.6b54b8","name":"ASOS Query","method":"GET","ret":"obj","url":"http://asoswebtest.bom.gov.au/reports/index.php?cmd=Omd&format=json","x":337,"y":135,"wires":[["d5f50c1.f2a0af","5198401e.ae67c"]]},{"id":"d5f50c1.f2a0af","type":"function","z":"94ab46e1.6b54b8","name":"Set MQTT Topic","func":"var out = [];\n \nmsg.payload.forEach(function(station){\n    if (station.stn_num) {\n      \tout.push({topic: \"asos/station/\"+ station.stn_num , payload:station});\n    }\n});\n\nreturn [out];","outputs":1,"noerr":0,"x":559,"y":234,"wires":[["2305be16.dcfa42","18495d0a.e7b6a3"]]},{"id":"f949fc0e.06b6","type":"comment","z":"94ab46e1.6b54b8","name":"publish api ","info":"","x":74,"y":64,"wires":[]},{"id":"18495d0a.e7b6a3","type":"debug","z":"94ab46e1.6b54b8","name":"","active":false,"console":"false","complete":"false","x":785,"y":233,"wires":[]},{"id":"5198401e.ae67c","type":"debug","z":"94ab46e1.6b54b8","name":"","active":false,"console":"false","complete":"false","x":546,"y":61,"wires":[]},{"id":"2305be16.dcfa42","type":"mqtt out","z":"94ab46e1.6b54b8","name":"","topic":"","qos":"1","retain":"true","broker":"7ec7cd8f.813834","x":791,"y":106,"wires":[]}]

Cheers

Adrian

[Adrian Brown]

The other thing we have noticed is that it does behave better when the broker is local e.g localhost, but it also eventually has the same issues

[Nicholas O'Leary]

Could you describe a bit more what you see? Is it seemingly random? Are you both publishing and subscribed to messages? Have you tried running with the 3.1 legacy support enabled/disabled?

I haven't seen the types of thing you describe, so would like to understand better the conditions you are hitting it under.

Nick

--
http://nodered.org
---
You received this message because you are subscribed to the Google Groups "Node-RED" group.
To unsubscribe from this group and stop receiving emails from it, send an email to node-red+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Adrian Brown]

Hi Nick at times it works fine, but If you leave it long enough it will start to misbehave. The MQTT client keeps randomly connecting and disconnecting

One node-red instance  is publishing from machine a and and another node-red instance is subscribing on machine b and updating a mongodb data base running on local host; (see flow below)

[{"id":"20dfcbb3.df2034","type":"mqtt-broker","z":"394bdae5.c6b426","broker":"localhost","port":"1883","clientid":"Cospac_Consumer1","usetls":false,"verifyservercert":false,"compatmode":false,"keepalive":"15","cleansession":false,"willTopic":"","willQos":"0","willRetain":"false","willPayload":"","birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":""},{"id":"2995b7e5.d66a48","type":"mongodb","z":"394bdae5.c6b426","hostname":"localhost","port":"27017","db":"cosppac","name":"localhost"},{"id":"a511afd5.5aee5","type":"function","z":"394bdae5.c6b426","name":"parse","func":"context.global.original_msg = context.global.original_msg || {};\n\nvar _ = context.global.lodash,\n    data = JSON.parse(msg.payload), \n    allowed = [\n        200891, \n        200892,\n        200859\n    ];\n\nif (_.includes(allowed, data.stn_num)) {\n    \n    context.global.original_msg[data.stn_num] = data;\n    \n    return _.assign({}, msg, {\n        payload: { \n            stn_num: data.stn_num \n        },\n        limit: 0,\n        skip:0\n    });\n}\n","outputs":1,"noerr":0,"x":358,"y":229,"wires":[["9ebe5619.6141a8"]]},{"id":"9ebe5619.6141a8","type":"mongodb in","z":"394bdae5.c6b426","mongodb":"2995b7e5.d66a48","name":"get","collection":"items","operation":"find","x":496,"y":228,"wires":[["570dc371.a8f23c","9184254.f6e7bd8"]]},{"id":"570dc371.a8f23c","type":"function","z":"394bdae5.c6b426","name":"update","func":"var _ = context.global.lodash,\n  doc = msg.payload.pop(),\n  data = context.global.original_msg[doc.stn_num];\n\nfunction updateDoc(doc, data) {\n  \n  doc.name = data.name;\n  doc.last_tx = data.x_tm;\n  doc.wla = data.wla;\n  doc.wla_2 = data.wla_2;\n  doc.wla_3 = data.wla_3;\n  doc.wla_4 = data.wla_4;\n  doc.sea_lvl_pred = data.sea_lvl_pred;\n  doc.residual_1 = data.residual_1;\n  doc.residual_2 = data.residual_2;\n  doc.residual_3 = data.residual_3;\n  doc.residual_4 = data.residual_4;\n  doc.t = data.t;\n  doc.wt = data.wt;\n  doc.qfe = data.qfe;\n  doc.da = data.da;\n  doc.ds = data.ds;\n  doc.sa = data.sa;\n  doc.sx = data.sx;\n  doc.da10 = data.da10;\n  doc.sa10 = data.sa10;\n  doc.sx10 = data.sx10;\n  doc.bv = data.bv;\n  doc.ti = data.ti;\n  doc.tat1 = data.tat1;\n  doc.tat2 = data.tat2;\n  \n}\n\nfunction chartSeries() {\n\n  return [{\n    name: \"sea_lvl_pred_3h\"\n  }, {\n    name: \"wla\"\n  }, {\n    name: \"wla_2\"\n  }, {\n    name: \"wla_3\"\n  }, {\n    name: \"wla_4\"\n  },  {\n    name: \"residual_1\"\n  }, {\n    name: \"residual_2\"\n  }, {\n    name: \"residual_3\"\n  }, {\n    name: \"residual_4\"\n  },{\n    name: \"t\"\n  }, {\n    name: \"tx\"\n  }, {\n    name: \"tn\"\n  }, {\n    name: \"wt\"\n  }, {\n    name: \"qfe\"\n  }, {\n    name: \"da\"\n  }, {\n    name: \"da10\"\n  }, {\n    name: \"sa\"\n  }, {\n    name: \"sa10\"\n  }, {\n    name: \"sx\"\n  }, {\n    name: \"sx10\"\n  }, {\n    name: \"bv\"\n  },{\n    name: \"ti\"\n  },{\n    name: \"tat1\"\n  },{\n    name: \"tat2\"\n  }];\n}\n\n\nfunction padNulls(data, value, currTm) {\n\n  if (data.length) {\n\n    var last = _.last(data),\n      mins = (currTm - last[0]) / (1000 * 60);\n    \n    //only need one null \n    if (mins > 1) {\n      data.push([last[0] + 60000, null]);\n    }\n    //for (var i = 0; i < mins - 1; ++i) {//}\n  }\n}\n\nfunction appendData(series, data, tm, len) {\n\n  padNulls(series.data, data[series.name], tm);\n\n  series.data.push([tm, data[series.name]]);\n\n  if (series.data.length > len) {\n    series.data = _.takeRight(series.data, len);\n  }\n}\n\n\nif (doc && data) {\n\n  //if it's a new message\n  if (!doc.last_tx || doc.last_tx && doc.last_tx !== data.x_tm) {\n\n    updateDoc(doc, data);\n    \n    doc.series = doc.series || chartSeries();\n\n    doc.series.forEach(function(s) {\n\n        s.data = s.data || [];\n\n        if (s.name === \"sea_lvl_pred_3h\") {\n          appendData(s, data, data.x_tm_3h, 3060); //3060\n\n        } else {\n          appendData(s, data, data.x_tm, 2880);\n        }\n        \n    });\n\n  }\n}\n\nreturn _.assign({}, msg, {\n      query: { \n          stn_num: data.stn_num \n      },\n      payload: doc,\n      skip: 0,\n      limit: 0\n    });\n","outputs":1,"noerr":0,"x":652,"y":229,"wires":[["2142e9ea.debd16","f51e4620.0ae1b8"]]},{"id":"2142e9ea.debd16","type":"mongodb out","z":"394bdae5.c6b426","mongodb":"2995b7e5.d66a48","name":"set","collection":"items","payonly":false,"upsert":false,"multi":false,"operation":"update","x":872,"y":143,"wires":[]},{"id":"6e4fff39.91b","type":"comment","z":"394bdae5.c6b426","name":"consume sea level stations","info":"","x":121,"y":44,"wires":[]},{"id":"9184254.f6e7bd8","type":"debug","z":"394bdae5.c6b426","name":"","active":true,"console":"false","complete":"false","x":676,"y":353,"wires":[]},{"id":"96a9046d.6956f8","type":"debug","z":"394bdae5.c6b426","name":"","active":true,"console":"false","complete":"false","x":102,"y":331,"wires":[]},{"id":"f51e4620.0ae1b8","type":"debug","z":"394bdae5.c6b426","name":"","active":true,"console":"false","complete":"false","x":823,"y":231,"wires":[]},{"id":"b056c59f.4fa938","type":"debug","z":"394bdae5.c6b426","name":"","active":true,"console":"false","complete":"false","x":103,"y":170,"wires":[]},{"id":"8c63a5c5.739c58","type":"mqtt in","z":"394bdae5.c6b426","name":"","topic":"asos/station/200891","broker":"20dfcbb3.df2034","x":120,"y":88,"wires":[["b056c59f.4fa938","a511afd5.5aee5"]]},{"id":"950d3d47.6af2c","type":"mqtt in","z":"394bdae5.c6b426","name":"","topic":"asos/station/200892","broker":"20dfcbb3.df2034","x":107,"y":235,"wires":[["96a9046d.6956f8","a511afd5.5aee5"]]},{"id":"c0356fa2.3fca9","type":"mqtt in","z":"394bdae5.c6b426","name":"","topic":"asos/station/200859","broker":"20dfcbb3.df2034","x":111,"y":403,"wires":[["3dc70c03.c238f4","a511afd5.5aee5"]]},{"id":"3dc70c03.c238f4","type":"debug","z":"394bdae5.c6b426","name":"","active":true,"console":"false","complete":"false","x":115,"y":500,"wires":[]}]

This flow causes issues on its own as very often the cpu is maxed out with node-red red consuming almost all CPU resources

legacy support makes no difference

Cheers

Adrian

[Adrian Brown]

ActiveMQ now and then complains about this

 WARN | Exception occurred processing:

MQTTFrame { type: unknown, qos: AT_MOST_ONCE, dup:false }: org.apache.activemq.transport.mqtt.MQTTProtocolException: Unknown MQTTFrame type: 0

 WARN | Transport Connection to: tcp://134.178.24.130:55882 failed: java.net.ProtocolException: Invalid CONNECT frame

 WARN | Exception occurred processing:

MQTTFrame { type: unknown, qos: AT_MOST_ONCE, dup:false }: org.apache.activemq.transport.mqtt.MQTTProtocolException: Unknown MQTTFrame type: 0

On Tuesday, 8 December 2015 10:46:04 UTC+11, Adrian Brown wrote:

[Adrian Brown]

ActiveMQ now and then complains about this

 WARN | Exception occurred processing:

MQTTFrame { type: unknown, qos: AT_MOST_ONCE, dup:false }: org.apache.activemq.transport.mqtt.MQTTProtocolException: Unknown MQTTFrame type: 0

 WARN | Transport Connection to: tcp://134.178.24.130:55882 failed: java.net.ProtocolException: Invalid CONNECT frame

 WARN | Exception occurred processing:

MQTTFrame { type: unknown, qos: AT_MOST_ONCE, dup:false }: org.apache.activemq.transport.mqtt.MQTTProtocolException: Unknown MQTTFrame type: 0

though like I said its not just an ActiveMQ issue.Its also very unstable on other brokers like rabbit and mosquitto

On Tuesday, 8 December 2015 10:57:18 UTC+11, Nick O'Leary wrote:

[Adrian Brown]

Top display of machine running mongodb update flow - it can hang like comes good for a a few minutes then hangs again

 6419 activemq  20   0 1152632 231180   7260 R 99.3 22.7  25:36.27 node-red

 3234 root      20   0 3135872 166704   6736 S  0.3 16.4   0:27.16 java

    1 root      20   0   54292   2616   1572 S  0.0  0.3   0:01.03 systemd

    2 root      20   0       0      0      0 S  0.0  0.0   0:00.00 kthreadd

    3 root      20   0       0      0      0 S  0.0  0.0   0:00.02 ksoftirqd+

    5 root       0 -20       0      0      0 S  0.0  0.0   0:00.00 kworker/0+

    6 root      20   0       0      0      0 S  0.0  0.0   0:00.06 kworker/u+

    7 root      rt   0       0      0      0 S  0.0  0.0   0:00.00 migration+

[Toshi Bass]

I had same issue repeated disconnect / connect on Mqtt, when I moved node red flows to same pi as Mqtt server it appeared to resolve the issue but I have had 1 seamingly random disconnect connect issue since, only happened after 0.12.2 update.

[Adrian Brown]

We have narrowed down the issue.

When we are running the mongodb node - node red goes into this high cpu usage mode ( essentially hangs) the connection to the broker then become unstable.

On Tuesday, 8 December 2015 10:46:04 UTC+11, Adrian Brown wrote:

[Adrian Brown]

Fix we changed the flow to utilize the mongodb2 node and  all works well. I would suggest the mongodb node has some issues. That need to be looked at.

new flow is below

[{"id":"3aaeb8a.fc55148","type":"mongodb2","z":"513a9521.aec56c","hostname":"iot-dev-activemq1.bom.gov.au","port":"27017","db":"cosppac","name":"Cosppac","options":"","parallelism":"-1"},{"id":"c78b28bb.3874d8","type":"mqtt-broker","z":"513a9521.aec56c","broker":"localhost","port":"1883","clientid":"Cospac_Consumer","usetls":true,"verifyservercert":false,"compatmode":false,"keepalive":"15","cleansession":false,"willTopic":"","willQos":"0","willRetain":"false","willPayload":"","birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":""},{"id":"1ea11585.e15eea","type":"function","z":"513a9521.aec56c","name":"parse","func":"context.global.original_msg = context.global.original_msg || {};\n\nvar _ = context.global.lodash,\n    data = JSON.parse(msg.payload), \n    allowed = [\n        200891, \n        200892,\n        200859\n    ];\n\nif (_.includes(allowed, data.stn_num)) {\n    \n    context.global.original_msg[data.stn_num] = data;\n    \n    return _.assign({}, msg, {\n        payload: { \n            stn_num: data.stn_num \n        },\n        limit: 0,\n        skip:0\n    });\n}\n","outputs":1,"noerr":0,"x":382,"y":267,"wires":[["89e755ec.7618a8"]]},{"id":"ce9776b6.316888","type":"function","z":"513a9521.aec56c","name":"update","func":"var _ = context.global.lodash,\n  doc = msg.payload,\n  data = context.global.original_msg[doc.stn_num];\n\nfunction updateDoc(doc, data) {\n  \n  doc.name = data.name;\n  doc.last_tx = data.x_tm;\n  doc.wla = data.wla;\n  doc.wla_2 = data.wla_2;\n  doc.wla_3 = data.wla_3;\n  doc.wla_4 = data.wla_4;\n  doc.sea_lvl_pred = data.sea_lvl_pred;\n  doc.residual_1 = data.residual_1;\n  doc.residual_2 = data.residual_2;\n  doc.residual_3 = data.residual_3;\n  doc.residual_4 = data.residual_4;\n  doc.t = data.t;\n  doc.wt = data.wt;\n  doc.qfe = data.qfe;\n  doc.da = data.da;\n  doc.ds = data.ds;\n  doc.sa = data.sa;\n  doc.sx = data.sx;\n  doc.da10 = data.da10;\n  doc.sa10 = data.sa10;\n  doc.sx10 = data.sx10;\n  doc.bv = data.bv;\n  doc.ti = data.ti;\n  doc.tat1 = data.tat1;\n  doc.tat2 = data.tat2;\n  \n}\n\nfunction chartSeries() {\n\n  return [{\n    name: \"sea_lvl_pred_3h\"\n  }, {\n    name: \"wla\"\n  }, {\n    name: \"wla_2\"\n  }, {\n    name: \"wla_3\"\n  }, {\n    name: \"wla_4\"\n  },  {\n    name: \"residual_1\"\n  }, {\n    name: \"residual_2\"\n  }, {\n    name: \"residual_3\"\n  }, {\n    name: \"residual_4\"\n  },{\n    name: \"t\"\n  }, {\n    name: \"tx\"\n  }, {\n    name: \"tn\"\n  }, {\n    name: \"wt\"\n  }, {\n    name: \"qfe\"\n  }, {\n    name: \"da\"\n  }, {\n    name: \"da10\"\n  }, {\n    name: \"sa\"\n  }, {\n    name: \"sa10\"\n  }, {\n    name: \"sx\"\n  }, {\n    name: \"sx10\"\n  }, {\n    name: \"bv\"\n  },{\n    name: \"ti\"\n  },{\n    name: \"tat1\"\n  },{\n    name: \"tat2\"\n  }];\n}\n\n\nfunction padNulls(data, value, currTm) {\n\n  if (data.length) {\n\n    var last = _.last(data),\n      mins = (currTm - last[0]) / (1000 * 60);\n    \n    //only need one null \n    if (mins > 1) {\n      data.push([last[0] + 60000, null]);\n    }\n    //for (var i = 0; i < mins - 1; ++i) {//}\n  }\n}\n\nfunction appendData(series, data, tm, len) {\n\n  padNulls(series.data, data[series.name], tm);\n\n  series.data.push([tm, data[series.name]]);\n\n  if (series.data.length > len) {\n    series.data = _.takeRight(series.data, len);\n  }\n}\n\n\nif (doc && data) {\n\n  //if it's a new message\n  if (!doc.last_tx || doc.last_tx && doc.last_tx !== data.x_tm) {\n\n    updateDoc(doc, data);\n    \n    doc.series = doc.series || chartSeries();\n\n    doc.series.forEach(function(s) {\n\n        s.data = s.data || [];\n\n        if (s.name === \"sea_lvl_pred_3h\") {\n          appendData(s, data, data.x_tm_3h, 3060); //3060\n\n        } else {\n          appendData(s, data, data.x_tm, 2880);\n        }\n        \n    });\n\n  }\n}\n\nreturn _.assign({}, msg, {\n    payload: [\n        { \n          stn_num: data.stn_num \n        },\n    doc\n]\n    });\n","outputs":1,"noerr":0,"x":735,"y":269,"wires":[["b08b8042.4f748","58ac92fe.20ebec"]]},{"id":"20533cbe.dfacc4","type":"debug","z":"513a9521.aec56c","name":"","active":false,"console":"false","complete":"false","x":1212,"y":268,"wires":[]},{"id":"bb00291d.44ffd8","type":"debug","z":"513a9521.aec56c","name":"","active":false,"console":"false","complete":"false","x":167,"y":334,"wires":[]},{"id":"62e44f29.9d1bb","type":"debug","z":"513a9521.aec56c","name":"","active":false,"console":"false","complete":"false","x":788,"y":473,"wires":[]},{"id":"96a9f02.f69561","type":"debug","z":"513a9521.aec56c","name":"","active":false,"console":"false","complete":"false","x":165,"y":209,"wires":[]},{"id":"22c18479.dd3e7c","type":"mqtt in","z":"513a9521.aec56c","name":"","topic":"asos/station/200891","broker":"c78b28bb.3874d8","x":171,"y":146,"wires":[["96a9f02.f69561","1ea11585.e15eea"]]},{"id":"b785bf53.487a4","type":"mqtt in","z":"513a9521.aec56c","name":"","topic":"asos/station/200892","broker":"c78b28bb.3874d8","x":174,"y":269,"wires":[["bb00291d.44ffd8","1ea11585.e15eea"]]},{"id":"42b573c5.bd4a8c","type":"mqtt in","z":"513a9521.aec56c","name":"","topic":"asos/station/200859","broker":"c78b28bb.3874d8","x":170,"y":392,"wires":[["5a43b7ba.a5bc48","1ea11585.e15eea"]]},{"id":"5a43b7ba.a5bc48","type":"debug","z":"513a9521.aec56c","name":"","active":false,"console":"false","complete":"false","x":161,"y":462,"wires":[]},{"id":"89e755ec.7618a8","type":"mongodb2 in","z":"513a9521.aec56c","service":"_ext_","configNode":"3aaeb8a.fc55148","name":"","collection":"items","operation":"findOne","x":551,"y":269,"wires":[["ce9776b6.316888","62e44f29.9d1bb"]]},{"id":"b08b8042.4f748","type":"mongodb2 in","z":"513a9521.aec56c","service":"_ext_","configNode":"3aaeb8a.fc55148","name":"","collection":"items","operation":"updateOne","x":939,"y":269,"wires":[["20533cbe.dfacc4"]]},{"id":"58ac92fe.20ebec","type":"debug","z":"513a9521.aec56c","name":"","active":false,"console":"false","complete":"false","x":1041,"y":367,"wires":[]}]

On Tuesday, 8 December 2015 10:46:04 UTC+11, Adrian Brown wrote:

[Julian Knight]

That's good info Adrian, thanks.

I've been seeing the same issue on my Pi2 which has 2xNR, MongoDB and Mosquitto running. Hadn't had time to track it down. It does appear that MQTT is still getting all the messages though.

I'll try swapping to the alternative MongoDB driver to see if that fixes it. Might explain a few other oddities with MongoDB too.

[Toshi Bass]

Just to be clear I was seeing repeated disconnect / connect on Mqtt using Mosquitto, but I am not using MongoDB

Toshi

[Dave C-J]

To be totally clear.. "Was seeing" or still are seeing ?

[Toshi Bass]

Well as I said, "Just to be clear I was seeing" ...... when I moved node red flows to same pi as Mqtt server it appeared to resolve the issue that was 2 days ago and mqtt has not disconnected. so for sure "Was seeing" ok not helpful to you, I just wanted to log the issue I had just incase someone else has same same problem and point out that although the problem appeared to be similar to the original post I was not using MongoDB 

[Matt]

I am having problems with connection to an external broker as well.

I have two raspberry pis with node-red installs, one of which has the mosquitto broker. On the other one I was having slowdowns of the node-red editor UI as well as the simple webpage it is hosting. Looking in ~/.pm/logs/ I have a 1.2gb node-red-out-0.log file!

The log is full of entries like:

5 Dec 05:01:13 - [info] [mqtt-broker:dbfede8e.24012] Disconnected from broker: mqtt://192.168.0.100:1883

5 Dec 05:01:13 - [info] [mqtt-broker:dbfede8e.24012] Connected to broker: mqtt://192.168.0.100:1883

5 Dec 05:01:13 - [info] [mqtt-broker:dbfede8e.24012] Disconnected from broker: mqtt://192.168.0.100:1883

5 Dec 05:01:13 - [info] [mqtt-broker:dbfede8e.24012] Connected to broker: mqtt://192.168.0.100:1883

Restarting node red (pm2 restart node-red) seems to fix it for a while. 

I havent had any problems with mqtt on the same pi that the broker resides on.

--

[Toshi Bass]

Hi Matt re the slow downs you refer to does that involve 3 to 4 minutes to deploy flows?
I had this problem as well but on the pi with mosquitto, I found that it didn't happen if used my pi zero so I rolled back node.js from v0.12.6 to v0.10.36 (because I couldn't find v0.10.29 the one that comes bundled with Jessie) and that cured the problem!

[Matt]

I wasn't doing much deploying so I can't say one way or the other.

It would take a couple of minutes to load the editor ui or a webpage served up by http nodes.

On Sun, Dec 13, 2015 at 1:50 PM, Toshi Bass <toshib...@gmail.com> wrote:

Hi Matt re the slow downs you refer to does that involve 3 to 4 minutes to deploy flows?
I had this problem as well but on the pi with mosquitto, I found that it didn't happen if used my pi zero so I rolled back node.js from  v0.12.6 to v0.10.36 (because I couldn't find v0.10.29 the one that comes bundled with Jessie) and that cured the problem!

[Pedro Mendes]

Hi guys, I'm having the same issues with instability of the MQTT node.

- using MySql

- no MongoDB

- mosquitto broker on localhost, no TLS, no legacy compatibility mode

This started happening after I updated to the following versions:

- node.js v0.12.10

- Node-RED v0.13.1

- mosquitto version 1.4.7

From:

- node.js v?

- Node-RED v0.12.?

- mosquitto version 1.4.2

After the instability starts (lots of connects and disconnects) node-red eventually stops responding. Sometimes it can go 1 or 2 days without a problem, sometimes it only takes an hour for it to start behaving like this. 

Any help would be greatly appreciated!

Cheers,

Pedro.

[Pedro Mendes]

It seems like this has been solved with v 0.13.2 --> https://groups.google.com/forum/#!searchin/node-red/mqtt$20connect$20disconnect/node-red/IWmNlkVeIYc/vnUxVTCYAwAJ

Just installed 0.13.3, so anxious to see the results :)

[Julian Knight]

Indeed, not seeing any issues at all now.

The only thing that might catch you out is if you end up with two MQTT clients with the same name. For example copying a flow that has the MQTT node with a name specified across to a second running instance of NR.

[Pedro Mendes]

Thanks for the info Julian. So far so good :)

[Pedro Mendes]

Just had a disconnect :( I was able to detect it and node-red didn't crash and didn't attempt reconnects forever, but wasn't able to resume the connection.

I had to manually restart the service in order to reconnect.

All other clients connected to mosquitto (external HW) were correctly connected and the service was up and running.

Are there any know issues with: 

mosquitto version 1.4.7 (build date 2016-02-08 20:05:57+0000)

Node-RED version: v0.13.3

Node.js  version: v0.12.10

?

I'm also using mosquito_auth_plug for authentication.

Any ideas?