Dashboard/Chart. Display an array from database as a curve. "Bad data inject"

[timothylegg]

Hello,

I found a post from late February that dealt close to the issue I have, but I'm not familiar enough with Node-RED to apply it to my scenario.

I have a database that contains two integers and a timestamp for each record.  I have a database query that returns an array of integers from the MySQL node.  In debug it looks like array[995] and then underneath a collection of arrays from [0..9] up to [990..994].  I attach this to the chart node and am told "Bad data inject", which isn't a particularly constructive criticism.

Let's check out the documentation for the node-red-dashboard...

https://github.com/node-red/node-red-dashboard/blob/master/Charts.md

In the documentation under Live Data -> Line Charts, I see a reference to the 'topic' property, which bewilders me because the Chart node doesn't have a field named topic.  When I look at the more relevant section of the document Stored Data -> Line Charts, I'm presented with a much more sophisticated structure, but I'm not certain where this would be entered either.  Also, my data is dynamically generated, not entered by hand into a formatted table form as in the example.

What suggestions do you have to help draw a smooth curve from these data points I generated/collected?

[Nick O'Leary]

Hi,

It would help if you could share an example of the data you are receiving.

When Debug receives a big array, it doesn't expand all 995 elements in one go, rather slices it up into sections with 10 elements each. This makes it a bit more manageable to examine the data. What you haven't shared is what any of those 10 element slices actually contain.

The topic it is refering to is the topic property of the message passed to the chart node - not a property of the node itself. It uses the msg.topic property to identify which data series the msg is providing a value for.

In you instance, you aren't feeding in live data, but you have an array of 995 values to feed in. In that case you need to look further down in the docs on the Stored Data section. It provides an example of the required data format needed to pass in a complete set of readings in one go.

If you can provide a more detailed example of the data you have, we can certainly help you do the necessary transformation (possibly using a JSONata expression in a Change node, or some JavaScript in a Function node...) to get to the required result.

Nick

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to the Google Groups "Node-RED" group.
To unsubscribe from this group and stop receiving emails from it, send an email to node-red+u...@googlegroups.com.
To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.
To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/1b78cb21-e2a5-4c98-9021-9ce15e29fcdf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[timothylegg]

Absolutely, full disclosure is part of how I learn and share knowledge.  I'm authoring a tutorial on this as I'm learning, mainly because I learn better by documenting my own process.  I successfully did this last year, but waited too long before writing about it.  Also, it is substantially different from what I made in November and December.

These are the exact steps:

https://timothylegg.com/wiki/index.php/Create_an_IOT_humidity_sensor_using_Arduino_and_Linux_machine_with_serial_communication

I tied a plain, ordinary debug node to the output of the database, in parallel to the chart to gather the data you are (hopefully) seeking.

I copied the path...

   payload

I copied the value

  [{"temp":18},{"temp":18},{"temp":18},{"temp":18}...  skip about 980 items ... {"temp":18},{"temp":18}]

It is temperature data recorded on one minute intervals with some 995 results from the database.  It can be downloaded in entirety from the URL:

https://timothylegg.com/data.txt

I did make one hazardous assumption and that is one dimensional data would be assumed to be linear by the chart node and graphed as such.  It's trivial for me to add a second dimension, such a time since epoch integer, but I worried that it would complicate the design.

For the fun of it, I uploaded an SQL file of the data to my website as well:

http://timothylegg.com/climate.sql

[Zenofmud]

Timothy
 

Just wondering why you don’t use MQTT to send the data from the arguing to the pi? It makes it very easy to scale up the number of sensors you can use. In my house I have 5 different sensors, three are attached to wemos’s and two hard wired to the pi and all send their data via MQTT and can be processed. 

If you are writing a tutorial, I hope you would make easy for someone to expand the inputs without having to start over.

Just food for thought.

Paul

Sent from my iPad

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/98cb6e7e-6abc-49cd-851c-22672fcd5068%40googlegroups.com.

[timothylegg]

The main reason I haven't used MQTT is because I've never used MQTT and I almost certainly do not have the required hardware to host that data channel.  USB cables are plentiful though and every Arduino has one.  The C code in the linux side isn't very friendly though.  What I have is an amalgam of about 3 different examples.  I don't understand some portions of the code to it's hard to minimize the code down to exactly what is essential for communication.

I would be open to authoring a second, modified document for MQTT once I get my hands on the required hardware and read up on the details of what it provides.

What has me stuck presently is the part that I thought would be the easiest; getting the data represented as an image in the Node RED dashboard.  I have no idea what the problem is.

[Garry Hayne]

On Thursday, March 15, 2018 at 4:40:10 PM UTC, timothylegg wrote:

The main reason I haven't used MQTT is because I've never used MQTT and I almost certainly do not have the required hardware to host that data channel.

Timothy, MQTT is a sofware solution that uses your existing Network, either hard-wired or Wifi.

Have a look here for a very good introduction to MQTT:

https://www.hivemq.com/mqtt-essentials/

Garry