persisting chart (0.16)

[Cor Bosman]

Hi all, im playing around with persisting graph data and not really succeeding. At the end is a simplified flow that shows what im doing. After some debugging it seems that immediately at startup, even though the top inject node has not injected data yet, empty data is being written to the output file node.  So the bottom section then reads an empty file. This seems like a bug to me. Or am I misunderstanding how this is supposed to work? 

I can probably fix this by setting some global variable that checks if data has been restored before writing back to file, but I sort of assumed this would work as is. 

I added a debug output file so you can see what is going on during startup. For me this is the output:

[]

READ STORAGE

[]

The first empty array is the output file node writing an empty array before the input file node has had a chance to read data from storage.

[{"id":"ab5e145d.962a88","type":"inject","z":"2f986470.46fcec","name":"","topic":"","payload":"","payloadType":"date","repeat":"10","crontab":"","once":false,"x":196.5,"y":437,"wires":[["166ce834.ad9588","a46aa01d.b0451"]]},{"id":"166ce834.ad9588","type":"random","z":"2f986470.46fcec","name":"random number","low":"1","high":"10","inte":"true","x":424.5,"y":437,"wires":[["9a4e44d5.548058"]]},{"id":"9a4e44d5.548058","type":"ui_chart","z":"2f986470.46fcec","name":"","group":"fc1757ac.916b98","order":0,"width":0,"height":0,"label":"chart","chartType":"line","legend":"false","xformat":"HH:mm:ss","interpolate":"linear","nodata":"","ymin":"0","ymax":"10","removeOlder":1,"removeOlderPoints":"","removeOlderUnit":"3600","cutout":0,"x":669.5,"y":436,"wires":[["16318cb1.56e013","a46aa01d.b0451"],[]]},{"id":"9f54c4b9.bc0788","type":"inject","z":"2f986470.46fcec","name":"","topic":"","payload":"READ STORAGE","payloadType":"str","repeat":"","crontab":"","once":true,"x":212.5,"y":605,"wires":[["f91fa71b.929df8","a46aa01d.b0451"]]},{"id":"f91fa71b.929df8","type":"file in","z":"2f986470.46fcec","name":"","filename":"/home/pi/storage/tmp.dat","format":"utf8","x":473.5,"y":607,"wires":[["5ce11390.636f7c"]]},{"id":"5ce11390.636f7c","type":"json","z":"2f986470.46fcec","name":"","x":571.5,"y":534,"wires":[["9a4e44d5.548058"]]},{"id":"16318cb1.56e013","type":"file","z":"2f986470.46fcec","name":"","filename":"/home/pi/storage/tmp.dat","appendNewline":true,"createDir":false,"overwriteFile":"true","x":977.5,"y":430,"wires":[]},{"id":"a46aa01d.b0451","type":"file","z":"2f986470.46fcec","name":"","filename":"/home/pi/storage/debug.dat","appendNewline":true,"createDir":false,"overwriteFile":"false","x":808.5,"y":744,"wires":[]},{"id":"fc1757ac.916b98","type":"ui_group","z":"","name":"Test","tab":"a92c5d74.e98a1","disp":true,"width":"6"},{"id":"a92c5d74.e98a1","type":"ui_tab","z":"","name":"Home","icon":"dashboard"}]

[Simon H]

hi Cor,

I just tested your flow in 15.2 - It works for me.  Added Debug nodes to the output of chart and the reader, and see the same data on both at deploy; and added a second chart, fed from the same restore, and the data restores to both charts, so maybe it is a bug in 16....

A while back I had a very short play with using the output to persist data,  but I ended up by keeping a complete dataset before the graph node, then updating the complete dataset on each change.  This was, for me, partly because my overall dataset was 30 days worth, and I wanted to be able to present only a day at a time to the graph and have control of how it looked.

The flow can be found in the library under cm160 mk3.  I actually construct two series, one with two points at start and end of day to force the display to always show the 24 hour period regardless of how many points there were for that day, and I save each day to disk for later recall.

The flow is actually very efficient with regard to the graph data.  But I did 'cheat' a little by requiring 'fs' in global and using it directly.

s

[Dave C-J]

well spotted...  on start the new charts seem to emit an empty array (when they have no need to) - so it will overwrite the file with a blank every start.

To stop it (for now, while I fix it) - insert a function between the chart and the file with

    if (msg.payload.length ===0 ) { return null; }

    return msg;

That will filter that out.

[Cor Bosman]

Thanks for verifying the bug. Added a temporary function node and now it works as I thought it should. 

return msg.payload.length ===0 ? null : msg;

[Paul Reed]

Now fixed in dashboard 2.2.2, and works well without the temp fix. Thanks Dave.

[Cor Bosman]

When do you expect this will be pushed to npm repo?