Time-shifted series

61 views
Skip to first unread message

MMM PPP

unread,
Dec 19, 2017, 4:37:39 PM12/19/17
to dygraphs-users
Hi,

I'm new to dygraph, but I think I'm starting to love it.

I'd like to ask if it's possible to do sth in particular, since I couldn't find anything related. Is it possible to show a time-shifted series in a graph, together with a "normal" series? I basically want to show today's data and compare it to yesterday's data in the same graph. This is really easy to do with Grafana, but I couldn't find a way to recreate it in dygraphs.

Any help will be greatly appreciated.

Bánhidi István

unread,
Dec 20, 2017, 3:05:53 AM12/20/17
to dygraph...@googlegroups.com
Hi,

Dygraph does not know this natively.
You have to use javascript for implementing this feature.
I think this is a simple task, because you have one data series.
You have to separate the data for two days and then replace the time stamp of one day with the other.
This is necessary because dygraph have just only one x axis, which is usually the time axis.
Ofcourse, after that you have two series in the graph. Finally, you have to mark somehow which day is yesterday and which is today's date.
For example: with colors, fullfill, annotations or other options.

ps.: Sorry for my English

Steve
--
You received this message because you are subscribed to the Google Groups "dygraphs-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dygraphs-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

MMM PPP

unread,
Dec 20, 2017, 6:06:51 AM12/20/17
to dygraphs-users
Thanks for your response.

I read that dygraph cannot handle two CSV data sources at the same time, so I guess the only option is to load the data from the CSV file into an array and add a new series by "shifting" that data by 24 hours. I also wanted to create series for comparing with the previous week, month and year, but I don't know if that is feasible since all the processing will be done by the web browser. Also, I have 0 experience with javascript.

PS: your English is fine. I hope mine is too...

El dimecres, 20 desembre de 2017 9:05:53 UTC+1, István Bánhidi va escriure:

Bánhidi István

unread,
Dec 20, 2017, 8:45:48 AM12/20/17
to dygraph...@googlegroups.com
Your are right.

When your JavaScript knowledge is low, then try to use another language on server side provided that you have access to the web server or what else create the CSV source file.
If you can not access to the other side, t
hen your only option that you write in JavaScript in browser (client side).
Otherwise never too late to learn something new!

I don't know what size of CSV files, but processing 1 million points is very fast. Check the examples, sample codes on dygraphs homepage.

Can you give more information about your problem?
For example: file sizes, number of records in files with special regard to files that include a larger time period, etc.


ps.: Your English is good.
At least I understand what you want and just that important ;)

Steve

MMM PPP

unread,
Dec 20, 2017, 9:19:00 AM12/20/17
to dygraphs-users
The data is from a series of sensors that append lines in several CSV files stored in a Raspberry Pi. There's more or less one value every minute or two, and some csv files (actually tab separated values) share several values (e.g. temperature, humidity, air pressure in a single file, preceded by the date and time). I currently use gnuplot to graph the data, and I compare the current versus the previous day, week and year. I generate the "previous" data series in the form of secondary csv files using bash scripts. The problem is that gnuplot graphs are not interactive (are just images displayed in a web server, I attached one to this post as an example) and cannot be zoomed. The other problem is that all the processing has to be done by the raspberry pi at regular intervals (e.g. updating every graph every 5 minutes), and when you have a lot of sensors, the CPU is generating graphs constantly, even if nobody is watching them, wasting resources that could be used for something else.

Using dygraphs, the graphs are generated when the user (it's just for me) visits the webpage, so it greatly reduces the CPU usage, and zooming in the graphs makes manually generating daily/weekly/monthly/yearly graphs uncessary, but each graphs take a few seconds to load in the website since they are processed on the go, compared to a static image which is instantaneous.

I guess I could prepare a CSV file with a series shifted by a day, a week, a month and a year every few minutes, and then show or hide the series in the graph using a few buttons, but I don't know if the raspberry pi could handle the manipulation of thousands of registers quickly enough.

The other option is letting the javascript do that, dynamically generating and showing the series when the graph is zoomed for day/week/month/year, but I have no experience in javascript and I don't know how long it would take to update the graph in the browser.

I'll explore the issue. That will sure keep me entertained during these christmas holidays :)

El dimecres, 20 desembre de 2017 14:45:48 UTC+1, István Bánhidi va escriure:
temp_day.png

Bánhidi István

unread,
Dec 23, 2017, 11:00:23 AM12/23/17
to dygraph...@googlegroups.com
Hi Marc,

Firstly:
How's your project?
Did you take a step forward?

Secondly:
What do you use for web server on RPI, do you have an installed php
?
You have to decide what do you want. I'm thinking about resource management.
Where do you want compute the data? On RPI or in your browser. Which is better for you?
I think you will never can choose until you don't try both version and you will see how is working the each version!

Thirdly:
I suggest that we go into private mode because this is a specific things not really about dygraphs.

Finally:
When you found a solution for your problem,
you will write it down to here as a thread closure.

MX
Steve

MMM PPP

unread,
Dec 26, 2017, 7:07:35 AM12/26/17
to dygraphs-users
Hi,

I spent some time working on it, although it's still a work in progress.

So far I've managed to read a CSV file using javascript and save it to a 2-dimension array. Then, I created a function to substract a number of milliseconds from the first column (the date) in order to get the previous day/week/month,etc. After that, I insert a new row containing the new date and its associated value in another column.

At this point, you have a bunch of rows containing dates and values (some at the 2nd row, some at the 3rd) with some empty values in between (you have to make sure these are blank). But dygraph needs to process the data in chronological order, so the next step is to sort all those dates.

Once they are in ascending order, you need to tell dygraph to connect all the datapoints regardless of the presence of empty datapoints in between.

I have assigned a button to call the functions that carry out all these processes, so every time I click the button, another column is added to the array and displayed as a series in the graph.

Right now it kinda works, but I have to optimize it. If the original CSV contains several tens of thousands of rows it can take a few seconds until the "previous" series is generated. And if called several times (which there's no need to), the array could grow exponentially and freeze your browser. Right now I wouldn't recommend it if you have less than 6GB of RAM ;) I've been thinking of only processing the range of data which is currently being displayed on the graph. Also, on CSV with multiple data series, I still have to tell the script which series should it use and where to place the new columns manually.


About the server, I'm using a Raspberry pi model B (1st gen) running raspbian stretch with the nginx web server and php7. I think it's better to compute the data on the client's side, since the device on which the browser will be run will almost sure be more powerfull than the raspberry's CPU, even if it means a little delay until the graph is displayed. Alternatively, I could use a mixed  approach: generate the time series dynamically with the raspberry pi and displaying the graph using dygraph. If I use the raspberry to generate the data files and the graphs (using gnuplot), the CPU is in use almost all the time (since there are quite a lot of sensors and data files that i'm processing...) and I can't use for anything else.

Anyway, you can close the thread if you want. I've more or less managed to do what I intented and I only need to optimize it now. I can share the code if you want, but right now it's a mess, since it's the first time i used javascript and I'm no programmer at all, just a hobbyist.

Thank you for your help, and I hope it can be useful to somebody in the future

Best regards,

Marc

El dissabte, 23 desembre de 2017 17:00:23 UTC+1, István Bánhidi va escriure:
Reply all
Reply to author
Forward
0 new messages