Continuous plotting of data in real time

[Anal Patel]

Hi,

I'm plotting data in real time, and would like to plot sets of data
over each other on same plot.
Currently, I'm plotting one set after another like:
gnuplot> plot "-"
input data ('e' ends) > 2 3
input data ('e' ends) > 4 5
input data ('e' ends) > 6 7
input data ('e' ends) > e

Each set of data that I plot clears the previous set and plots only
new set.
I would like to take this one step further, where I would like to see
the history of sets of data on same plot, i.e. all the sets of data
are plotted on same graph. I took an approach of buffering old sets of
data and then issuing plot command with old and new set of data to
achieve this. But quickly it came to a point where gnuplot was very
slow and could not handle writing so many sets of data. In my case
each set of data arrive in 20ms interval, and there could be thousands
of sets.

Is there a way that gnuplot can remember the previously plotted data
and then only thing I have to do is issue new set. The users do need
to see the points in real time, i.e. can't wait until all the sets are
collected and then issue plot command.

Thanks-
Andy Patel

[sfeam]

Anal Patel wrote:

So long as you can define the axis ranges in advance so that all plots
use the same axes, you should be able to use multiplot.
Assume that "datasource" is a program or script that will wait for and
then return the next set of data points when they are available.

set xrange [...]
set yrange [...]

set print "repeat_forever.gp"
print "replot"
print "reread"
unset print

set multiplot
plot "< datasource" with lines notitle
load "repeat_forever.gp"
# we'll never get here unless someone externally kills the loop

If the "datasource" script is not smart enough to wait before returning,
you could instead put a "pause" command at the start of the
repeat_forever loop.

> Thanks-
> Andy Patel

[Anal Patel]

> > Andy Patel- Hide quoted text -
>
> - Show quoted text -- Hide quoted text -
>
> - Show quoted text -

Hmmm.... that's an intresting approach. However, the mechanism
requires a separate script or a program to plot the data. I have the
data to plot in memory (c++ program), and this program is doing lot
many thing including spitting out data for other gnuplots. On a real
time, it may be too much to write the data to file and then other
script or program can read it and issue the plot command.

What I have currently is from c++ program, there is a pipe open for
gnuplot to stream out data as they arrive. Writing data to a file and
have another program pick up will slow things down on a real time
system.

Thanks,
Andy Patel

[sfeam]

Anal Patel wrote:

>

> Hmmm.... that's an intresting approach. However, the mechanism
> requires a separate script or a program to plot the data. I have the
> data to plot in memory (c++ program), and this program is doing lot
> many thing including spitting out data for other gnuplots. On a real
> time, it may be too much to write the data to file and then other
> script or program can read it and issue the plot command.
>
> What I have currently is from c++ program, there is a pipe open for
> gnuplot to stream out data as they arrive. Writing data to a file and
> have another program pick up will slow things down on a real time
> system.
>
> Thanks,
> Andy Patel

Is this pipe open as a command source or a data source?
If it's a command source, you can do the same thing as above by
issuing successive "plot" commands rather than "replot".
From the c++ side:

gp = popen("gnuplot","w");
fprintf(gp,"set multiplot\n");
while (1) do {
fprintf(gp,"plot '-' with lines\n");
... dump lots of data ...
fprintf(gp,"E\n");
}

[untested! but not that far from code I am using in production]

If it's only open as a data source the problem becomes a little
more involved. I think you could go back to the original approach
only using a named pipe rather than a real file for the input.
But synchronizing named pipes turns out to be a bit tricky, so I'd
recommending trying the other approach first.

Ethan

[muhammad....@gmail.com]

Hi Andy, I am currently facing the same problem. Did you find solution to the problem?

Thanks

[Karl]

Unlikely, although there are a few ways to speed things up a bit.

Firstly some terminals are faster with displaying that others, depending
on your platform. Just give it a try with qt, wxt, even a bitmap driver
with external viewer? If you have a slow terminal, that might be the
limiting factor.

On the long run, this won´t help, because gnuplot always reprocesses all
data on every "plot" or "replot", including parsing and converting each
line of text data into columns and binary numbers.
It might be an option for you to feed binary data to gnuplot. That could
speed things up to some extent. (i´ve never tried it myself, admittedly,
anyone with first-hand experience?)

Otherwise I'd recommend compressing each incoming dataset into a few
numbers (using "stats" or "fit", for example), print those to a logfile
("set print logfname append"), and then only plot the current dataset
and that logfile. Of course, plotting the logfile also takes
increasingly longer, but you could increase the step width with time

plot logfname every (int(n/100)+1) # n is number of datasets so far

lines that are not read in take nearly no time.

Or you choose a clever method to always plot only a set of ten or so of
the stored datasets, something like this:

n = 123 # number of datasets so far, in a while loop?
fname(n) = "name_".n.".dat"
imax=10 # plot imax datasets
num(i) = int(n-n*10**(i/(imax))/10) # or similar/better ;-)
plot for [i=1:10] fname(num(i)) title sprintf("set nr %.f",num(i)),\
"-" title "last set"


Karl