Problem using command line argument with grep in plot file

[Marcelo Jara]

Hello,

I hope someone can assist me with this. I am plotting a file based on conditional data. To accomplish this, I'm using grep when plotting the file. The path of the data file is dynamic so I'm passing it in the command line. I'm having trouble passing the entire file with path to grep. Here's an example:

I am calling gnuplot as follows:

gnuplot -e "result_path='/test/'" GNU.plt


GNU.plt has

plot '<grep -E "System/CPU/System/percent.*655291" '.result_path.'Metrics.txt'. using 2:4 every ::2 with lines


This doesn't work because result_path is not being evaluated.



If I change GNU.plt to the following, then it works:

plot result_path.'Metrics.txt' using 2:4 every ::2 with lines


So I know result_path works, but not sure how to put it in the middle of the "grep" command.


thanks,


Marcelo

[sfeam]

Marcelo Jara wrote:

> Hello,
>
> I hope someone can assist me with this. I am plotting a file based on
> conditional data. To accomplish this, I'm using grep when plotting the
> file. The path of the data file is dynamic so I'm passing it in the
> command line. I'm having trouble passing the entire file with path to
> grep. Here's an example:
>
> I am calling gnuplot as follows:
>
> gnuplot -e "result_path='/test/'" GNU.plt
>
>
> GNU.plt has
>
> plot '<grep -E "System/CPU/System/percent.*655291"
> '.result_path.'Metrics.txt'. using 2:4 every ::2 with lines
>
>
> This doesn't work because result_path is not being evaluated.

I think it is not working for other reasons.
- the grep command seems to have only 1 argument rather than 2
- there is an extra period immediately before "using"
- I am uncertain about whether grep is guaranteed to not
return extraneous whitespace, which would mess up your
string processing

Ethan

[Marcelo Jara]

Thanks Ethan. This was a mistake in posting to this group. Here's the command I'm running:

gnuplot -e "result_path='/tmp/'" GNU.plt


In the /tmp directory, I have a file called Metrics.txt with the data.


GNU.plt has the following (formatting removed):

plot '<grep -E "System/CPU/System/percent.*655291" '.result_path.'Metrics.txt' using 2:4 every ::2 with lines


The error I'm getting is

---------------------------
plot '<grep -E "System/CPU/System/percent.*655291" '.result_path.'Metrics.txt' using 2:4 every ::2 with lines
^
"GNU.plt", line 10: warning: Skipping data file with no valid points

plot '<grep -E "System/CPU/System/percent.*655291" '.result_path.'Metrics.txt' using 2:4 every ::2 with lines
^
"GNU.plt", line 10: x range is invalid
------------------------------

Notice that result_path is not being evaluated.

If I remove the grep command and just try to plot everything, then it's ok:

plot result_path.'Metrics.txt' using 2:4 every ::2 with lines

So I know I'm passing result_path correctly. I'm just having issue concatenating the result_path within the grep command.






To test this yourself. Here's the full plot file.
-----------
set xdata time
set format x "%H:%M:%S"
set timefmt x "%Y-%m-%d %H:%M:%S"
set datafile separator ","
set grid
set autoscale y
set term png size 800,400 enhanced font 'Verdana,10'

set output "test.png"
#plot '<grep -E "System/CPU/System/percent.*655291" '.result_path.'Metrics.txt' using 2:4 every ::2 with lines

plot result_path.'Metrics.txt' using 2:4 every ::2 with lines

----------------

Here is a sample of the data from Metrics.txt


---------------Begin time,End time,Metric,Average value,Agent,Agent ID
2013-03-21 18:32:00,2013-03-21 18:33:00,System/CPU/System/percent,0.09333333373069763,"",655292
2013-03-21 18:32:00,2013-03-21 18:33:00,System/CPU/System/percent,0.10182222061687046,"",1565949
2013-03-21 18:32:00,2013-03-21 18:33:00,System/CPU/System/percent,0.1011791339205725,"",655291
2013-03-21 18:33:00,2013-03-21 18:34:00,System/CPU/System/percent,0.8489555319150287,"",1565949
2013-03-21 18:33:00,2013-03-21 18:34:00,System/CPU/System/percent,0.08623333772023518,"",655292
2013-03-21 18:33:00,2013-03-21 18:34:00,System/CPU/System/percent,2.7617832819620767,"",655291
2013-03-21 18:32:00,2013-03-21 18:33:00,System/CPU/User/percent,0.13710000117619833,"",655292
2013-03-21 18:32:00,2013-03-21 18:33:00,System/CPU/User/percent,0.10526666707462735,"",1565949
2013-03-21 18:32:00,2013-03-21 18:33:00,System/CPU/User/percent,0.13043180434151763,"",655291
2013-03-21 18:33:00,2013-03-21 18:34:00,System/CPU/User/percent,0.09056666493415833,"",655292
2013-03-21 18:33:00,2013-03-21 18:34:00,System/CPU/User/percent,42.89093017578125,"",655291
2013-03-21 18:33:00,2013-03-21 18:34:00,System/CPU/User/percent,16.11038972934087,"",1565949
2013-03-21 18:32:00,2013-03-21 18:33:00,System/Load,0.6477777693006728,"",1565949
2013-03-21 18:32:00,2013-03-21 18:33:00,System/Load,0.5380407037929118,"",655291
2013-03-21 18:32:00,2013-03-21 18:33:00,System/Load,0.12000000476837158,"",655292
2013-03-21 18:33:00,2013-03-21 18:34:00,System/Load,1.6044443978203669,"",1565949
2013-03-21 18:33:00,2013-03-21 18:34:00,System/Load,0.29333333174387616,"",655292
2013-03-21 18:33:00,2013-03-21 18:34:00,System/Load,3.7316665649414062,"",655291
2013-03-21 18:32:00,2013-03-21 18:33:00,System/Memory/Used/bytes,711324558.2222222,"",1565949
2013-03-21 18:32:00,2013-03-21 18:33:00,System/Memory/Used/bytes,585834496.0,"",655292
2013-03-21 18:32:00,2013-03-21 18:33:00,System/Memory/Used/bytes,527178421.0890585,"",655291
2013-03-21 18:33:00,2013-03-21 18:34:00,System/Memory/Used/bytes,997482951.1111112,"",1565949
2013-03-21 18:33:00,2013-03-21 18:34:00,System/Memory/Used/bytes,585681578.6666666,"",655292
2013-03-21 18:33:00,2013-03-21 18:34:00,System/Memory/Used/bytes,1248720896.0,"",655291
----------------------------

[sfeam]

Marcelo Jara wrote:

> Thanks Ethan. This was a mistake in posting to this group. Here's the
> command I'm running:
>
> gnuplot -e "result_path='/tmp/'" GNU.plt
>
>
> In the /tmp directory, I have a file called Metrics.txt with the data.
>
>
> GNU.plt has the following (formatting removed):
>
> plot '<grep -E "System/CPU/System/percent.*655291"
> '.result_path.'Metrics.txt' using 2:4 every ::2 with lines
>
>
> The error I'm getting is
>
> ---------------------------
> plot '<grep -E "System/CPU/System/percent.*655291"
> '.result_path.'Metrics.txt' using 2:4 every ::2 with lines
> ^
> "GNU.plt", line 10: warning: Skipping data file with no valid points

Yes. I see that error message also.

But it plots with no error if I remove the "every ::2" spec.
Note that there are only two lines returned from grep in the example
you provided, but "every ::2" asks for the third point (which doesn't exist).

So evaluation of result_path is correct,
the construction of the grep command is correct,
but apparently the command does not return what you were expecting
or else your "every" clause is not constructed correctly.

Here's what I get after adding to your script
print "result_path = ", result_path
set term dumb
and removing the "every ::2" part


[123] gnuplot -e "result_path='./'" Metrics.plt
result_path = ./


3 ++----+----+-----+-----+-----+----+-----+-----+-----+----+-----+----++
+ + + + + + +
| + + + + + +----+-|
2.5 ++.............................................................----.++
| : : : : : ---- |
| : : : : ----- |
2 ++................................................----..............++
| : : : ---- : |
| : : : -----: : |
| : : : ---- : : |
1.5 ++...............................----...............................++
| : : ---- : : : |
| : :----- : : : |
1 ++..................----............................................++
| : ---- : : : : |
| ----- : : : : |
0.5 ++.....----.........................................................++
| ---- : : : : : |
+-- + + + + + +
0 ++----+----+-----+-----+-----+----+-----+-----+-----+----+-----+----++
18:33:00 18:33:10 18:33:20 18:33:30 18:33:40 18:33:50 18:34:00



Ethan

>
> plot '<grep -E "System/CPU/System/percent.*655291"
> '.result_path.'Metrics.txt' using 2:4 every ::2 with lines
> ^
> "GNU.plt", line 10: x range is invalid
> ------------------------------
>
> Notice that result_path is not being evaluated.
>
> If I remove the grep command and just try to plot everything, then it's
> ok:
>
> plot result_path.'Metrics.txt' using 2:4 every ::2 with lines
>
>
> So I know I'm passing result_path correctly. I'm just having issue
> concatenating the result_path within the grep command.
>
>
>
>
>
>
> To test this yourself. Here's the full plot file.
> -----------
> set xdata time
> set format x "%H:%M:%S"
> set timefmt x "%Y-%m-%d %H:%M:%S"
> set datafile separator ","
> set grid
> set autoscale y
> set term png size 800,400 enhanced font 'Verdana,10'
>
> set output "test.png"
> #plot '<grep -E "System/CPU/System/percent.*655291"

> #'.result_path.'Metrics.txt' using 2:4 every ::2 with lines

[Marcelo Jara]

Thanks! Yes, in my rush to get the script working, I didn't get enough samples. I thought ::2 meant to only skip the first line (header)?

What was confusing me was that result_path wasn't being evaluated in the error message so I assumed that meant it was not set correctly. If result_path was evaluated correctly in the error message, then I would have started looking somewhere else.