info
Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Canvas precision
26 views
Skip to first unread message
tombert
Jun 14, 2013, 10:13:18 AM6/14/13
to
Before digging deep into gnuplot internals, I would like to know if it's normal that I get a resolution of only 3 digits after the comma when using canvas 2D plots?
I have this data file:
25785.0000998 -68.8343006527
25785.0001028012 -80.5616032527
25785.0001058025 -87.8913075527
25785.0001088037 -87.6995350527
25785.000111805 -87.0571553527
25785.0001148063 -86.7244984527
25785.0001178075 -84.5539128527
...
it has ~18000 entries. For those who are interested, the firs columns are frequencies (MHz), the seconds ones are gains (dB).
Now when zooming deep into the canvas plot the points are drawn only to precision of 0.001 - which means that several points are drawn on the same x-position.
This precision seems to change depending on the number of points I use.
I am using gnuplot 4.6.
Anybody an idea?
thx
I have this data file:
25785.0000998 -68.8343006527
25785.0001028012 -80.5616032527
25785.0001058025 -87.8913075527
25785.0001088037 -87.6995350527
25785.000111805 -87.0571553527
25785.0001148063 -86.7244984527
25785.0001178075 -84.5539128527
...
it has ~18000 entries. For those who are interested, the firs columns are frequencies (MHz), the seconds ones are gains (dB).
Now when zooming deep into the canvas plot the points are drawn only to precision of 0.001 - which means that several points are drawn on the same x-position.
This precision seems to change depending on the number of points I use.
I am using gnuplot 4.6.
Anybody an idea?
thx
Karl
Jun 14, 2013, 11:08:02 AM6/14/13
to
differ in the 9-10th position, i悲 say that is 32 bit precision.
- Karl
tombert
Jun 14, 2013, 3:38:18 PM6/14/13
to
I think I found the problem.
With the following data:
25785.0001 -68.997037
25785.00010027 -69.1994067
25785.00010054 -69.5961962
25785.00010081 -70.1889926
25785.00010108 -70.9806719
25785.00010135 -71.9761248
... till ...
25785.99586 -62.0013247
25785.99668 -61.3820468
25785.9975 -60.851285
25785.99832 -60.4307604
25785.99914 -60.136098
25785.99996 -59.9775515
... 6000 points
gnuplot creates 6000 path points:
M(941,2093);
L(941,2107);
L(941,2133);
L(941,2172);
L(941,2225);
L(941,2292);
... till ...
L(10459,1627);
L(10467,1586);
L(10475,1550);
L(10483,1522);
L(10491,1503);
L(10499,1492);
One sees that the first data points all got translated into 941. This seems to be dependent on the total number of points.
When I delete the last 5000 points, then gnuplot translates into:
M(940,627);
L(944,651);
L(947,697);
L(951,766);
L(955,858);
L(958,974);
...
So I would like to know how to increase the value range for these translations?
thx
With the following data:
25785.0001 -68.997037
25785.00010027 -69.1994067
25785.00010054 -69.5961962
25785.00010081 -70.1889926
25785.00010108 -70.9806719
25785.00010135 -71.9761248
... till ...
25785.99586 -62.0013247
25785.99668 -61.3820468
25785.9975 -60.851285
25785.99832 -60.4307604
25785.99914 -60.136098
25785.99996 -59.9775515
... 6000 points
gnuplot creates 6000 path points:
M(941,2093);
L(941,2107);
L(941,2133);
L(941,2172);
L(941,2225);
L(941,2292);
... till ...
L(10459,1627);
L(10467,1586);
L(10475,1550);
L(10483,1522);
L(10491,1503);
L(10499,1492);
One sees that the first data points all got translated into 941. This seems to be dependent on the total number of points.
When I delete the last 5000 points, then gnuplot translates into:
M(940,627);
L(944,651);
L(947,697);
L(951,766);
L(955,858);
L(958,974);
...
So I would like to know how to increase the value range for these translations?
thx
tombert
Jun 14, 2013, 3:48:04 PM6/14/13
to
And probably here is the solution:
from canvas.trm:
#define CANVAS_OVERSAMPLE 10.
#define CANVAS_XMAX (600 * CANVAS_OVERSAMPLE)
If I increase the sampling would this solve my problem? Would it be somehow nice if this value could be changed via gnuplot command?
thx
from canvas.trm:
#define CANVAS_OVERSAMPLE 10.
#define CANVAS_XMAX (600 * CANVAS_OVERSAMPLE)
If I increase the sampling would this solve my problem? Would it be somehow nice if this value could be changed via gnuplot command?
thx
tombert
Jun 15, 2013, 3:28:53 AM6/15/13
to
0 new messages