information about measurement results on bench

[jl.gil...@gmail.com]

Dear Joel,

I manage to install NT2 with Boost_1.57. Benches, tests, examples seems to be executed in the right way.
My question is about the results obtained: what is the signification (units used) of the measurement on the different tests and bench ?

For instance, I've executed the following bench:   "core.signal.simd.static_fft.bench"

Here is below some resulting measurements on an Ubuntu 14.04 /  Intel(R) dual-Core(TM) i5-2520M CPU @ 2.50GHz platform:

--------------------------------------------------------------------------------
Benchmark    Size    Result    Unit    Samples #
--------------------------------------------------------------------------------
fft_forward    (32)    3.25899    cpe (med)    4827323
fft_forward    (32)    1.62357e-23    s. (med)    4827323
fft_forward    (64)    3.48903    cpe (med)    2894104
fft_forward    (64)    6.74605e-18    s. (med)    2894104
[...]
fft_forward    (16384)    11.3755    cpe (med)    4001
fft_forward    (16384)    0.000149858    s. (med)    4001
--------------------------------------------------------------------------------


My question is:  What is the meaning of symbol "s." (seconds ?) and "cpe" ??
And, more generally, are there are any doc/infos giving the required information about the way measurements are done  ?
Thanks for your response.

JL

[Joel FALCOU]

On 16/10/2015 18:13, jl.gil...@gmail.com wrote:
> Dear Joel,
>
> I manage to install NT2 with Boost_1.57. Benches, tests, examples seems
> to be executed in the right way.
> My question is about the results obtained: what is the signification
> (units used) of the measurement on the different tests and bench ?
>
> For instance, I've executed the following bench:
> "core.signal.simd.static_fft.bench"
>
> Here is below some resulting measurements on an Ubuntu 14.04 / Intel(R)
> dual-Core(TM) i5-2520M CPU @ 2.50GHz platform:
>
> --------------------------------------------------------------------------------
> Benchmark Size Result Unit Samples #
> --------------------------------------------------------------------------------
> fft_forward (32) 3.25899 cpe (med) 4827323
> fft_forward (32) 1.62357e-23 s. (med) 4827323
> fft_forward (64) 3.48903 cpe (med) 2894104
> fft_forward (64) 6.74605e-18 s. (med) 2894104
> [...]
> fft_forward (16384) 11.3755 cpe (med) 4001
> fft_forward (16384) 0.000149858 s. (med) 4001
> --------------------------------------------------------------------------------
>
>
> My question is: What is the meaning of symbol "s." (seconds ?) and
> "cpe" ??

cpe is cycles per elements, ie ow many CPU cycles did it consume to
compute one value of the output. The less , the better

s is indeed second and Samples # is the number of sample run by the
benchmark.

> And, more generally, are there are any doc/infos giving the required
> information about the way measurements are done ?

I'm working on extracting the bench system to be standalone (and
documented properly). Basically, what we do is run a function to measure
for N seconds (N being 5 or 10s, anythgn significant in term of time).
Durign these N seconds, we perform #samples run fo the function and
compute the median, average or min or max of all the samples acquired
(as indicte by the (med) output).

Running for N seconds and accumualtign samples is better as we can
control the total bench time and assume a minimal viable set of samples.
Median is used as approximation of "most frequent time".