unhalted vs. RTDSC time

Nichols A. Romero

unread,

Jun 6, 2023, 2:32:45 PM6/6/23

to likwid...@googlegroups.com

Hi,

I know this question gets asked many times, but I could find the answer on the wiki. The close thing I could find was this old GitHub issue:

https://github.com/RRZE-HPC/likwid/issues/443

What is the definition of RDTSC vs. unhalted times?

I am pretty sure that RDTSC is the "real time".

--

Nichols A. Romero, Ph.D.

Nichols A. Romero

unread,

Jun 7, 2023, 1:54:19 AM6/7/23

to likwid...@googlegroups.com

A couple of follow-up questions:

- Which "time" is used in the memory bandwidth and flop rate formulas?

- Another observation regarding timings, if I do "-g MEM" vs. "-g FLOPS_DP" the runtimes as reported by LIKWID can differ by as much as 30%. Is this "noise" (OS, other users, etc.) or hardware counter overhead?

moebiusband

unread,

Jun 7, 2023, 3:15:37 AM6/7/23

to likwid-users

Hi,

The time using the RDTSC count is the walltime (or real time as you called it). This walltime is used in all rate metrics.

Unhalted time is a time using the real cycle counter but using the nominal clock frequency to calculate time. It is debatable if this is useful. It tells you if either the core was in halted state for some duration during measurement or if the actual frequency was different from the base clock frequency (which on modern processors is probably always the case).

You can check all groups yourself (here for ICX):

https://github.com/RRZE-HPC/likwid/blob/master/groups/ICX/FLOPS_DP.txt

With regards to runtime deviations. There are many influences and I would need more information on your actual case. But in general with current kernels context switches to the kernel are very expensive. The more counters a group requires the more costly it is. Therefore it is expected that the MEM group has more overhead than the Flops groups. If you are using the marker api this does not influence the measurement itself, but of course the overall runtime is affected. In general the advice is to only measure stuff that has some minimum duration (at least in the seconds range) and to not use the regions excessively.

There are ways to reduce the overhead, but with regard to this I have to ask Tom (the main Likwid developer).

Best Regards,

Jan

Nichols A. Romero

unread,

Jun 8, 2023, 1:35:07 AM6/8/23

to likwid...@googlegroups.com

Hi Jan,

Here is an example where I see the runtimes deviate a lot. I am using the Marker API.

For the FLOP_COUNT group, "main" (which encompases the entire application) measures 264 seconds (average).

For the MEM group, "main" measures 200 seconds (average).

I apologize that I couldn't just upload the data as an attachment and had to copy paste it instead. So, its quite lengthy.

===== FLOP_COUNT =====

Region: main
Group: 1
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Event | Counter | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Region calls | CTR | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| INSTR_RETIRED_ANY | FIXC0 | 216122800000 | 237395200000 | 250409700000 | 227082200000 | 245403000000 | 251279300000 | 257923000000 | 206187300000 |
| CPU_CLK_UNHALTED_CORE | FIXC1 | 450437600000 | 429546100000 | 441051500000 | 411170700000 | 435504900000 | 382634400000 | 431907200000 | 433996600000 |
| CPU_CLK_UNHALTED_REF | FIXC2 | 392926200000 | 385816600000 | 390442600000 | 384654800000 | 390231600000 | 364838700000 | 381210700000 | 389652200000 |
| FP_ARITH_INST_RETIRED_128B_PACKED_DOUBLE | PMC0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_SCALAR_DOUBLE | PMC1 | 6097907000 | 6227147000 | 5909099000 | 6236822000 | 6142802000 | 5913296000 | 6073966000 | 5969415000 |
| FP_ARITH_INST_RETIRED_256B_PACKED_DOUBLE | PMC2 | 9721618000 | 9968190000 | 9998267000 | 10201970000 | 9617871000 | 9819643000 | 10008740000 | 10050330000 |
| FP_ARITH_INST_RETIRED_512B_PACKED_DOUBLE | PMC3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+-----------------------------------------------+---------+---------------+--------------+--------------+--------------+
| Event | Counter | Sum | Min | Max | Avg |
+-----------------------------------------------+---------+---------------+--------------+--------------+--------------+
| Region calls STAT | CTR | 8 | 1 | 1 | 1 |
| INSTR_RETIRED_ANY STAT | FIXC0 | 1891802500000 | 206187300000 | 257923000000 | 236475312500 |
| CPU_CLK_UNHALTED_CORE STAT | FIXC1 | 3416249000000 | 382634400000 | 450437600000 | 427031125000 |
| CPU_CLK_UNHALTED_REF STAT | FIXC2 | 3079773400000 | 364838700000 | 392926200000 | 384971675000 |
| FP_ARITH_INST_RETIRED_128B_PACKED_DOUBLE STAT | PMC0 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_SCALAR_DOUBLE STAT | PMC1 | 48570454000 | 5909099000 | 6236822000 | 6071306750 |
| FP_ARITH_INST_RETIRED_256B_PACKED_DOUBLE STAT | PMC2 | 79386629000 | 9617871000 | 10201970000 | 9923328625 |
| FP_ARITH_INST_RETIRED_512B_PACKED_DOUBLE STAT | PMC3 | 0 | 0 | 0 | 0 |
+-----------------------------------------------+---------+---------------+--------------+--------------+--------------+

+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Metric | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Runtime (RDTSC) [s] | 200.2063 | 200.2553 | 200.2221 | 200.2688 | 200.2048 | 200.2609 | 200.2043 | 200.1922 |
| Runtime unhalted [s] | 180.1756 | 171.8188 | 176.4237 | 164.4687 | 174.2022 | 153.0542 | 172.7633 | 173.5991 |
| Clock [MHz] | 2865.9088 | 2783.3503 | 2823.9988 | 2672.3285 | 2790.0370 | 2621.9355 | 2832.4636 | 2784.5059 |
| CPI | 2.0842 | 1.8094 | 1.7613 | 1.8107 | 1.7747 | 1.5227 | 1.6746 | 2.1049 |
| DP [MFLOP/s] | 224.6901 | 230.2057 | 229.2562 | 234.9078 | 222.8432 | 225.6650 | 230.3094 | 230.6320 |
| AVX DP [MFLOP/s] | 194.2320 | 199.1096 | 199.7435 | 203.7655 | 192.1606 | 196.1370 | 199.9705 | 200.8136 |
| AVX512 DP [MFLOP/s] | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Packed [MUOPS/s] | 48.5580 | 49.7774 | 49.9359 | 50.9414 | 48.0402 | 49.0342 | 49.9926 | 50.2034 |
| Scalar [MUOPS/s] | 30.4581 | 31.0960 | 29.5127 | 31.1423 | 30.6826 | 29.5280 | 30.3388 | 29.8184 |
| Vectorization ratio | 61.4533 | 61.5498 | 62.8531 | 62.0603 | 61.0245 | 62.4145 | 62.2329 | 62.7371 |
+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+
| Metric | Sum | Min | Max | Avg | %ile 25 | %ile 50 | %ile 75 |
+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+
| Runtime (RDTSC) [s] STAT | 1601.8147 | 200.1922 | 200.2688 | 200.2268 | 200.2043 | 200.2063 | 200.2553 |
| Runtime unhalted [s] STAT | 1366.5056 | 153.0542 | 180.1756 | 170.8132 | 164.4687 | 172.7633 | 174.2022 |
| Clock [MHz] STAT | 22174.5284 | 2621.9355 | 2865.9088 | 2771.8160 | 2672.3285 | 2784.5059 | 2823.9988 |
| CPI STAT | 14.5425 | 1.5227 | 2.1049 | 1.8178 | 1.6746 | 1.7747 | 1.8107 |
| DP [MFLOP/s] STAT | 1828.5094 | 222.8432 | 234.9078 | 228.5637 | 224.6901 | 229.2562 | 230.3094 |
| AVX DP [MFLOP/s] STAT | 1585.9323 | 192.1606 | 203.7655 | 198.2415 | 194.2320 | 199.1096 | 199.9705 |
| AVX512 DP [MFLOP/s] STAT | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Packed [MUOPS/s] STAT | 396.4831 | 48.0402 | 50.9414 | 49.5604 | 48.5580 | 49.7774 | 49.9926 |
| Scalar [MUOPS/s] STAT | 242.5769 | 29.5127 | 31.1423 | 30.3221 | 29.5280 | 30.3388 | 30.6826 |
| Vectorization ratio STAT | 496.3255 | 61.0245 | 62.8531 | 62.0407 | 61.4533 | 62.0603 | 62.4145 |
+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+

Region: allocate
Group: 1
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Event | Counter | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Region calls | CTR | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| INSTR_RETIRED_ANY | FIXC0 | 8589502000 | 8587624000 | 8591225000 | 8582688000 | 8587360000 | 8589137000 | 8592157000 | 8591681000 |
| CPU_CLK_UNHALTED_CORE | FIXC1 | 3396911000 | 3399919000 | 3386465000 | 3391948000 | 3382283000 | 3394513000 | 3386290000 | 3391718000 |
| CPU_CLK_UNHALTED_REF | FIXC2 | 3449694000 | 3449184000 | 3456464000 | 3445711000 | 3448208000 | 3465283000 | 3432963000 | 3439204000 |
| FP_ARITH_INST_RETIRED_128B_PACKED_DOUBLE | PMC0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_SCALAR_DOUBLE | PMC1 | 34295190 | 34263290 | 34299810 | 34248720 | 34280270 | 34285540 | 34283430 | 34234590 |
| FP_ARITH_INST_RETIRED_256B_PACKED_DOUBLE | PMC2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_512B_PACKED_DOUBLE | PMC3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+-----------------------------------------------+---------+-------------+------------+------------+------------+
| Event | Counter | Sum | Min | Max | Avg |
+-----------------------------------------------+---------+-------------+------------+------------+------------+
| Region calls STAT | CTR | 8 | 1 | 1 | 1 |
| INSTR_RETIRED_ANY STAT | FIXC0 | 68711374000 | 8582688000 | 8592157000 | 8588921750 |
| CPU_CLK_UNHALTED_CORE STAT | FIXC1 | 27130047000 | 3382283000 | 3399919000 | 3391255875 |
| CPU_CLK_UNHALTED_REF STAT | FIXC2 | 27586711000 | 3432963000 | 3465283000 | 3448338875 |
| FP_ARITH_INST_RETIRED_128B_PACKED_DOUBLE STAT | PMC0 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_SCALAR_DOUBLE STAT | PMC1 | 274190840 | 34234590 | 34299810 | 34273855 |
| FP_ARITH_INST_RETIRED_256B_PACKED_DOUBLE STAT | PMC2 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_512B_PACKED_DOUBLE STAT | PMC3 | 0 | 0 | 0 | 0 |
+-----------------------------------------------+---------+-------------+------------+------------+------------+

+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Metric | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Runtime (RDTSC) [s] | 3.2111 | 3.2099 | 3.2133 | 3.2074 | 3.2123 | 3.2144 | 3.1936 | 3.1996 |
| Runtime unhalted [s] | 1.3588 | 1.3600 | 1.3546 | 1.3568 | 1.3529 | 1.3578 | 1.3545 | 1.3567 |
| Clock [MHz] | 2461.7408 | 2464.2866 | 2449.3280 | 2460.9862 | 2452.1997 | 2448.9373 | 2466.0053 | 2465.4758 |
| CPI | 0.3955 | 0.3959 | 0.3942 | 0.3952 | 0.3939 | 0.3952 | 0.3941 | 0.3948 |
| DP [MFLOP/s] | 10.6801 | 10.6743 | 10.6742 | 10.6781 | 10.6717 | 10.6663 | 10.7349 | 10.6995 |
| AVX DP [MFLOP/s] | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| AVX512 DP [MFLOP/s] | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Packed [MUOPS/s] | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Scalar [MUOPS/s] | 10.6801 | 10.6743 | 10.6742 | 10.6781 | 10.6717 | 10.6663 | 10.7349 | 10.6995 |
| Vectorization ratio | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+
| Metric | Sum | Min | Max | Avg | %ile 25 | %ile 50 | %ile 75 |
+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+
| Runtime (RDTSC) [s] STAT | 25.6616 | 3.1936 | 3.2144 | 3.2077 | 3.1996 | 3.2099 | 3.2123 |
| Runtime unhalted [s] STAT | 10.8521 | 1.3529 | 1.3600 | 1.3565 | 1.3545 | 1.3567 | 1.3578 |
| Clock [MHz] STAT | 19668.9597 | 2448.9373 | 2466.0053 | 2458.6200 | 2449.3280 | 2460.9862 | 2464.2866 |
| CPI STAT | 3.1588 | 0.3939 | 0.3959 | 0.3948 | 0.3941 | 0.3948 | 0.3952 |
| DP [MFLOP/s] STAT | 85.4791 | 10.6663 | 10.7349 | 10.6849 | 10.6717 | 10.6743 | 10.6801 |
| AVX DP [MFLOP/s] STAT | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| AVX512 DP [MFLOP/s] STAT | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Packed [MUOPS/s] STAT | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Scalar [MUOPS/s] STAT | 85.4791 | 10.6663 | 10.7349 | 10.6849 | 10.6717 | 10.6743 | 10.6801 |
| Vectorization ratio STAT | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+

Region: driver
Group: 1
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Event | Counter | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Region calls | CTR | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| INSTR_RETIRED_ANY | FIXC0 | 201025000000 | 222319600000 | 235412900000 | 211961700000 | 230348000000 | 236157500000 | 242875100000 | 190549700000 |
| CPU_CLK_UNHALTED_CORE | FIXC1 | 439905500000 | 420154000000 | 431734400000 | 400533000000 | 424957600000 | 373202000000 | 421397700000 | 424524000000 |
| CPU_CLK_UNHALTED_REF | FIXC2 | 383495000000 | 376350800000 | 381027700000 | 376138000000 | 381779600000 | 355317700000 | 372800400000 | 380108100000 |
| FP_ARITH_INST_RETIRED_128B_PACKED_DOUBLE | PMC0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_SCALAR_DOUBLE | PMC1 | 6063608000 | 6192881000 | 5874798000 | 6202572000 | 6108521000 | 5879010000 | 6039682000 | 5935180000 |
| FP_ARITH_INST_RETIRED_256B_PACKED_DOUBLE | PMC2 | 9721618000 | 9968190000 | 9998267000 | 10201970000 | 9617871000 | 9819642000 | 10008740000 | 10050330000 |
| FP_ARITH_INST_RETIRED_512B_PACKED_DOUBLE | PMC3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+-----------------------------------------------+---------+---------------+--------------+--------------+--------------+
| Event | Counter | Sum | Min | Max | Avg |
+-----------------------------------------------+---------+---------------+--------------+--------------+--------------+
| Region calls STAT | CTR | 8 | 1 | 1 | 1 |
| INSTR_RETIRED_ANY STAT | FIXC0 | 1770649500000 | 190549700000 | 242875100000 | 221331187500 |
| CPU_CLK_UNHALTED_CORE STAT | FIXC1 | 3336408200000 | 373202000000 | 439905500000 | 417051025000 |
| CPU_CLK_UNHALTED_REF STAT | FIXC2 | 3007017300000 | 355317700000 | 383495000000 | 375877162500 |
| FP_ARITH_INST_RETIRED_128B_PACKED_DOUBLE STAT | PMC0 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_SCALAR_DOUBLE STAT | PMC1 | 48296252000 | 5874798000 | 6202572000 | 6037031500 |
| FP_ARITH_INST_RETIRED_256B_PACKED_DOUBLE STAT | PMC2 | 79386628000 | 9617871000 | 10201970000 | 9923328500 |
| FP_ARITH_INST_RETIRED_512B_PACKED_DOUBLE STAT | PMC3 | 0 | 0 | 0 | 0 |
+-----------------------------------------------+---------+---------------+--------------+--------------+--------------+

+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Metric | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Runtime (RDTSC) [s] | 193.6845 | 193.6857 | 193.6823 | 193.6882 | 193.6834 | 193.6812 | 193.7019 | 193.6961 |
| Runtime unhalted [s] | 175.9627 | 168.0620 | 172.6968 | 160.2136 | 169.9833 | 149.2812 | 168.5595 | 169.8100 |
| Clock [MHz] | 2867.7310 | 2790.9668 | 2832.6473 | 2662.1341 | 2782.7374 | 2625.8264 | 2825.8869 | 2792.1200 |
| CPI | 2.1883 | 1.8899 | 1.8339 | 1.8896 | 1.8449 | 1.5803 | 1.7350 | 2.2279 |
| DP [MFLOP/s] | 232.0789 | 237.8371 | 236.8201 | 242.7120 | 230.1695 | 233.1542 | 237.8637 | 238.1901 |
| AVX DP [MFLOP/s] | 200.7722 | 205.8632 | 206.4880 | 210.6885 | 198.6308 | 202.8001 | 206.6834 | 207.5484 |
| AVX512 DP [MFLOP/s] | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Packed [MUOPS/s] | 50.1931 | 51.4658 | 51.6220 | 52.6721 | 49.6577 | 50.7000 | 51.6708 | 51.8871 |
| Scalar [MUOPS/s] | 31.3066 | 31.9739 | 30.3321 | 32.0235 | 31.5387 | 30.3541 | 31.1803 | 30.6417 |
| Vectorization ratio | 61.5868 | 61.6803 | 62.9889 | 62.1899 | 61.1575 | 62.5509 | 62.3659 | 62.8715 |
+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+
| Metric | Sum | Min | Max | Avg | %ile 25 | %ile 50 | %ile 75 |
+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+
| Runtime (RDTSC) [s] STAT | 1549.5033 | 193.6812 | 193.7019 | 193.6879 | 193.6823 | 193.6845 | 193.6882 |
| Runtime unhalted [s] STAT | 1334.5691 | 149.2812 | 175.9627 | 166.8211 | 160.2136 | 168.5595 | 169.9833 |
| Clock [MHz] STAT | 22180.0499 | 2625.8264 | 2867.7310 | 2772.5062 | 2662.1341 | 2790.9668 | 2825.8869 |
| CPI STAT | 15.1898 | 1.5803 | 2.2279 | 1.8987 | 1.7350 | 1.8449 | 1.8899 |
| DP [MFLOP/s] STAT | 1888.8256 | 230.1695 | 242.7120 | 236.1032 | 232.0789 | 236.8201 | 237.8637 |
| AVX DP [MFLOP/s] STAT | 1639.4746 | 198.6308 | 210.6885 | 204.9343 | 200.7722 | 205.8632 | 206.6834 |
| AVX512 DP [MFLOP/s] STAT | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Packed [MUOPS/s] STAT | 409.8686 | 49.6577 | 52.6721 | 51.2336 | 50.1931 | 51.4658 | 51.6708 |
| Scalar [MUOPS/s] STAT | 249.3509 | 30.3321 | 32.0235 | 31.1689 | 30.3541 | 31.1803 | 31.5387 |
| Vectorization ratio STAT | 497.3917 | 61.1575 | 62.9889 | 62.1740 | 61.5868 | 62.1899 | 62.5509 |
+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+

Region: check_sum
Group: 1
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Event | Counter | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Region calls | CTR | 80040 | 80040 | 80040 | 80040 | 80040 | 80040 | 80040 | 80040 |
| INSTR_RETIRED_ANY | FIXC0 | 72911620000 | 78213930000 | 84310410000 | 79251790000 | 84980130000 | 83333770000 | 81530540000 | 81307940000 |
| CPU_CLK_UNHALTED_CORE | FIXC1 | 74693610000 | 71836980000 | 74032850000 | 68728800000 | 72549520000 | 63478730000 | 71770180000 | 72458500000 |
| CPU_CLK_UNHALTED_REF | FIXC2 | 65960080000 | 65106900000 | 66113460000 | 65462800000 | 66096620000 | 61496080000 | 64285650000 | 65731350000 |
| FP_ARITH_INST_RETIRED_128B_PACKED_DOUBLE | PMC0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_SCALAR_DOUBLE | PMC1 | 5447646000 | 5631802000 | 5318135000 | 5599531000 | 5529465000 | 5339540000 | 5488072000 | 5305763000 |
| FP_ARITH_INST_RETIRED_256B_PACKED_DOUBLE | PMC2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_512B_PACKED_DOUBLE | PMC3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+-----------------------------------------------+---------+--------------+-------------+-------------+-------------+
| Event | Counter | Sum | Min | Max | Avg |
+-----------------------------------------------+---------+--------------+-------------+-------------+-------------+
| Region calls STAT | CTR | 640320 | 80040 | 80040 | 80040 |
| INSTR_RETIRED_ANY STAT | FIXC0 | 645840130000 | 72911620000 | 84980130000 | 80730016250 |
| CPU_CLK_UNHALTED_CORE STAT | FIXC1 | 569549170000 | 63478730000 | 74693610000 | 71193646250 |
| CPU_CLK_UNHALTED_REF STAT | FIXC2 | 520252940000 | 61496080000 | 66113460000 | 65031617500 |
| FP_ARITH_INST_RETIRED_128B_PACKED_DOUBLE STAT | PMC0 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_SCALAR_DOUBLE STAT | PMC1 | 43659954000 | 5305763000 | 5631802000 | 5457494250 |
| FP_ARITH_INST_RETIRED_256B_PACKED_DOUBLE STAT | PMC2 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_512B_PACKED_DOUBLE STAT | PMC3 | 0 | 0 | 0 | 0 |
+-----------------------------------------------+---------+--------------+-------------+-------------+-------------+

+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Metric | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Runtime (RDTSC) [s] | 31.2874 | 31.4540 | 31.4976 | 31.6679 | 31.3865 | 31.5122 | 31.2856 | 31.3705 |
| Runtime unhalted [s] | 29.8775 | 28.7349 | 29.6137 | 27.4916 | 29.0199 | 25.3916 | 28.7081 | 28.9835 |
| Clock [MHz] | 2831.0074 | 2758.4178 | 2799.4131 | 2624.7203 | 2744.0666 | 2580.5941 | 2791.0587 | 2755.8510 |
| CPI | 1.0244 | 0.9185 | 0.8781 | 0.8672 | 0.8537 | 0.7617 | 0.8803 | 0.8912 |
| DP [MFLOP/s] | 174.1160 | 179.0485 | 168.8428 | 176.8203 | 176.1731 | 169.4435 | 175.4187 | 169.1321 |
| AVX DP [MFLOP/s] | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| AVX512 DP [MFLOP/s] | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Packed [MUOPS/s] | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Scalar [MUOPS/s] | 174.1160 | 179.0485 | 168.8428 | 176.8203 | 176.1731 | 169.4435 | 175.4187 | 169.1321 |
| Vectorization ratio | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+
| Metric | Sum | Min | Max | Avg | %ile 25 | %ile 50 | %ile 75 |
+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+
| Runtime (RDTSC) [s] STAT | 251.4617 | 31.2856 | 31.6679 | 31.4327 | 31.2874 | 31.3865 | 31.4976 |
| Runtime unhalted [s] STAT | 227.8208 | 25.3916 | 29.8775 | 28.4776 | 27.4916 | 28.7349 | 29.0199 |
| Clock [MHz] STAT | 21885.1290 | 2580.5941 | 2831.0074 | 2735.6411 | 2624.7203 | 2755.8510 | 2791.0587 |
| CPI STAT | 7.0751 | 0.7617 | 1.0244 | 0.8844 | 0.8537 | 0.8781 | 0.8912 |
| DP [MFLOP/s] STAT | 1388.9950 | 168.8428 | 179.0485 | 173.6244 | 169.1321 | 174.1160 | 176.1731 |
| AVX DP [MFLOP/s] STAT | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| AVX512 DP [MFLOP/s] STAT | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Packed [MUOPS/s] STAT | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Scalar [MUOPS/s] STAT | 1388.9950 | 168.8428 | 179.0485 | 173.6244 | 169.1321 | 174.1160 | 176.1731 |
| Vectorization ratio STAT | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+

Region: refine
Group: 1
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Event | Counter | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Region calls | CTR | 21 | 21 | 21 | 21 | 21 | 21 | 21 | 21 |
| INSTR_RETIRED_ANY | FIXC0 | 32735640000 | 39811230000 | 41225390000 | 39262690000 | 40858600000 | 45121720000 | 43885800000 | 15436230000 |
| CPU_CLK_UNHALTED_CORE | FIXC1 | 16441320000 | 16054790000 | 15973080000 | 14955950000 | 15844360000 | 14088680000 | 15778390000 | 14120350000 |
| CPU_CLK_UNHALTED_REF | FIXC2 | 13697680000 | 13540930000 | 13443570000 | 13146200000 | 13622380000 | 12988170000 | 12969920000 | 12643910000 |
| FP_ARITH_INST_RETIRED_128B_PACKED_DOUBLE | PMC0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_SCALAR_DOUBLE | PMC1 | 154514 | 116575 | 122027 | 259625 | 119173 | 229999 | 201756 | 796399 |
| FP_ARITH_INST_RETIRED_256B_PACKED_DOUBLE | PMC2 | 2421728 | 2778620 | 865845 | 17365030 | 897493 | 14251740 | 12884540 | 72685680 |
| FP_ARITH_INST_RETIRED_512B_PACKED_DOUBLE | PMC3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+-----------------------------------------------+---------+--------------+-------------+-------------+--------------+
| Event | Counter | Sum | Min | Max | Avg |
+-----------------------------------------------+---------+--------------+-------------+-------------+--------------+
| Region calls STAT | CTR | 168 | 21 | 21 | 21 |
| INSTR_RETIRED_ANY STAT | FIXC0 | 298337300000 | 15436230000 | 45121720000 | 37292162500 |
| CPU_CLK_UNHALTED_CORE STAT | FIXC1 | 123256920000 | 14088680000 | 16441320000 | 15407115000 |
| CPU_CLK_UNHALTED_REF STAT | FIXC2 | 106052760000 | 12643910000 | 13697680000 | 13256595000 |
| FP_ARITH_INST_RETIRED_128B_PACKED_DOUBLE STAT | PMC0 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_SCALAR_DOUBLE STAT | PMC1 | 2000068 | 116575 | 796399 | 250008.5000 |
| FP_ARITH_INST_RETIRED_256B_PACKED_DOUBLE STAT | PMC2 | 124150676 | 865845 | 72685680 | 1.551883e+07 |
| FP_ARITH_INST_RETIRED_512B_PACKED_DOUBLE STAT | PMC3 | 0 | 0 | 0 | 0 |
+-----------------------------------------------+---------+--------------+-------------+-------------+--------------+

+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Metric | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Runtime (RDTSC) [s] | 6.4831 | 6.4831 | 6.4831 | 6.4830 | 6.4831 | 6.4830 | 6.4831 | 6.4831 |
| Runtime unhalted [s] | 6.5765 | 6.4219 | 6.3893 | 5.9824 | 6.3378 | 5.6355 | 6.3114 | 5.6482 |
| Clock [MHz] | 3000.7402 | 2964.1156 | 2970.3420 | 2844.1507 | 2907.7767 | 2711.8224 | 3041.3357 | 2791.9203 |
| CPI | 0.5022 | 0.4033 | 0.3875 | 0.3809 | 0.3878 | 0.3122 | 0.3595 | 0.9148 |
| DP [MFLOP/s] | 1.5180 | 1.7324 | 0.5530 | 10.7542 | 0.5721 | 8.8287 | 7.9808 | 44.9693 |
| AVX DP [MFLOP/s] | 1.4942 | 1.7144 | 0.5342 | 10.7141 | 0.5537 | 8.7933 | 7.9497 | 44.8465 |
| AVX512 DP [MFLOP/s] | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Packed [MUOPS/s] | 0.3735 | 0.4286 | 0.1336 | 2.6785 | 0.1384 | 2.1983 | 1.9874 | 11.2116 |
| Scalar [MUOPS/s] | 0.0238 | 0.0180 | 0.0188 | 0.0400 | 0.0184 | 0.0355 | 0.0311 | 0.1228 |
| Vectorization ratio | 94.0023 | 95.9735 | 87.6475 | 98.5269 | 88.2781 | 98.4118 | 98.4583 | 98.9162 |
+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+
| Metric | Sum | Min | Max | Avg | %ile 25 | %ile 50 | %ile 75 |
+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+
| Runtime (RDTSC) [s] STAT | 51.8646 | 6.4830 | 6.4831 | 6.4831 | 6.4830 | 6.4831 | 6.4831 |
| Runtime unhalted [s] STAT | 49.3030 | 5.6355 | 6.5765 | 6.1629 | 5.6482 | 6.3114 | 6.3893 |
| Clock [MHz] STAT | 23232.2036 | 2711.8224 | 3041.3357 | 2904.0255 | 2791.9203 | 2907.7767 | 2970.3420 |
| CPI STAT | 3.6482 | 0.3122 | 0.9148 | 0.4560 | 0.3595 | 0.3875 | 0.4033 |
| DP [MFLOP/s] STAT | 76.9085 | 0.5530 | 44.9693 | 9.6136 | 0.5721 | 1.7324 | 44.9693 |
| AVX DP [MFLOP/s] STAT | 76.6001 | 0.5342 | 44.8465 | 9.5750 | 0.5537 | 1.7144 | 44.8465 |
| AVX512 DP [MFLOP/s] STAT | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Packed [MUOPS/s] STAT | 19.1499 | 0.1336 | 11.2116 | 2.3937 | 0.1384 | 0.4286 | 11.2116 |
| Scalar [MUOPS/s] STAT | 0.3084 | 0.0180 | 0.1228 | 0.0386 | 0.0184 | 0.0238 | 0.0355 |
| Vectorization ratio STAT | 760.2146 | 87.6475 | 98.9162 | 95.0268 | 88.2781 | 95.9735 | 98.4583 |
+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+

Region: comm
Group: 1
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Event | Counter | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Region calls | CTR | 2000 | 2000 | 2000 | 2000 | 2000 | 2000 | 2000 | 2000 |
| INSTR_RETIRED_ANY | FIXC0 | 34926810000 | 44562060000 | 48051160000 | 35729770000 | 42262630000 | 48086090000 | 57012110000 | 32507280000 |
| CPU_CLK_UNHALTED_CORE | FIXC1 | 255584300000 | 244003500000 | 250486700000 | 231963000000 | 245703700000 | 217289500000 | 245090300000 | 248335400000 |
| CPU_CLK_UNHALTED_REF | FIXC2 | 214488800000 | 210949100000 | 213143600000 | 209857600000 | 212798600000 | 199088200000 | 209190200000 | 213585200000 |
| FP_ARITH_INST_RETIRED_128B_PACKED_DOUBLE | PMC0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_SCALAR_DOUBLE | PMC1 | 611752100 | 556986900 | 552774500 | 599016900 | 574984700 | 535348300 | 547629900 | 624682100 |
| FP_ARITH_INST_RETIRED_256B_PACKED_DOUBLE | PMC2 | 22550300 | 15646620 | 14030050 | 15597530 | 15537040 | 17605890 | 14709460 | 20803990 |
| FP_ARITH_INST_RETIRED_512B_PACKED_DOUBLE | PMC3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+-----------------------------------------------+---------+---------------+--------------+--------------+--------------+
| Event | Counter | Sum | Min | Max | Avg |
+-----------------------------------------------+---------+---------------+--------------+--------------+--------------+
| Region calls STAT | CTR | 16000 | 2000 | 2000 | 2000 |
| INSTR_RETIRED_ANY STAT | FIXC0 | 343137910000 | 32507280000 | 57012110000 | 42892238750 |
| CPU_CLK_UNHALTED_CORE STAT | FIXC1 | 1938456400000 | 217289500000 | 255584300000 | 242307050000 |
| CPU_CLK_UNHALTED_REF STAT | FIXC2 | 1683101300000 | 199088200000 | 214488800000 | 210387662500 |
| FP_ARITH_INST_RETIRED_128B_PACKED_DOUBLE STAT | PMC0 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_SCALAR_DOUBLE STAT | PMC1 | 4603175400 | 535348300 | 624682100 | 575396925 |
| FP_ARITH_INST_RETIRED_256B_PACKED_DOUBLE STAT | PMC2 | 136480880 | 14030050 | 22550300 | 17060110 |
| FP_ARITH_INST_RETIRED_512B_PACKED_DOUBLE STAT | PMC3 | 0 | 0 | 0 | 0 |
+-----------------------------------------------+---------+---------------+--------------+--------------+--------------+

+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Metric | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Runtime (RDTSC) [s] | 101.5719 | 101.7193 | 101.5686 | 101.2608 | 101.2740 | 101.7142 | 101.8224 | 101.9163 |
| Runtime unhalted [s] | 102.2340 | 97.6016 | 100.1964 | 92.7854 | 98.2816 | 86.9160 | 98.0364 | 99.3344 |
| Clock [MHz] | 2978.9847 | 2891.7276 | 2937.9525 | 2763.3306 | 2886.5712 | 2728.5513 | 2929.0296 | 2906.7416 |
| CPI | 7.3177 | 5.4756 | 5.2129 | 6.4921 | 5.8137 | 4.5188 | 4.2989 | 7.6394 |
| DP [MFLOP/s] | 6.9109 | 6.0910 | 5.9949 | 6.5317 | 6.2912 | 5.9556 | 5.9561 | 6.9459 |
| AVX DP [MFLOP/s] | 0.8881 | 0.6153 | 0.5525 | 0.6161 | 0.6137 | 0.6924 | 0.5778 | 0.8165 |
| AVX512 DP [MFLOP/s] | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Packed [MUOPS/s] | 0.2220 | 0.1538 | 0.1381 | 0.1540 | 0.1534 | 0.1731 | 0.1445 | 0.2041 |
| Scalar [MUOPS/s] | 6.0228 | 5.4757 | 5.4424 | 5.9156 | 5.6775 | 5.2633 | 5.3783 | 6.1294 |
| Vectorization ratio | 3.5551 | 2.7324 | 2.4753 | 2.5378 | 2.6311 | 3.1840 | 2.6158 | 3.2230 |
+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+
| Metric | Sum | Min | Max | Avg | %ile 25 | %ile 50 | %ile 75 |
+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+
| Runtime (RDTSC) [s] STAT | 812.8475 | 101.2608 | 101.9163 | 101.6059 | 101.2740 | 101.5719 | 101.7193 |
| Runtime unhalted [s] STAT | 775.3858 | 86.9160 | 102.2340 | 96.9232 | 102.2340 | 92.7854 | 98.0364 |
| Clock [MHz] STAT | 23022.8891 | 2728.5513 | 2978.9847 | 2877.8611 | 2763.3306 | 2891.7276 | 2929.0296 |
| CPI STAT | 46.7691 | 4.2989 | 7.6394 | 5.8461 | 4.5188 | 5.4756 | 6.4921 |
| DP [MFLOP/s] STAT | 50.6773 | 5.9556 | 6.9459 | 6.3347 | 5.9561 | 6.0910 | 6.5317 |
| AVX DP [MFLOP/s] STAT | 5.3724 | 0.5525 | 0.8881 | 0.6716 | 0.5778 | 0.6153 | 0.6924 |
| AVX512 DP [MFLOP/s] STAT | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Packed [MUOPS/s] STAT | 1.3430 | 0.1381 | 0.2220 | 0.1679 | 0.1445 | 0.1538 | 0.1731 |
| Scalar [MUOPS/s] STAT | 45.3050 | 5.2633 | 6.1294 | 5.6631 | 5.3783 | 5.4757 | 5.9156 |
| Vectorization ratio STAT | 22.9545 | 2.4753 | 3.5551 | 2.8693 | 2.5378 | 2.6311 | 3.1840 |
+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+

Region: stencil_driver
Group: 1
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Event | Counter | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Region calls | CTR | 80000 | 80000 | 80000 | 80000 | 80000 | 80000 | 80000 | 80000 |
| INSTR_RETIRED_ANY | FIXC0 | 59731300000 | 58976430000 | 61112400000 | 56961930000 | 61492240000 | 58896780000 | 59583520000 | 60494260000 |
| CPU_CLK_UNHALTED_CORE | FIXC1 | 92078380000 | 87188380000 | 90154850000 | 83858640000 | 89792670000 | 77414560000 | 87675080000 | 88536690000 |
| CPU_CLK_UNHALTED_REF | FIXC2 | 88347050000 | 85762660000 | 87334890000 | 86691280000 | 88254900000 | 80793450000 | 85358980000 | 87126500000 |
| FP_ARITH_INST_RETIRED_128B_PACKED_DOUBLE | PMC0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_SCALAR_DOUBLE | PMC1 | 1064038 | 1043044 | 994035 | 988149 | 1043275 | 1027032 | 991873 | 1033976 |
| FP_ARITH_INST_RETIRED_256B_PACKED_DOUBLE | PMC2 | 9696646000 | 9949764000 | 9983371000 | 10169010000 | 9601436000 | 9787785000 | 9981142000 | 9956843000 |
| FP_ARITH_INST_RETIRED_512B_PACKED_DOUBLE | PMC3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+-----------------------------------------------+---------+--------------+-------------+-------------+--------------+
| Event | Counter | Sum | Min | Max | Avg |
+-----------------------------------------------+---------+--------------+-------------+-------------+--------------+
| Region calls STAT | CTR | 640000 | 80000 | 80000 | 80000 |
| INSTR_RETIRED_ANY STAT | FIXC0 | 477248860000 | 56961930000 | 61492240000 | 59656107500 |
| CPU_CLK_UNHALTED_CORE STAT | FIXC1 | 696699250000 | 77414560000 | 92078380000 | 87087406250 |
| CPU_CLK_UNHALTED_REF STAT | FIXC2 | 689669710000 | 80793450000 | 88347050000 | 86208713750 |
| FP_ARITH_INST_RETIRED_128B_PACKED_DOUBLE STAT | PMC0 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_SCALAR_DOUBLE STAT | PMC1 | 8185422 | 988149 | 1064038 | 1.023178e+06 |
| FP_ARITH_INST_RETIRED_256B_PACKED_DOUBLE STAT | PMC2 | 79125997000 | 9601436000 | 10169010000 | 9890749625 |
| FP_ARITH_INST_RETIRED_512B_PACKED_DOUBLE STAT | PMC3 | 0 | 0 | 0 | 0 |
+-----------------------------------------------+---------+--------------+-------------+-------------+--------------+

+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Metric | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Runtime (RDTSC) [s] | 41.8199 | 41.4384 | 41.6346 | 41.8118 | 41.9949 | 41.4165 | 41.5603 | 41.4016 |
| Runtime unhalted [s] | 36.8315 | 34.8754 | 36.0626 | 33.5435 | 35.9171 | 30.9659 | 35.0701 | 35.4148 |
| Clock [MHz] | 2605.5796 | 2541.5542 | 2580.6774 | 2418.3059 | 2543.5566 | 2395.4405 | 2567.8280 | 2540.4577 |
| CPI | 1.5415 | 1.4784 | 1.4752 | 1.4722 | 1.4602 | 1.3144 | 1.4715 | 1.4636 |
| DP [MFLOP/s] | 927.4928 | 960.4644 | 959.1652 | 972.8609 | 914.5575 | 945.3268 | 960.6654 | 962.0011 |
| AVX DP [MFLOP/s] | 927.4674 | 960.4392 | 959.1413 | 972.8373 | 914.5327 | 945.3020 | 960.6415 | 961.9762 |
| AVX512 DP [MFLOP/s] | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Packed [MUOPS/s] | 231.8668 | 240.1098 | 239.7853 | 243.2093 | 228.6332 | 236.3255 | 240.1604 | 240.4940 |
| Scalar [MUOPS/s] | 0.0254 | 0.0252 | 0.0239 | 0.0236 | 0.0248 | 0.0248 | 0.0239 | 0.0250 |
| Vectorization ratio | 99.9890 | 99.9895 | 99.9900 | 99.9903 | 99.9891 | 99.9895 | 99.9901 | 99.9896 |
+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+
| Metric | Sum | Min | Max | Avg | %ile 25 | %ile 50 | %ile 75 |
+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+
| Runtime (RDTSC) [s] STAT | 333.0780 | 41.4016 | 41.9949 | 41.6347 | 41.4165 | 41.5603 | 41.8118 |
| Runtime unhalted [s] STAT | 278.6809 | 30.9659 | 36.8315 | 34.8351 | 33.5435 | 35.0701 | 35.9171 |
| Clock [MHz] STAT | 20193.3999 | 2395.4405 | 2605.5796 | 2524.1750 | 2418.3059 | 2541.5542 | 2567.8280 |
| CPI STAT | 11.6770 | 1.3144 | 1.5415 | 1.4596 | 1.4602 | 1.4715 | 1.4752 |
| DP [MFLOP/s] STAT | 7602.5341 | 914.5575 | 972.8609 | 950.3168 | 927.4928 | 959.1652 | 960.6654 |
| AVX DP [MFLOP/s] STAT | 7602.3376 | 914.5327 | 972.8373 | 950.2922 | 927.4674 | 959.1413 | 960.6415 |
| AVX512 DP [MFLOP/s] STAT | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Packed [MUOPS/s] STAT | 1900.5843 | 228.6332 | 243.2093 | 237.5730 | 231.8668 | 239.7853 | 240.1604 |
| Scalar [MUOPS/s] STAT | 0.1966 | 0.0236 | 0.0254 | 0.0246 | 0.0239 | 0.0248 | 0.0250 |
| Vectorization ratio STAT | 799.9171 | 99.9890 | 99.9903 | 99.9896 | 99.9891 | 99.9895 | 99.9900 |
+---------------------------+------------+-----------+-----------+-----------+-----------+-----------+-----------+

Region: deallocate
Group: 1
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Event | Counter | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Region calls | CTR | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| INSTR_RETIRED_ANY | FIXC0 | 4164819000 | 4159108000 | 4168127000 | 4168276000 | 4156504000 | 4156429000 | 4160237000 | 4169006000 |
| CPU_CLK_UNHALTED_CORE | FIXC1 | 2950611000 | 2453817000 | 2502762000 | 2949253000 | 2961944000 | 2442241000 | 2955786000 | 2369950000 |
| CPU_CLK_UNHALTED_REF | FIXC2 | 2454773000 | 2452044000 | 2496167000 | 2052929000 | 2058153000 | 2454661000 | 2059206000 | 2376438000 |
| FP_ARITH_INST_RETIRED_128B_PACKED_DOUBLE | PMC0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_SCALAR_DOUBLE | PMC1 | 11 | 11 | 11 | 11 | 11 | 7 | 4 | 11 |
| FP_ARITH_INST_RETIRED_256B_PACKED_DOUBLE | PMC2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_512B_PACKED_DOUBLE | PMC3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+------------------------------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+-----------------------------------------------+---------+-------------+------------+------------+------------+
| Event | Counter | Sum | Min | Max | Avg |
+-----------------------------------------------+---------+-------------+------------+------------+------------+
| Region calls STAT | CTR | 8 | 1 | 1 | 1 |
| INSTR_RETIRED_ANY STAT | FIXC0 | 33302506000 | 4156429000 | 4169006000 | 4162813250 |
| CPU_CLK_UNHALTED_CORE STAT | FIXC1 | 21586364000 | 2369950000 | 2961944000 | 2698295500 |
| CPU_CLK_UNHALTED_REF STAT | FIXC2 | 18404371000 | 2052929000 | 2496167000 | 2300546375 |
| FP_ARITH_INST_RETIRED_128B_PACKED_DOUBLE STAT | PMC0 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_SCALAR_DOUBLE STAT | PMC1 | 77 | 4 | 11 | 9.6250 |
| FP_ARITH_INST_RETIRED_256B_PACKED_DOUBLE STAT | PMC2 | 0 | 0 | 0 | 0 |
| FP_ARITH_INST_RETIRED_512B_PACKED_DOUBLE STAT | PMC3 | 0 | 0 | 0 | 0 |
+-----------------------------------------------+---------+-------------+------------+------------+------------+

+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Metric | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Runtime (RDTSC) [s] | 1.5078 | 1.5097 | 1.5278 | 1.5060 | 1.5102 | 1.5013 | 1.5183 | 1.4053 |
| Runtime unhalted [s] | 1.1802 | 0.9815 | 1.0011 | 1.1797 | 1.1848 | 0.9769 | 1.1823 | 0.9480 |
| Clock [MHz] | 3004.9645 | 2501.8019 | 2506.5612 | 3591.5088 | 3597.8126 | 2487.3442 | 3588.4936 | 2493.1686 |
| CPI | 0.7085 | 0.5900 | 0.6005 | 0.7075 | 0.7126 | 0.5876 | 0.7105 | 0.5685 |
| DP [MFLOP/s] | 7.295397e-06 | 7.286139e-06 | 7.199886e-06 | 7.304049e-06 | 7.283934e-06 | 4.662533e-06 | 2.634557e-06 | 7.827566e-06 |
| AVX DP [MFLOP/s] | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| AVX512 DP [MFLOP/s] | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Packed [MUOPS/s] | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Scalar [MUOPS/s] | 7.295397e-06 | 7.286139e-06 | 7.199886e-06 | 7.304049e-06 | 7.283934e-06 | 4.662533e-06 | 2.634557e-06 | 7.827566e-06 |
| Vectorization ratio | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+----------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+---------------------------+------------+--------------+--------------+--------------+--------------+--------------+--------------+
| Metric | Sum | Min | Max | Avg | %ile 25 | %ile 50 | %ile 75 |
+---------------------------+------------+--------------+--------------+--------------+--------------+--------------+--------------+
| Runtime (RDTSC) [s] STAT | 11.9864 | 1.4053 | 1.5278 | 1.4983 | 1.5013 | 1.5078 | 1.5102 |
| Runtime unhalted [s] STAT | 8.6345 | 0.9480 | 1.1848 | 1.0793 | 0.9769 | 1.0011 | 1.1802 |
| Clock [MHz] STAT | 23771.6554 | 2487.3442 | 3597.8126 | 2971.4569 | 2493.1686 | 2506.5612 | 3588.4936 |
| CPI STAT | 5.1857 | 0.5685 | 0.7126 | 0.6482 | 0.5876 | 0.6005 | 0.7085 |
| DP [MFLOP/s] STAT | 0.0001 | 2.634557e-06 | 7.827566e-06 | 6.436758e-06 | 4.662533e-06 | 7.283934e-06 | 7.295397e-06 |
| AVX DP [MFLOP/s] STAT | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| AVX512 DP [MFLOP/s] STAT | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Packed [MUOPS/s] STAT | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Scalar [MUOPS/s] STAT | 0.0001 | 2.634557e-06 | 7.827566e-06 | 6.436758e-06 | 4.662533e-06 | 7.283934e-06 | 7.295397e-06 |
| Vectorization ratio STAT | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+---------------------------+------------+--------------+--------------+--------------+--------------+--------------+--------------+

===== MEM =====

Region: main
Group: 1
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Event | Counter | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Region calls | CTR | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| INSTR_RETIRED_ANY | FIXC0 | 230127500000 | 1190373000000 | 1192053000000 | 1178686000000 | 1180478000000 | 1190281000000 | 1203824000000 | 1134422000000 |
| CPU_CLK_UNHALTED_CORE | FIXC1 | 508085400000 | 745939000000 | 745831700000 | 746522500000 | 745396300000 | 746099000000 | 744994700000 | 745872200000 |
| CPU_CLK_UNHALTED_REF | FIXC2 | 436101500000 | 640613400000 | 640660100000 | 640618700000 | 640696200000 | 640448400000 | 640374000000 | 639736800000 |
| CAS_COUNT_RD | MBOX0C0 | 24650910000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX0C1 | 8147471000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX1C0 | 24507430000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX1C1 | 8139855000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX2C0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX2C1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX3C0 | 24627320000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX3C1 | 8143066000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX4C0 | 24533700000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX4C1 | 8140053000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX5C0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX5C1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+----------------------------+---------+---------------+--------------+---------------+---------------+
| Event | Counter | Sum | Min | Max | Avg |
+----------------------------+---------+---------------+--------------+---------------+---------------+
| Region calls STAT | CTR | 8 | 1 | 1 | 1 |
| INSTR_RETIRED_ANY STAT | FIXC0 | 8500244500000 | 230127500000 | 1203824000000 | 1062530562500 |
| CPU_CLK_UNHALTED_CORE STAT | FIXC1 | 5728740800000 | 508085400000 | 746522500000 | 716092600000 |
| CPU_CLK_UNHALTED_REF STAT | FIXC2 | 4919249100000 | 436101500000 | 640696200000 | 614906137500 |
| CAS_COUNT_RD STAT | MBOX0C0 | 24650910000 | 0 | 24650910000 | 3081363750 |
| CAS_COUNT_WR STAT | MBOX0C1 | 8147471000 | 0 | 8147471000 | 1018433875 |
| CAS_COUNT_RD STAT | MBOX1C0 | 24507430000 | 0 | 24507430000 | 3063428750 |
| CAS_COUNT_WR STAT | MBOX1C1 | 8139855000 | 0 | 8139855000 | 1017481875 |
| CAS_COUNT_RD STAT | MBOX2C0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR STAT | MBOX2C1 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD STAT | MBOX3C0 | 24627320000 | 0 | 24627320000 | 3078415000 |
| CAS_COUNT_WR STAT | MBOX3C1 | 8143066000 | 0 | 8143066000 | 1017883250 |
| CAS_COUNT_RD STAT | MBOX4C0 | 24533700000 | 0 | 24533700000 | 3066712500 |
| CAS_COUNT_WR STAT | MBOX4C1 | 8140053000 | 0 | 8140053000 | 1017506625 |
| CAS_COUNT_RD STAT | MBOX5C0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR STAT | MBOX5C1 | 0 | 0 | 0 | 0 |
+----------------------------+---------+---------------+--------------+---------------+---------------+

+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Metric | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Runtime (RDTSC) [s] | 263.8005 | 263.8043 | 263.7778 | 263.8562 | 263.8025 | 263.8686 | 269.4805 | 263.7719 |
| Runtime unhalted [s] | 203.2347 | 298.3768 | 298.3338 | 298.6100 | 298.1588 | 298.4404 | 297.9988 | 298.3491 |
| Clock [MHz] | 2912.6474 | 2911.0226 | 2910.3926 | 2913.2776 | 2908.5378 | 2912.4004 | 2908.4267 | 2914.7597 |
| CPI | 2.2078 | 0.6266 | 0.6257 | 0.6334 | 0.6314 | 0.6268 | 0.6189 | 0.6575 |
| Memory read bandwidth [MBytes/s] | 23853.0217 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory read data volume [GBytes] | 6292.4390 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory write bandwidth [MBytes/s] | 7901.8367 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory write data volume [GBytes] | 2084.5085 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory bandwidth [MBytes/s] | 31754.8584 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory data volume [GBytes] | 8376.9475 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+
| Metric | Sum | Min | Max | Avg | %ile 25 | %ile 50 | %ile 75 |
+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+
| Runtime (RDTSC) [s] STAT | 2116.1623 | 263.7719 | 269.4805 | 264.5203 | 263.7778 | 263.8025 | 263.8562 |
| Runtime unhalted [s] STAT | 2291.5024 | 203.2347 | 298.6100 | 286.4378 | 297.9988 | 298.3338 | 298.3768 |
| Clock [MHz] STAT | 23291.4648 | 2908.4267 | 2914.7597 | 2911.4331 | 2908.5378 | 2911.0226 | 2912.6474 |
| CPI STAT | 6.6281 | 0.6189 | 2.2078 | 0.8285 | 0.6257 | 0.6268 | 0.6334 |
| Memory read bandwidth [MBytes/s] STAT | 23853.0217 | 0 | 23853.0217 | 2981.6277 | 0 | 0 | 0 |
| Memory read data volume [GBytes] STAT | 6292.4390 | 0 | 6292.4390 | 786.5549 | 0 | 0 | 0 |
| Memory write bandwidth [MBytes/s] STAT | 7901.8367 | 0 | 7901.8367 | 987.7296 | 0 | 0 | 0 |
| Memory write data volume [GBytes] STAT | 2084.5085 | 0 | 2084.5085 | 260.5636 | 0 | 0 | 0 |
| Memory bandwidth [MBytes/s] STAT | 31754.8584 | 0 | 31754.8584 | 3969.3573 | 0 | 0 | 0 |
| Memory data volume [GBytes] STAT | 8376.9475 | 0 | 8376.9475 | 1047.1184 | 0 | 0 | 0 |
+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+

Region: allocate
Group: 1
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Event | Counter | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Region calls | CTR | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| INSTR_RETIRED_ANY | FIXC0 | 10021360000 | 10021350000 | 10021350000 | 10021350000 | 10021350000 | 10021350000 | 10021350000 | 10021350000 |
| CPU_CLK_UNHALTED_CORE | FIXC1 | 4741292000 | 4748385000 | 4766731000 | 4745253000 | 4740622000 | 4741186000 | 4747495000 | 4746461000 |
| CPU_CLK_UNHALTED_REF | FIXC2 | 4008135000 | 4031600000 | 4042033000 | 4029167000 | 4006969000 | 4016471000 | 4012866000 | 4024050000 |
| CAS_COUNT_RD | MBOX0C0 | 124324200 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX0C1 | 122322100 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX1C0 | 124146100 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX1C1 | 122303300 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX2C0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX2C1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX3C0 | 124163800 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX3C1 | 122285500 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX4C0 | 124088500 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX4C1 | 122302900 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX5C0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX5C1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+----------------------------+---------+-------------+-------------+-------------+--------------+
| Event | Counter | Sum | Min | Max | Avg |
+----------------------------+---------+-------------+-------------+-------------+--------------+
| Region calls STAT | CTR | 8 | 1 | 1 | 1 |
| INSTR_RETIRED_ANY STAT | FIXC0 | 80170810000 | 10021350000 | 10021360000 | 10021351250 |
| CPU_CLK_UNHALTED_CORE STAT | FIXC1 | 37977425000 | 4740622000 | 4766731000 | 4747178125 |
| CPU_CLK_UNHALTED_REF STAT | FIXC2 | 32171291000 | 4006969000 | 4042033000 | 4021411375 |
| CAS_COUNT_RD STAT | MBOX0C0 | 124324200 | 0 | 124324200 | 15540525 |
| CAS_COUNT_WR STAT | MBOX0C1 | 122322100 | 0 | 122322100 | 1.529026e+07 |
| CAS_COUNT_RD STAT | MBOX1C0 | 124146100 | 0 | 124146100 | 1.551826e+07 |
| CAS_COUNT_WR STAT | MBOX1C1 | 122303300 | 0 | 122303300 | 1.528791e+07 |
| CAS_COUNT_RD STAT | MBOX2C0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR STAT | MBOX2C1 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD STAT | MBOX3C0 | 124163800 | 0 | 124163800 | 15520475 |
| CAS_COUNT_WR STAT | MBOX3C1 | 122285500 | 0 | 122285500 | 1.528569e+07 |
| CAS_COUNT_RD STAT | MBOX4C0 | 124088500 | 0 | 124088500 | 1.551106e+07 |
| CAS_COUNT_WR STAT | MBOX4C1 | 122302900 | 0 | 122302900 | 1.528786e+07 |
| CAS_COUNT_RD STAT | MBOX5C0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR STAT | MBOX5C1 | 0 | 0 | 0 | 0 |
+----------------------------+---------+-------------+-------------+-------------+--------------+

+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Metric | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Runtime (RDTSC) [s] | 3.1917 | 3.1982 | 3.2022 | 3.2031 | 3.1931 | 3.1917 | 3.2598 | 3.1973 |
| Runtime unhalted [s] | 1.8965 | 1.8994 | 1.9067 | 1.8981 | 1.8963 | 1.8965 | 1.8990 | 1.8986 |
| Clock [MHz] | 2957.2848 | 2944.4675 | 2948.2153 | 2944.3042 | 2957.7330 | 2951.0810 | 2957.6616 | 2948.8061 |
| CPI | 0.4731 | 0.4738 | 0.4757 | 0.4735 | 0.4731 | 0.4731 | 0.4737 | 0.4736 |
| Memory read bandwidth [MBytes/s] | 9960.2771 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory read data volume [GBytes] | 31.7902 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory write bandwidth [MBytes/s] | 9809.7107 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory write data volume [GBytes] | 31.3097 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory bandwidth [MBytes/s] | 19769.9879 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory data volume [GBytes] | 63.0999 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+
| Metric | Sum | Min | Max | Avg | %ile 25 | %ile 50 | %ile 75 |
+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+
| Runtime (RDTSC) [s] STAT | 25.6371 | 3.1917 | 3.2598 | 3.2046 | 3.1917 | 3.1973 | 3.2022 |
| Runtime unhalted [s] STAT | 15.1911 | 1.8963 | 1.9067 | 1.8989 | 1.8965 | 1.8981 | 1.8990 |
| Clock [MHz] STAT | 23609.5535 | 2944.3042 | 2957.7330 | 2951.1942 | 2944.4675 | 2948.8061 | 2957.2848 |
| CPI STAT | 3.7896 | 0.4731 | 0.4757 | 0.4737 | 0.4731 | 0.4735 | 0.4737 |
| Memory read bandwidth [MBytes/s] STAT | 9960.2771 | 0 | 9960.2771 | 1245.0346 | 0 | 0 | 0 |
| Memory read data volume [GBytes] STAT | 31.7902 | 0 | 31.7902 | 3.9738 | 0 | 0 | 0 |
| Memory write bandwidth [MBytes/s] STAT | 9809.7107 | 0 | 9809.7107 | 1226.2138 | 0 | 0 | 0 |
| Memory write data volume [GBytes] STAT | 31.3097 | 0 | 31.3097 | 3.9137 | 0 | 0 | 0 |
| Memory bandwidth [MBytes/s] STAT | 19769.9879 | 0 | 19769.9879 | 2471.2485 | 0 | 0 | 0 |
| Memory data volume [GBytes] STAT | 63.0999 | 0 | 63.0999 | 7.8875 | 0 | 0 | 0 |
+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+

Region: driver
Group: 1
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Event | Counter | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Region calls | CTR | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| INSTR_RETIRED_ANY | FIXC0 | 212641500000 | 1172643000000 | 1174192000000 | 1160866000000 | 1162664000000 | 1172541000000 | 1186056000000 | 1115991000000 |
| CPU_CLK_UNHALTED_CORE | FIXC1 | 494893500000 | 732703000000 | 732611000000 | 733143200000 | 732194600000 | 732851200000 | 731810100000 | 732523800000 |
| CPU_CLK_UNHALTED_REF | FIXC2 | 425020600000 | 629476700000 | 629550300000 | 629371400000 | 629604000000 | 629313200000 | 629302400000 | 628518000000 |
| CAS_COUNT_RD | MBOX0C0 | 24284030000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX0C1 | 7828170000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX1C0 | 24140860000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX1C1 | 7820637000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX2C0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX2C1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX3C0 | 24260840000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX3C1 | 7823895000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX4C0 | 24167300000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX4C1 | 7820839000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX5C0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX5C1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+----------------------------+---------+---------------+--------------+---------------+---------------+
| Event | Counter | Sum | Min | Max | Avg |
+----------------------------+---------+---------------+--------------+---------------+---------------+
| Region calls STAT | CTR | 8 | 1 | 1 | 1 |
| INSTR_RETIRED_ANY STAT | FIXC0 | 8357594500000 | 212641500000 | 1186056000000 | 1044699312500 |
| CPU_CLK_UNHALTED_CORE STAT | FIXC1 | 5622730400000 | 494893500000 | 733143200000 | 702841300000 |
| CPU_CLK_UNHALTED_REF STAT | FIXC2 | 4830156600000 | 425020600000 | 629604000000 | 603769575000 |
| CAS_COUNT_RD STAT | MBOX0C0 | 24284030000 | 0 | 24284030000 | 3035503750 |
| CAS_COUNT_WR STAT | MBOX0C1 | 7828170000 | 0 | 7828170000 | 978521250 |
| CAS_COUNT_RD STAT | MBOX1C0 | 24140860000 | 0 | 24140860000 | 3017607500 |
| CAS_COUNT_WR STAT | MBOX1C1 | 7820637000 | 0 | 7820637000 | 977579625 |
| CAS_COUNT_RD STAT | MBOX2C0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR STAT | MBOX2C1 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD STAT | MBOX3C0 | 24260840000 | 0 | 24260840000 | 3032605000 |
| CAS_COUNT_WR STAT | MBOX3C1 | 7823895000 | 0 | 7823895000 | 977986875 |
| CAS_COUNT_RD STAT | MBOX4C0 | 24167300000 | 0 | 24167300000 | 3020912500 |
| CAS_COUNT_WR STAT | MBOX4C1 | 7820839000 | 0 | 7820839000 | 977604875 |
| CAS_COUNT_RD STAT | MBOX5C0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR STAT | MBOX5C1 | 0 | 0 | 0 | 0 |
+----------------------------+---------+---------------+--------------+---------------+---------------+

+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Metric | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Runtime (RDTSC) [s] | 257.2544 | 257.2491 | 257.2446 | 257.2439 | 257.2536 | 257.2550 | 262.8132 | 257.2495 |
| Runtime unhalted [s] | 197.9580 | 293.0824 | 293.0455 | 293.2582 | 292.8781 | 293.1413 | 292.7250 | 293.0098 |
| Clock [MHz] | 2910.9889 | 2909.9572 | 2909.2525 | 2912.1945 | 2907.3591 | 2911.3052 | 2907.2183 | 2913.6923 |
| CPI | 2.3274 | 0.6248 | 0.6239 | 0.6315 | 0.6298 | 0.6250 | 0.6170 | 0.6564 |
| Memory read bandwidth [MBytes/s] | 24095.1911 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory read data volume [GBytes] | 6198.5939 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory write bandwidth [MBytes/s] | 7785.2376 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory write data volume [GBytes] | 2002.7866 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory bandwidth [MBytes/s] | 31880.4286 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory data volume [GBytes] | 8201.3805 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+
| Metric | Sum | Min | Max | Avg | %ile 25 | %ile 50 | %ile 75 |
+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+
| Runtime (RDTSC) [s] STAT | 2063.5633 | 257.2439 | 262.8132 | 257.9454 | 257.2446 | 257.2495 | 257.2544 |
| Runtime unhalted [s] STAT | 2249.0983 | 197.9580 | 293.2582 | 281.1373 | 292.7250 | 293.0098 | 293.0824 |
| Clock [MHz] STAT | 23281.9680 | 2907.2183 | 2913.6923 | 2910.2460 | 2907.3591 | 2909.9572 | 2911.3052 |
| CPI STAT | 6.7358 | 0.6170 | 2.3274 | 0.8420 | 0.6239 | 0.6250 | 0.6315 |
| Memory read bandwidth [MBytes/s] STAT | 24095.1911 | 0 | 24095.1911 | 3011.8989 | 0 | 0 | 0 |
| Memory read data volume [GBytes] STAT | 6198.5939 | 0 | 6198.5939 | 774.8242 | 0 | 0 | 0 |
| Memory write bandwidth [MBytes/s] STAT | 7785.2376 | 0 | 7785.2376 | 973.1547 | 0 | 0 | 0 |
| Memory write data volume [GBytes] STAT | 2002.7866 | 0 | 2002.7866 | 250.3483 | 0 | 0 | 0 |
| Memory bandwidth [MBytes/s] STAT | 31880.4286 | 0 | 31880.4286 | 3985.0536 | 0 | 0 | 0 |
| Memory data volume [GBytes] STAT | 8201.3805 | 0 | 8201.3805 | 1025.1726 | 0 | 0 | 0 |
+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+

Region: check_sum
Group: 1
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Event | Counter | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Region calls | CTR | 80040 | 80040 | 80040 | 80040 | 80040 | 80040 | 80040 | 80040 |
| INSTR_RETIRED_ANY | FIXC0 | 57821080000 | 986477200000 | 986072600000 | 985102900000 | 985856000000 | 981928300000 | 985741400000 | 981870300000 |
| CPU_CLK_UNHALTED_CORE | FIXC1 | 64361380000 | 299748700000 | 300371200000 | 301071200000 | 301356200000 | 300640200000 | 299434000000 | 300477200000 |
| CPU_CLK_UNHALTED_REF | FIXC2 | 53999930000 | 256612000000 | 257193000000 | 257335400000 | 258278600000 | 257114800000 | 256692400000 | 256644100000 |
| CAS_COUNT_RD | MBOX0C0 | 1591492000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX0C1 | 138888900 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX1C0 | 1607186000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX1C1 | 167624100 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX2C0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX2C1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX3C0 | 1783279000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX3C1 | 224080800 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX4C0 | 1826428000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX4C1 | 253658900 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX5C0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX5C1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+----------------------------+---------+---------------+-------------+--------------+--------------+
| Event | Counter | Sum | Min | Max | Avg |
+----------------------------+---------+---------------+-------------+--------------+--------------+
| Region calls STAT | CTR | 640320 | 80040 | 80040 | 80040 |
| INSTR_RETIRED_ANY STAT | FIXC0 | 6950869780000 | 57821080000 | 986477200000 | 868858722500 |
| CPU_CLK_UNHALTED_CORE STAT | FIXC1 | 2167460080000 | 64361380000 | 301356200000 | 270932510000 |
| CPU_CLK_UNHALTED_REF STAT | FIXC2 | 1853870230000 | 53999930000 | 258278600000 | 231733778750 |
| CAS_COUNT_RD STAT | MBOX0C0 | 1591492000 | 0 | 1591492000 | 198936500 |
| CAS_COUNT_WR STAT | MBOX0C1 | 138888900 | 0 | 138888900 | 1.736111e+07 |
| CAS_COUNT_RD STAT | MBOX1C0 | 1607186000 | 0 | 1607186000 | 200898250 |
| CAS_COUNT_WR STAT | MBOX1C1 | 167624100 | 0 | 167624100 | 2.095301e+07 |
| CAS_COUNT_RD STAT | MBOX2C0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR STAT | MBOX2C1 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD STAT | MBOX3C0 | 1783279000 | 0 | 1783279000 | 222909875 |
| CAS_COUNT_WR STAT | MBOX3C1 | 224080800 | 0 | 224080800 | 28010100 |
| CAS_COUNT_RD STAT | MBOX4C0 | 1826428000 | 0 | 1826428000 | 228303500 |
| CAS_COUNT_WR STAT | MBOX4C1 | 253658900 | 0 | 253658900 | 3.170736e+07 |
| CAS_COUNT_RD STAT | MBOX5C0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR STAT | MBOX5C1 | 0 | 0 | 0 | 0 |
+----------------------------+---------+---------------+-------------+--------------+--------------+

+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Metric | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Runtime (RDTSC) [s] | 21.4971 | 103.0638 | 103.2683 | 103.3105 | 103.7147 | 103.2443 | 105.3066 | 103.0096 |
| Runtime unhalted [s] | 25.7446 | 119.9000 | 120.1489 | 120.4289 | 120.5426 | 120.2564 | 119.7740 | 120.1910 |
| Clock [MHz] | 2979.6890 | 2920.2406 | 2919.6955 | 2924.8814 | 2916.9658 | 2923.2015 | 2916.2633 | 2926.9810 |
| CPI | 1.1131 | 0.3039 | 0.3046 | 0.3056 | 0.3057 | 0.3062 | 0.3038 | 0.3060 |
| Memory read bandwidth [MBytes/s] | 20269.5356 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory read data volume [GBytes] | 435.7366 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory write bandwidth [MBytes/s] | 2334.8324 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory write data volume [GBytes] | 50.1922 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory bandwidth [MBytes/s] | 22604.3681 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory data volume [GBytes] | 485.9288 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+
| Metric | Sum | Min | Max | Avg | %ile 25 | %ile 50 | %ile 75 |
+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+
| Runtime (RDTSC) [s] STAT | 746.4149 | 21.4971 | 105.3066 | 93.3019 | 103.0638 | 103.2683 | 103.7147 |
| Runtime unhalted [s] STAT | 866.9864 | 25.7446 | 120.5426 | 108.3733 | 119.9000 | 120.1910 | 120.4289 |
| Clock [MHz] STAT | 23427.9181 | 2916.2633 | 2979.6890 | 2928.4898 | 2916.9658 | 2920.2406 | 2924.8814 |
| CPI STAT | 3.2489 | 0.3038 | 1.1131 | 0.4061 | 0.3039 | 0.3056 | 0.3060 |
| Memory read bandwidth [MBytes/s] STAT | 20269.5356 | 0 | 20269.5356 | 2533.6919 | 0 | 0 | 0 |
| Memory read data volume [GBytes] STAT | 435.7366 | 0 | 435.7366 | 54.4671 | 0 | 0 | 0 |
| Memory write bandwidth [MBytes/s] STAT | 2334.8324 | 0 | 2334.8324 | 291.8540 | 0 | 0 | 0 |
| Memory write data volume [GBytes] STAT | 50.1922 | 0 | 50.1922 | 6.2740 | 0 | 0 | 0 |
| Memory bandwidth [MBytes/s] STAT | 22604.3681 | 0 | 22604.3681 | 2825.5460 | 0 | 0 | 0 |
| Memory data volume [GBytes] STAT | 485.9288 | 0 | 485.9288 | 60.7411 | 0 | 0 | 0 |
+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+

Region: refine
Group: 1
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Event | Counter | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Region calls | CTR | 21 | 21 | 21 | 21 | 21 | 21 | 21 | 21 |
| INSTR_RETIRED_ANY | FIXC0 | 40505850000 | 53951360000 | 53000940000 | 54082280000 | 49277810000 | 56353150000 | 54135570000 | 19089440000 |
| CPU_CLK_UNHALTED_CORE | FIXC1 | 19856670000 | 19966850000 | 19978990000 | 19650540000 | 19978580000 | 19811710000 | 19827360000 | 18191300000 |
| CPU_CLK_UNHALTED_REF | FIXC2 | 16547290000 | 16639980000 | 16649200000 | 16375660000 | 16649330000 | 16509910000 | 16522920000 | 15160470000 |
| CAS_COUNT_RD | MBOX0C0 | 213922200 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX0C1 | 71378700 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX1C0 | 213571700 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX1C1 | 71282540 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX2C0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX2C1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX3C0 | 213505700 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX3C1 | 71296170 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX4C0 | 213283600 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX4C1 | 71297040 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX5C0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX5C1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+----------------------------+---------+--------------+-------------+-------------+--------------+
| Event | Counter | Sum | Min | Max | Avg |
+----------------------------+---------+--------------+-------------+-------------+--------------+
| Region calls STAT | CTR | 168 | 21 | 21 | 21 |
| INSTR_RETIRED_ANY STAT | FIXC0 | 380396400000 | 19089440000 | 56353150000 | 47549550000 |
| CPU_CLK_UNHALTED_CORE STAT | FIXC1 | 157262000000 | 18191300000 | 19978990000 | 19657750000 |
| CPU_CLK_UNHALTED_REF STAT | FIXC2 | 131054760000 | 15160470000 | 16649330000 | 16381845000 |
| CAS_COUNT_RD STAT | MBOX0C0 | 213922200 | 0 | 213922200 | 26740275 |
| CAS_COUNT_WR STAT | MBOX0C1 | 71378700 | 0 | 71378700 | 8.922338e+06 |
| CAS_COUNT_RD STAT | MBOX1C0 | 213571700 | 0 | 213571700 | 2.669646e+07 |
| CAS_COUNT_WR STAT | MBOX1C1 | 71282540 | 0 | 71282540 | 8.910318e+06 |
| CAS_COUNT_RD STAT | MBOX2C0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR STAT | MBOX2C1 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD STAT | MBOX3C0 | 213505700 | 0 | 213505700 | 2.668821e+07 |
| CAS_COUNT_WR STAT | MBOX3C1 | 71296170 | 0 | 71296170 | 8.912021e+06 |
| CAS_COUNT_RD STAT | MBOX4C0 | 213283600 | 0 | 213283600 | 26660450 |
| CAS_COUNT_WR STAT | MBOX4C1 | 71297040 | 0 | 71297040 | 8912130 |
| CAS_COUNT_RD STAT | MBOX5C0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR STAT | MBOX5C1 | 0 | 0 | 0 | 0 |
+----------------------------+---------+--------------+-------------+-------------+--------------+

+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Metric | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Runtime (RDTSC) [s] | 6.7158 | 6.7262 | 6.7262 | 6.7262 | 6.7262 | 6.7262 | 6.8715 | 6.7262 |
| Runtime unhalted [s] | 7.9427 | 7.9868 | 7.9916 | 7.8602 | 7.9914 | 7.9247 | 7.9310 | 7.2765 |
| Clock [MHz] | 2999.9797 | 2999.8189 | 2999.9814 | 2999.9516 | 2999.9049 | 2999.9639 | 2999.9687 | 2999.7892 |
| CPI | 0.4902 | 0.3701 | 0.3770 | 0.3633 | 0.4054 | 0.3516 | 0.3663 | 0.9530 |
| Memory read bandwidth [MBytes/s] | 8141.0605 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory read data volume [GBytes] | 54.6741 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory write bandwidth [MBytes/s] | 2718.3886 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory write data volume [GBytes] | 18.2563 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory bandwidth [MBytes/s] | 10859.4491 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory data volume [GBytes] | 72.9304 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+
| Metric | Sum | Min | Max | Avg | %ile 25 | %ile 50 | %ile 75 |
+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+
| Runtime (RDTSC) [s] STAT | 53.9445 | 6.7158 | 6.8715 | 6.7431 | 6.7262 | 6.7262 | 6.7262 |
| Runtime unhalted [s] STAT | 62.9049 | 7.2765 | 7.9916 | 7.8631 | 7.8602 | 7.9310 | 7.9868 |
| Clock [MHz] STAT | 23999.3583 | 2999.7892 | 2999.9814 | 2999.9198 | 2999.8189 | 2999.9516 | 2999.9687 |
| CPI STAT | 3.6769 | 0.3516 | 0.9530 | 0.4596 | 0.3633 | 0.3701 | 0.4054 |
| Memory read bandwidth [MBytes/s] STAT | 8141.0605 | 0 | 8141.0605 | 1017.6326 | 0 | 0 | 0 |
| Memory read data volume [GBytes] STAT | 54.6741 | 0 | 54.6741 | 6.8343 | 0 | 0 | 0 |
| Memory write bandwidth [MBytes/s] STAT | 2718.3886 | 0 | 2718.3886 | 339.7986 | 0 | 0 | 0 |
| Memory write data volume [GBytes] STAT | 18.2563 | 0 | 18.2563 | 2.2820 | 0 | 0 | 0 |
| Memory bandwidth [MBytes/s] STAT | 10859.4491 | 0 | 10859.4491 | 1357.4311 | 0 | 0 | 0 |
| Memory data volume [GBytes] STAT | 72.9304 | 0 | 72.9304 | 9.1163 | 0 | 0 | 0 |
+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+

Region: comm
Group: 1
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Event | Counter | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Region calls | CTR | 2000 | 2000 | 2000 | 2000 | 2000 | 2000 | 2000 | 2000 |
| INSTR_RETIRED_ANY | FIXC0 | 40880980000 | 59464750000 | 62339260000 | 48985910000 | 54664190000 | 61463880000 | 73327960000 | 42300550000 |
| CPU_CLK_UNHALTED_CORE | FIXC1 | 301523300000 | 305431400000 | 304041700000 | 303948400000 | 302712900000 | 304572900000 | 304711100000 | 305390800000 |
| CPU_CLK_UNHALTED_REF | FIXC2 | 251279100000 | 254538600000 | 253378400000 | 253300500000 | 252271800000 | 253830800000 | 253936600000 | 254502700000 |
| CAS_COUNT_RD | MBOX0C0 | 15020690000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX0C1 | 5047785000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX1C0 | 15002850000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX1C1 | 5048156000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX2C0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX2C1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX3C0 | 15021080000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX3C1 | 5050146000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX4C0 | 15006060000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX4C1 | 5052006000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX5C0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX5C1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+----------------------------+---------+---------------+--------------+--------------+--------------+
| Event | Counter | Sum | Min | Max | Avg |
+----------------------------+---------+---------------+--------------+--------------+--------------+
| Region calls STAT | CTR | 16000 | 2000 | 2000 | 2000 |
| INSTR_RETIRED_ANY STAT | FIXC0 | 443427480000 | 40880980000 | 73327960000 | 55428435000 |
| CPU_CLK_UNHALTED_CORE STAT | FIXC1 | 2432332500000 | 301523300000 | 305431400000 | 304041562500 |
| CPU_CLK_UNHALTED_REF STAT | FIXC2 | 2027038500000 | 251279100000 | 254538600000 | 253379812500 |
| CAS_COUNT_RD STAT | MBOX0C0 | 15020690000 | 0 | 15020690000 | 1877586250 |
| CAS_COUNT_WR STAT | MBOX0C1 | 5047785000 | 0 | 5047785000 | 630973125 |
| CAS_COUNT_RD STAT | MBOX1C0 | 15002850000 | 0 | 15002850000 | 1875356250 |
| CAS_COUNT_WR STAT | MBOX1C1 | 5048156000 | 0 | 5048156000 | 631019500 |
| CAS_COUNT_RD STAT | MBOX2C0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR STAT | MBOX2C1 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD STAT | MBOX3C0 | 15021080000 | 0 | 15021080000 | 1877635000 |
| CAS_COUNT_WR STAT | MBOX3C1 | 5050146000 | 0 | 5050146000 | 631268250 |
| CAS_COUNT_RD STAT | MBOX4C0 | 15006060000 | 0 | 15006060000 | 1875757500 |
| CAS_COUNT_WR STAT | MBOX4C1 | 5052006000 | 0 | 5052006000 | 631500750 |
| CAS_COUNT_RD STAT | MBOX5C0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR STAT | MBOX5C1 | 0 | 0 | 0 | 0 |
+----------------------------+---------+---------------+--------------+--------------+--------------+

+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Metric | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Runtime (RDTSC) [s] | 101.0896 | 102.3272 | 101.8555 | 101.8366 | 101.4149 | 102.0533 | 104.2811 | 102.2396 |
| Runtime unhalted [s] | 120.6097 | 122.1730 | 121.6171 | 121.5798 | 121.0853 | 121.8295 | 121.8848 | 122.1564 |
| Clock [MHz] | 2999.8759 | 2999.8415 | 2999.8668 | 2999.8697 | 2999.8660 | 2999.7545 | 2999.8642 | 2999.8754 |
| CPI | 7.3756 | 5.1363 | 4.8772 | 6.2048 | 5.5377 | 4.9553 | 4.1555 | 7.2195 |
| Memory read bandwidth [MBytes/s] | 38018.1890 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory read data volume [GBytes] | 3843.2435 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory write bandwidth [MBytes/s] | 12787.4475 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory write data volume [GBytes] | 1292.6780 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory bandwidth [MBytes/s] | 50805.6365 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory data volume [GBytes] | 5135.9215 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+
| Metric | Sum | Min | Max | Avg | %ile 25 | %ile 50 | %ile 75 |
+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+
| Runtime (RDTSC) [s] STAT | 817.0978 | 101.0896 | 104.2811 | 102.1372 | 101.4149 | 101.8555 | 102.2396 |
| Runtime unhalted [s] STAT | 972.9356 | 120.6097 | 122.1730 | 121.6169 | 121.0853 | 121.6171 | 121.8848 |
| Clock [MHz] STAT | 23998.8140 | 2999.7545 | 2999.8759 | 2999.8518 | 2999.8415 | 2999.8660 | 2999.8697 |
| CPI STAT | 45.4619 | 4.1555 | 7.3756 | 5.6827 | 4.8772 | 5.1363 | 6.2048 |
| Memory read bandwidth [MBytes/s] STAT | 38018.1890 | 0 | 38018.1890 | 4752.2736 | 0 | 0 | 0 |
| Memory read data volume [GBytes] STAT | 3843.2435 | 0 | 3843.2435 | 480.4054 | 0 | 0 | 0 |
| Memory write bandwidth [MBytes/s] STAT | 12787.4475 | 0 | 12787.4475 | 1598.4309 | 0 | 0 | 0 |
| Memory write data volume [GBytes] STAT | 1292.6780 | 0 | 1292.6780 | 161.5848 | 0 | 0 | 0 |
| Memory bandwidth [MBytes/s] STAT | 50805.6365 | 0 | 50805.6365 | 6350.7046 | 0 | 0 | 0 |
| Memory data volume [GBytes] STAT | 5135.9215 | 0 | 5135.9215 | 641.9902 | 0 | 0 | 0 |
+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+

Region: stencil_driver
Group: 1
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Event | Counter | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Region calls | CTR | 80000 | 80000 | 80000 | 80000 | 80000 | 80000 | 80000 | 80000 |
| INSTR_RETIRED_ANY | FIXC0 | 72211680000 | 71937910000 | 71985650000 | 71902180000 | 72007320000 | 71926290000 | 71971050000 | 71886360000 |
| CPU_CLK_UNHALTED_CORE | FIXC1 | 107042700000 | 106508900000 | 107188200000 | 107406700000 | 107035300000 | 106763200000 | 106756900000 | 107392000000 |
| CPU_CLK_UNHALTED_REF | FIXC2 | 101347700000 | 100790300000 | 101446200000 | 101451100000 | 101452100000 | 100952600000 | 101225700000 | 101293000000 |
| CAS_COUNT_RD | MBOX0C0 | 5675768000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX0C1 | 2307106000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX1C0 | 5644058000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX1C1 | 2263641000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX2C0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX2C1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX3C0 | 5699052000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX3C1 | 2167892000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX4C0 | 5647237000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX4C1 | 2120758000 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX5C0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX5C1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+----------------------------+---------+--------------+--------------+--------------+--------------+
| Event | Counter | Sum | Min | Max | Avg |
+----------------------------+---------+--------------+--------------+--------------+--------------+
| Region calls STAT | CTR | 640000 | 80000 | 80000 | 80000 |
| INSTR_RETIRED_ANY STAT | FIXC0 | 575828440000 | 71886360000 | 72211680000 | 71978555000 |
| CPU_CLK_UNHALTED_CORE STAT | FIXC1 | 856093900000 | 106508900000 | 107406700000 | 107011737500 |
| CPU_CLK_UNHALTED_REF STAT | FIXC2 | 809958700000 | 100790300000 | 101452100000 | 101244837500 |
| CAS_COUNT_RD STAT | MBOX0C0 | 5675768000 | 0 | 5675768000 | 709471000 |
| CAS_COUNT_WR STAT | MBOX0C1 | 2307106000 | 0 | 2307106000 | 288388250 |
| CAS_COUNT_RD STAT | MBOX1C0 | 5644058000 | 0 | 5644058000 | 705507250 |
| CAS_COUNT_WR STAT | MBOX1C1 | 2263641000 | 0 | 2263641000 | 282955125 |
| CAS_COUNT_RD STAT | MBOX2C0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR STAT | MBOX2C1 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD STAT | MBOX3C0 | 5699052000 | 0 | 5699052000 | 712381500 |
| CAS_COUNT_WR STAT | MBOX3C1 | 2167892000 | 0 | 2167892000 | 270986500 |
| CAS_COUNT_RD STAT | MBOX4C0 | 5647237000 | 0 | 5647237000 | 705904625 |
| CAS_COUNT_WR STAT | MBOX4C1 | 2120758000 | 0 | 2120758000 | 265094750 |
| CAS_COUNT_RD STAT | MBOX5C0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR STAT | MBOX5C1 | 0 | 0 | 0 | 0 |
+----------------------------+---------+--------------+--------------+--------------+--------------+

+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Metric | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Runtime (RDTSC) [s] | 40.7161 | 40.5976 | 40.8463 | 40.8128 | 40.8305 | 40.6872 | 41.6959 | 40.7221 |
| Runtime unhalted [s] | 42.8172 | 42.6037 | 42.8754 | 42.9628 | 42.8142 | 42.7054 | 42.7029 | 42.9568 |
| Clock [MHz] | 2640.4743 | 2641.8335 | 2641.4939 | 2646.7516 | 2637.5798 | 2643.8867 | 2636.5972 | 2650.5266 |
| CPI | 1.4823 | 1.4806 | 1.4890 | 1.4938 | 1.4865 | 1.4843 | 1.4833 | 1.4939 |
| Memory read bandwidth [MBytes/s] | 35627.9633 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory read data volume [GBytes] | 1450.6314 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory write bandwidth [MBytes/s] | 13925.7332 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory write data volume [GBytes] | 567.0014 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory bandwidth [MBytes/s] | 49553.6965 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory data volume [GBytes] | 2017.6328 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+
| Metric | Sum | Min | Max | Avg | %ile 25 | %ile 50 | %ile 75 |
+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+
| Runtime (RDTSC) [s] STAT | 326.9085 | 40.5976 | 41.6959 | 40.8636 | 40.6872 | 40.7221 | 40.8305 |
| Runtime unhalted [s] STAT | 342.4384 | 42.6037 | 42.9628 | 42.8048 | 42.7029 | 42.8142 | 42.8754 |
| Clock [MHz] STAT | 21139.1436 | 2636.5972 | 2650.5266 | 2642.3929 | 2637.5798 | 2641.4939 | 2643.8867 |
| CPI STAT | 11.8937 | 1.4806 | 1.4939 | 1.4867 | 1.4823 | 1.4843 | 1.4890 |
| Memory read bandwidth [MBytes/s] STAT | 35627.9633 | 0 | 35627.9633 | 4453.4954 | 0 | 0 | 0 |
| Memory read data volume [GBytes] STAT | 1450.6314 | 0 | 1450.6314 | 181.3289 | 0 | 0 | 0 |
| Memory write bandwidth [MBytes/s] STAT | 13925.7332 | 0 | 13925.7332 | 1740.7167 | 0 | 0 | 0 |
| Memory write data volume [GBytes] STAT | 567.0014 | 0 | 567.0014 | 70.8752 | 0 | 0 | 0 |
| Memory bandwidth [MBytes/s] STAT | 49553.6965 | 0 | 49553.6965 | 6194.2121 | 0 | 0 | 0 |
| Memory data volume [GBytes] STAT | 2017.6328 | 0 | 2017.6328 | 252.2041 | 0 | 0 | 0 |
+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+

Region: deallocate
Group: 1
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Event | Counter | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Region calls | CTR | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| INSTR_RETIRED_ANY | FIXC0 | 4852692000 | 4852716000 | 4852688000 | 4852685000 | 4852688000 | 4852720000 | 4852688000 | 4852546000 |
| CPU_CLK_UNHALTED_CORE | FIXC1 | 3609514000 | 3461289000 | 3435704000 | 3454584000 | 3456480000 | 3445579000 | 3459705000 | 3291940000 |
| CPU_CLK_UNHALTED_REF | FIXC2 | 3008053000 | 2889959000 | 2863238000 | 2879076000 | 2881577000 | 2871648000 | 2883221000 | 2743519000 |
| CAS_COUNT_RD | MBOX0C0 | 122717100 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX0C1 | 81900730 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX1C0 | 122634300 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX1C1 | 81876530 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX2C0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX2C1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX3C0 | 122583800 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX3C1 | 81870940 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX4C0 | 122559500 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX4C1 | 81878940 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD | MBOX5C0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR | MBOX5C1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-----------------------+---------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+----------------------------+---------+-------------+------------+------------+--------------+
| Event | Counter | Sum | Min | Max | Avg |
+----------------------------+---------+-------------+------------+------------+--------------+
| Region calls STAT | CTR | 8 | 1 | 1 | 1 |
| INSTR_RETIRED_ANY STAT | FIXC0 | 38821423000 | 4852546000 | 4852720000 | 4852677875 |
| CPU_CLK_UNHALTED_CORE STAT | FIXC1 | 27614795000 | 3291940000 | 3609514000 | 3451849375 |
| CPU_CLK_UNHALTED_REF STAT | FIXC2 | 23020291000 | 2743519000 | 3008053000 | 2877536375 |
| CAS_COUNT_RD STAT | MBOX0C0 | 122717100 | 0 | 122717100 | 1.533964e+07 |
| CAS_COUNT_WR STAT | MBOX0C1 | 81900730 | 0 | 81900730 | 1.023759e+07 |
| CAS_COUNT_RD STAT | MBOX1C0 | 122634300 | 0 | 122634300 | 1.532929e+07 |
| CAS_COUNT_WR STAT | MBOX1C1 | 81876530 | 0 | 81876530 | 1.023457e+07 |
| CAS_COUNT_RD STAT | MBOX2C0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR STAT | MBOX2C1 | 0 | 0 | 0 | 0 |
| CAS_COUNT_RD STAT | MBOX3C0 | 122583800 | 0 | 122583800 | 15322975 |
| CAS_COUNT_WR STAT | MBOX3C1 | 81870940 | 0 | 81870940 | 1.023387e+07 |
| CAS_COUNT_RD STAT | MBOX4C0 | 122559500 | 0 | 122559500 | 1.531994e+07 |
| CAS_COUNT_WR STAT | MBOX4C1 | 81878940 | 0 | 81878940 | 1.023487e+07 |
| CAS_COUNT_RD STAT | MBOX5C0 | 0 | 0 | 0 | 0 |
| CAS_COUNT_WR STAT | MBOX5C1 | 0 | 0 | 0 | 0 |
+----------------------------+---------+-------------+------------+------------+--------------+

+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Metric | sal-nplnpl04:0:0 | sal-nplnpl04:1:1 | sal-nplnpl04:2:2 | sal-nplnpl04:3:3 | sal-nplnpl04:4:4 | sal-nplnpl04:5:5 | sal-nplnpl04:6:6 | sal-nplnpl04:7:7 |
+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+
| Runtime (RDTSC) [s] | 1.5596 | 1.5215 | 1.4993 | 1.5055 | 1.5071 | 1.5189 | 1.5478 | 1.3980 |
| Runtime unhalted [s] | 1.4438 | 1.3845 | 1.3743 | 1.3818 | 1.3826 | 1.3782 | 1.3839 | 1.3168 |
| Clock [MHz] | 2999.8672 | 2994.2252 | 2999.8304 | 2999.7234 | 2998.7720 | 2999.6444 | 2999.8515 | 2999.7399 |
| CPI | 0.7438 | 0.7133 | 0.7080 | 0.7119 | 0.7123 | 0.7100 | 0.7129 | 0.6784 |
| Memory read bandwidth [MBytes/s] | 20128.3045 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory read data volume [GBytes] | 31.3917 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory write bandwidth [MBytes/s] | 13440.6467 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory write data volume [GBytes] | 20.9617 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory bandwidth [MBytes/s] | 33568.9512 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Memory data volume [GBytes] | 52.3534 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-----------------------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+------------------+

+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+
| Metric | Sum | Min | Max | Avg | %ile 25 | %ile 50 | %ile 75 |
+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+
| Runtime (RDTSC) [s] STAT | 12.0577 | 1.3980 | 1.5596 | 1.5072 | 1.4993 | 1.5071 | 1.5215 |
| Runtime unhalted [s] STAT | 11.0459 | 1.3168 | 1.4438 | 1.3807 | 1.3743 | 1.3818 | 1.3839 |
| Clock [MHz] STAT | 23991.6540 | 2994.2252 | 2999.8672 | 2998.9568 | 2998.7720 | 2999.7234 | 2999.8304 |
| CPI STAT | 5.6906 | 0.6784 | 0.7438 | 0.7113 | 0.7080 | 0.7119 | 0.7129 |
| Memory read bandwidth [MBytes/s] STAT | 20128.3045 | 0 | 20128.3045 | 2516.0381 | 0 | 0 | 0 |
| Memory read data volume [GBytes] STAT | 31.3917 | 0 | 31.3917 | 3.9240 | 0 | 0 | 0 |
| Memory write bandwidth [MBytes/s] STAT | 13440.6467 | 0 | 13440.6467 | 1680.0808 | 0 | 0 | 0 |
| Memory write data volume [GBytes] STAT | 20.9617 | 0 | 20.9617 | 2.6202 | 0 | 0 | 0 |
| Memory bandwidth [MBytes/s] STAT | 33568.9512 | 0 | 33568.9512 | 4196.1189 | 0 | 0 | 0 |
| Memory data volume [GBytes] STAT | 52.3534 | 0 | 52.3534 | 6.5442 | 0 | 0 | 0 |
+----------------------------------------+------------+-----------+------------+-----------+-----------+-----------+-----------+

--

---
You received this message because you are subscribed to the Google Groups "likwid-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to likwid-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/likwid-users/23f0a34c-bd03-4e6c-b7ae-1456c7677677n%40googlegroups.com.

Thomas Gruber

unread,

Jun 9, 2023, 7:17:54 AM6/9/23

to likwid-users

Hi,

According to your text, the "main" region contains all other regions. The "stencil_driver" and "check_sum" regions are traversed 80K times. The outside markers contain all the operations within them, so the other region markers as well. Each marker causes reads (and potentially writes) to the hardware counters. While the access times to hwthread-local counters are "small", the access times to the memory controller counters are "slow". LIKWID does not apply any access optimizations yet but it's on the list. Each access causes system calls. While the user-space part of the system call (inside glibc) is counted, the kernel-space part is not counted. During the kernel time, the two unhalted cycle events do not increment while rdtsc keeps counting. By default, LIKWID configures the events to count only in user-space. For some counters, you can activate kernel counting with <event>:<counter>:KERNEL (see likwid-perfctr -e or the counter options & architecture documentation). System call times have increased dramatically in the last years due to more and more operations required to mitigate the CPU vulnerabilities (cache flushing, TLB flushing, ...). See https://blogs.fau.de/hager/archives/8189. Of course, if the hardware thread changes to a halting state like for I/O wait or barriers, the two unhalted cycle events are not incremented. RDTSC is not affected by halts, so it keeps counting.

Best,

Thomas

P.S. This is a lot of unreadable data. Please change the font to "Courier New" for command outputs or code snippets in future posts.

Reply all

Reply to author

Forward