Uncore counter support for Sapphire Rapids

147 views

Skip to first unread message

Knapp, Rashawn L

unread,

Aug 1, 2023, 12:37:10 PM8/1/23

to perfap...@icl.utk.edu, ptools-perfapi, Knapp, Rashawn L

Hello,

I have a couple questions regarding PAPI support for uncore counters for Sapphire Rapids.

Does PAPI support Sapphire Rapids uncore counters? If so, is this provided in the version of libpfm4 which ships with PAPI 7.0.1?

Kind Regards,

-Rashawn

Rashawn L. Knapp

Development Tools Software Engineer, Intel Corporation

Software and Advanced Technology Group

Parallel Runtimes Engineering DevOps for oneAPI

Nystrom, William David

unread,

Sep 4, 2024, 7:25:57 PM9/4/24

to Knapp, Rashawn L, perfap...@icl.utk.edu, ptools-perfapi, Nystrom, William David

I just built the master branch of PAPI on one of our Sapphire Rapids compute nodes but papi_native_avail does not seem to list any spr_unc_imc* counters. Is there something
special that I need to do to build PAPI with the spr_unc_imc* counter support?

Thanks,

Dave

________________________________________
From: ptools-...@icl.utk.edu <ptools-...@icl.utk.edu> on behalf of Knapp, Rashawn L <rashawn...@intel.com>
Sent: Tuesday, August 1, 2023 10:36 AM
To: perfap...@icl.utk.edu; ptools-perfapi
Cc: Knapp, Rashawn L
Subject: [EXTERNAL] [ptools-perfapi] Uncore counter support for Sapphire Rapids

--
You received this message because you are subscribed to the Google Groups "ptools-perfapi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ptools-perfap...@icl.utk.edu<mailto:ptools-perfap...@icl.utk.edu>.
To view this discussion on the web visit https://groups.google.com/a/icl.utk.edu/d/msgid/ptools-perfapi/SJ0PR11MB579279FA7888BC1A9988EAFCB70AA%40SJ0PR11MB5792.namprd11.prod.outlook.com<https://urldefense.com/v3/__https://groups.google.com/a/icl.utk.edu/d/msgid/ptools-perfapi/SJ0PR11MB579279FA7888BC1A9988EAFCB70AA*40SJ0PR11MB5792.namprd11.prod.outlook.com?utm_medium=email&utm_source=footer__;JQ!!Bt8fGhp8LhKGRg!CcoxYtf3Hnwkg2RCkwtANVW_2G0BbHZ2DZrmXFvq6YSTMrrd6WERYmw4iGot5Fl-YX-fML_zQ9iBEuSND-l-URQ$>.

Heike Jagode

unread,

Sep 4, 2024, 8:05:20 PM9/4/24

to Nystrom, William David, Knapp, Rashawn L, perfap...@icl.utk.edu, ptools-perfapi

To unsubscribe from this group and stop receiving emails from it, send an email to ptools-perfap...@icl.utk.edu.
To view this discussion on the web visit https://groups.google.com/a/icl.utk.edu/d/msgid/ptools-perfapi/SJ0PR09MB9651EAEB00F934BE50224FADD89C2%40SJ0PR09MB9651.namprd09.prod.outlook.com.

__________________________________________
Heike Jagode, Ph.D., Research Associate Professor

Innovative Computing Laboratory (ICL)

University of Tennessee Knoxville

http://icl.utk.edu/~jagode/

Nystrom, William David

unread,

Sep 4, 2024, 9:02:04 PM9/4/24

to Heike Jagode, Knapp, Rashawn L, perfap...@icl.utk.edu, ptools-perfapi, Nystrom, William David

Heike,

The setting for /proc/sys/kernel/perf_event_paranoid is "-1" which I believe is the
most permissive setting. I have accessed the uncore memory controller counters
on this same very heterogeneous cluster on our Cascade Lake and Skylake nodes
as a normal user. For this recent build, I was building PAPI with the GNU compiler
and running papi_native_avail both as a normal user. It has been a year or two
since I had built PAPI on this research cluster - so, I was wondering if somehow
something had changed such that I was not building PAPI correctly in order to get
the spr_unc_imc* support. So, at this point, I am wondering why I am having this
problem.

Thanks,

Dave

________________________________________
From: Heike Jagode <jag...@icl.utk.edu>
Sent: Wednesday, September 4, 2024 6:04 PM
To: Nystrom, William David
Cc: Knapp, Rashawn L; perfap...@icl.utk.edu; ptools-perfapi
Subject: [EXTERNAL] Re: [ptools-perfapi] Re: Uncore counter support for Sapphire Rapids

Dave,

These are uncore events, which require elevated privileges to access. Chances are that perf_event_uncore is not listed as an active component due to missing permissions when you call papi_component_avail. Could you please check?

Also, what is the paranoid flag set to on your machine?

If you get /proc/sys/kernel/perf_event_paranoid set to zero, you should be able to access these uncore events.

Thanks,
Heike

On Wed, Sep 4, 2024 at 7:25 PM 'Nystrom, William David' via ptools-perfapi <ptools-...@icl.utk.edu<mailto:ptools-...@icl.utk.edu>> wrote:
I just built the master branch of PAPI on one of our Sapphire Rapids compute nodes but papi_native_avail does not seem to list any spr_unc_imc* counters. Is there something
special that I need to do to build PAPI with the spr_unc_imc* counter support?

Thanks,

Dave

________________________________________

From: ptools-...@icl.utk.edu<mailto:ptools-...@icl.utk.edu> <ptools-...@icl.utk.edu<mailto:ptools-...@icl.utk.edu>> on behalf of Knapp, Rashawn L <rashawn...@intel.com<mailto:rashawn...@intel.com>>

Sent: Tuesday, August 1, 2023 10:36 AM

To: perfap...@icl.utk.edu<mailto:perfap...@icl.utk.edu>; ptools-perfapi

Cc: Knapp, Rashawn L
Subject: [EXTERNAL] [ptools-perfapi] Uncore counter support for Sapphire Rapids

Hello,

I have a couple questions regarding PAPI support for uncore counters for Sapphire Rapids.

Does PAPI support Sapphire Rapids uncore counters? If so, is this provided in the version of libpfm4 which ships with PAPI 7.0.1?

Kind Regards,

-Rashawn

Rashawn L. Knapp
Development Tools Software Engineer, Intel Corporation
Software and Advanced Technology Group
Parallel Runtimes Engineering DevOps for oneAPI

--
You received this message because you are subscribed to the Google Groups "ptools-perfapi" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ptools-perfap...@icl.utk.edu<mailto:ptools-perfapi%2Bunsu...@icl.utk.edu><mailto:ptools-perfap...@icl.utk.edu<mailto:ptools-perfapi%2Bunsu...@icl.utk.edu>>.
To view this discussion on the web visit https://groups.google.com/a/icl.utk.edu/d/msgid/ptools-perfapi/SJ0PR11MB579279FA7888BC1A9988EAFCB70AA%40SJ0PR11MB5792.namprd11.prod.outlook.com<https://urldefense.com/v3/__https://groups.google.com/a/icl.utk.edu/d/msgid/ptools-perfapi/SJ0PR11MB579279FA7888BC1A9988EAFCB70AA*40SJ0PR11MB5792.namprd11.prod.outlook.com__;JQ!!Bt8fGhp8LhKGRg!DVW7-pSgHJEIM4nelS031xJnxCtD56v_-vTI6xEG_gb4D1IjvTr_dpUY2VNiU0tgQomJTASi8DEgnYw$><https://urldefense.com/v3/__https://groups.google.com/a/icl.utk.edu/d/msgid/ptools-perfapi/SJ0PR11MB579279FA7888BC1A9988EAFCB70AA*40SJ0PR11MB5792.namprd11.prod.outlook.com?utm_medium=email&utm_source=footer__;JQ!!Bt8fGhp8LhKGRg!CcoxYtf3Hnwkg2RCkwtANVW_2G0BbHZ2DZrmXFvq6YSTMrrd6WERYmw4iGot5Fl-YX-fML_zQ9iBEuSND-l-URQ$>.

--
You received this message because you are subscribed to the Google Groups "ptools-perfapi" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ptools-perfap...@icl.utk.edu<mailto:ptools-perfapi%2Bunsu...@icl.utk.edu>.
To view this discussion on the web visit https://groups.google.com/a/icl.utk.edu/d/msgid/ptools-perfapi/SJ0PR09MB9651EAEB00F934BE50224FADD89C2%40SJ0PR09MB9651.namprd09.prod.outlook.com<https://urldefense.com/v3/__https://groups.google.com/a/icl.utk.edu/d/msgid/ptools-perfapi/SJ0PR09MB9651EAEB00F934BE50224FADD89C2*40SJ0PR09MB9651.namprd09.prod.outlook.com__;JQ!!Bt8fGhp8LhKGRg!DVW7-pSgHJEIM4nelS031xJnxCtD56v_-vTI6xEG_gb4D1IjvTr_dpUY2VNiU0tgQomJTASiLnjSDcw$>.

--
__________________________________________
Heike Jagode, Ph.D., Research Associate Professor
Innovative Computing Laboratory (ICL)
University of Tennessee Knoxville

http://icl.utk.edu/~jagode/<https://urldefense.com/v3/__http://icl.utk.edu/*jagode/__;fg!!Bt8fGhp8LhKGRg!DVW7-pSgHJEIM4nelS031xJnxCtD56v_-vTI6xEG_gb4D1IjvTr_dpUY2VNiU0tgQomJTASi77Rs5M0$>

Heike Jagode

unread,

Sep 4, 2024, 9:12:05 PM9/4/24

to Nystrom, William David, Knapp, Rashawn L, perfap...@icl.utk.edu, ptools-perfapi

Again, what does your papi_component_avail output look like?

With the latest version, you should see something like the following on Sapphire Rapids:

$: bin>./papi_component_avail
Available components and hardware information.
--------------------------------------------------------------------------------
PAPI version : 7.2.0.0
Operating system : Linux 4.18.0-477.27.1.el8_8.x86_64
Vendor string and code : GenuineIntel (1, 0x1)
Model string and code : Intel(R) Xeon(R) Gold 6430 (143, 0x8f)
CPU revision : 8.000000
CPUID : Family/Model/Stepping 6/143/8, 0x06/0x8f/0x08
CPU Max MHz : 3400
CPU Min MHz : 800
Total cores : 64
SMT threads per core : 1
Cores per socket : 32
Sockets : 2
Cores per NUMA region : 8
NUMA regions : 8
Running in a VM : no
Number Hardware Counters : 19
Max Multiplex Counters : 384
Fast counter read (rdpmc): yes
--------------------------------------------------------------------------------

Compiled-in components:
Name: perf_event Linux perf_event CPU counters
Name: perf_event_uncore Linux perf_event CPU uncore and northbridge
Name: sysdetect System info detection component

Active components:
Name: perf_event Linux perf_event CPU counters
Native: 167, Preset: 42, Counters: 19
PMUs supported: ix86arch, perf, perf_raw, spr

Name: perf_event_uncore Linux perf_event CPU uncore and northbridge
Native: 3754, Preset: 0, Counters: 287
PMUs supported: rapl, spr_unc_imc0, spr_unc_imc1, spr_unc_imc2, spr_unc_imc3, spr_unc_imc4
spr_unc_imc5, spr_unc_imc6, spr_unc_imc7, spr_unc_upi0, spr_unc_upi1, spr_unc_upi2
spr_unc_cha0, spr_unc_cha1, spr_unc_cha2, spr_unc_cha3, spr_unc_cha4, spr_unc_cha5
spr_unc_cha6, spr_unc_cha7, spr_unc_cha8, spr_unc_cha9, spr_unc_cha10
spr_unc_cha11, spr_unc_cha12, spr_unc_cha13, spr_unc_cha14, spr_unc_cha15
spr_unc_cha16, spr_unc_cha17, spr_unc_cha18, spr_unc_cha19, spr_unc_cha20
spr_unc_cha21, spr_unc_cha22, spr_unc_cha23, spr_unc_cha24, spr_unc_cha25
spr_unc_cha26, spr_unc_cha27, spr_unc_cha28, spr_unc_cha29, spr_unc_cha30
spr_unc_cha31, spr_unc_cha32, spr_unc_cha33, spr_unc_cha34, spr_unc_cha35
spr_unc_cha36, spr_unc_cha37, spr_unc_cha38, spr_unc_cha39, spr_unc_cha40
spr_unc_cha41, spr_unc_cha42, spr_unc_cha43, spr_unc_cha44, spr_unc_cha45
spr_unc_cha46, spr_unc_cha47, spr_unc_cha48, spr_unc_cha49, spr_unc_cha50
spr_unc_cha51, spr_unc_cha52, spr_unc_cha53, spr_unc_cha54, spr_unc_cha55
spr_unc_cha56, spr_unc_cha57, spr_unc_cha58, spr_unc_cha59

Name: sysdetect System info detection component
Native: 0, Preset: 0, Counters: 0

Thanks,

Heike

Nystrom, William David

unread,

Sep 4, 2024, 10:52:02 PM9/4/24

to Heike Jagode, Knapp, Rashawn L, perfap...@icl.utk.edu, ptools-perfapi, Nystrom, William David

Heike,

Sorry. I misread your previous email and did not realize that you were asking for output
from papi_component_avail. I mentally inserted papi_native_avail. Here is the output you
asked for.

% papi_component_avail

Available components and hardware information.
--------------------------------------------------------------------------------
PAPI version : 7.2.0.0

Operating system : Linux 4.18.0-553.8.1.el8_10.x86_64

Vendor string and code : GenuineIntel (1, 0x1)

Model string and code : Intel(R) Xeon(R) CPU Max 9480 (143, 0x8f)

CPU revision : 8.000000
CPUID : Family/Model/Stepping 6/143/8, 0x06/0x8f/0x08

CPU Max MHz : 3500
CPU Min MHz : 800
Total cores : 224
SMT threads per core : 2
Cores per socket : 56
Sockets : 2
Cores per NUMA region : 28

NUMA regions : 8
Running in a VM : no
Number Hardware Counters : 19
Max Multiplex Counters : 384
Fast counter read (rdpmc): yes
--------------------------------------------------------------------------------

Compiled-in components:
Name: perf_event Linux perf_event CPU counters
Name: perf_event_uncore Linux perf_event CPU uncore and northbridge
Name: sysdetect System info detection component

Active components:
Name: perf_event Linux perf_event CPU counters
Native: 167, Preset: 42, Counters: 19
PMUs supported: ix86arch, perf, perf_raw, spr

Name: perf_event_uncore Linux perf_event CPU uncore and northbridge

Native: 3446, Preset: 0, Counters: 259
PMUs supported: rapl, spr_unc_upi0, spr_unc_upi1, spr_unc_upi2, spr_unc_upi3, spr_unc_cha0
spr_unc_cha1, spr_unc_cha2, spr_unc_cha3, spr_unc_cha4, spr_unc_cha5, spr_unc_cha6
spr_unc_cha7, spr_unc_cha8, spr_unc_cha9, spr_unc_cha10, spr_unc_cha11
spr_unc_cha12, spr_unc_cha13, spr_unc_cha14, spr_unc_cha15, spr_unc_cha16
spr_unc_cha17, spr_unc_cha18, spr_unc_cha19, spr_unc_cha20, spr_unc_cha21
spr_unc_cha22, spr_unc_cha23, spr_unc_cha24, spr_unc_cha25, spr_unc_cha26
spr_unc_cha27, spr_unc_cha28, spr_unc_cha29, spr_unc_cha30, spr_unc_cha31
spr_unc_cha32, spr_unc_cha33, spr_unc_cha34, spr_unc_cha35, spr_unc_cha36
spr_unc_cha37, spr_unc_cha38, spr_unc_cha39, spr_unc_cha40, spr_unc_cha41
spr_unc_cha42, spr_unc_cha43, spr_unc_cha44, spr_unc_cha45, spr_unc_cha46
spr_unc_cha47, spr_unc_cha48, spr_unc_cha49, spr_unc_cha50, spr_unc_cha51
spr_unc_cha52, spr_unc_cha53, spr_unc_cha54, spr_unc_cha55, spr_unc_cha56

spr_unc_cha57, spr_unc_cha58, spr_unc_cha59

Name: sysdetect System info detection component
Native: 0, Preset: 0, Counters: 0

--------------------------------------------------------------------------------

I also think I see what the problem is. I was building and running on an HBM only node. Here at
LANL, our SPR nodes are either DDR only or HBM only with the latter being more interesting to
understand performance of memory bandwidth utilization for. If I run papi_component_avail on a
DDR only node, this is the output that I get.

% papi_component_avail

Available components and hardware information.
--------------------------------------------------------------------------------
PAPI version : 7.2.0.0

Operating system : Linux 4.18.0-553.8.1.el8_10.x86_64

Vendor string and code : GenuineIntel (1, 0x1)

Model string and code : Intel(R) Xeon(R) Platinum 8480L (143, 0x8f)
CPU revision : 7.000000
CPUID : Family/Model/Stepping 6/143/7, 0x06/0x8f/0x07
CPU Max MHz : 3800
CPU Min MHz : 800
Total cores : 224
SMT threads per core : 2
Cores per socket : 56
Sockets : 2
Cores per NUMA region : 28

Native: 3790, Preset: 0, Counters: 291

PMUs supported: rapl, spr_unc_imc0, spr_unc_imc1, spr_unc_imc2, spr_unc_imc3, spr_unc_imc4
spr_unc_imc5, spr_unc_imc6, spr_unc_imc7, spr_unc_upi0, spr_unc_upi1, spr_unc_upi2

spr_unc_upi3, spr_unc_cha0, spr_unc_cha1, spr_unc_cha2, spr_unc_cha3, spr_unc_cha4
spr_unc_cha5, spr_unc_cha6, spr_unc_cha7, spr_unc_cha8, spr_unc_cha9
spr_unc_cha10, spr_unc_cha11, spr_unc_cha12, spr_unc_cha13, spr_unc_cha14
spr_unc_cha15, spr_unc_cha16, spr_unc_cha17, spr_unc_cha18, spr_unc_cha19
spr_unc_cha20, spr_unc_cha21, spr_unc_cha22, spr_unc_cha23, spr_unc_cha24
spr_unc_cha25, spr_unc_cha26, spr_unc_cha27, spr_unc_cha28, spr_unc_cha29
spr_unc_cha30, spr_unc_cha31, spr_unc_cha32, spr_unc_cha33, spr_unc_cha34
spr_unc_cha35, spr_unc_cha36, spr_unc_cha37, spr_unc_cha38, spr_unc_cha39
spr_unc_cha40, spr_unc_cha41, spr_unc_cha42, spr_unc_cha43, spr_unc_cha44
spr_unc_cha45, spr_unc_cha46, spr_unc_cha47, spr_unc_cha48, spr_unc_cha49
spr_unc_cha50, spr_unc_cha51, spr_unc_cha52, spr_unc_cha53, spr_unc_cha54
spr_unc_cha55, spr_unc_cha56, spr_unc_cha57, spr_unc_cha58, spr_unc_cha59

Name: sysdetect System info detection component
Native: 0, Preset: 0, Counters: 0

--------------------------------------------------------------------------------

From this, I can see that the DDR only node reports availability of the spr_unc_imc* counters.

So, does this mean that PAPI does not support measuring memory controller counters for
HBM? Or does it mean that perhaps the Linux kernel on the nodes is not recent enough.

This is the output of "uname -a" on the node.

% uname -a
Linux cn895 4.18.0-553.8.1.el8_10.x86_64 #1 SMP Fri Jun 14 03:19:37 EDT 2024 x86_64 x86_64 x86_64 GNU/Linux

I believe we are running some version of RHEL8 on our SPR nodes for this cluster.

Thanks,

Dave

________________________________________
From: Heike Jagode <jag...@icl.utk.edu>

Sent: Wednesday, September 4, 2024 7:11 PM

To: Nystrom, William David
Cc: Knapp, Rashawn L; perfap...@icl.utk.edu; ptools-perfapi

Subject: Re: [EXTERNAL] Re: [ptools-perfapi] Re: Uncore counter support for Sapphire Rapids

Again, what does your papi_component_avail output look like?

With the latest version, you should see something like the following on Sapphire Rapids:

$: bin>./papi_component_avail
Available components and hardware information.
--------------------------------------------------------------------------------

PAPI version : 7.2.0.0<https://urldefense.com/v3/__http://7.2.0.0__;!!Bt8fGhp8LhKGRg!Eafy9bZcYl2wenkaHQ10wkzpgHooV9uwUpWD_NlZqrB38yy6bvuVNs3grMAuqxUfQp0RLKWlK0hIudQ$>

On Wed, Sep 4, 2024 at 9:02 PM Nystrom, William David <w...@lanl.gov<mailto:w...@lanl.gov>> wrote:
Heike,

The setting for /proc/sys/kernel/perf_event_paranoid is "-1" which I believe is the
most permissive setting. I have accessed the uncore memory controller counters
on this same very heterogeneous cluster on our Cascade Lake and Skylake nodes
as a normal user. For this recent build, I was building PAPI with the GNU compiler
and running papi_native_avail both as a normal user. It has been a year or two
since I had built PAPI on this research cluster - so, I was wondering if somehow
something had changed such that I was not building PAPI correctly in order to get
the spr_unc_imc* support. So, at this point, I am wondering why I am having this
problem.

Thanks,

Dave

________________________________________

From: Heike Jagode <jag...@icl.utk.edu<mailto:jag...@icl.utk.edu>>

Sent: Wednesday, September 4, 2024 6:04 PM
To: Nystrom, William David

Cc: Knapp, Rashawn L; perfap...@icl.utk.edu<mailto:perfap...@icl.utk.edu>; ptools-perfapi

Subject: [EXTERNAL] Re: [ptools-perfapi] Re: Uncore counter support for Sapphire Rapids

Dave,

These are uncore events, which require elevated privileges to access. Chances are that perf_event_uncore is not listed as an active component due to missing permissions when you call papi_component_avail. Could you please check?

Also, what is the paranoid flag set to on your machine?

If you get /proc/sys/kernel/perf_event_paranoid set to zero, you should be able to access these uncore events.

Thanks,
Heike

On Wed, Sep 4, 2024 at 7:25 PM 'Nystrom, William David' via ptools-perfapi <ptools-...@icl.utk.edu<mailto:ptools-...@icl.utk.edu><mailto:ptools-...@icl.utk.edu<mailto:ptools-...@icl.utk.edu>>> wrote:
I just built the master branch of PAPI on one of our Sapphire Rapids compute nodes but papi_native_avail does not seem to list any spr_unc_imc* counters. Is there something
special that I need to do to build PAPI with the spr_unc_imc* counter support?

Thanks,

Dave

________________________________________

From: ptools-...@icl.utk.edu<mailto:ptools-...@icl.utk.edu><mailto:ptools-...@icl.utk.edu<mailto:ptools-...@icl.utk.edu>> <ptools-...@icl.utk.edu<mailto:ptools-...@icl.utk.edu><mailto:ptools-...@icl.utk.edu<mailto:ptools-...@icl.utk.edu>>> on behalf of Knapp, Rashawn L <rashawn...@intel.com<mailto:rashawn...@intel.com><mailto:rashawn...@intel.com<mailto:rashawn...@intel.com>>>

Sent: Tuesday, August 1, 2023 10:36 AM

To: perfap...@icl.utk.edu<mailto:perfap...@icl.utk.edu><mailto:perfap...@icl.utk.edu<mailto:perfap...@icl.utk.edu>>; ptools-perfapi

To unsubscribe from this group and stop receiving emails from it, send an email to ptools-perfap...@icl.utk.edu<mailto:ptools-perfapi%2Bunsu...@icl.utk.edu><mailto:ptools-perfapi%2Bunsu...@icl.utk.edu<mailto:ptools-perfapi%252Buns...@icl.utk.edu>><mailto:ptools-perfap...@icl.utk.edu<mailto:ptools-perfapi%2Bunsu...@icl.utk.edu><mailto:ptools-perfapi%2Bunsu...@icl.utk.edu<mailto:ptools-perfapi%252Buns...@icl.utk.edu>>>.
To view this discussion on the web visit https://groups.google.com/a/icl.utk.edu/d/msgid/ptools-perfapi/SJ0PR11MB579279FA7888BC1A9988EAFCB70AA%40SJ0PR11MB5792.namprd11.prod.outlook.com<https://urldefense.com/v3/__https://groups.google.com/a/icl.utk.edu/d/msgid/ptools-perfapi/SJ0PR11MB579279FA7888BC1A9988EAFCB70AA*40SJ0PR11MB5792.namprd11.prod.outlook.com__;JQ!!Bt8fGhp8LhKGRg!Eafy9bZcYl2wenkaHQ10wkzpgHooV9uwUpWD_NlZqrB38yy6bvuVNs3grMAuqxUfQp0RLKWlrDxDLkU$><https://urldefense.com/v3/__https://groups.google.com/a/icl.utk.edu/d/msgid/ptools-perfapi/SJ0PR11MB579279FA7888BC1A9988EAFCB70AA*40SJ0PR11MB5792.namprd11.prod.outlook.com__;JQ!!Bt8fGhp8LhKGRg!DVW7-pSgHJEIM4nelS031xJnxCtD56v_-vTI6xEG_gb4D1IjvTr_dpUY2VNiU0tgQomJTASi8DEgnYw$><https://urldefense.com/v3/__https://groups.google.com/a/icl.utk.edu/d/msgid/ptools-perfapi/SJ0PR11MB579279FA7888BC1A9988EAFCB70AA*40SJ0PR11MB5792.namprd11.prod.outlook.com?utm_medium=email&utm_source=footer__;JQ!!Bt8fGhp8LhKGRg!CcoxYtf3Hnwkg2RCkwtANVW_2G0BbHZ2DZrmXFvq6YSTMrrd6WERYmw4iGot5Fl-YX-fML_zQ9iBEuSND-l-URQ$>.

--
You received this message because you are subscribed to the Google Groups "ptools-perfapi" group.

--
__________________________________________
Heike Jagode, Ph.D., Research Associate Professor
Innovative Computing Laboratory (ICL)
University of Tennessee Knoxville

http://icl.utk.edu/~jagode/<https://urldefense.com/v3/__http://icl.utk.edu/*jagode/__;fg!!Bt8fGhp8LhKGRg!Eafy9bZcYl2wenkaHQ10wkzpgHooV9uwUpWD_NlZqrB38yy6bvuVNs3grMAuqxUfQp0RLKWlRGOfLV0$><https://urldefense.com/v3/__http://icl.utk.edu/*jagode/__;fg!!Bt8fGhp8LhKGRg!DVW7-pSgHJEIM4nelS031xJnxCtD56v_-vTI6xEG_gb4D1IjvTr_dpUY2VNiU0tgQomJTASi77Rs5M0$>

--
__________________________________________
Heike Jagode, Ph.D., Research Associate Professor
Innovative Computing Laboratory (ICL)
University of Tennessee Knoxville

http://icl.utk.edu/~jagode/<https://urldefense.com/v3/__http://icl.utk.edu/*jagode/__;fg!!Bt8fGhp8LhKGRg!Eafy9bZcYl2wenkaHQ10wkzpgHooV9uwUpWD_NlZqrB38yy6bvuVNs3grMAuqxUfQp0RLKWlRGOfLV0$>

Thomas Roehl

unread,

Sep 5, 2024, 5:55:22 AM9/5/24

to Nystrom, William David, Heike Jagode, Knapp, Rashawn L, perfap...@icl.utk.edu, ptools-perfapi

Hi @ all,

As far as I can see in its repo, libpfm4 does not have support for the
HBM units of SPR yet. But even if, there is probably the same issue we
had with LIKWID on these SPR HBM only nodes at LANL. The RHEL Linux
kernel 4.18 publishes these HBM units as 'uncore_type_14_*' while
LIKWID [1] and libpfm4 [2] commonly use the actual names like in this
case 'uncore_hbm_*'.

Best,
Thomas

[1]
https://github.com/RRZE-HPC/likwid/blob/master/src/includes/perfmon_sapphirerapids_counters.h#L1488
[2] unit name registered for SPR iMC:
https://sourceforge.net/p/perfmon2/libpfm4/ci/master/tree/lib/pfmlib_intel_spr_unc_imc.c#l58

Am 05.09.24 um 04:51 schrieb 'Nystrom, William David' via ptools-perfapi:

Nystrom, William David

unread,

Sep 5, 2024, 11:13:48 AM9/5/24

to Thomas Roehl, Heike Jagode, Knapp, Rashawn L, perfap...@icl.utk.edu, ptools-perfapi, Nystrom, William David

Hi Tom,

Thanks for chiming in. I suspect that is the underlying problem with why the support
is not showing up for the HBM uncore memory controller counters. Not sure what the
resolution is but it would be nice to somehow get the problem fixed in a way that was
acceptable to the performance tool community so we could measure HBM memory
counters on our large HBM only cluster here at LANL.

Thanks,

Dave

________________________________________
From: Thomas Roehl <thomas...@googlemail.com>
Sent: Thursday, September 5, 2024 3:55 AM
To: Nystrom, William David; Heike Jagode

Cc: Knapp, Rashawn L; perfap...@icl.utk.edu; ptools-perfapi
Subject: Re: [EXTERNAL] Re: [ptools-perfapi] Re: Uncore counter support for Sapphire Rapids

https://urldefense.com/v3/__https://github.com/RRZE-HPC/likwid/blob/master/src/includes/perfmon_sapphirerapids_counters.h*L1488__;Iw!!Bt8fGhp8LhKGRg!BWsPl7qGBWtRJEOZJ2aXEA18tcMX16xOiArkAsc8uTNprAIjzLjV1J8-rAK5m0z8pHFCTBw77eB9KP3ngLytVVc$

[2] unit name registered for SPR iMC:

https://urldefense.com/v3/__https://sourceforge.net/p/perfmon2/libpfm4/ci/master/tree/lib/pfmlib_intel_spr_unc_imc.c*l58__;Iw!!Bt8fGhp8LhKGRg!BWsPl7qGBWtRJEOZJ2aXEA18tcMX16xOiArkAsc8uTNprAIjzLjV1J8-rAK5m0z8pHFCTBw77eB9KP3nhnBaizo$

Am 05.09.24 um 04:51 schrieb 'Nystrom, William David' via ptools-perfapi:
> Heike,
>
> Sorry. I misread your previous email and did not realize that you were asking for output
> from papi_component_avail. I mentally inserted papi_native_avail. Here is the output you
> asked for.
>
> % papi_component_avail
> Available components and hardware information.
> --------------------------------------------------------------------------------

> PAPI version : https://urldefense.com/v3/__http://7.2.0.0__;!!Bt8fGhp8LhKGRg!BWsPl7qGBWtRJEOZJ2aXEA18tcMX16xOiArkAsc8uTNprAIjzLjV1J8-rAK5m0z8pHFCTBw77eB9KP3nti3IZv4$

> PAPI version : https://urldefense.com/v3/__http://7.2.0.0__;!!Bt8fGhp8LhKGRg!BWsPl7qGBWtRJEOZJ2aXEA18tcMX16xOiArkAsc8uTNprAIjzLjV1J8-rAK5m0z8pHFCTBw77eB9KP3nti3IZv4$ <https://urldefense.com/v3/__http://7.2.0.0__;!!Bt8fGhp8LhKGRg!Eafy9bZcYl2wenkaHQ10wkzpgHooV9uwUpWD_NlZqrB38yy6bvuVNs3grMAuqxUfQp0RLKWlK0hIudQ$>

> To view this discussion on the web visit https://urldefense.com/v3/__https://groups.google.com/a/icl.utk.edu/d/msgid/ptools-perfapi/SJ0PR11MB579279FA7888BC1A9988EAFCB70AA*40SJ0PR11MB5792.namprd11.prod.outlook.com__;JQ!!Bt8fGhp8LhKGRg!BWsPl7qGBWtRJEOZJ2aXEA18tcMX16xOiArkAsc8uTNprAIjzLjV1J8-rAK5m0z8pHFCTBw77eB9KP3nL0Orw2Y$ <https://urldefense.com/v3/__https://groups.google.com/a/icl.utk.edu/d/msgid/ptools-perfapi/SJ0PR11MB579279FA7888BC1A9988EAFCB70AA*40SJ0PR11MB5792.namprd11.prod.outlook.com__;JQ!!Bt8fGhp8LhKGRg!Eafy9bZcYl2wenkaHQ10wkzpgHooV9uwUpWD_NlZqrB38yy6bvuVNs3grMAuqxUfQp0RLKWlrDxDLkU$><https://urldefense.com/v3/__https://groups.google.com/a/icl.utk.edu/d/msgid/ptools-perfapi/SJ0PR11MB579279FA7888BC1A9988EAFCB70AA*40SJ0PR11MB5792.namprd11.prod.outlook.com__;JQ!!Bt8fGhp8LhKGRg!DVW7-pSgHJEIM4nelS031xJnxCtD56v_-vTI6xEG_gb4D1IjvTr_dpUY2VNiU0tgQomJTASi8DEgnYw$><https://urldefense.com/v3/__https://groups.google.com/a/icl.utk.edu/d/msgid/ptools-perfapi/SJ0PR11MB579279FA7888BC1A9988EAFCB70AA*40SJ0PR11MB5792.namprd11.prod.outlook.com?utm_medium=email&utm_source=footer__;JQ!!Bt8fGhp8LhKGRg!CcoxYtf3Hnwkg2RCkwtANVW_2G0BbHZ2DZrmXFvq6YSTMrrd6WERYmw4iGot5Fl-YX-fML_zQ9iBEuSND-l-URQ$>.

>
> --
> You received this message because you are subscribed to the Google Groups "ptools-perfapi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to ptools-perfap...@icl.utk.edu<mailto:ptools-perfapi%2Bunsu...@icl.utk.edu><mailto:ptools-perfapi%2Bunsu...@icl.utk.edu<mailto:ptools-perfapi%252Buns...@icl.utk.edu>>.

> To view this discussion on the web visit https://urldefense.com/v3/__https://groups.google.com/a/icl.utk.edu/d/msgid/ptools-perfapi/SJ0PR09MB9651EAEB00F934BE50224FADD89C2*40SJ0PR09MB9651.namprd09.prod.outlook.com__;JQ!!Bt8fGhp8LhKGRg!BWsPl7qGBWtRJEOZJ2aXEA18tcMX16xOiArkAsc8uTNprAIjzLjV1J8-rAK5m0z8pHFCTBw77eB9KP3nsrPRm1U$ <https://urldefense.com/v3/__https://groups.google.com/a/icl.utk.edu/d/msgid/ptools-perfapi/SJ0PR09MB9651EAEB00F934BE50224FADD89C2*40SJ0PR09MB9651.namprd09.prod.outlook.com__;JQ!!Bt8fGhp8LhKGRg!Eafy9bZcYl2wenkaHQ10wkzpgHooV9uwUpWD_NlZqrB38yy6bvuVNs3grMAuqxUfQp0RLKWlbCsfh2w$><https://urldefense.com/v3/__https://groups.google.com/a/icl.utk.edu/d/msgid/ptools-perfapi/SJ0PR09MB9651EAEB00F934BE50224FADD89C2*40SJ0PR09MB9651.namprd09.prod.outlook.com__;JQ!!Bt8fGhp8LhKGRg!DVW7-pSgHJEIM4nelS031xJnxCtD56v_-vTI6xEG_gb4D1IjvTr_dpUY2VNiU0tgQomJTASiLnjSDcw$>.

>
>
> --
> __________________________________________
> Heike Jagode, Ph.D., Research Associate Professor
> Innovative Computing Laboratory (ICL)
> University of Tennessee Knoxville

> https://urldefense.com/v3/__http://icl.utk.edu/*jagode/__;fg!!Bt8fGhp8LhKGRg!BWsPl7qGBWtRJEOZJ2aXEA18tcMX16xOiArkAsc8uTNprAIjzLjV1J8-rAK5m0z8pHFCTBw77eB9KP3npGHCN3o$ <https://urldefense.com/v3/__http://icl.utk.edu/*jagode/__;fg!!Bt8fGhp8LhKGRg!Eafy9bZcYl2wenkaHQ10wkzpgHooV9uwUpWD_NlZqrB38yy6bvuVNs3grMAuqxUfQp0RLKWlRGOfLV0$><https://urldefense.com/v3/__http://icl.utk.edu/*jagode/__;fg!!Bt8fGhp8LhKGRg!DVW7-pSgHJEIM4nelS031xJnxCtD56v_-vTI6xEG_gb4D1IjvTr_dpUY2VNiU0tgQomJTASi77Rs5M0$>

>
>
> --
> __________________________________________
> Heike Jagode, Ph.D., Research Associate Professor
> Innovative Computing Laboratory (ICL)
> University of Tennessee Knoxville

Reply all

Reply to author

Forward

0 new messages