Subject: Query regarding programmatic energy measurements and Intel RAPL availability

[Aidan Dakhama]

Hi all,

I am currently planning some experiments on CloudLab and need to collect power and energy consumption metrics for my workloads. I have a couple of questions regarding the supported energy measurement capabilities on the clusters:

External Energy Measurements: I understand that some of the clusters/machines have external power monitoring capabilities. I believe you can often see this on the graphs via the web interface. Is there a supported API, CLI tool, or programmatic method for users to fetch these external energy measurements during an experiment and if so where could I find the information on using this?

Intel RAPL Availability: I am also interested in using Intel RAPL. Can you confirm if RAPL is supported and enabled across Intel-based machines, and specifically on the m510 nodes? I primarily need to know if the necessary MSRs or perf_events are exposed to the OS/user space, or if they are disabled/restricted at the BIOS level.

Thank you for your time and help!

Best regards,

Aidan

[ajma...@gmail.com]

Hi Aidan,

External Energy Measurements:  At one point I believe we had a project running power monitoring at one of the clusters, but that project hasn't been active in over 6 years.  Unless I'm mistaken, right now we do not collect external power measurements at any of the CloudLab sites.  The experiment status pages just report load averages from the OS and network statistics from our switch port counters.  We also do not have a way for users to fetch such metrics themselves, as that information would come from servers' out of band management systems.  Depending on the scope/scale of what you need, we could potentially scrape these metrics for you over the course of an experiment and send them to you at the conclusion.

RAPL Availability:  I had to look into this more closely.  It looks like running `sudo rdmsr -X --bitfield 63:63 0x610` (following a `modprobe msr`) will print a `1` if RAPL is locked and `0` otherwise.  That command on both a m510 and a c6620 node printed `0`, so I think you're probably good to go?  If you have issues, then I can look more closely at the BIOS options.

Best,

 - Aleks

[Aidan Dakhama]

Hi Aleks,

Thanks again for checking the RAPL status previously.

We are preparing for our project and will definitely benefit from the external power measurements. Could you let us know which specific nodes/clusters have this available, and how we can arrange for that data to be collected and sent to us?

Additionally, we hope to evaluate our workloads across Intel, and perhaps also AMD, and ARM nodes. Since we will be extensively accessing Linux interfaces like perf_event_open and powercap sysfs, we need to know if the underlying hardware features are fully exposed and enabled at the BIOS/firmware level.

Specifically, can you confirm if the following are universally enabled on your bare-metal machines, or if there are any platform-wide restrictions?

• Intel: PMU, RAPL, PEBS, Processor Trace (PT), and LBR.

• AMD: Core/Data Fabric PMU, RAPL (via AMD MSRs), IBS, and LBR/BRS.

• ARM: PMUv3, AMU, SPE, and CoreSight/ETM.

• General: Any other BIOS/firmware features required to fully utilise hardware performance counters and power telemetry.

Thanks again for your time and support!

Best regards,

Aidan