alternatives to loading msr via modprobe ?

[Sajid]

Hi,

As per the docs, I loaded msr as root but likwid still doesn’t have access to msr.

[sajid@xrmlite ~]$ sudo -s
[root@xrmlite sajid]# modprobe msr
[root@xrmlite sajid]# exit
[sajid@xrmlite ~]$ likwid-perfctr -g FLOPS_AVX -C 3 -S 20ms
--------------------------------------------------------------------------------
CPU name:       Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
CPU type:       Intel Xeon Broadwell EN/EP/EX processor
CPU clock:      2.10 GHz
Cannot get access to MSRs. Please check permissions to the MSRs

Is there an alternative ?
Thank You.

​

[Thomas Röhl]

Can you please supply more information:

Operating system including version

likwid version

have you changed anything in the build configuration of likwid? Accessmode?

output of ls -la /dev/cpu/*/msr

security solution (apparmor, ..) active?

The alternative is to use the perf_event backend. ACCESSMODE=perf_event, in config.mk make distclean, make, sudo make install and ensure that /proc/sys/kernel/perf_event_paranoid is 0 (for Uncore measurements) or 1 (for only core-local measurements)

Best,
Thomas

[Sajid]

OS : CentOS 7.4

likwid version : 4.3.2 

Build recipe: used spack package manager
[sajid@xrmlite ~]$ spack find -ldf likwid
==> 1 installed packages.
-- linux-centos7-x86_64 / g...@7.3.0 -----------------------------
r76dngv    lik...@4.3.2%gcc cflags="-O3 -mavx2" cxxflags="-O3 -mavx2"
zmy36jg        ^l...@5.3.4%gcc cflags="-O3 -mavx2" cxxflags="-O3 -mavx2"
7zig44d            ^ncurses@6.1%gcc cflags="-O3 -mavx2" cxxflags="-O3 -mavx2"
mcblpx4            ^readline@7.0%gcc cflags="-O3 -mavx2" cxxflags="-O3 -mavx2"
tdjcr4u        ^pe...@5.26.2%gcc cflags="-O3 -mavx2" cxxflags="-O3 -mavx2"
jim3cxh            ^gd...@1.14.1%gcc cflags="-O3 -mavx2" cxxflags="-O3 -mavx2"

as requested  : 
[sajid@xrmlite ~]$  ls -la /dev/cpu/*/msr                                       
crw-------. 1 root root 202,  0 Jul 23 15:14 /dev/cpu/0/msr                     
crw-------. 1 root root 202,  1 Jul 23 15:14 /dev/cpu/1/msr                     
crw-------. 1 root root 202, 10 Jul 23 15:14 /dev/cpu/10/msr                    
crw-------. 1 root root 202, 11 Jul 23 15:14 /dev/cpu/11/msr                    
crw-------. 1 root root 202, 12 Jul 23 15:14 /dev/cpu/12/msr                    
crw-------. 1 root root 202, 13 Jul 23 15:14 /dev/cpu/13/msr                    
crw-------. 1 root root 202, 14 Jul 23 15:14 /dev/cpu/14/msr                    
crw-------. 1 root root 202, 15 Jul 23 15:14 /dev/cpu/15/msr                    
crw-------. 1 root root 202, 16 Jul 23 15:14 /dev/cpu/16/msr                    
crw-------. 1 root root 202, 17 Jul 23 15:14 /dev/cpu/17/msr                    
crw-------. 1 root root 202, 18 Jul 23 15:14 /dev/cpu/18/msr                    
crw-------. 1 root root 202, 19 Jul 23 15:14 /dev/cpu/19/msr                    
crw-------. 1 root root 202,  2 Jul 23 15:14 /dev/cpu/2/msr                     
crw-------. 1 root root 202, 20 Jul 23 15:14 /dev/cpu/20/msr                    
crw-------. 1 root root 202, 21 Jul 23 15:14 /dev/cpu/21/msr                    
crw-------. 1 root root 202, 22 Jul 23 15:14 /dev/cpu/22/msr                    
crw-------. 1 root root 202, 23 Jul 23 15:14 /dev/cpu/23/msr                    
crw-------. 1 root root 202, 24 Jul 23 15:14 /dev/cpu/24/msr                    
crw-------. 1 root root 202, 25 Jul 23 15:14 /dev/cpu/25/msr                    
crw-------. 1 root root 202, 26 Jul 23 15:14 /dev/cpu/26/msr                    
crw-------. 1 root root 202, 27 Jul 23 15:14 /dev/cpu/27/msr                    
crw-------. 1 root root 202, 28 Jul 23 15:14 /dev/cpu/28/msr                    
crw-------. 1 root root 202, 29 Jul 23 15:14 /dev/cpu/29/msr                    
crw-------. 1 root root 202,  3 Jul 23 15:14 /dev/cpu/3/msr                     
crw-------. 1 root root 202, 30 Jul 23 15:14 /dev/cpu/30/msr                    
crw-------. 1 root root 202, 31 Jul 23 15:14 /dev/cpu/31/msr                    
crw-------. 1 root root 202,  4 Jul 23 15:14 /dev/cpu/4/msr                     
crw-------. 1 root root 202,  5 Jul 23 15:14 /dev/cpu/5/msr                     
crw-------. 1 root root 202,  6 Jul 23 15:14 /dev/cpu/6/msr                     
crw-------. 1 root root 202,  7 Jul 23 15:14 /dev/cpu/7/msr                     
crw-------. 1 root root 202,  8 Jul 23 15:14 /dev/cpu/8/msr                     
crw-------. 1 root root 202,  9 Jul 23 15:14 /dev/cpu/9/msr                     

No apparmor as far as I can  see

​

[Thomas Röhl]

I'm not the maintainer of the LIKWID spack package but I checked the package.py at https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/likwid/package.py and it seem like it installs the package with your username (see INSTALL_CHOWN). But that's wrong, the username should be a root user. Access to the registers is only allowed as root and LIKWID uses an access daemon to allow users to access the registers. LIKWID's default setting is -g root -o root

You can change the user of likwid-accessD and likwid-setFreq to root (chown -g root -o root <PATH_TO_LIKWID>/sbin/likwid-*) and give it the suid bit again (is deleted at chown, chmod u+s
<PATH_TO_LIKWID>/sbin/likwid-*)

I'll create a pull request at spack although spack packages should be installable as user and this is not possible with LIKWID with access daemon.

If you change ACCESSMODE=perf_event in config.mk you can install it as user. With perf_event it depends on the /proc/sys/kernel/perf_event_paranoid setting how much LIKWID can access. 1 means only core-local counters, 0 allows everything. Some features are not supported by perf_event like reading the turbo frequencies, changing CPU and Uncore frequencies, ... anything where it needs access to registers that are not provided by the perf_event interface.

Best,

Thomas

[Sajid]

Thank you! I'll just build likwid as usual since I do have root on the system.

[Thomas Röhl]

I'm glad it works now. I added a pull request to spack.