Prometheus RHEL 7.4 Node exporter Issue - scaling_cur_freq not found

[dst]

Hello guys,

  I am facing this issue while running node exporter in RHEL 7.4

Error:

"ERRO[0953] ERROR: cpu collector failed after 0.014791s: open /sys/bus/cpu/devices/cpu0/cpufreq/scaling_cur_freq: no such file or directory source="collector.go:123" 

Investigation:

I found that for this version of RHEL 7.4 - it is updated from scaling_cur_freq to cpuinfo_cur_freq which is causing the issue.

Kernel Version: 

Linux  3.10.0-514.10.2.el7.x86_64 #1 SMP Mon Feb 20 02:37:52 EST 2017 x86_64 x86_64 x86_64 GNU/Linux

  

  Please let me know if anyone has faced such issues in the past or can help me in resolving this.

Thanks,

Dhruv

[Ben Kochie]

I thought we had fixed that, because the issue is closed, but I guess we did not fix this for very old kernels.

https://github.com/prometheus/node_exporter/issues/644

I have re-opened the issue.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/f17658d9-9ba0-4340-af66-131b76964db5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dhruv Thakkar]

Didn't reply to group by mistake !!

Hello Ben,

 Thanks for looking into it quickly.

 Yes I had gone through this particular issue. 

 But got stuck as I was confused - whether I need to get latest source code and covert to python binary ?

 Because I had tried by downloading latest version of node exporter but the issue still persists.

Regards,

Dhruv 

On Tue, Jun 12, 2018 at 12:55 PM, Ben Kochie <sup...@gmail.com> wrote:

I thought we had fixed that, because the issue is closed, but I guess we did not fix this for very old kernels.

https://github.com/prometheus/node_exporter/issues/644

I have re-opened the issue.

On Tue, Jun 12, 2018 at 9:06 AM dst <dhruv.th...@gmail.com> wrote:

Hello guys,

  I am facing this issue while running node exporter in RHEL 7.4

Error:

"ERRO[0953] ERROR: cpu collector failed after 0.014791s: open /sys/bus/cpu/devices/cpu0/cpufreq/scaling_cur_freq: no such file or directory source="collector.go:123" 

Investigation:

I found that for this version of RHEL 7.4 - it is updated from scaling_cur_freq to cpuinfo_cur_freq which is causing the issue.

Kernel Version: 

Linux  3.10.0-514.10.2.el7.x86_64 #1 SMP Mon Feb 20 02:37:52 EST 2017 x86_64 x86_64 x86_64 GNU/Linux

  

  Please let me know if anyone has faced such issues in the past or can help me in resolving this.

Thanks,

Dhruv

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.

[Ben Kochie]

I setup a test RHEL 7 machine, but it looks like there is already a fix for this in the current kernel.

Linux centos-test 3.10.0-693.17.1.el7.x86_64 #1 SMP Thu Jan 25 20:13:58 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

I am able to see the correct files:

$ ls -l /sys/devices/system/cpu/cpu0/cpufreq/

total 0

-r--r--r-- 1 root root 4096 Jun 12 12:44 affected_cpus

-r--r--r-- 1 root root 4096 Jun 12 12:44 bios_limit

-r-------- 1 root root 4096 Jun 12 12:44 cpuinfo_cur_freq

-r--r--r-- 1 root root 4096 Jun 12 12:44 cpuinfo_max_freq

-r--r--r-- 1 root root 4096 Jun 12 12:44 cpuinfo_min_freq

-r--r--r-- 1 root root 4096 Jun 12 12:44 cpuinfo_transition_latency

-r--r--r-- 1 root root 4096 Jun 12 12:44 freqdomain_cpus

-r--r--r-- 1 root root 4096 Jun 12 12:44 related_cpus

-r--r--r-- 1 root root 4096 Jun 12 12:44 scaling_available_frequencies

-r--r--r-- 1 root root 4096 Jun 12 12:43 scaling_available_governors

-r--r--r-- 1 root root 4096 Jun 12 12:44 scaling_cur_freq

-r--r--r-- 1 root root 4096 Jun 12 12:44 scaling_driver

-rw-r--r-- 1 root root 4096 Jun 12 12:54 scaling_governor

-rw-r--r-- 1 root root 4096 Jun 12 12:44 scaling_max_freq

-rw-r--r-- 1 root root 4096 Jun 12 12:44 scaling_min_freq

-rw-r--r-- 1 root root 4096 Jun 12 12:44 scaling_setspeed

I will try downgrading the kernel if I can to reproduce.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/f17658d9-9ba0-4340-af66-131b76964db5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAN-Z2Qg9eN%2ByFLKgJyyhxeWbcE67t%2BQ21B_eMFTA9eLe1bxhvw%40mail.gmail.com.

[dst]

Hello Ben,

Thanks for validating quickly from your side.

 

Just For your reference (from my machine):

 

 ls -l /sys/devices/system/cpu/cpu0/cpufreq/

-r--r--r--. 1 root root 4096 Jun 13 07:06 affected_cpus

-r--------. 1 root root 4096 Jun 13 07:06 cpuinfo_cur_freq

-r--r--r--. 1 root root 4096 Jul 23  2017 cpuinfo_max_freq

-r--r--r--. 1 root root 4096 Jun 13 07:06 cpuinfo_min_freq

-r--r--r--. 1 root root 4096 Jun 13 07:06 cpuinfo_transition_latency

-r--r--r--. 1 root root 4096 Jun 13 07:06 related_cpus

-r--r--r--. 1 root root 4096 Jul 22  2017 scaling_available_governors

-r--r--r--. 1 root root 4096 Jun 13 07:06 scaling_driver

-rw-r--r--. 1 root root 4096 Jul 22  2017 scaling_governor

-rw-r--r--. 1 root root 4096 Jun 13 07:06 scaling_max_freq

-rw-r--r--. 1 root root 4096 Jun 13 07:06 scaling_min_freq

-rw-r--r--. 1 root root 4096 Jun 13 07:06 scaling_setspeed

Please do let me know in case of kernel upgrade is required. 

Thanks,

Dhruv  

[Ben Kochie]

Would you mind posting the contents of those files:

for file in /sys/bus/cpu/devices/cpu0/cpufreq/*; do echo "${file}: '$(< ${file})'" ; done

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/6aafa450-6a84-4833-9205-5b80bc79cae8%40googlegroups.com.

[Dhruv Thakkar]

Sure.

Here it is:

for file in /sys/bus/cpu/devices/cpu0/cpufreq/*; do echo "${file}: '$(< ${file})'" ; done

/sys/bus/cpu/devices/cpu0/cpufreq/affected_cpus: '0'

/sys/bus/cpu/devices/cpu0/cpufreq/cpuinfo_cur_freq: '1200000'

/sys/bus/cpu/devices/cpu0/cpufreq/cpuinfo_max_freq: '2600000'

/sys/bus/cpu/devices/cpu0/cpufreq/cpuinfo_min_freq: '1200000'

/sys/bus/cpu/devices/cpu0/cpufreq/cpuinfo_transition_latency: '4294967295'

/sys/bus/cpu/devices/cpu0/cpufreq/related_cpus: '0'

/sys/bus/cpu/devices/cpu0/cpufreq/scaling_available_governors: 'performance powersave'

/sys/bus/cpu/devices/cpu0/cpufreq/scaling_driver: 'intel_pstate'

/sys/bus/cpu/devices/cpu0/cpufreq/scaling_governor: 'performance'

/sys/bus/cpu/devices/cpu0/cpufreq/scaling_max_freq: '2600000'

/sys/bus/cpu/devices/cpu0/cpufreq/scaling_min_freq: '1200000'

/sys/bus/cpu/devices/cpu0/cpufreq/scaling_setspeed: '<unsupported>'

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.

[Ben Kochie]

FYI, this was fixed by RedHat in 3.10.0-518.el7

* Tue Nov 01 2016 Rafael Aquini <aqu...@redhat.com> [3.10.0-518.el7]

- [cpufreq] expose scaling_cur_freq sysfs file for set_policy() drivers (Oleksandr Natalenko) [1382608]

I am working on changes to make the node_exporter robust, but for now, upgrading the kernel to the latest patch release is the best option.

[dst]

Hello Ben,

 

Thanks a lot for your findings and further help on this. 

I will check the feasibility of the kernel upgrade and proceed further.

Regards,

Dhruv