context deadline exceeded

[BHARATH KUMAR]

Hi all,

I am installing node exporter on many servers and I a monitoring those servers using prometheus. Initially i didn't face any kind of errors. but on few servers I am facing the error "Get "http://some_ip:9100/metrics": context deadline exceeded ". I tried changing scrape interval and tried disabling firewall. but nothing worked. I am not using any docker or kubernetes. can anyone please help me out of this?

Thanks in advance.

Thanks & regards,

Bharath.

[Evelyn Pereira Souza]

On 09.04.22 19:16, BHARATH KUMAR wrote:
> Hi all,

Hi Kumar

>
> I am installing node exporter on many servers and I a monitoring those
> servers using prometheus. Initially i didn't face any kind of errors.
> but on few servers I am facing the error "Get
> "http://some_ip:9100/metrics": context deadline exceeded ". I tried
> changing scrape interval and tried disabling firewall. but nothing
> worked. I am not using any docker or kubernetes. can anyone please help
> me out of this?

Try increase scrape_timeout

best regards
Evelyn

[BHARATH KUMAR]

heyy thanks.... i changed the timeout to 2m still I am getting the same error... will the timeout depends upon the scrape interval.

global:
  scrape_interval: 2m
  evaluation_interval: 15s
  scrape_timeout: 2m

this is my prometheus.yml

could you please tell me?

thanks & regards

Bharath Kumar

[Brian Candler]

> on few servers I am facing the error "Get "http://some_ip:9100/metrics": context deadline exceeded

Does this happen consistently, i.e. there are no metrics returned at all for those servers, or intermittent?

You can replicate the problem without using Prometheus like this:

curl -h 'http://some_ip:9100/metrics'

If it just hangs, then you know the problem is either with the network (e.g. firewall rules dropping the traffic) or with the host where the exporter is running (e.g. iptables dropping the traffic)

[BHARATH KUMAR]

Yes... I checked on the servers where I am getting that error. I run this command  curl -h 'http://some_ip:9100/metrics' it is displaying the metrics s output.

[BHARATH KUMAR]

I changed scrape_timeout to 5m,15s,1m,2m,3m.. But still I am getting the same error. Can anyone please help me out of this?

Thanks and regards,

Bharath Kumar