Context Deadline Exceeded

[bamb...@gmail.com]

I have prometheus configuration with many jobs where i am scraping metrics over http. But I have one job where i need to scrape the metrics over https.When i access:

https://ip-address:port/metrics

I can see the metrics. The job that I have added in the prometheus.yml configuration is:

- job_name: 'test-jvm-metrics'
    scheme: https
    static_configs:
      - targets: ['ip:port']

When i restart the prometheus I can see error on my target that says:

context deadline exceeded

I have read that maybe the scrape_timeout is the problem, but I have set it to 50 sec and still the same problem.

What can cause this problem and how to fix it? Thank you!

[Julius Volz]

Can you curl that metrics endpoint from the same machine / container in which Prometheus is running? Sounds like Prometheus might be running into some network issues that cause a timeout.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/d4158d57-19ae-46e1-8f34-176e0174568a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[bamb...@gmail.com]

I have tried on another machine from which i am scraping metrics with node_exporter and I have the same error for my app that is with https... The application was on http but when i change it to go over https, then the prometheus stop to scrape the metrics... As you can see i think my configuration is okay but there is still a problem. Any idea how to fix it?

On Monday, April 16, 2018 at 1:08:34 PM UTC+2, Julius Volz wrote:

Can you curl that metrics endpoint from the same machine / container in which Prometheus is running? Sounds like Prometheus might be running into some network issues that cause a timeout.

On Mon, Apr 16, 2018 at 8:11 AM, <bamb...@gmail.com> wrote:

I have prometheus configuration with many jobs where i am scraping metrics over http. But I have one job where i need to scrape the metrics over https.When i access:

https://ip-address:port/metrics

I can see the metrics. The job that I have added in the prometheus.yml configuration is:

- job_name: 'test-jvm-metrics'
    scheme: https
    static_configs:
      - targets: ['ip:port']

When i restart the prometheus I can see error on my target that says:

context deadline exceeded

I have read that maybe the scrape_timeout is the problem, but I have set it to 50 sec and still the same problem.

What can cause this problem and how to fix it? Thank you!

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.

[bamb...@gmail.com]

From the same machine with prometheus I am exposing metrics via node_exporter and that is working without errors...

[Julius Volz]

You should really try to manually curl the metrics URL from the same machine as Prometheus (or from the same container, if Prometheus is running in a container). It's most likely some networking problem, assuming that the /metrics endpoint loads really fast in a browser.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/659561f1-4a97-407d-b7c6-3d692e34385b%40googlegroups.com.

[bamb...@gmail.com]

I placed the app in the same container and now I have this error: 

Get https://IP:PORT/metrics: x509: cannot validate certificate for IP because it doesn't contain any IP SANs

[bamb...@gmail.com]

The application has .jks certificate with a password. So how to setup the job configuration?

I have tried like this, but its not working:

- job_name: 'test-jvm-metrics'

    scheme: https

    tls_config:

      cert_file: /opt/monitoring/prometheus/prometheus-1.7.1.linux-amd64/server1.jks

    basic_auth:

      password: 'password'

    scrape_interval: 60s

    scrape_timeout: 60s

    static_configs:

      - targets: ['IP:PORT']

[bamb...@gmail.com]

I have fixed the problem, adding:

tls_config:

      insecure_skip_verify: true