prometheus targets server returned http status 401 unauthorized

[parwez khan]

Hi All,

I am trying to scrape weblogic metrics using weblogic monitoring exporter and prometheus. I am not using kubernetes or docker, prometheus and weblogic are configured on RHEL servers only. I am able to get metrics from browser with url "http://host:port/wls-exporter/metrics". My prometheus configuration file for weblogic looks like as below:

 - job_name: 'weblogic' 

metrics_path: '/wls-exporter/metrics' 

static_configs: - targets: ['xxx:7001']

but for the above configuration prometheus gives error as target down and http, 401 error. I tried adding basic auth in prometehus for weblogic, but it gives some strconv error.

Can anyone please suggest me on how and where to add authentication or scarpe metrics without authentication ?

[Simon Pasquier]

On Wed, Sep 11, 2019 at 8:39 PM parwez khan <parwez...@gmail.com> wrote:
>
> Hi All,
>
> I am trying to scrape weblogic metrics using weblogic monitoring exporter and prometheus. I am not using kubernetes or docker, prometheus and weblogic are configured on RHEL servers only. I am able to get metrics from browser with url "http://host:port/wls-exporter/metrics". My prometheus configuration file for weblogic looks like as below:
> - job_name: 'weblogic'
> metrics_path: '/wls-exporter/metrics'
> static_configs: - targets: ['xxx:7001']
>
> but for the above configuration prometheus gives error as target down and http, 401 error. I tried adding basic auth in prometehus for weblogic, but it gives some strconv error.

Can you share the exact error message?

>
> Can anyone please suggest me on how and where to add authentication or scarpe metrics without authentication ?
>

> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/67a5e97b-bce0-4285-9360-6b946b7a1f16%40googlegroups.com.

[Lijing Zhang]

Will this help?

https://github.com/oracle/weblogic-monitoring-exporter/blob/master/samples/kubernetes/end2end/prometheus/values.yaml 

About basic_auth in scrape job

在 2019年9月12日星期四 UTC+8上午2:39:35，parwez khan写道：

[parwez khan]

Hi Simon and Zhang, thanks for the prompt reply. There is actually no error in prometheus server logs. Any error message is displayed on target console itself. Please note I have tried below till now:

Note: All documentation from oracle or github provides details for using prometheus for weblogic on kubernetes, but I am not using kubernetes, feeling helpless as no documentation is available.

1. with basic auth added in scrape confihurations, prometheus.yml file looks like

- job_name: 'weblogic'

    metrics_path: '/wls-exporter/metrics'

    static_configs:

    - targets: ['localhost:7001']

    basic_auth:

     username: 'weblogic'

     password: 'welcome1'

with this configuration,  target console display error as "strconv.ParseFloat: parsing "base_domain": invalid syntax"

2. Adding authentication in target itself like below:

    job_name: 'weblogic'

    metrics_path: '/wls-exporter/metrics'

    static_configs:

    - targets: ['weblogic:welcome1@localhost:7001'], I get following error.

parse http://weblogic:welcome1%40host:8001/wls-exporter/metrics: invalid URL escape "%40"

3. And with no auth added, it simply displays error as "error server returned HTTP status 401 Unauthorized"

[Simon Pasquier]

On Thu, Sep 12, 2019 at 2:32 PM parwez khan <parwez...@gmail.com> wrote:
>
> Hi Simon and Zhang, thanks for the prompt reply. There is actually no error in prometheus server logs. Any error message is displayed on target console itself. Please note I have tried below till now:
>
> Note: All documentation from oracle or github provides details for using prometheus for weblogic on kubernetes, but I am not using kubernetes, feeling helpless as no documentation is available.
>
> 1. with basic auth added in scrape confihurations, prometheus.yml file looks like
> - job_name: 'weblogic'
> metrics_path: '/wls-exporter/metrics'
> static_configs:
> - targets: ['localhost:7001']
> basic_auth:
> username: 'weblogic'
> password: 'welcome1'
> with this configuration, target console display error as "strconv.ParseFloat: parsing "base_domain": invalid syntax"

The probable reason is that your endpoint doesn't expose valid data.
You can check whether this is the case with promtool:
curl http://weblogic:welcome1@localhost:7001/wls-exporter/metrics |
promtool check metrics

>
> 2. Adding authentication in target itself like below:
> job_name: 'weblogic'
> metrics_path: '/wls-exporter/metrics'
> static_configs:
> - targets: ['weblogic:welcome1@localhost:7001'], I get following error.
>
> parse http://weblogic:welcome1%40host:8001/wls-exporter/metrics: invalid URL escape "%40"
>
> 3. And with no auth added, it simply displays error as "error server returned HTTP status 401 Unauthorized"
>
> On Thursday, 12 September 2019 00:09:35 UTC+5:30, parwez khan wrote:
>>
>> Hi All,
>>
>> I am trying to scrape weblogic metrics using weblogic monitoring exporter and prometheus. I am not using kubernetes or docker, prometheus and weblogic are configured on RHEL servers only. I am able to get metrics from browser with url "http://host:port/wls-exporter/metrics". My prometheus configuration file for weblogic looks like as below:
>> - job_name: 'weblogic'
>> metrics_path: '/wls-exporter/metrics'
>> static_configs: - targets: ['xxx:7001']
>>
>> but for the above configuration prometheus gives error as target down and http, 401 error. I tried adding basic auth in prometehus for weblogic, but it gives some strconv error.
>>
>> Can anyone please suggest me on how and where to add authentication or scarpe metrics without authentication ?
>

> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/f3435824-a1fb-4dc1-9d57-77d06f890afc%40googlegroups.com.

[parwez khan]

Thanks a lot Simon, I checked with promtool command and found following error.

"error while linting: text format parsing error in line 1: unexpected end of input stream" and this is how the metrics page(http://host:7001/wls-exporter/metrics)  displays in browser:

jvm_heap_free_current{domain="pdc_domain",server="AdminServer",name="AdminServer"} 207368856
jvm_heap_free_percent{domain="pdc_domain",server="AdminServer",name="AdminServer"} 47
jvm_heap_size_current{domain="pdc_domain",server="AdminServer",name="AdminServer"} 704643072
wls_scrape_mbeans_count_total{instance="slc14ujw.us.oracle.com:8001"} 3
wls_scrape_duration_seconds{instance="slc14ujw.us.oracle.com:8001"} 0.01
wls_scrape_cpu_seconds{instance="slc14ujw.us.oracle.com:8001"} 0.01

any help on how can I resolve this please ?

On Thursday, 12 September 2019 00:09:35 UTC+5:30, parwez khan wrote:

[Simon Pasquier]

On Thu, Sep 12, 2019 at 5:57 PM parwez khan <parwez...@gmail.com> wrote:
>
> Thanks a lot Simon, I checked with promtool command and found following error.
>
> "error while linting: text format parsing error in line 1: unexpected end of input stream" and this is how the metrics page(http://host:7001/wls-exporter/metrics) displays in browser:
>
> jvm_heap_free_current{domain="pdc_domain",server="AdminServer",name="AdminServer"} 207368856
> jvm_heap_free_percent{domain="pdc_domain",server="AdminServer",name="AdminServer"} 47
> jvm_heap_size_current{domain="pdc_domain",server="AdminServer",name="AdminServer"} 704643072
> wls_scrape_mbeans_count_total{instance="slc14ujw.us.oracle.com:8001"} 3
> wls_scrape_duration_seconds{instance="slc14ujw.us.oracle.com:8001"} 0.01
> wls_scrape_cpu_seconds{instance="slc14ujw.us.oracle.com:8001"} 0.01
>
>
> any help on how can I resolve this please ?

The help/type text is missing. See
https://prometheus.io/docs/instrumenting/exposition_formats/#text-format-example

>
>
>
>
> On Thursday, 12 September 2019 00:09:35 UTC+5:30, parwez khan wrote:
>>
>> Hi All,
>>
>> I am trying to scrape weblogic metrics using weblogic monitoring exporter and prometheus. I am not using kubernetes or docker, prometheus and weblogic are configured on RHEL servers only. I am able to get metrics from browser with url "http://host:port/wls-exporter/metrics". My prometheus configuration file for weblogic looks like as below:
>> - job_name: 'weblogic'
>> metrics_path: '/wls-exporter/metrics'
>> static_configs: - targets: ['xxx:7001']
>>
>> but for the above configuration prometheus gives error as target down and http, 401 error. I tried adding basic auth in prometehus for weblogic, but it gives some strconv error.
>>
>> Can anyone please suggest me on how and where to add authentication or scarpe metrics without authentication ?
>

> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/08bf98f3-2e4e-4272-be37-35e528b2db32%40googlegroups.com.

[Brian Brazil]

On Fri, 13 Sep 2019 at 09:01, Simon Pasquier <spas...@redhat.com> wrote:

On Thu, Sep 12, 2019 at 5:57 PM parwez khan <parwez...@gmail.com> wrote:
>
> Thanks a lot Simon, I checked with promtool command and found following error.
>
> "error while linting: text format parsing error in line 1: unexpected end of input stream" and this is how the metrics page(http://host:7001/wls-exporter/metrics)  displays in browser:
>
> jvm_heap_free_current{domain="pdc_domain",server="AdminServer",name="AdminServer"} 207368856
> jvm_heap_free_percent{domain="pdc_domain",server="AdminServer",name="AdminServer"} 47
> jvm_heap_size_current{domain="pdc_domain",server="AdminServer",name="AdminServer"} 704643072
> wls_scrape_mbeans_count_total{instance="slc14ujw.us.oracle.com:8001"} 3
> wls_scrape_duration_seconds{instance="slc14ujw.us.oracle.com:8001"} 0.01
> wls_scrape_cpu_seconds{instance="slc14ujw.us.oracle.com:8001"} 0.01
>
>
> any help on how can I resolve this please ?

The help/type text is missing. See
https://prometheus.io/docs/instrumenting/exposition_formats/#text-format-example

They're optional. The likely issue is it's missing a final newline.

You should use client_java to generate your metrics, it'll take care of things like this for you.

Brian

 

>
>
>
>
> On Thursday, 12 September 2019 00:09:35 UTC+5:30, parwez khan wrote:
>>
>> Hi All,
>>
>> I am trying to scrape weblogic metrics using weblogic monitoring exporter and prometheus. I am not using kubernetes or docker, prometheus and weblogic are configured on RHEL servers only. I am able to get metrics from browser with url "http://host:port/wls-exporter/metrics". My prometheus configuration file for weblogic looks like as below:
>>  - job_name: 'weblogic'
>> metrics_path: '/wls-exporter/metrics'
>> static_configs: - targets: ['xxx:7001']
>>
>> but for the above configuration prometheus gives error as target down and http, 401 error. I tried adding basic auth in prometehus for weblogic, but it gives some strconv error.
>>
>> Can anyone please suggest me on how and where to add authentication or scarpe metrics without authentication ?
>
> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/08bf98f3-2e4e-4272-be37-35e528b2db32%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAM6RFu7LZ7q0OJQiOyuN7989k%2BzR47zTOd%3DYBa8HR2Fpj%3Dt95A%40mail.gmail.com.

--

Brian Brazil

www.robustperception.io

[parwez khan]

Hi Simon and Brian, Thanks for all the help till now. I am not really a java guy, I have administration background. I went through the client_java documentations but could not understand much. I am actually confused on how to use this client to generate metrics from weblogic. I already have wls-exporter deployed on weblogic , which exposes metrics over rest api and I was trying to pull those metrices into prometheus. For the below jvm sample, can you plesae tell me on how can I use java client, if you can help me with one example, I will manage to figure out rest of them. Request you to help with some example or guide on how can I achieve it. Many many thanks for your time and help.

On Thursday, 12 September 2019 00:09:35 UTC+5:30, parwez khan wrote:

[Simon Pasquier]

This looks similar to
https://github.com/oracle/weblogic-monitoring-exporter/issues/54
I would try to reach out the weblogic-monitoring-exporter project directly.

> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/9f924a12-f559-49fe-9526-e482ee1f6b1a%40googlegroups.com.

[parwez khan]

Thanks Simon, I will open issue on wls-exporter page on github. Thanks for the help.

On Thursday, 12 September 2019 00:09:35 UTC+5:30, parwez khan wrote: