not able to see metrics from query browser even though end point is up and showing metrics through curl

[Kumar]

not able to see metrics from query browser even though end point is up and showing metrics through curl

Using custom exporter.

[Brian Candler]

Then you're probably not scraping it, or the custom exporter is returning invalid data.

Check the value of the "up" metric in query browser for the given job and instance, i.e. up{job="foo",instance="bar"}

"up" missing - scraping not configured.

"up" has value 0 - unable to communicate or data is bad.

"up" has value 1 - maybe partial scrape

Check logs at prometheus and exporter; check traffic between prometheus and exporter with tcpdump.  Also show example of the curl output.

[Wesley Peng]

Hi

Brian Candler wrote:
> "up" missing - scraping not configured.
> "up" has value 0 - unable to communicate or data is bad.
> "up" has value 1 - maybe partial scrape
>
> Check logs at prometheus and exporter; check traffic between prometheus
> and exporter with tcpdump.

Can you show a sample tcpdump syntax for this purpose?

Thanks.

[Brian Candler]

tcpdump -i eth0 -nn -s0 -A host x.x.x.x and tcp port y

where x.x.x.x is the IP address of the remote host, and y is the port you're running the exporter on.  Replace "eth0" with your network interface name.  If prometheus and the exporter are on the same machine then use "lo" (loopback) as the interface name.

[Kumar]

up metric showing value 1 and alos targets it is showing up as well.

is there nay time mismatch between prometheus server and exporter? if yes,, how could we disable it.

[Brian Candler]

I don't really understand what you're trying to say.

Does your exporter include timestamps for each metric?  These are optional.  It's almost always best not to include them unless you are *really* sure you know what you're doing.  Notice that none of the standard exporters, like node_exporter, include timestamps.  In that case, clock sync between prometheus server and exporter is irrelevant.

If you do include timestamps, and the values are too far in the past, or out of sequence, then prometheus will discard the metrics and log an error.  You might need to run prometheus with --log.level=debug to see them.

If you're trying to export the time when some event happened - e.g. the time when a job was last run - then *don't* use the prometheus metric timestamp for this.  Instead, expose a separate metric which is the timestamp as a Unix time.  For an example of this in action, see how the node_exporter textfile collector returns the timestamp of the textfile it returned.