How do I re-export scraped metrics?

[jay...@gmail.com]

I scrape several node exporters and create a new metric in a rule. For example, each node exporter exports a node_active_jobs metric, and let's say that one of the labels is job_class, and I my rules file says:

groups:
 - name: alljobs 
   rules:
   - record: active_jobs 
     expr: sum (node_active_jobs) by (job_class)

How can I make the resulting active_jobs metric visible to some other scraper (in myh case: the Google Compute Engine autoscaler). The closest I have come to is use http://localhost:9090/federate?match[]=active__jobs, but (a) this also appends a timestamp to the metric, which confuses the autoscaler, and (b) does not allow me to specify labels (curl 'http://localhost:9090/federate?match[]=active__jobs{job_class="priority"} fails with 1:35: parse error: could not parse remaining input "=\"priority\""...

Help?

[b.ca...@pobox.com]

Federation is fine - although you could also just query the regular 'query' API endpoint.

Your label query is failing because of curl interpreting metacharacters specially. Add the "-g" flag to curl and you should be OK.

[jay...@gmail.com]

On Tuesday, December 1, 2020 at 4:25:13 PM UTC-5 b.ca...@pobox.com wrote:

Federation is fine - although you could also just query the regular 'query' API endpoint.

Your label query is failing because of curl interpreting metacharacters specially. Add the "-g" flag to curl and you should be OK.

Oops. Asleep at the wheel :(

The query API won't do, as it returns a JSON object, and I need a single line that looks like the output of /metrics. 

Thanks!

/ji