dynamic targets file with api token

[Andrew Meyer]

I am trying to set up dynamic targets for our storage environment.  I am able to access it using the static_configs but it requires an API token.

This is what I have currently configured but had to revert back to static_configs.  It wouldn't read the API token at all.

[
  {
    "targets": [ "1.2.3.4", "2.3.45" ],
    "labels": {
      "env": "prod",
      "__storage_apitoken": "something-something-something-something",
      "job": "storage_exporter"
    }
  }
]

[Brian Candler]

1. How do you need to provide the API token - as a HTTP header (if so what)? As a URL query parameter? Something else?

2. What do you mean by "set up dynamic targets"?  What service discovery mechanism are you using?

static_configs and file_sd_configs are the same, when it comes to setting labels and targets.

3. What relabelling configs do you have?   That "__storage_apitoken" label must be rewritten to have some effect.  Otherwise, labels which begin with double underscore are thrown away.

4. Aside: you shouldn't set "job" as a target label.  This is added automatically by the scrape job itself.

[Andrew Meyer]

So this pure storage.  Here is more details:

[
  {
    "targets": [ "1.2.3.4", "2.3.4.5" ],
    "labels": {
      "env": "prod",
      "__pure_apitoken: API KEY,
      "job": "pure_flashblade"
    }
  }
]

[Andrew Meyer]

This is basically going to be set up as

file_sd_configs:

- files: [ '/path/to/file.json' ]

And I will comment out the static config.  But I'm having some issues with the json file.

[Brian Candler]

file_sd_configs (with a YAML/JSON file) and static_configs (with inline list of targets) are identical format.  What you've written looks valid; we can't really help unless you describe what "having some issues" actually means, and how it's different from what you did with static_configs.

Note that simply adding a target label called "__pure_apitoken" won't make any difference to how the scraping is done; it certainly won't pass any token to the target in the scrape.  This is true for both static_configs and file_sd_configs.

So I suggest you:

1. Show your full (working) config with static_configs

2. Show your non-working config with file_sd_configs

and it will probably be clear what you've done differently in the second case.

[Andrew Meyer]

Sorry about that.  Here is the actual config for the pure_flashblade and pure_flasharray for prometheus.

  - job_name: 'pure_flashblade'
    #scrape_timeout: 1m
    #scrape_interval: 15s
    metrics_path: /metrics/flashblade/array
    relabel_configs:
    # meta label of target address --> get parameter "pure_host"
    - source_labels: [__address__]
      target_label: __param_endpoint
    # label of target api token --> get parameter "pure_apitoken"
    - source_labels: [__pure_apitoken]
      target_label: __param_apitoken
    # display the pure host as the instance label
    - source_labels: [__address__]
      target_label: instance
    # point the exporter to the scraping endpoint of the exporter
    - target_label: __address__
      replacement: 127.0.0.1:9491 # address of the exporter, in debug mode
                                  # THIS NEEDS TO BE CHANGED TO YOUR ENVIRONMENT

    # Actual pure hosts (without a prometheus endpoint) as targets
#    file_sd_configs:
#    - files: [ '/etc/prometheus/pure_flashblade_targets.json' ]
    static_configs:
    - targets: [ "1.2.3.4" ]
      labels:
        __pure_apitoken: Some-api-token

I would like to comment out the static_configs section and move to a file_sd_configs setup.

I have mirrored it after some of the other exporters I did this too, but falling short on the API token.

[
  {
    "targets": [ "1.2.3.4" ],
    "labels": {
      "env": "prod",
      "__pure_apitoken": "some-api-token",
      "job": "pure_flashblade"
    }
  }
]

[Andrew Meyer]

Converted to yml file.  It is working now.  No more json.

[Brian Candler]

Glad it's working.  But JSON should have worked fine (all JSON is valid YAML automatically)

[
  {
    "targets": [ "1.2.3.4" ],
    "labels": {
      "env": "prod",
      "__pure_apitoken": "some-api-token",
      "job": "pure_flashblade"
    }
  }
]

That looks OK to me, except you need to remove the "job" here. Also there's no need to add "env" since you weren't doing so in the static_configs version - although it doesn't do any harm to add it, it will just be included in your scraped timeseries.

    - source_labels: [__pure_apitoken]
      target_label: __param_apitoken

That explains how you're providing the apitoken: as a parameter in the request URL.