Maintaining dns_sd_configs in prometheus yaml through json files

[Agarwal ,Naveen]

Hello, 

We use file_sd_configs extensively. We have multiple target files (json) and it is refered in prometheus yaml. The targets in json are maintained by our customized discovery agents. 

We now have usecase to support dns_sd_configs. Does dns_sd_config also allow cases where the dns domain names can be specified in a separate file(json Or yaml) and it can be refered in prometheus file. 

Reason: We have large number of targets and want to maintain smaller files. This avoids having lot of config in prometheus yaml. 

Also each json file is maintained by separate teams. In case we need to put all dns_sd_configs in prometheus yaml, we will need to think of a way to maintain it across multiple teams. 

Any other suggestions if this is not feasible in prometheus currently. 

Thanks, 

Naveen

[Brian Candler]

> Reason: We have large number of targets and want to maintain smaller files.

What's the reason you are using dns_sd_configs in the first place? It sounds like you should be using file_sd_configs.  You should only use dns_sd_configs if you need to do an unusual DNS lookup like SRV or MX record, and/or when a single DNS query returns multiple targets which need to be scraped individually.

If you are just using DNS names to refer to targets, each of which has a single A or AAAA record (or you don't care which one is used for each target), then file_sd_configs will work just fine. With file_sd_configs the lists of targets are in separate files (JSON or YAML), and are automatically re-read whenever they change - there's no need to signal prometheus.

If there is some specific reason for you to use dns_sd_configs, then can you give an example of what you're currently doing?  If it's something like this:

- job_name: foo

  dns_sd_configs:

  - type: SRV
    names:

    - foo.com

    - bar.com

    - qux.com

then those domain names do indeed have to go into prometheus.yml.  You can build this from a script if you like, and send prometheus a reload signal when it changes. All JSON is valid YAML, so you can write the whole config in JSON if you prefer, and you can transform it or generate it with something like JSONNET.