We copied Host A in Cloud A and moved it to Cloud B. Now:
Cloud A: Old Host A
Cloud B: Host A
But we noticed that Prometheus was continuing to scrape metrics for:
Cloud A: Old Host A
(We could tell by device identifiers in disk utilization queries, for example).
It was not until after we restarted the prometheus service itself, that we noticed it was finally scraping the new host.
When this host was copied/migrated, it's A record was updated with the new IP address, the hostname stayed the same. We are also scraping metrics via file_sd.
Does Prometheus know to cache DNS entries? If so, where can I see or flush this? I am not confident it's host related, when I scraped metrics using the hostname from Prometheus, it returned the correct device (new host) just fine.