I tired using Docker Swarm Service Discovery in prometheus, but facing problems using it. I followed the docker swarm support documentation. Created a daemon.json file and mounted /var/run/docker.sock in prometheus container. Container is giving permission denied error as prometheus is running as nobody and doesn't have access to mounted /var/run/docker.sock. Below is my prometheus.yml.
Prometheus Version : v2.20.1
prometheus: image: prom/prometheus networks: - monitor ports: - "9090:9090" command: - '--config.file=/etc/prometheus/prometheus.yml' - '--storage.tsdb.path=/prometheus' - '--storage.tsdb.retention=${PROMETHEUS_RETENTION:-24h}' volumes: - prometheus:/prometheus - /home/efs/devops/dsm:/etc/prometheus:ro - /var/run/docker.sock:/var/run/docker.sock:ro deploy: mode: replicated replicas: 1 resources: limits: memory: 1024M reservations: memory: 128M
Prometheus.yml
scrape_configs: - job_name: 'docker' dockerswarm_sd_configs: - host: unix:///var/run/docker.sock role: nodes
Error:monitor_prometheus.1.uwda89...@monitoring.dev.ai | level=error ts=2020-08-06T07:21:19.106Z caller=refresh.go:98 component="discovery manager scrape" discovery=dockerswarm msg="Unable to refresh target groups" err="error while listing swarm nodes: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get \"http://%2Fvar%2Frun%2Fdocker.sock/v1.24/nodes\": dial unix /var/run/docker.sock: connect: permission denied