When I originally configured Prometheus with a variety of exporters I had it scraping ports on a specific docker swarm host. This is dangerous as if that host goes down the underlying service will pop back up on a different host but Prometheus won’t be able to scrape it. I considered using haproxy to round robin onto the docker swarm nodes, but Kubernetes can resolve services by service name – is there no way to do this in Docker Swarm?
There is, but unlike Kubernetes the services can’t resolve each other by default. We must create a specific network and attach the services to it.
/prometheus $ nslookup unifi_exporter Server: 127.0.0.11 Address 1: 127.0.0.11 nslookup: can't resolve 'unifi_exporter'
Create overlay network:
sudo docker network create -d overlay monitoring tb3iw12k7xaw5olz7rasdcnm0
Redeploy Prometheus on network:
docker service create --replicas 1 --name prometheus \ --mount type=bind,source=/data/docker/prometheus/config/prometheus.yml,destination=/etc/prometheus/prometheus.yml \ --mount type=bind,src=/data/docker/prometheus/data,dst=/prometheus \ --publish published=9090,target=9090,protocol=tcp \ --network monitoring \ prom/prometheus
Redeploy our exporter, this time attached to the overlay network. Note we no longer need to publish a port.
docker service create --replicas 1 --name unifi_exporter \ --mount type=bind,src=/data/docker/unifi-exporter/config.yml,dst=/config.yml \ --mount type=bind,src=/etc/ssl,dst=/etc/ssl,readonly \ --replicas=1 \ --network monitoring \ louisvernon/unifi_exporter:0.4.0-18-g85455df -config.file=/config.yml
Confirm Prometheus can resolve the exporter by service name:
/prometheus $ nslookup unifi_exporter Server: 127.0.0.11 Address 1: 127.0.0.11 Name: unifi_exporter Address 1: 10.0.1.15