Browsed by
Category: grafana

Attaching Docker Swarm Services to an Overlay Network

Attaching Docker Swarm Services to an Overlay Network

When I originally configured Prometheus with a variety of exporters I had it scraping ports on a specific docker swarm host. This is dangerous as if that host goes down the underlying service will pop back up on a different host but Prometheus won’t be able to scrape it. I considered using haproxy to round robin onto the docker swarm nodes, but Kubernetes can resolve services by service name – is there no way to do this in Docker Swarm?

There is, but unlike Kubernetes the services can’t resolve each other by default. We must create a specific network and attach the services to it.

Before:

/prometheus $ nslookup unifi_exporter
Server:    127.0.0.11
Address 1: 127.0.0.11

nslookup: can't resolve 'unifi_exporter'

Create overlay network:

sudo docker network create -d overlay monitoring
tb3iw12k7xaw5olz7rasdcnm0

Redeploy Prometheus on network:

docker service create --replicas 1 --name prometheus \
    --mount type=bind,source=/data/docker/prometheus/config/prometheus.yml,destination=/etc/prometheus/prometheus.yml \
    --mount type=bind,src=/data/docker/prometheus/data,dst=/prometheus \
    --publish published=9090,target=9090,protocol=tcp \
    --network monitoring \
    prom/prometheus

Redeploy our exporter, this time attached to the overlay network. Note we no longer need to publish a port.

docker service create --replicas 1 --name unifi_exporter \
    --mount type=bind,src=/data/docker/unifi-exporter/config.yml,dst=/config.yml \
    --mount type=bind,src=/etc/ssl,dst=/etc/ssl,readonly \
    --replicas=1 \
    --network monitoring \
    louisvernon/unifi_exporter:0.4.0-18-g85455df -config.file=/config.yml

Confirm Prometheus can resolve the exporter by service name:

/prometheus $ nslookup unifi_exporter
Server:    127.0.0.11
Address 1: 127.0.0.11

Name:      unifi_exporter
Address 1: 10.0.1.15