Browsed by
Category: Networking

Unifi to Grafana (using Prometheus and unifi_exporter)

Unifi to Grafana (using Prometheus and unifi_exporter)

Documenting the process of getting this up and running. We already had Prometheus and Grafana running on our docker swarm cluster (we promise to document this all one day).

There was only one up to date image of unifi_exporter in DockerHub and it had no documentation so we were not comfortable using it.

1) Download, build and push unifi_exporter.

$ git clone [email protected]:mdlayher/unifi_exporter.git
...
$ cd unifi_exporter
$ sudo docker build -t louisvernon/unifi_exporter:$(git describe --tags) . # yields a tag like 0.4.0-18-g85455df
$ sudo docker push louisvernon/unifi_exporter:$(git describe --tags)

2) Create read only admin user for unifi_exporter service:

3) Create config.yml on storage mounted on dockerswarm node. In our case we have a glusterfs volume mounted across all nodes. If you are using the self-signed cert on your unifi controller then you will need to set insecure to true.

$ $ cat /data/docker/unifi-exporter/config.yml 
listen:
  address: :9130
  metricspath: /metrics
unifi:
  address: https://unifi.vern.space
  username: unifiexporter
  password: random_password
  site: Default 
  insecure: false
  timeout: 5s

4) Deploy to docker swarm. The docker image does not contain any trusted certs, so we mounted the host certs as readonly.

$ docker service create --replicas 1 --name unifi_exporter \
    --mount type=bind,src=/data/docker/unifi-exporter/config.yml,dst=/config.yml \
    --mount type=bind,src=/etc/ssl,dst=/etc/ssl,readonly \
    --publish 9130:9130 \
    --replicas=1 \
    louisvernon/unifi_exporter:0.4.0-18-g85455df -config.file=/config.yml

5) You should see something like this from the logs (we use portainer to quickly inspect our services).

2018/06/12 01:10:47 [INFO] successfully authenticated to UniFi controller
2018/06/12 01:10:47 Starting UniFi exporter on ":9130" for site(s): Default

First time around (before we bind mounted /etc/ssl) we had an x509 error due to the missing trusted certs..

6) Add unifi_exporter as a new target for prometheus.

$ cat /data/docker/prometheus/config/prometheus.yml
...
  - job_name: 'unifi_exporter'
    static_configs:
      - targets: ['dockerswarm:9130']
        labels:
          alias: unifi_exporter
...

7) Point your browser at http://dockerswarm:9130/metrics and make sure you see stats. In our case the payload was 267 lines.

8) Restart the prometheus service: `docker service update –force prometheus`

9) Hop on over to prometheus to make sure the new target is listed and UP: http://dockerswarm:9090/targets

10) Finally we import the dashboard into Grafana. Our options are a little sparse right now, but this dashboard gives us somewhere to start. we made some tweaks to this to make it multi-AP friendly with some some extra stats:
Unifi-1516201148080

The result:

Used LB4M – Finding the Management Interface

Used LB4M – Finding the Management Interface

I bought an LB4M on ebay. I did not have a USB to RJ45 console cable.

Power port – Green
Status port – Orange

I read online that by default the management port pulls an IP via DHCP but that wasn’t working. I cycled through several ports on the device connecting them to my network and actually missed that one port pulled an IP address.

I then connected my laptop to the management port and watched traffic with Wireshark. The LB4M was sending data using the CDP protocol (new to me).

So the firmware is 1.1.0.8. Understanding the span of features vs stability for LB4M firmware versions seems to be a minefield.

The packet data indicated that it had picked up (or statically held) an IP on some interface. As I plugged in ports I ran:

$sudo nmap -F 192.168.1.215
Nmap scan report for 192.168.1.215
Host is up (0.010s latency).
Not shown: 99 closed ports
PORT   STATE SERVICE
23/tcp open  telnet

I was in.
Username: admin
Password:

My first experience with this CLI, so lets poke around.
(type show ? for options)

(Switching) >show network

Interface Status............................... Up
IP Address..................................... 192.168.1.215
Subnet Mask.................................... 255.255.255.0
Default Gateway................................ 192.168.1.1
Burned In MAC Address.......................... XX:XX:XX:XX:XX:XX
Locally Administered MAC address............... 00:00:00:00:00:00
MAC Address Type............................... Burned In
Configured IPv4 Protocol....................... DHCP
Management VLAN ID............................. 1
(Switching) >show serviceport

Interface Status............................... Up
IP Address..................................... 0.0.0.0
Subnet Mask.................................... 0.0.0.0
Default Gateway................................ 0.0.0.0
Configured IPv4 Protocol....................... None
Burned In MAC Address.......................... XX:XX:XX:XX:XX:XX
iperf for testing HDMI balun runs

iperf for testing HDMI balun runs

It seems reasonable that jitter will be a big factor in the performance of CAT5e/6 runs used for HDMI baluns. I ran two high quality unshielded CAT5e cables and two high quality shielded (but ungrounded by me) CAT6 cables.

They cross over multiple power lines and unfortunately run parallel to power lines for about 11 foot of the ~ 30 foot run.

Testing Jitter:

On the server:

<pre>iperf -s -w 128k -u</pre>

This means start iperf server with a 128KB buffer in UDP mode. UDP best reflects the nature of traffic of HDMI over ethernet.

On the client:

<pre>iperf -c serverip -u -b 1000m -w 128k -t 120</pre>

This means start the client and connect to serverip, use UDP and send traffic at gigabit speeds for 120 seconds. I needed to run the test for at least 120 seconds to get consistent jitter results.

First I tested with a 3 foot CAT5e cable connected directly to the switch, this represents ‘ideal’ performance. The same short cable was connected to the ends of the long CAT5e and CAT6 runs.

[table]
[ ID], Interval, Transfer, Bandwidth, Jitter, Lost/Total Datagrams
IDEAL, 0.0-120.0 sec, 9.65 GBytes, 691 Mbits/sec, 0.013 ms, 47713/7095556 (0.67%)
STP-CAT6, 0.0-120.0 sec, 9.62 GBytes, 689 Mbits/sec, 0.015 ms, 26399/7052134 (0.37%)
CAT5e, 0.0-120.0 sec, 9.57 GBytes, 685 Mbits/sec, 0.180 ms, 108956/7100704 (1.5%)
[/table]

So there we have it. From a simple bandwidth perspective the long CAT5e run offers basically the same performance as the STP-CAT6 run, however when it comes to jitter, the CAT6 cable is marginally worse than the ideal case, while the CAT5e run is an order of magnitude worse.

Very interesting! In the end I’m sure it’s the shielding rather than the CAT6 rating that makes the STP-CAT6 superior.