Automatically adding UptimeRobot to Gitlab's monitoring whitelist with Ansible

GitLab has a number of monitoring endpoints you can ping to make sure that not only is the rails process running (and not just your webserver) but that also the background services Gitlab depends on are running successfully too.

I want to monitor my personal Gitlab instance with UptimeRobot hitting those endpoints for a more accurate uptime measurement. Gitlab does have a monitoring token I could append to the URL but that is deprecated. Instead, you should add the monitoring machines to an IP whitelist. Thankfully, UptimeRobot provides a list of all of their incoming IP addresses.

Thing is, I'm lazy. I don't want to have to manually add all of those IP addresses to the whitelist. I already have the Omnibus GitLab configuration managed as a template via Ansible, so the best way is to have Ansible automatically download those lists.

First we'll set the IP URLs as variables:

uptimerobot_ipv4_url: "https://uptimerobot.com/inc/files/ips/IPv4.txt"
uptimerobot_ipv6_url: "https://uptimerobot.com/inc/files/ips/IPv6.txt"

Then have Ansible download the remote files and register a local variable:

- name: Download the list of IPv4 address of UptimeRobot network
  uri:
    url: "{{ uptimerobot_ipv4_url }}"
    return_content: yes
  register: _ur_ipv4_uri

- name: Download the list of IPv6 address of UptimeRobot network
  uri:
    url: "{{ uptimerobot_ipv6_url }}"
    return_content: yes
  register: _ur_ipv6_uri

Now that we have the two variables, uripv4_uri and uripv6_uri, we can split up the plain text list of IP addresses:

- name: Split up the list of IPs
  set_fact:
    _ur_ipv4_ip_list: "{{ _ur_ipv4_uri.content.split() }}"
    _ur_ipv6_ip_list: "{{ _ur_ipv6_uri.content.split() }}"

Then we merge the two lists into a single one of all of UptimeRobot's IP addresses:

- name: Combining the list of IPv4 and IPv6 addresses
  set_fact:
    _ur_all_ips: "{{ _ur_ipv4_ip_list | union(_ur_ipv6_ip_list) }}"

Lastly, here's the tricky part. I want to include this list of IP addresses into the monitoring whitelist, while not overriding the current values that may be included already. Thus, I want to take these two dictionaries:

_ur_all_ips:
  - 216.144.250.150
  - 69.162.124.226
  - ...

gitlab_rb:
  gitlab_rails:
    monitoring_whitelist:
      - 127.0.0.1
      - 10.0.0.0/8
      - ...

And merge them together into the sub-key gitlab_rb.gitlab_rails.monitoring_whitelist. This is done via the lovely combine filter:

- name: Inject UptimeRobot IP list into monitoring whitelist
  set_fact:
    gitlab_rb: "{{ gitlab_rb | combine({'gitlab_rails': {'monitoring_whitelist': gitlab_rb.gitlab_rails.monitoring_whitelist | default([]) | union(_ur_all_ips) }}, recursive=True) }}"

The important magic happens all in that one line. I take the existing gitlab configuration dictionary and build a new dictionary that matches it's structure. I then take the current monitoring whitelist (which might not already exist so I default to an empty list) and do a set union with the UptimeRobot IP addresses. Both dictionaries are passed to the combine filter, with recursive set to true to not override any other sibling keys in the original dictionary.

Simple in concept, but Ansible's defaults don't make it easy. It took a while to figure out the appropriate combination of filters to make this merge work out. Hopefully it'll save someone else some time.