Pi-hole and resolv.conf

Discovering the problem

I’ve had an issue sitting open for a long while that I’d yet to address. If Pi-hole starts up and system-apps’ /etc/resolv.conf had been overwritten to use the DHCP settings of nameserver 10.0.0.11, Pi-hole would fail to initialize. The work-around I’d discovered was to manually modify /etc/resolv.conf, start Pi-hole, and then not care about whatever changes DHCP would make to it until I next had to restart Pi-hole, which was seldom-to-never. Low risk, functional work-around, nothing depending on it: low priority.

That was until I started seeing other oddities that all came back to this one issue. I was working on getting Prometheus Alertmanager up and running and found it was unable to connect to the endpoints that were otherwise accessible from within my browser.

Prometheus failures

Okay, this is really starting to sound like a DNS issue, which is what clues me in to the problem I had been having with Pi-hole. I cat the Prometheus container’s resolv.conf and see it has nameserver 127.0.0.11. Odd, but I suspect that’s perhaps specific to Docker and just a coincidence that it’s .11 matches system-apps. A quick google reveals that yes, this is the expected nameserver for a Docker container.

So let’s try something, because I now suspect that Docker spun up the container using the incorrect nameserver allocated to system-apps. I adjust system-apps’ resolv.conf, restart Prometheus, and everything’s working fine.

An inelegant fix that introduces complexity

Checking my original GitLab issue for the problem, I see I’d previously found Debian’s relevant documentation for pinning changes in resolv.conf. Alertnatively, it’s possible to configure the nameserver used by a Docker container. I opt for the former as it’s less hacky and implement it with a supersede command in /etc/dhcp/dhclient.conf as per Debian’s documentation.

root@system-apps:~# cat /etc/dhcp/dhclient.conf | grep "^supersede"
supersede domain-name "hosts.lan";
supersede domain-search "hosts.lan";
supersede domain-name-servers 127.0.0.1;

Restarting the Prometheus container and everything starts polling without issue.

Prometheus success

Trying not to go too deep into the rabbit hole

I could just stop here, but there is nagging thought that I can’t address without going down a rabbit hole for which I obviously have no time, but… Is this solution the most elegant? I suspect that there’s a way of doing this while introducing less complexity. Maybe there’s some Pi-hole option that allows it to start up with resolv.conf nameserver 10.0.0.11 instead of requiring nameserver 127.0.0.1? Attempting to do so causes DNS resolution is currently unavailable.

Others report the same problem and workaround, and a solution I should investigate has been posted. The Pi-hole documentation even marks this option as recommended.

--dns=127.0.0.1 Recommended

Sets your container’s resolve settings to localhost so it can resolve DHCP hostnames from Pi-hole’s DNSMasq, also fixes common resolution errors on container restart.

As a quick check, I revert the changes I’ve made to my dhclient.conf and resolv.conf, then I apply the following to my Pi-hole’s docker-compose.yml:

services:
  pihole:
+    dns:
+      - "127.0.0.1"

That seems to do it and is notably more elegant than modifying dhclient.conf and resolv.conf.

Three points come out of this whole exercise: