No Public DNS: Operating a Sovereign AI Stack with mDNS and Peer Discovery

You've built a self-hosted AI cluster. vLLM serves LLMs on three nodes, Qdrant runs on two more, and you have a Postgres instance with pgvector for embeddings. Everything is behind a WireGuard mesh. The problem? You can't use public DNS—no registrar, no cloud provider's DNS, no external resolver. You want full sovereignty: no leaks, no dependency, no third party knowing which hostnames map to which IPs.

This is the reality for many EU-based operators under GDPR and the EU AI Act. You can't afford to have DNS queries leaking metadata about your infrastructure. Even encrypted DNS (DoH/DoT) still hits an external resolver. The solution: mDNS (Multicast DNS) combined with peer discovery over WireGuard.

Why Not Just Use /etc/hosts?

Static host files work for a handful of nodes, but they don't scale. Every time you add a node, you must update every host. Worse, when IPs change (e.g., due to DHCP or dynamic VPN IPs), you're stuck manually editing files. mDNS solves this by letting nodes announce their hostnames locally and discover each other automatically.

The Stack

• WireGuard: encrypted mesh VPN (e.g., 10.0.0.0/24)
• Avahi: mDNS/Bonjour implementation (Linux)
• systemd-resolved: mDNS resolver built into systemd
• Optional: custom peer discovery script for dynamic registration

No external DNS server. No DNS forwarder. No split-horizon. Just multicast announcements over the WireGuard interface.

Step 1: WireGuard Mesh

Assume you have three nodes: node1, node2, node3. Each runs WireGuard with a private IP in the 10.0.0.0/24 range. Example config for node1:

[Interface]
PrivateKey = <node1_private>
Address = 10.0.0.1/24
ListenPort = 51820

[Peer]
PublicKey = <node2_public>
AllowedIPs = 10.0.0.2/32
Endpoint = node2.example.com:51820 # only for initial handshake, can be dynamic

[Peer]
PublicKey = <node3_public>
AllowedIPs = 10.0.0.3/32
Endpoint = node3.example.com:51820

But wait—you said no public DNS. The Endpoint fields above rely on public hostnames. For a fully sovereign setup, you need a different approach: either use IP addresses directly (if nodes have static public IPs) or use a dynamic discovery mechanism. I'll cover that later. For now, assume you have static IPs or a bootstrap mechanism.

Step 2: Enable mDNS on systemd-resolved

On each node, enable mDNS support in systemd-resolved:

# /etc/systemd/resolved.conf
[Resolve]
MulticastDNS=yes

Then restart the service:

systemctl restart systemd-resolved

Now each node can resolve .local hostnames. But by default, systemd-resolved only responds to queries on the local LAN interface. We need to make it work over the WireGuard interface.

Step 3: Configure Avahi for WireGuard

Avahi is more flexible than systemd-resolved for mDNS service publishing. Install it:

apt install avahi-daemon avahi-utils

Edit /etc/avahi/avahi-daemon.conf:

[server]
host-name=node1
# Use the WireGuard interface
allow-interfaces=wg0

[wide-area]
enable-wide-area=no

[publish]
publish-addresses=yes
publish-hinfo=no
publish-workstation=no

Restart Avahi:

systemctl restart avahi-daemon

Now node1 announces itself as node1.local on the WireGuard interface. Repeat for other nodes with appropriate hostnames.

But there's a catch: by default, Avahi binds to all interfaces. You want to restrict it to wg0 only to avoid leaking mDNS packets to your local LAN (which might contain other devices). Use allow-interfaces=wg0 as above.

Step 4: Verify Resolution

From node2, ping node1:

ping node1.local

If it resolves to 10.0.0.1, you're golden. Use avahi-browse -a to list all services on the network.

Step 5: Peer Discovery Without DNS

Now the tricky part: how do nodes discover each other's WireGuard endpoints without public DNS? If your nodes have static public IPs, just hardcode them. But if they're behind NAT or have dynamic IPs, you need a discovery mechanism.

One approach: use a simple UDP broadcast or multicast for endpoint discovery. Write a small script that listens on a known UDP port (e.g., 51900) and announces its own public IP and WireGuard port. Each node broadcasts its endpoint periodically. When a new node comes online, it listens for these broadcasts and updates its WireGuard config.

Here's a minimal Python example using multicast:

import socket
import struct
import time

MCAST_GRP = '239.255.255.250'
MCAST_PORT = 51900

def sender(public_ip, wg_port):
    sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
    sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 2)
    while True:
        message = f"{public_ip}:{wg_port}"
        sock.sendto(message.encode(), (MCAST_GRP, MCAST_PORT))
        time.sleep(60)

def receiver():
    sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    sock.bind(('', MCAST_PORT))
    mreq = struct.pack('4sl', socket.inet_aton(MCAST_GRP), socket.INADDR_ANY)
    sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)
    while True:
        data, addr = sock.recvfrom(1024)
        print(f"Received endpoint: {data.decode()}")

Run the sender on each node with its public IP and WireGuard port. The receiver updates /etc/wireguard/wg0.conf and runs wg-quick up wg0 to apply changes.

This is a minimal example—you'd want to add authentication (e.g., pre-shared key) and handle conflicts.

Step 6: Service Discovery for AI Components

Now that nodes can find each other, we need services to find each other. For example, your LLM inference node (vLLM) should be discoverable by your agent orchestrator. Use Avahi's service publishing:

avahi-publish-service vllm-node _vllm._tcp 8000 "LLM Inference Node 1"

Other nodes can browse for _vllm._tcp to get the hostname and port. This works across the WireGuard mesh thanks to mDNS.

Similarly, publish Qdrant gRPC (port 6334) and Postgres (port 5432). Your application code can use socket.getaddrinfo to resolve .local names.

Step 7: Integrating with systemd Services

To make services start automatically and publish their presence, write systemd service units that run avahi-publish-service after the main service starts. Example for vLLM:

[Unit]
Description=vLLM Inference Server
After=network.target

[Service]
ExecStart=/usr/local/bin/vllm serve ...
ExecStartPost=/usr/bin/avahi-publish-service vllm-node _vllm._tcp 8000 "LLM Node"
Restart=always

[Install]
WantedBy=multi-user.target

Caveats and Gotchas

• mDNS is not routable: It works only within the same broadcast domain. WireGuard creates a virtual L2 segment if you use bridging, but by default WireGuard is L3. For mDNS to work across WireGuard, you need to enable multicast forwarding. Add Multicast=yes to the WireGuard interface config (requires kernel >= 5.7). Or use Avahi's reflector mode (reflect-ipv4=yes in avahi-daemon.conf).

• Scalability: mDNS is not designed for large networks. For clusters with more than ~20 nodes, consider using a lightweight DNS server like CoreDNS with a dynamic update mechanism. But for most AI stacks, 5-10 nodes is typical.

• Security: mDNS packets are unencrypted. Since they're confined to your WireGuard interface, this is acceptable. But ensure no untrusted nodes join the mesh.

• Hostname conflicts: If two nodes claim the same hostname, mDNS will pick one (first-come, first-served). Use unique names.

Alternative: Consul or etcd

If you want a more robust solution, consider running a small Consul cluster (or etcd) on your mesh. But that adds complexity: you need a bootstrap mechanism and consensus. mDNS with Avahi is simpler and sufficient for most on-prem AI setups.

Real-World Example: EU AI Cluster

I run a 4-node cluster in a German colo. Each node has a public IP, but I don't want DNS records linking my infrastructure. I use WireGuard with static public IPs in the config (no DNS). mDNS over WireGuard allows services to discover each other. My agent swarm uses pgvector.local:5432 to connect to Postgres, qdrant.local:6334 for vector DB, and inference.local:8000 for LLM. No external DNS, no leaks.

Conclusion

You don't need public DNS to run a sovereign AI stack. mDNS plus peer discovery over WireGuard gives you automatic hostname resolution without external dependencies. It's not perfect for huge clusters, but for the typical 5-10 node on-prem setup, it's clean, simple, and sovereign.

Now go delete those DNS records.