No Public Endpoints: WireGuard-Meshing Every Service in a Sovereign AI Stack

If you’re running a sovereign AI stack—LLM inference, vector databases, embedding pipelines, agent orchestrators—you’ve probably got a bunch of services that need to talk to each other. The default is to expose each one on a public IP with a firewall rule, maybe behind a reverse proxy. That’s a lot of surface area. Every public endpoint is a target. Every open port is a risk.

I’m going to show you how to eliminate public endpoints entirely by building a WireGuard mesh. Every service gets a private IP on a mesh network. No ports open to the internet. Zero attack surface from the outside. All communication happens over encrypted tunnels.

Why Mesh, Not Hub-and-Spoke?

A typical VPN setup uses a central server (hub) that all clients connect to. Traffic between two clients goes through the hub. That’s fine for a small office, but for an AI stack with multiple nodes—say, a GPU server running vLLM, a storage node with Postgres, an embedding server with BGE-M3, and a control plane for agent orchestration—hub-and-spoke adds latency and a single point of failure. A mesh gives every node a direct encrypted tunnel to every other node. No central bottleneck. Lower latency. Better resilience.

WireGuard is perfect for this: it’s simple, fast, and built into the Linux kernel since 5.6. No bloated daemons. No certificate authorities. Just a few config files and a pair of keys per node.

The Architecture

We’ll create a private subnet, say 10.0.0.0/16. Each service gets a unique IP in that range. WireGuard peers connect directly to each other. We’ll use a small orchestration tool or a script to distribute peer configs, but the mesh itself is fully decentralized.

Here’s a three-node example:

• Node A (GPU server): Runs vLLM for LLM inference. IP: 10.0.0.1
• Node B (storage): Runs Postgres with pgvector. IP: 10.0.0.2
• Node C (control plane): Runs an agent orchestrator (e.g., LangChain + FastAPI). IP: 10.0.0.3

Each node has a WireGuard interface (wg0) with its private key and a config that lists all other nodes as peers.

Step-by-Step: Building the Mesh

1. Install WireGuard

On every node (assuming Debian/Ubuntu):

sudo apt update && sudo apt install wireguard

For kernel < 5.6, install wireguard-dkms as well. On modern kernels, it’s built-in.

2. Generate Keys

On each node, generate a key pair:

wg genkey | tee /etc/wireguard/private.key | wg pubkey > /etc/wireguard/public.key
chmod 600 /etc/wireguard/private.key

3. Configure the Interface

Create /etc/wireguard/wg0.conf on node A:

[Interface]
Address = 10.0.0.1/16
PrivateKey = <NodeA_private_key>
ListenPort = 51820

# Node B
[Peer]
PublicKey = <NodeB_public_key>
Endpoint = <NodeB_public_ip>:51820
AllowedIPs = 10.0.0.2/32
PersistentKeepalive = 25

# Node C
[Peer]
PublicKey = <NodeC_public_key>
Endpoint = <NodeC_public_ip>:51820
AllowedIPs = 10.0.0.3/32
PersistentKeepalive = 25

On node B:

[Interface]
Address = 10.0.0.2/16
PrivateKey = <NodeB_private_key>
ListenPort = 51820

[Peer]
PublicKey = <NodeA_public_key>
Endpoint = <NodeA_public_ip>:51820
AllowedIPs = 10.0.0.1/32
PersistentKeepalive = 25

[Peer]
PublicKey = <NodeC_public_key>
Endpoint = <NodeC_public_ip>:51820
AllowedIPs = 10.0.0.3/32
PersistentKeepalive = 25

And similarly for node C.

Note: Endpoint is the public IP of the peer. If nodes are behind NAT, you’ll need to configure port forwarding or use a relay. For cloud VMs with public IPs, it’s straightforward.

4. Enable and Start

sudo systemctl enable wg-quick@wg0
sudo systemctl start wg-quick@wg0

5. Verify

From any node, ping the others:

ping 10.0.0.2
ping 10.0.0.3

Check the handshake:

sudo wg show

You should see all peers with a recent handshake time.

Running Services Over the Mesh

Now that every node is on 10.0.0.0/16, you configure your services to listen only on the WireGuard IP. For example:

• Postgres on node B: Edit postgresql.conf to set listen_addresses = '10.0.0.2'. No public port.
• vLLM on node A: Start the server with --host 10.0.0.1.
• Agent orchestrator on node C: Connect to Postgres at 10.0.0.2:5432 and to vLLM at 10.0.0.1:8000.

No firewall rules needed except to allow UDP on port 51820 from peer public IPs. That’s it. The rest of the ports are closed to the internet.

Scaling the Mesh

For a larger stack (say, 10+ nodes), manually editing configs becomes tedious. Use a simple inventory file and a script to generate configs. Here’s a minimal approach:

Create a YAML file mesh.yml:

nodes:
  - name: gpu-server
    public_ip: 203.0.113.1
    private_ip: 10.0.0.1
    private_key: ... (or generate at runtime)
  - name: db-server
    public_ip: 203.0.113.2
    private_ip: 10.0.0.2
    private_key: ...
  # ...

Then a script reads this, generates keys if missing, and writes wg0.conf for each node. Distribute the configs via SSH or Ansible.

For true mesh automation, consider tools like netmaker or headscale (Tailscale-compatible control server) but that adds complexity. I prefer a bare-bones script—less moving parts, easier to audit.

Handling NAT and Dynamic IPs

If nodes are behind NAT (e.g., home servers), you need a public relay or a STUN-like solution. One option: run a small WireGuard “hub” on a cheap VPS that forwards traffic. But that reintroduces a central point. Better: use a dynamic DNS service and set PersistentKeepalive = 25 to keep NAT mappings alive. For full mesh behind NAT, consider boringtun or tailscale, but those are heavier. For sovereign AI stacks, I recommend all nodes have public IPs (even if just a /32 from a cloud provider). It’s simpler and more reliable.

Security Considerations

• Firewall: Only allow UDP 51820 from known peer IPs. Nothing else.
• Key management: Store private keys securely. Use a secrets manager or encrypted files. Rotate keys periodically.
• AllowedIPs: Be precise. Only allow the exact /32 for each peer. No wildcards.
• No DNS leak: Ensure your services don’t fall back to public DNS for internal names. Use /etc/hosts or a private DNS resolver on the mesh.
• Audit: Periodically run wg show to verify peer states. Log any failed handshakes.

Real-World Example: Replacing SaaS with On-Prem

I run a sovereign AI stack for a client that replaced OpenAI + Pinecone with self-hosted alternatives. The stack:

• Inference: vLLM (Llama 3 70B) on a single A100 node.
• Embeddings: BGE-M3 on a separate node with 4x RTX 4090s.
• Vector DB: Qdrant on a node with NVMe SSDs.
• Postgres: TimescaleDB with pgvector for metadata and hybrid search.
• Orchestration: Custom agent swarm (Python + Celery) on a control node.

All nodes are in a WireGuard mesh. The only public endpoint is the control node’s API, but even that is behind a reverse proxy (Caddy) that terminates TLS and proxies to the mesh IP. The inference and database nodes have zero public exposure. If the control node is compromised, the attacker still can’t reach the database or GPU nodes without a WireGuard key.

Result: Pentest found zero open ports on the inference and database nodes. Compliance auditors were happy. Latency between nodes is under 1ms (same datacenter).

When Not to Mesh

If you have only two nodes, a simple point-to-point WireGuard is fine. If you have many nodes with dynamic IPs or heavy churn, consider a mesh networking tool like tinc or nebula. But for a stable AI stack with <20 nodes, manual WireGuard mesh is unbeatable.

Conclusion

WireGuard meshing gives you a private, encrypted network for your sovereign AI stack with zero public endpoints. It’s simple to set up, fast, and secure. Stop exposing services to the internet. Mesh them.

Further Reading

• WireGuard official site: https://www.wireguard.com/
• systemd network namespaces for isolation: systemd-nspawn or networkd
• For containerized services: run WireGuard in a sidecar container or use podman with network plugins.