Running Agent Swarms in Docker Without Kubernetes: An Egress-Only Networking Approach

When I started building autonomous agent swarms, Kubernetes seemed like the obvious choice. But the overhead—both cognitive and operational—wasn't justified for my use case. I needed something simpler: a way to run dozens of agents that communicate, coordinate, and execute tasks without a heavyweight orchestrator. Docker Compose with an egress-only networking pattern turned out to be the sweet spot.

This article walks through the architecture, networking design, and deployment of agent swarms using plain Docker. No Kubernetes, no service mesh, no fluff. Just containers, networks, and a few environment variables.

The Problem with Kubernetes for Agent Swarms

Kubernetes excels at running stateless microservices that need auto-scaling, rolling updates, and service discovery. But agent swarms are different:

• Agents are long-running, stateful processes that maintain internal state (conversation history, task queues, model caches).
• Communication patterns are peer-to-peer, not request-response. Agents broadcast messages, subscribe to topics, and negotiate tasks.
• The swarm topology is dynamic: agents join and leave based on workload, not HTTP traffic.

Kubernetes' service mesh and ingress controllers add latency and complexity that hurt agent responsiveness. I wanted a network where agents could talk to each other without going through a load balancer or sidecar proxy.

Egress-Only Networking: The Core Idea

In an egress-only network, containers can initiate outbound connections to the outside world, but inbound connections from outside are blocked. This is a common pattern for security, but it also simplifies agent communication.

Here's the twist: agents don't need to be reachable from the outside. They only need to reach:

• A shared message broker (like NATS or Redis) for inter-agent communication.
• External APIs or databases for task execution.
• A control plane (optional) for monitoring and debugging.

By making the swarm egress-only, we eliminate the need for complex ingress configuration, port mapping, and DNS-based service discovery. Agents find each other through the message broker, not through IP addresses.

Architecture Overview

Our swarm consists of three components:

1. Message Broker: A lightweight pub/sub system (NATS or Redis) that agents use to communicate.
2. Agent Containers: Each agent runs in its own container, with a unique role (e.g., planner, executor, monitor).
3. Control Plane (optional): A container that provides a CLI or web interface to inspect swarm state. It connects to the broker but doesn't expose ports.

All containers share a Docker network that is egress-only: they can reach the internet and the broker, but the broker's port is not exposed to the host. The control plane, if needed, can be accessed via docker exec or a separate management network.

Docker Compose Configuration

Here's a docker-compose.yml that sets up the swarm:

version: '3.8'

services:
  broker:
    image: nats:2.9-alpine
    command: "-js"  # JetStream enabled for persistence
    networks:
      - swarm-net
    # No ports exposed to host

  agent-planner:
    build: ./agent-planner
    environment:
      - BROKER_URL=nats://broker:4222
      - AGENT_ROLE=planner
    depends_on:
      - broker
    networks:
      - swarm-net
    # No ports exposed

  agent-executor:
    build: ./agent-executor
    environment:
      - BROKER_URL=nats://broker:4222
      - AGENT_ROLE=executor
    depends_on:
      - broker
    networks:
      - swarm-net

  agent-monitor:
    build: ./agent-monitor
    environment:
      - BROKER_URL=nats://broker:4222
      - AGENT_ROLE=monitor
    depends_on:
      - broker
    networks:
      - swarm-net

networks:
  swarm-net:
    driver: bridge
    internal: false  # Allows egress to internet

Key points:

• No ports are exposed on any container. The broker is only reachable within swarm-net.
• Agents use environment variables to discover the broker URL. No DNS resolution beyond Docker's internal DNS.
• The bridge network is not internal: true, so containers can reach the internet (e.g., to call external APIs).

Agent Communication Pattern

Agents communicate via NATS subjects. Each agent subscribes to a subject based on its role and publishes messages to subjects for coordination. For example:

• Planner publishes tasks to tasks.new.
• Executors subscribe to tasks.new and publish results to tasks.completed.
• Monitor subscribes to all subjects for logging.

Here's a minimal Python agent using nats-py:

import asyncio
import nats
import os

BROKER_URL = os.getenv("BROKER_URL", "nats://localhost:4222")
ROLE = os.getenv("AGENT_ROLE", "unknown")

async def main():
    nc = await nats.connect(BROKER_URL)
    js = nc.jetstream()
    
    if ROLE == "planner":
        # Publish a task every 10 seconds
        async def publish_task():
            while True:
                await asyncio.sleep(10)
                await js.publish("tasks.new", b"{\"type\": \"compute\", \"data\": [1,2,3]}")
        asyncio.create_task(publish_task())
    elif ROLE == "executor":
        # Subscribe to tasks
        async def handle_task(msg):
            data = msg.data.decode()
            print(f"Received task: {data}")
            # Process...
            await js.publish("tasks.completed", b"{\"status\": \"done\"}")
        await js.subscribe("tasks.new", cb=handle_task)
    
    await asyncio.Future()  # Run forever

if __name__ == "__main__":
    asyncio.run(main())

Agents don't know each other's IP addresses. They only know the broker. This decouples agents and makes scaling trivial: just start more containers with the same role.

Scaling Without Kubernetes

To scale the executor pool, you can use Docker Compose's --scale flag:

docker-compose up -d --scale agent-executor=5

Each new executor container gets a unique hostname (e.g., agent-executor_2) but subscribes to the same NATS subject. The broker distributes messages across subscribers. This is a simple form of work queue.

For dynamic scaling based on load, you can write a small script that monitors NATS queue depth and adjusts the scale factor:

#!/bin/bash
# Pseudo-code: check queue depth and scale
QUEUE_DEPTH=$(docker exec broker nats sub --count tasks.new 2>&1 | tail -1)
if [ "$QUEUE_DEPTH" -gt 100 ]; then
    docker-compose up -d --scale agent-executor=10
fi

This is not as sophisticated as Kubernetes' HPA, but it works for many scenarios without the overhead.

Security Implications

Egress-only networking reduces the attack surface. Since no ports are exposed, there's no way for an attacker to directly connect to an agent container. The only entry point is the message broker, which can be secured with authentication and TLS.

To further isolate agents, you can use Docker's internal network for the broker and a separate external network for internet access. But for simplicity, a single bridge network with internal: false is sufficient for most use cases.

Lessons Learned

1. Broker selection matters: NATS with JetStream is ideal because it provides persistence and exactly-once delivery semantics. Redis Pub/Sub is simpler but doesn't persist messages if a subscriber is offline.
2. Health checks: Use Docker's healthcheck to ensure agents are connected to the broker. If an agent loses connection, it should restart.
3. Logging: Centralize logs using Docker's logging driver (e.g., fluentd or gelf) rather than relying on agents to write to files.
4. State persistence: Agents that need to persist state should use volumes or an external database. The egress-only network allows them to reach a database outside Docker.

When This Approach Breaks Down

This pattern is not suitable for every scenario. Avoid it if:

• You need agents to be directly accessible from outside (e.g., for webhooks).
• You require fine-grained service discovery with DNS SRV records.
• Your swarm spans multiple hosts without a Docker Swarm or overlay network.

For single-host or small multi-host setups, Docker Compose with an egress-only network is a pragmatic choice. For larger deployments, consider Docker Swarm (which adds multi-host networking) or Nomad.

Conclusion

Running agent swarms without Kubernetes is not only possible but often simpler. By leveraging Docker's native networking and an egress-only design, you get a secure, scalable, and maintainable orchestration layer with minimal overhead. The key is to embrace the broker pattern and let agents communicate asynchronously through a shared message bus.

Next time you're tempted to spin up a Kubernetes cluster for a swarm of 10 agents, ask yourself: do I really need all that complexity? Sometimes a docker-compose up is all you need.