Raft for Agent Consensus: When Leaderless Coordination Beats Central Orchestration

Central orchestration is the default for agent swarms. A single coordinator decides who does what, when. It's simple to reason about, but it's a single point of failure and a bottleneck. I've been there: you scale to fifty agents, and the orchestrator starts choking. You add a retry loop, then a queue, then a database to track state, and suddenly you've built a distributed system without the tools for it.

We hit that wall on a self-hosted project. The fix was to stop orchestrating and start coordinating with Raft. This is the story of that transition and the lessons we learned.

Why central orchestration fails at scale

Central orchestration means one process—the orchestrator—holds all state and makes all decisions. Agents ask for tasks, report results, and wait for the next command. It works for small systems, but as you grow:

• Reliability: If the orchestrator dies, everything stops. You can run it in HA mode with a standby, but failover is never instant and often loses in-flight tasks.
• Throughput: Every decision goes through one process. Even with async I/O, you hit CPU or network limits. Requests queue up, latency spikes, and agents idle.
• State management: The orchestrator tracks every agent's progress, task status, and resource allocation. That's a lot of mutable state in one place. You end up with a database that becomes the real bottleneck.
• Coupling: Agents are dumb clients. They can't act without instruction, which means they can't handle edge cases or adapt to partial failures.

We saw all three. Our orchestrator (a Python service with a Postgres backend) would get stuck in a loop, or the database connection pool would exhaust, and the whole swarm would stall. We needed something that didn't have a single point of failure.

Enter Raft: leaderless coordination

Raft is a consensus algorithm that provides replicated state machines across a cluster of nodes. At any moment, one node is the leader, but if it fails, the rest elect a new one automatically. The key is that the cluster as a whole is the authority, not any single process.

For agent coordination, we don't need a leader at all in the sense of "the one who decides." Instead, we use Raft to maintain a shared log of tasks and decisions. Each agent is a Raft node. They all participate in consensus, so no single agent is special. When a task needs to be assigned, agents propose it to the log, and the log order determines who does what.

This is leaderless in the sense that there's no central orchestrator. There is a Raft leader for the consensus protocol, but that leader is just a coordinator for the log, not a decision-maker. If it dies, another agent takes over, and the swarm continues seamlessly.

How we implemented it

We used etcd, which implements Raft, as the coordination layer. Each agent runs an etcd client and participates in the cluster. The state we keep in etcd:

• Task queue: A list of pending tasks, each with a unique ID and parameters.
• Agent registry: Which agents are alive and their capabilities.
• Task assignments: Which agent is working on which task.
• Shared config: Parameters that agents need, like model endpoints or timeouts.

We use etcd's atomic operations to claim tasks. When an agent wants a task, it issues a transaction that checks if a task is unassigned and then writes its own ID to the assignment key. Only one agent wins the transaction, so no two agents ever pick up the same task.

Here's a simplified example of how a task gets claimed:

# Each agent attempts to claim task 'task-123'
etcdctl txn --interactive <<'EOF'
mod key("/tasks/task-123/assignee") = ""
put key "/tasks/task-123/assignee" value "agent-42"
EOF

If the transaction succeeds, the agent owns the task. If it fails, someone else got it first.

The benefits we saw

Fault tolerance: When an agent dies, its tasks remain in the queue with no assignee. Another agent picks them up. When the orchestrator died, we had to restart it manually and replay tasks from a database. Now we lose nothing.

Horizontal scaling: Adding agents is trivial. Each new agent joins the etcd cluster and starts claiming tasks. No central server to reconfigure. We went from 5 to 50 agents without touching the coordination layer.

Decentralized decision-making: Agents can make local decisions based on the shared state. For example, if a task requires a GPU, an agent with a GPU can claim it preferentially. The cluster doesn't need to know about GPU availability; the agents self-select.

Simpler code: The orchestrator was thousands of lines of business logic. Now, each agent has a simple loop: watch for tasks, claim one, execute, write result. The coordination logic is in etcd's transactions, not in custom code.

When Raft is overkill

Raft isn't a silver bullet. For small swarms (under 10 agents) where tasks are infrequent and reliability isn't critical, a central orchestrator is simpler and easier to debug. Raft adds latency (every consensus write requires a round-trip to the majority) and operational complexity (you need to manage an etcd cluster).

Also, if your agents are stateless and tasks are independent, you might not need consensus at all. A simple work queue like Redis or RabbitMQ with competing consumers gives you fault tolerance without the overhead. Raft shines when you need a shared, consistent view of state—like task assignments, progress tracking, or configuration—across many nodes.

Lessons learned

1. Design for consensus from the start. We retrofitted Raft onto an existing system, and it was painful. The old code assumed a single orchestrator. We had to refactor everything to think in terms of shared state and atomic operations. If you're building a swarm, decide early whether you need consensus.

2. Keep the state small. etcd has a write limit (default 1.5 MB per request, but keep it small for performance). We store only task metadata, not results. Results go to object storage or a database. The coordination layer should be lean.

3. Watch for split-brain. Raft ensures consensus within a cluster, but if you have a network partition, you can have two clusters. We use etcd's built-in cluster membership and set a quorum size. If you lose quorum, the cluster stops accepting writes, which is better than splitting.

4. Test failure scenarios. We simulate node failures, network partitions, and slow nodes. It's humbling to see how many subtle bugs surface. Use chaos testing tools like Chaostoolkit.

5. Use transactions for anything that must be atomic. Claiming a task, updating status, and recording results should be atomic. etcd's transactions are your friend.

Example: Agent task loop with etcd

Here's a pseudo-code of the agent loop:

import etcd3

# Connect to etcd cluster
client = etcd3.client(host='etcd-1', port=2379)

while True:
    # Watch for pending tasks
    tasks = client.get_prefix('/tasks/pending/')
    for task_key, task_value in tasks:
        # Try to claim the task
        txn = client.transaction()
        txn.compare(
            key=task_key + '/assignee',
            value=None,  # unassigned
            op='eq'
        )
        txn.put(task_key + '/assignee', agent_id)
        if txn.commit():
            # We got the task
            execute_task(task_value)
            client.delete(task_key + '/assignee')
            client.put(task_key + '/status', 'done')
            break

This loop runs in every agent. The transaction ensures only one agent claims a task. If an agent crashes mid-task, the assignment remains, but we have a watchdog that clears stale assignments after a timeout.

Real-world results

We run a swarm of 30 agents on a mix of on-prem servers and a few cloud VMs. Each agent runs a container with a large language model (we use vLLM for inference) and a set of tools. The etcd cluster runs on three dedicated nodes.

Before, the orchestrator would fail about once a week, causing a stall of 10-15 minutes. Now, we haven't had a single full outage in three months. The system gracefully handles node reboots, network blips, and even a full datacenter maintenance window.

Throughput improved because agents don't wait for a central server to respond. They claim tasks as soon as they're free. In similar deployments, teams typically observe significant reductions in task completion time for large batches.

Alternatives to etcd

We chose etcd because it's battle-tested and has a simple API. But there are other Raft implementations:

• Consul (HashiCorp) uses Raft and offers service discovery, which pairs well with agents.
• Apache ZooKeeper is older but still used; it's more complex.
• Hashicorp's memberlist provides gossip-based membership, but not consensus.

If you want to embed Raft directly in your agents, you could use a library like hasicorp/raft or etcd-io/raft. We chose etcd as a separate service for simplicity and because we wanted a battle-tested system.

When central orchestration is still better

Central orchestration isn't always wrong. If you have a small, fixed set of agents that rarely change, and you need strict control over execution order, a central orchestrator is simpler. Also, if you need to enforce complex business rules that span multiple tasks, a single decision-maker is easier to code.

But for autonomous agents that need to be resilient and scale, leaderless coordination with Raft is a game-changer. It's the difference between a system that survives failures and one that fails on its own.

Conclusion

Raft gave us a way to build an agent swarm that is truly self-organizing. The coordination is done through a replicated log, not a single brain. This shift eliminated our single point of failure and let us scale horizontally without rearchitecting.

If you're building a swarm that needs to be reliable, consider Raft. It's not the easiest path, but it's the one that leads to a system that doesn't fall apart when a node goes down. Start with etcd, keep your state minimal, and test failure scenarios until you're confident. Your agents will thank you.

Damir Radulić writes about self-hosted AI and distributed systems. This article is based on a real deployment of a self-hosted agent swarm using etcd and vLLM.