Leaderless Swarm Coordination: CRDT Action Registers Over Raft

When you put more than a handful of autonomous agents in a room, the first thing they start fighting over is who gets to do what. The classic answer is a leader: pick one agent, give it a lock, and let everyone else ask permission. That works until the leader dies, or the network partitions, or the leader simply becomes the bottleneck. The engineering instinct is to reach for Raft or Paxos and build a replicated log with a single elected leader. But for a swarm of agents that are supposed to be self-organizing, that's often the wrong tool. What you actually want is a data structure that lets every agent record its actions without asking anyone, and lets every other agent converge on the same view eventually. That's a CRDT, and specifically, an action register.

The Problem with Leader-Based Coordination

Raft solves a real problem: it gives you a totally ordered log that every node agrees on. The cost is that one node is the leader, and every write goes through it. For a database, that's fine. For a swarm of agents that need to coordinate in real time, it introduces a single point of failure and a coordination bottleneck. If the leader is partitioned away, the swarm is stuck. If the leader is slow, every agent waits.

More importantly, leader-based systems impose a mental model on your agents: they have to ask before they act. That's a serialization of behavior that fights against the very autonomy you're trying to build. Agents should be able to make local decisions and reconcile them later, not wait for a central authority to bless every move.

CRDTs 101: State-Based and Operation-Based

A Conflict-Free Replicated Data Type (CRDT) is a data structure that can be replicated across multiple nodes, updated independently, and merged without conflicts. There are two flavors: state-based (convergent) and operation-based (commutative).

State-based CRDTs merge by sending the full state. They're simpler to reason about, but can be expensive if the state is large. Operation-based CRDTs send only the operations, but require a reliable broadcast mechanism.

For an action register, we want something that tracks the latest action for a given key, but also allows for concurrent updates. A simple last-write-wins (LWW) register is a CRDT: store the timestamp and the value, and on merge, take the one with the highest timestamp. But timestamps have issues with clock skew. A better approach is to use a vector clock or a unique ID per agent to break ties.

Designing an Action Register

An action register maps an agent ID (or a task ID) to the last action that agent performed. Each entry has a value (the action), a timestamp (logical or physical), and a unique ID (agent ID + sequence number). When two agents concurrently update the same key, the merge rule is: compare timestamps, then compare IDs to break ties deterministically.

Here's a minimal implementation in Python using a dictionary and a custom merge function:

class ActionRegister:
    def __init__(self):
        self._entries = {}  # key -> (timestamp, agent_id, seq, action)

    def set(self, key, action, agent_id, timestamp):
        self._entries[key] = (timestamp, agent_id, timestamp, action)

    def merge(self, other):
        for key, (ts, agent, seq, action) in other._entries.items():
            if key not in self._entries:
                self._entries[key] = (ts, agent, seq, action)
            else:
                cur_ts, cur_agent, cur_seq, cur_action = self._entries[key]
                if (ts, agent, seq) > (cur_ts, cur_agent, cur_seq):
                    self._entries[key] = (ts, agent, seq, action)

This is a simplified version, but it captures the essence. The merge function is idempotent, commutative, and associative, which are the three properties a CRDT must satisfy.

Why Not Just Use a Log?

A log gives you a total order, but that's often more than you need. For action coordination, you typically only care about the latest state, not the entire history. A log forces you to replay all events to get the current state, which is wasteful. An action register gives you the current state directly, and the merge logic is trivial.

However, there's a tradeoff. With a log, you can detect conflicts easily because you see both operations. With a register, you lose that history—you only see the last write. If you need to audit what happened, you need to keep a separate log of all actions, but then you're back to storing the full history.

Handling Concurrent Actions: The Merge Rule Matters

In a swarm, two agents might try to claim the same task simultaneously. With an action register, both writes happen, and the merge rule decides which one wins. If you use a simple timestamp, the one with the later timestamp wins. But what if the timestamps are the same? You need a tiebreaker, like the agent ID.

This is where the design gets interesting. You have to decide what "winning" means for your use case. For task assignment, you might want the first come, first served. But with CRDTs, you can't have a total order without consensus. So you need to define a deterministic rule that all agents will agree on, even if it's not the "true" order.

One common approach is to use a hybrid logical clock (HLC) that combines a physical timestamp with a logical counter. This gives you a consistent ordering across nodes without needing a global clock. But even then, concurrent updates can have the same HLC, so you still need a tiebreaker.

Service Discovery Without a Leader

Action registers aren't just for task assignment. They can also serve as a distributed registry for agent capabilities. Each agent writes its current capabilities to a key-value store, and other agents read that to discover who can do what. Since it's a CRDT, any agent can update its own entry, and the rest of the swarm will eventually see it.

In practice, you might combine an action register with a membership list. Each agent periodically writes a heartbeat to its own key. If another agent sees a stale heartbeat, it can mark that agent as dead. This is a form of failure detection that doesn't require a leader.

The Role of Anti-Entropy and Gossip

For state-based CRDTs, you need a way to propagate state changes. Gossip protocols are a natural fit: each agent periodically sends its entire state (or a delta) to a random subset of peers. The peers merge and forward. This is how RiNET's WireGuard mesh might work if we were to implement this—though we haven't, so don't quote me on that.

The key insight is that you don't need perfect propagation. As long as the network eventually delivers messages, the CRDT will converge. The convergence is guaranteed by the mathematical properties of the merge function, not by any particular delivery order.

Practical Considerations: Storage and Sync

If you're running a swarm on three Hetzner servers, you might store the action register in a database like Postgres, but that's a single point of failure. Better to keep the register in memory, or in a replicated key-value store like etcd—but etcd uses Raft, which brings back the leader problem.

You could implement your own replication using CRDTs and a message bus like NATS or Redis Pub/Sub. Each agent maintains a local copy of the register, and merges incoming updates. This gives you eventual consistency with no leader.

The tradeoff is that you might have temporary inconsistencies. For many agent coordination tasks, that's acceptable. For example, if two agents both think they have the same task, the merge rule will resolve it, and one can back off.

Real-World Patterns: What Teams Typically Do

In industry, teams building agent swarms often start with a central coordinator, then move to a leaderless design as they scale. They typically observe that the central coordinator becomes a bottleneck, and they look for alternatives. Some adopt CRDTs for specific coordination tasks, like distributed counters or sets, but few use them for full action logs.

A common pattern is to use a CRDT for the swarm's shared state, but keep a separate audit log that is eventually consistent. The audit log can be a simple append-only log, which is itself a CRDT (a grow-only set).

Another pattern is to use a hybrid approach: use Raft for the control plane (like cluster membership) and CRDTs for the data plane (like task assignments). This gives you the best of both worlds: strong consistency for critical decisions, and eventual consistency for high-throughput actions.

When to Use Raft vs. CRDTs

Raft is the right choice when you need linearizable operations—when every read sees the latest write, and every write is totally ordered. That's essential for distributed databases, but for agent coordination, you often don't need that level of consistency.

CRDTs are the right choice when you can tolerate eventual consistency, and when you need high availability and partition tolerance. In a swarm, if the network partitions, you still want agents to act autonomously. With CRDTs, they can continue to work, and when the partition heals, they merge.

The catch is that you have to design your application to handle conflicts. That's not always easy. For example, if two agents both try to move a physical robot to the same location, the merge rule might pick one, but the robot can't be in two places at once. In that case, you need a higher-level conflict resolution strategy, like having the losing agent retry with a different location.

Implementation Sketch: A Simple Swarm Coordinator

Let's put it together. Suppose you have a swarm of agents, each with a unique ID. They need to coordinate on a set of tasks. Each task has a key, and the action register maps the task key to the agent that claimed it.

Here's a sketch of the agent logic:

class SwarmAgent:
    def __init__(self, agent_id, register):
        self.id = agent_id
        self.register = register

    def claim_task(self, task_id):
        # Try to claim the task by writing our ID to the register
        self.register.set(task_id, self.id, self.id, time.time())
        # Later, after merging, we can check if we actually got it
        # For now, we assume we might not win

    def check_claim(self, task_id):
        # After a gossip round, see who owns the task
        owner = self.register.get(task_id)
        return owner == self.id

This is a simplified version, but you get the idea. The agent writes its claim, then waits for gossip to propagate. It can't be sure it won until it sees the merged state.

Conclusion: The Tradeoff Is Inevitability

Leaderless coordination with CRDTs is a tradeoff: you give up the strong consistency of a single leader, but you gain autonomy and availability. For a swarm of agents, that's often the right call. The agents can act independently, and the system heals itself after partitions.

The key is to design your merge rules carefully and to understand that conflicts are part of life. You can't avoid them, but you can make them deterministic and easy to resolve.

So next time you're tempted to reach for Raft to coordinate your agents, ask yourself: do you really need a leader, or can you let the agents figure it out themselves? Sometimes the best way to lead is not to lead at all.