When Agents Loop: Detecting and Breaking Infinite Cycles Without a Kill Switch

We ran a 50-agent swarm for an ETL pipeline. After 12 minutes of silence, I checked the logs. Agent C was calling Agent D, which called Agent E, which called Agent C again. 47,000 messages deep, no output. No kill switch—because we deliberately built without one. Here's how we fixed it.

The Anatomy of an Agent Loop

Agent loops aren't just infinite while(true) bugs. They're subtle: agent A sends a message, agent B processes and sends to C, C sends back to A because of a misrouted event type. In our case, a data_ready event was being forwarded instead of consumed. The loop looked like:

C -> D (transform_request)
D -> E (transform_result)
E -> C (data_ready)

Each iteration consumed 0.3s and 2MB of memory. After 1000 iterations, we had 2GB of heap pressure and zero throughput.

Detection Strategies That Work

1. Topological Ordering with Cycle Detection

Before deploying a swarm, we now compute a dependency graph and check for cycles using DFS-based topological sort. If a cycle exists, we fail deployment.

from collections import defaultdict

def has_cycle(edges):
    graph = defaultdict(list)
    for u, v in edges:
        graph[u].append(v)
    
    WHITE, GRAY, BLACK = 0, 1, 2
    color = {node: WHITE for node in graph}
    
    def dfs(node):
        color[node] = GRAY
        for neighbor in graph[node]:
            if color[neighbor] == GRAY:
                return True
            if color[neighbor] == WHITE and dfs(neighbor):
                return True
        color[node] = BLACK
        return False
    
    for node in list(graph):
        if color[node] == WHITE:
            if dfs(node):
                return True
    return False

# Example: C->D, D->E, E->C
edges = [('C','D'), ('D','E'), ('E','C')]
print(has_cycle(edges))  # True

This catches static cycles at deploy time. But dynamic cycles—where agents subscribe to topics based on runtime conditions—need runtime detection.

2. Message TTL and Hop Count

Every message carries a max_hops field. We set a default of 10 for most workflows. Each agent decrements the hop count. If it reaches zero, the message is dropped and logged.

class Message:
    def __init__(self, payload, max_hops=10):
        self.payload = payload
        self.max_hops = max_hops
        self.hop_count = 0
    
    def increment_hop(self):
        self.hop_count += 1
        return self.hop_count <= self.max_hops

In practice, we saw loops where messages survived 50+ hops before hitting the limit. The cost: each hop adds ~1ms latency. Acceptable.

3. State Machine with Timeout

Instead of a kill switch, each agent runs a state machine with a RUNNING state that transitions to TIMEOUT if no progress is made within a configurable window. We track last output timestamp.

import time

class AgentStateMachine:
    def __init__(self, timeout_seconds=30):
        self.state = 'IDLE'
        self.last_output = time.time()
        self.timeout = timeout_seconds
    
    def on_message(self, msg):
        self.state = 'RUNNING'
        # process...
        self.last_output = time.time()
        self.state = 'IDLE'
    
    def check_timeout(self):
        if self.state == 'RUNNING' and time.time() - self.last_output > self.timeout:
            self.state = 'TIMEOUT'
            self.cleanup()

We set a 30-second timeout per agent. In the loop scenario, each agent's last_output never updates because the loop doesn't produce final output. After 30 seconds, the agent self-terminates and emits a dead_letter event.

Breaking the Loop Without Killing

Once detected, breaking gracefully is key. We use three techniques:

1. Circuit Breaker Pattern

Each agent maintains a counter of consecutive repeated message types. If the same message type appears more than 5 times in a row, the agent enters a CIRCUIT_OPEN state and drops all messages of that type for 60 seconds.

class CircuitBreaker:
    def __init__(self, threshold=5, cooldown=60):
        self.threshold = threshold
        self.cooldown = cooldown
        self.counts = defaultdict(int)
        self.open_until = {}
    
    def allow(self, msg_type):
        now = time.time()
        if msg_type in self.open_until and now < self.open_until[msg_type]:
            return False
        return True
    
    def record(self, msg_type):
        self.counts[msg_type] += 1
        if self.counts[msg_type] >= self.threshold:
            self.open_until[msg_type] = time.time() + self.cooldown
            self.counts[msg_type] = 0

2. Dead Letter Queue

Instead of dropping, we redirect looping messages to a dead letter queue (DLQ). A separate monitor agent analyzes DLQ patterns and can alert or auto-adjust routing.

# In agent
if msg.max_hops <= 0:
    dlq.put(msg)
    return

3. Backpressure via Token Bucket

Each agent has a token bucket that limits processing rate. If an agent is flooded by a loop, the bucket empties and new messages are queued with backpressure. This prevents memory blowup and gives the circuit breaker time to open.

import time

class TokenBucket:
    def __init__(self, rate=10, capacity=20):
        self.rate = rate
        self.capacity = capacity
        self.tokens = capacity
        self.last_refill = time.time()
    
    def consume(self, tokens=1):
        self._refill()
        if self.tokens >= tokens:
            self.tokens -= tokens
            return True
        return False
    
    def _refill(self):
        now = time.time()
        elapsed = now - self.last_refill
        self.tokens = min(self.capacity, self.tokens + elapsed * self.rate)
        self.last_refill = now

Real-World Numbers

After implementing these, we saw:

• Loop detection latency: <10ms per message (hop count check)
• False positives: 0 in 2 weeks (we tuned threshold to 15 for some agents)
• Compute waste reduction: 94% (from 12 minutes of loop to <30s timeout)
• DLQ volume: 0.01% of total messages

The system now self-heals: if a loop forms, the circuit breaker opens, messages go to DLQ, and an alert fires. No kill switch needed.

Lessons Learned

1. Static cycle detection is mandatory for deploy-time safety. Use topological sort.
2. Hop counts are cheap but effective. Start with 10, adjust per workflow.
3. Timeouts must be per-agent, not per-swarm. A single stuck agent shouldn't stall everything.
4. Circuit breakers prevent cascading failures. Reset cooldown to give time for manual fix.
5. Backpressure is your friend. Without it, a loop can OOM the entire node.

We now ship every agent with these patterns baked into the base class. The result: zero manual interventions for loops in the last 3 months.