Unit Tests for Agent Behavior: Validating Swarm Actions Before They Hit Production

Agent swarms are notoriously difficult to test. Unlike traditional software, where a function returns a predictable output for a given input, agents make decisions based on context, history, and sometimes randomness. A single agent might call a tool, generate text, or delegate to another agent. A swarm of agents compounds this complexity: actions are interleaved, state is distributed, and failures cascade. Yet, shipping untested agent code is a recipe for disaster. In production, a misbehaving agent can delete data, overcharge customers, or leak sensitive information.

At RINET, we've developed a methodology for unit-testing agent behavior that catches bugs before they hit production. This article walks through our approach: deterministic decision testing for individual agents, and interaction testing for multi-agent swarms. We'll use a concrete example: a swarm that processes support tickets by classifying intent, fetching context, and composing a reply. All code snippets are in Python, using pytest and a custom agent framework.

Why Traditional Testing Falls Short

Unit tests for functions are straightforward: mock dependencies, call the function, assert the result. But agents are stateful and non-deterministic. An LLM call might return different responses for the same prompt due to temperature. Tool calls depend on external APIs. Swarm interactions depend on message ordering. Traditional mocks help, but they don't capture the agent's decision logic.

We need tests that verify:

• Given a specific state and input, does the agent choose the correct action?
• Given a sequence of messages, does the swarm produce the expected final state?
• Are error paths handled gracefully?

Deterministic Decision Testing for Individual Agents

The core of an agent is its decision loop: given a context (messages, state, tools), it decides what to do next. We can unit-test this decision by making the LLM call deterministic. How? By mocking the LLM to return a predefined response that triggers a specific tool call or output.

Consider an agent that classifies a support ticket. It has two tools: fetch_customer_info and escalate_to_human. The agent should call fetch_customer_info if the ticket mentions a known customer ID, otherwise escalate.

Here's the agent code (simplified):

class TicketClassifier:
    def __init__(self, llm, tools):
        self.llm = llm
        self.tools = tools

    def decide(self, messages):
        # LLM returns a structured decision: tool_name and arguments
        response = self.llm.chat(messages, tools=self.tools)
        if response.tool_call:
            return response.tool_call
        else:
            return None

To test this, we mock the LLM to return a specific tool call. We use pytest's monkeypatch or a mock library:

import pytest
from unittest.mock import MagicMock

class MockLLM:
    def chat(self, messages, tools):
        # Simulate LLM deciding to call fetch_customer_info with customer_id=123
        return MagicMock(tool_call={"name": "fetch_customer_info", "arguments": {"customer_id": "123"}})

def test_classifier_calls_fetch_customer_info():
    llm = MockLLM()
    tools = [fetch_customer_info, escalate_to_human]
    classifier = TicketClassifier(llm, tools)
    messages = [{"role": "user", "content": "My customer ID is 123, please help"}]
    decision = classifier.decide(messages)
    assert decision["name"] == "fetch_customer_info"
    assert decision["arguments"]["customer_id"] == "123"

Similarly, we can test that the agent returns None (no tool call) when the LLM decides to reply directly. By controlling the mock's return value, we cover all decision branches.

Testing Tool Execution and Side Effects

Once the agent decides to call a tool, we need to test that the tool executes correctly and that the agent handles the result. This is where integration testing with mocked external services comes in. For example, fetch_customer_info might call a CRM API. We mock that API and test the agent's next decision after receiving the result.

def test_agent_handles_tool_result():
    # Arrange: mock LLM to first call fetch_customer_info, then after result, call escalate
    mock_llm = MagicMock()
    mock_llm.chat.side_effect = [
        MagicMock(tool_call={"name": "fetch_customer_info", "arguments": {"customer_id": "123"}}),
        MagicMock(tool_call={"name": "escalate_to_human", "arguments": {"reason": "VIP customer"}})
    ]
    agent = TicketClassifier(mock_llm, [fetch_customer_info, escalate_to_human])
    messages = [{"role": "user", "content": "My customer ID is 123"}]
    
    # Act: first decision
    decision1 = agent.decide(messages)
    # Simulate tool execution and append result
    tool_result = {"role": "tool", "content": "{\"name\": \"Alice\", \"tier\": \"VIP\"}"}
    messages.append(tool_result)
    decision2 = agent.decide(messages)
    
    # Assert
    assert decision2["name"] == "escalate_to_human"
    assert decision2["arguments"]["reason"] == "VIP customer"

This pattern verifies the agent's response to tool outputs. We can also test error handling: what if the API returns a 500? The agent should retry or escalate.

Multi-Agent Swarm Interaction Testing

Now, the swarm. A swarm is a graph of agents that pass messages. We need to test that the correct sequence of actions occurs. Our approach: run the swarm in a test harness with deterministic LLM mocks for each agent, and assert the final state or message log.

Consider a two-agent swarm: ClassifierAgent and ResponderAgent. Classifier decides whether to fetch info or escalate. Responder composes a reply. The swarm orchestrator passes messages between them.

We mock each agent's LLM to return specific decisions. Then we run the swarm for a fixed number of steps or until termination. Finally, we assert the messages sent and received.

def test_swarm_escalates_vip_customer():
    # Mock LLMs for both agents
    classifier_llm = MagicMock()
    classifier_llm.chat.side_effect = [
        MagicMock(tool_call={"name": "fetch_customer_info", "arguments": {"customer_id": "123"}}),
        MagicMock(tool_call={"name": "escalate_to_human", "arguments": {"reason": "VIP"}})
    ]
    responder_llm = MagicMock(return_value=MagicMock(content="Escalated to human."))
    
    # Create agents
    classifier = TicketClassifier(classifier_llm, [fetch_customer_info, escalate_to_human])
    responder = ResponderAgent(responder_llm)
    
    # Create swarm with a simple graph: classifier -> responder
    swarm = Swarm([classifier, responder], adjacency=[(0,1)])
    
    # Run with initial message
    initial_msg = {"role": "user", "content": "Customer ID 123 needs help"}
    final_state = swarm.run(initial_msg, max_steps=5)
    
    # Assert that the final message contains escalation
    assert "Escalated to human" in final_state.messages[-1]["content"]
    # Assert that the tool was called
    assert any("fetch_customer_info" in str(m) for m in final_state.messages)

This test validates the entire flow without touching any real LLM. The mocks make it fast and deterministic.

Handling Non-Determinism: Seeding and Controlled Randomness

Some agents use temperature > 0 or random sampling for diversity. For unit tests, we force determinism by setting temperature=0 and seeding random number generators. Alternatively, we can mock the LLM entirely as above. For production, we run a separate suite with real LLMs to catch regressions, but those are slower and more expensive.

Testing Error Paths and Recovery

Critical: agents fail. APIs timeout, tools throw exceptions, LLMs return malformed JSON. Your tests should cover these. For example, if fetch_customer_info raises an exception, the agent should retry or escalate. We inject exceptions into mocked tools and assert the agent's next decision.

def test_agent_retries_on_timeout():
    # Mock fetch_customer_info to raise TimeoutError the first time, succeed the second
    tool_mock = MagicMock(side_effect=[TimeoutError("API timeout"), {"name": "Alice"}])
    # ... rest of setup
    # Assert that the agent called the tool twice
    assert tool_mock.call_count == 2

Tradeoffs and Lessons Learned

1. Mock granularity: Mocking the LLM is powerful but can lead to false positives if the real LLM behaves differently. Mitigate by running integration tests with a small, fast model (e.g., GPT-4o-mini) in CI.
2. Test maintenance: As agent logic evolves, mocks must be updated. Keep mock fixtures centralized.
3. Coverage: Not all behaviors are testable via mocks (e.g., prompt sensitivity). Supplement with property-based tests and adversarial testing.
4. Speed: Unit tests run in milliseconds. We run them on every commit. Integration tests run nightly.

Conclusion

Unit testing agent behavior is not only possible but essential. By making decisions deterministic through mocking, we can verify individual actions and swarm interactions before they hit production. This methodology has caught numerous bugs in our swarms: agents calling wrong tools, infinite loops, and mishandled errors. The code snippets above are from our production test suite. Adapt them to your framework.

Start with decision tests for single agents. Then build interaction tests for your swarm. Your future self—and your users—will thank you.