Circuit Breaker Pattern for External Services

Introduction

In our previous lesson, we established a robust system for creating and propagating structured, data-rich errors. This is the foundation of a resilient system, as it allows us to know when and why things fail. Now, we will build on that foundation to control how our system reacts to repeated failures.

This lesson addresses a classic problem in distributed systems: cascading failures. Imagine one of your agent's critical tools relies on an external API that suddenly becomes slow or unavailable. If your system continues to hammer this failing service with requests, you'll not only waste resources and increase latency but also risk overwhelming your own application, potentially causing a system-wide outage.

We will tackle this by implementing the Circuit Breaker pattern. As an engineer with a background in electronics, you'll immediately recognize the analogy: just as an electrical circuit breaker trips to protect appliances from a damaging power surge, a software circuit breaker "trips" to protect your application from a failing dependency. This lesson will guide you through the pattern's mechanics and its practical implementation in your Node.js agentic infrastructure.

The Circuit Breaker State Machine

At its core, the circuit breaker is a state machine that wraps calls to an external service. It monitors for failures and uses the outcome to decide whether to allow future calls, fail them immediately, or test if the service has recovered.

The three states are:

1. CLOSED: This is the normal operational state. All requests are allowed to pass through to the external service. The circuit breaker monitors for failures (like timeouts or the structured errors we designed in the last lesson). If the failure count exceeds a configured threshold within a specific time window, the breaker "trips" and transitions to the OPEN state.
2. OPEN: In this state, the circuit breaker immediately rejects all requests for the external service without even attempting to call it. This is the "fail fast" principle in action. It prevents your application from waiting on a service that is likely to fail, freeing up resources and providing a near-instant response (often a fallback or an error message). After a configured timeout period, the breaker transitions to the HALF-OPEN state.
3. HALF-OPEN: Here, the breaker allows a single "probe" request to go through to the external service. This is a test to see if the dependency has recovered.
   • If this probe request succeeds, the breaker assumes the service is healthy again and transitions back to CLOSED.
   • If the probe request fails, the breaker immediately transitions back to OPEN, starting the timeout period over again.

To see these concepts explained in the context of microservice APIs, the following video provides an excellent overview.

Implementing a Circuit Breaker in Node.js

While it's possible to build a circuit breaker from scratch, for a production system, it's best to use a battle-tested library. Opossum is a popular and robust choice for Node.js. It allows you to wrap any asynchronous function with circuit breaker logic easily.

The ideal place to apply this in our architecture is around the execution of an agent's tools, especially those that make network calls to external APIs or databases.

Let's start by studying a guide on how to use Opossum.

Now, let's see how you would integrate Opossum into your tool execution logic. First, you'll need to install it:
npm install opossum

Here is a practical example of wrapping a tool function with a circuit breaker.

import CircuitBreaker from 'opossum';
import { z } from 'zod';

// This is our original async tool function that might fail
async function externalWeatherAPITool(params: { city: string }): Promise<{ temp: number }> {
  console.log(`Calling external weather API for ${params.city}...`);
  // Simulate an API call that can fail or time out
  const response = await fetch(`https://api.weather.example.com?city=${params.city}`);
  if (!response.ok) {
    throw new Error(`API failed with status ${response.status}`);
  }
  return response.json();
}

// Configuration options for the circuit breaker
const options: CircuitBreaker.Options = {
  timeout: 3000, // If the function takes longer than 3 seconds, trigger a failure
  errorThresholdPercentage: 50, // When 50% of requests fail, trip the circuit
  resetTimeout: 30000 // After 30 seconds in OPEN state, try again (HALF-OPEN)
};

// Create a new circuit breaker that wraps our tool
const breaker = new CircuitBreaker(externalWeatherAPITool, options);

// Define a fallback function. This is called when the circuit is OPEN.
breaker.fallback(() => {
  console.log('Circuit is open. Executing fallback.');
  return { temp: 20, source: 'cached' }; // Return a default or cached value
});

// Event listeners for logging and monitoring (connects to our next module!)
breaker.on('open', () => console.log(`Circuit OPENED: The breaker is now tripped.`));
breaker.on('halfOpen', () => console.log(`Circuit HALF_OPEN: The breaker will attempt a probe request.`));
breaker.on('close', () => console.log(`Circuit CLOSED: The breaker is now closed and functional.`));
breaker.on('failure', (error) => console.error(`Circuit FAILURE: Execution failed with:`, error.message));

// Now, the agent calls the tool via the breaker's `fire` method
async function runAgentTask() {
  try {
    const result = await breaker.fire({ city: 'London' });
    console.log('Tool execution successful:', result);
  } catch (err: any) {
    console.error('Breaker fire() call failed:', err.message);
  }
}

// Example execution
runAgentTask();

In this example, the breaker.fire() method is what the agent's logic would call instead of externalWeatherAPITool() directly. The fallback function is particularly powerful. It integrates seamlessly with the resilience patterns we've discussed, allowing you to define a graceful degradation path. Instead of the agent failing completely, it can proceed with cached data, a default value, or a message indicating reduced functionality.

Beyond Classic Failures: Circuit Breakers for Agentic Systems

The classic circuit breaker pattern is excellent for handling explicit, "hard" failures like API timeouts, network errors, or HTTP 5xx status codes. However, in an agentic system, some of the most dangerous failures are subtle.

An LLM can return a 200 OK response that is syntactically perfect but semantically nonsensical—a confident hallucination. A classic circuit breaker, which only tracks explicit errors, would see this as a success and remain CLOSED, allowing the "successful garbage" to propagate through your system.

This is where we must extend our thinking. To build a truly resilient AI system, the circuit breaker needs to be aware of different classes of failures.

The article introduces a critical concept: your circuit breaker's definition of "failure" must evolve.

• Hard Failures: Handled by default (timeouts, exceptions).
• Structural Failures: We can detect these using zod schemas on the LLM output, as we've already discussed. A validation failure can and should be counted as a failure by the circuit breaker.
• Semantic Failures: Detecting hallucinations or nonsensical answers is much harder. The article suggests strategies like "LLM-as-Judge" (using another model to validate the output), which could be a tool that your main agent calls. A failure from this validation tool would then count towards tripping the circuit.

You don't need to implement detection for all failure types today. The key takeaway is to design your system with this taxonomy in mind. By wrapping your tool calls with a library like Opossum, you have the hook you need. As you develop more sophisticated validation logic (e.g., for semantic checks), you can simply have that logic throw an error, which the circuit breaker will correctly interpret as a failure to be counted.

Conclusion

In this lesson, we've added a powerful resilience pattern to our toolkit. The circuit breaker is an essential component for building stable, production-grade distributed systems, and it's particularly vital in an agentic architecture where dependencies on external models and tools are common.

Key Takeaways:

• The Circuit Breaker pattern protects your system from cascading failures by wrapping dependencies in a state machine (CLOSED, OPEN, HALF-OPEN).
• It allows your system to "fail fast" when a dependency is unhealthy, preventing wasted resources and improving responsiveness.
• In Node.js, libraries like Opossum provide a robust and easy-to-use implementation, which should be applied to tool executions involving network calls.
• For agentic systems, the definition of "failure" must be expanded beyond simple network errors to include structural and semantic issues, such as malformed outputs and hallucinations. Your circuit breaker implementation should be able to count these validation failures.

The state changes and events emitted by the circuit breaker are critical signals about the health of your system's dependencies. In our next module, "Observability and Debugging," we will learn how to capture these signals through structured logging and tracing, giving us a real-time dashboard of our system's resilience.