Robust Tooling: Handling Failures and Unexpected Results

Recommended resources

The freely available resources this lesson will direct you to.

Hello! Welcome back.

In our last lesson, we designed deterministic and reliable logic for an agent to select the right tool for a job. We established that for production-grade systems, moving core logic from probabilistic LLM reasoning to deterministic code is paramount for consistency and auditability.

However, even with perfect tool selection, the real world is messy. APIs go down, networks get congested, and tools return unexpected or incorrect results. This brings us to today's learning outcome: Design retry and error-handling patterns for when tool calls fail or return unexpected results.

This is not just about writing try/catch blocks. It's about designing a resilient system that anticipates failure as a normal part of its operation. For the agentic AI services you plan to offer clients, robustness is a primary feature, not an afterthought. A system that can gracefully handle and recover from errors is significantly more valuable and trustworthy.

1. A Taxonomy of Tool Failures

Before designing solutions, we must first understand the problem space. Tool call failures are not monolithic; they manifest in different ways, each requiring a distinct handling strategy. Drawing from software engineering principles and recent research in agentic systems, we can classify these failures into three main categories.

The research paper "Structured Handling of Exceptions in LLM-Driven Agentic Workflows" (SHIELDA) provides a formal taxonomy that is highly relevant here.

[PDF] Structured Handling of Exceptions in LLM-Driven Agentic ...

To build a robust system, we first need a structured understanding of what can go wrong. This paper provides a comprehensive taxonomy of exceptions in agentic workflows. We won't read it all, but we will use its classifications to build our mental model.

Please review the descriptions of the following exception types from the paper. You don't need to read the full text, just focus on understanding these specific failure modes. You will find them in section 3.1: Tool Invocation Exception: The agent fails to correctly format or plan the tool call. Tool Output Exception: The tool runs but returns semantically incorrect or malformed output. Unavailable Tool: The tool cannot be reached due to network errors, downtime, or rate limits. API Invocation Exception: A lower-level error related to the API interaction itself (e.g., wrong parameters). Notice the distinction between an invocation failing versus the output being wrong. This is a critical separation for designing handlers.

Based on this, let's establish our working categories:

Invocation Failures (The Call Fails): The agent's request to the tool never successfully completes.
- Transient Errors: Temporary issues like network timeouts, API rate limits, or a service being momentarily unavailable (e.g., HTTP 503 Service Unavailable). These are often recoverable by simply trying again.
- Permanent Errors: Unrecoverable issues with the request itself, such as an invalid API key (HTTP 401 Unauthorized), a malformed request (HTTP 400 Bad Request), or calling a non-existent endpoint (HTTP 404 Not Found).
Execution Failures (The Tool Breaks): The tool successfully receives the request but fails during its internal processing, potentially crashing or timing out before returning a result.
Output Failures (The Result is Wrong): The tool executes and returns a response, but the response is unusable.
- Structural Errors: The response violates the expected data contract. For example, returning a string when a JSON object is expected, or a JSON object missing required fields.
- Semantic Errors: The response is structurally valid but logically incorrect or nonsensical. For instance, a calculator tool returning 2 + 2 = 5, or a weather API returning sunny skies during a hurricane.

Understanding this taxonomy allows us to move from generic error handling to designing specific, targeted recovery patterns.

2. Core Patterns for Error Handling and Resiliency

Now, let's explore the architectural patterns to address these failures. These are design blueprints you can incorporate into any agentic system.

Pattern 1: Intelligent Retries for Transient Failures

The most common failure type is transient. The simplest response is to "try again." However, a naive retry loop can cause more problems, like overwhelming a struggling service. A more intelligent approach is necessary.

Agents That Don't Break: Production Patterns - Byte AI Blog

This blog post, 'Agents That Don't Break: Production Patterns,' outlines several best practices for building robust agent systems. We'll start with its excellent explanation of retry strategies.

Please read section 1, 'Retry Strategies with Exponential Backoff.' Pay attention to the three key components of a robust retry mechanism: Exponential Backoff: Why you should increase the delay between retries. Jitter: The reason for adding randomness to the delay. Max Retry Limits: The importance of capping the number of attempts.

This pattern is your first line of defense against temporary glitches. By implementing retries with exponential backoff and jitter, you give a failing service time to recover while preventing a "thundering herd" of simultaneous retry attempts from your own systems. You only apply this pattern to errors you've identified as transient. Retrying a permanent error like an invalid API key is futile and wastes resources.

Pattern 2: The Circuit Breaker for System Stability

What if a service is down for an extended period? Continuously retrying will waste resources and slow down your entire system. The Circuit Breaker pattern, a classic in distributed systems, prevents this.

Agents That Don't Break: Production Patterns - Byte AI Blog

Let's continue with the 'Agents That Don't Break' article to see how to handle more persistent failures.

Now read section 3, 'Circuit Breaker Pattern for Agent Systems.' Focus on the state transitions: Closed: The normal state where calls are permitted. Open: The state entered after a threshold of failures, where calls are blocked immediately. Half-Open: A test state to check if the service has recovered.

By wrapping a tool call in a circuit breaker, you protect your agentic system from cascading failures. If a critical API is down, the circuit "trips" (opens), and subsequent calls fail immediately without waiting for a timeout. This frees up resources and allows the agent to pursue an alternative path, such as using a fallback tool or escalating to a human, which leads us to our next pattern.

Pattern 3: Fallbacks and Graceful Degradation

When a tool call fails permanently (i.e., retries are exhausted or the circuit is open), the workflow shouldn't just crash. A robust system will have a predefined backup plan. This is the essence of graceful degradation.

Your design can include several layers of fallbacks:

Alternate Tool: If WeatherAPI_A fails, try WeatherAPI_B.
Use Cached Data: If you can't get a live stock price, use the last known price from 5 minutes ago (and clearly state that it's cached).
Default Value: If a personalization service is down, return a generic, non-personalized experience.
Human-in-the-Loop: If the agent cannot proceed, it should escalate the task to a human operator with all the relevant context.

The video below on agentic design patterns mentions this concept of having a backup or fallback method as a recurring theme.

Master ALL 20 Agentic AI Design Patterns [Complete Course]

This video covers a wide range of agentic design patterns. We'll focus on a segment that explicitly discusses handling exceptions and having backup options.

Watch the segment on 'Exception Handling and Recovery' from 31:26 to 33:01. Notice how the presenter classifies errors into 'permanent' and 'temporary'. For permanent errors, they discuss the importance of having a 'Plan B' or 'backup options' like using a simpler method, saved data, or escalating to a human.

Pattern 4: Reflection and Self-Correction

This is a more advanced, "agentic" pattern for handling errors, particularly output failures. What if the tool call fails because the agent formatted the input incorrectly? Instead of just failing, we can design the system to allow the agent to correct its own mistake.

The process looks like this:

The agent calls a tool with invalid parameters.
The system catches the error (e.g., a validation error from the tool's API).
Instead of just logging the error, the system feeds the error message back to the agent as part of a new prompt.
The new prompt instructs the agent: "Your previous attempt failed with this error. Please analyze the error, correct your tool call parameters, and try again."

This creates a "reflection loop" where the agent learns from its immediate failures.

How to Build AI Agents with PydanticAI (Beginner Tutorial)

The PydanticAI library provides a very elegant implementation of this self-correction pattern. This video demonstrates it clearly.

Watch the final example on 'Reflection and Self-Correction' from 22:15 to 26:15. This is the most crucial part. Observe how a 'ModelRetry' exception is raised with a specific instruction string when a database lookup fails. This instruction is then fed back into the LLM's context, allowing it to self-correct its query and succeed on the next attempt. This is a powerful design pattern for handling errors that originate from the agent's own reasoning.

This pattern is incredibly powerful because it turns certain classes of failures into opportunities for the agent to adapt, making the system more resilient without requiring hard-coded fixes for every possible mistake.

3. A Framework for Designing Resilient Workflows

We have a set of patterns. Now, how do we organize them into a coherent system design? The SHIELDA paper offers a brilliant conceptual framework for this, breaking down exception handling into three orthogonal dimensions.

Multi-Stage Research Agent with Retry Logic and Checkpointing — A visual representation of how retry logic can be integrated into a multi-stage agentic workflow. When the 'VALIDATING' step fails, an error handler checks a retry count and loops back to the 'SEARCHING' step if the limit is not exceeded. Source: ranjankumar.in

This flowchart illustrates a practical implementation. When validation fails, the system doesn't just stop; it enters a defined error-handling path that involves a retry loop with a maximum count. Your design should map out these paths for all critical tool calls.

The SHIELDA framework suggests a formal model for designing these paths.

[PDF] Structured Handling of Exceptions in LLM-Driven Agentic ...

Let's return to the SHIELDA paper to see how it proposes structuring our handling patterns into a composable framework. This provides a high-level, architectural view.

Please read section 4.1 'System Architecture' and the descriptions of the three dimensions of a handler pattern. Then, look at Table 2, 'Local Handling Mechanisms.' Focus on the triadic model: Local Handling: The immediate action (e.g., Retry with Backoff, Fallback to Alternate API). You'll see these in Table 2. Flow Control: The decision about the workflow's next step (e.g., continue, skip, abort). State Recovery: How to clean up the agent's state (e.g., rollback, compensate). This structure provides a powerful mental model for designing a complete error-handling response.

By thinking in these three dimensions, you can design a complete response for any given failure:

Exception: A tool call times out.
Your Design:
1. Local Handling: Retry with Backoff (up to 3 times).
2. Flow Control: If successful, Continue. If all retries fail, Skip this step.
3. State Recovery: If skipped, Compensate by logging the failure and noting in the agent's memory that the information could not be retrieved.

This structured approach ensures you have a comprehensive plan, transforming your agent from a brittle script into a resilient, production-ready system.

Conclusion

Today we've moved from simply using tools to designing systems that can withstand tool failures. This is a critical step in engineering reliable agentic workflows that you can confidently deploy for clients.

Key Takeaways:

Failure is Normal: In agentic systems interacting with external tools, failures are not edge cases; they are part of normal operation and must be designed for.
A Taxonomy of Failure: Differentiating between invocation, execution, and output failures (both structural and semantic) is key to applying the correct handling pattern.
A Portfolio of Patterns: You now have a set of core design patterns for resilience:
- Intelligent Retries: For transient errors.
- Circuit Breakers: To protect your system from cascading failures.
- Fallbacks: To ensure graceful degradation when a tool is unavailable.
- Self-Correction: To empower the agent to fix its own mistakes.
Structured Design: Using a framework like SHIELDA's Local Handling -> Flow Control -> State Recovery model allows you to systematically design comprehensive error-handling logic for any situation.

In our next lesson, we will shift from reactive error handling to proactive quality control. We will focus on how to apply output validation techniques to verify an agent's work before it reaches the end user. Even if a tool call succeeds without error, how do we ensure the output is correct, relevant, and meets our quality standards?