Robust Output Handling: Fallback & Retry Strategies

Beyond Retry: Fallback and Self-Correction

A robust system doesn't just retry; it has a plan for what to do when retries don't work. We're building a workflow that can not only attempt to fix its mistakes but also knows when to give up and what to do next.

1. Defining "Quality Thresholds"

Before our agent can retry or fall back, it needs a clear definition of what constitutes a "low-quality" output. In the last lesson, the threshold was simple: any claim with a Contradicted or Unsubstantiated status. Now, we'll incorporate the other quality metrics we discussed in Module 7, Lesson 1, such as brand voice, clarity, and structural requirements.

To do this, we can use a combination of different evaluation types.

The key takeaway is that we can create a suite of tests that run after each content generation step. Some will be simple code checks, and others will be LLM calls. The combined result of this test suite will determine if the output meets our quality threshold.

Your Turn: Brainstorming Quality Checks

Based on the brand voice guide you created in Module 1 and your goals for each platform, take a moment to brainstorm a few potential quality checks. For example:

• LinkedIn Article:
  • Rule-based: Must be between 500 and 1500 words. Must contain at least two section headings.
  • LLM-based: Does the article's tone sound professional and authoritative? Rate from 1-5.
• X (Twitter) Thread:
  • Rule-based: Does each tweet in the thread stay under 280 characters?
  • LLM-based: Is the hook in the first tweet engaging and intriguing?
• Instagram Carousel:
  • Rule-based: Does the generated content contain exactly 5 slides of text?
  • LLM-based: Is the narrative across the slides coherent and easy to follow?

This exercise helps translate abstract brand goals into concrete, automatable checks.

2. The General Self-Correction Pattern in LangGraph

The "Generate → Verify → Decide → Revise" loop from our previous lesson is a specific instance of a more general pattern. Let's adapt our LangGraph implementation to handle a variety of quality issues.

The core idea is to have a centralized evaluation node that runs all our quality checks and populates a list of issues in the agent's state.

Let's adapt this pattern to our TypeScript project.

Step 1: Generalize the Agent State

Instead of a specific verified_claims field, we'll use a more generic list to hold any and all quality issues found. We'll also rename revision_number to retry_count to reflect its general purpose.

// in your state.ts file
import { TypedDocument } from "@langchain/core/documents";

// ... other imports

export interface AgentState {
  // ... other state properties like 'topic', 'outline'
  article: string;
  quality_issues: string[]; // A list of feedback strings
  retry_count: number;
  // This can still be useful if you run fact-checking separately
  verified_claims?: ClaimVerificationResult[];
}

Step 2: Create a General Evaluation Node

We'll create a new node, evaluateContentNode, that consolidates all our checks. This node will run the fact-checker from the previous lesson, plus any new rule-based or LLM-based evaluations. If any check fails, it adds a descriptive string to the quality_issues list.

// in your nodes.ts file

async function checkBrandVoice(article: string): Promise<string | null> {
  // LLM call to evaluate brand voice
  // Returns a feedback string if it fails, e.g., "The tone is too casual."
  // Returns null if it passes.
  const isOk = false; // dummy check
  if (!isOk) {
      return "The tone is too casual for a LinkedIn thought-leadership piece.";
  }
  return null;
}

async function checkWordCount(article: string): Promise<string | null> {
  const wordCount = article.split(/\s+/).length;
  if (wordCount < 400 || wordCount > 2000) {
    return `Article word count is ${wordCount}, which is outside the acceptable range of 400-2000 words.`;
  }
  return null;
}

export async function evaluateContentNode(state: AgentState): Promise<Partial<AgentState>> {
  console.log("--- EVALUATING CONTENT QUALITY ---");
  const issues: string[] = [];
  const { article } = state;

  // Run all checks in parallel
  const results = await Promise.all([
    checkBrandVoice(article),
    checkWordCount(article),
    // You would also trigger your fact-checking logic here and format its output
    // For example: runFactChecker(article).then(formatFactCheckIssues)
  ]);

  for (const result of results) {
    if (result) {
      issues.push(result);
    }
  }

  console.log(`Found ${issues.length} quality issues.`);
  return { quality_issues: issues };
}

Step 3: Generalize the Revision Node and Decision Logic

The reviseArticleNode and decideNextStep functions can now be made more generic. The decision logic simply checks if the quality_issues list is empty, and the revision node formats all issues into the prompt.

// in your graph.ts or nodes.ts

const MAX_RETRIES = 2;

export function decideOnRetry(state: AgentState): "revise" | "finish" {
  console.log("--- DECIDING ON RETRY ---");
  const retryCount = state.retry_count || 0;

  if (retryCount >= MAX_RETRIES) {
    console.log("Maximum retries reached. Finishing.");
    return "finish";
  }

  if (state.quality_issues && state.quality_issues.length > 0) {
    console.log("Quality issues found. Routing to revision.");
    return "revise";
  }

  console.log("No quality issues. Finishing.");
  return "finish";
}

export async function reviseContentNode(state: AgentState): Promise<Partial<AgentState>> {
    console.log("--- REVISING CONTENT BASED ON FEEDBACK ---");

    const feedback = state.quality_issues.map(issue => `- ${issue}`).join('\n');
    
    const system_message = `You are an editor revising a draft. Address the following issues and provide a new, complete version of the article. Focus only on fixing the specified issues.
    
    **ISSUES TO FIX:**
    ${feedback}`;

    const user_message = `**ORIGINAL ARTICLE:**\n${state.article}`;
    
    // ... (LLM call with messages) ...
    // const revisedArticle = response.choices[0].message.content || "";

    const revisedArticle = "A newly revised article..."; // dummy response

    return { 
        article: revisedArticle, 
        retry_count: (state.retry_count || 0) + 1,
        quality_issues: [] // Clear issues for the next evaluation cycle
    };
}

By wiring these generalized nodes into your LangGraph using addConditionalEdges (just as we did in the last lesson), you create a flexible and powerful quality assurance loop.

3. Implementing Fallback Strategies

What happens when decideOnRetry returns "finish" because retry_count has reached MAX_RETRIES, but there are still unresolved quality issues? This is where your fallback strategy comes in. The agent must fail gracefully.

Here are some common fallback strategies:

1. Discard and Log (Safest): If the content isn't good enough after 2-3 retries, simply discard it. Log the topic, the generated article, and the final list of quality_issues for later analysis. This prevents low-quality content from ever reaching the final approval stage.
2. Route to Human-in-the-Loop (Most Robust): The finish branch of your conditional edge doesn't have to lead directly to END. It can lead to a humanApprovalNode. This is an excellent fallback, as it presents the problematic article and its issues to you for a final decision.
3. Use a Different Strategy: A more advanced fallback could involve trying a different LLM (e.g., switching from a fast model to a more powerful one like GPT-4) or using a simpler generation prompt as a last resort.

For our project, routing to a human is the ideal fallback. We just need to modify our graph logic slightly. Our decideOnRetry function remains the same, but the edge connection changes.

// In your main graph definition file

// ... (node definitions)

workflow.addConditionalEdges(
  "evaluate_content",
  decideOnRetry,
  {
    revise: "revise_content", // If issues exist and retries < max, go to revision
    finish: "human_approval"  // If no issues OR retries >= max, go to human approval
  }
);

// The loop is created by this edge
workflow.addEdge("revise_content", "evaluate_content");

// ...

With this setup, any piece of content, whether perfect on the first try or still flawed after maximum retries, ends up in the human_approval queue. This ensures nothing gets published automatically unless it passes all quality gates, and nothing gets stuck in an infinite loop.

Conclusion

You have now designed a comprehensive quality assurance system that is both resilient and robust. By generalizing the self-correction pattern and implementing clear fallback strategies, you have built a critical component for any production-grade autonomous agent.

Key Takeaways:

• Retry loops are a general pattern applicable to any measurable quality metric, including rule-based checks (word count, structure) and LLM-based evaluations (brand voice, tone).
• Agent state is key for managing the loop. Tracking quality_issues and retry_count allows for stateful, intelligent control flow.
• Fallback strategies are essential for safety. A MAX_RETRIES limit combined with a fallback path (like routing to a human or discarding the content) prevents infinite loops and ensures system stability.
• The Generate -> Evaluate -> Decide -> Revise cycle, implemented with LangGraph's conditional edges, is the fundamental building block for creating self-correcting AI systems.

Preview of the next lesson:
Our agent can now generate, analyze, and iteratively improve content until it meets our high standards, with a safe fallback for when it can't. The content pipeline is complete. In the next module, we will shift our focus from generation and refinement to distribution. Our first lesson will be "Authenticate and post content via the LinkedIn API," where we'll start building the tools to share our agent's work with the world.