Implementing a Verification Pipeline for Content Accuracy

The Self-Correction Loop: From Verification to Revision

The pattern we are about to implement is often called a "Reflexion" or "self-correction" loop. Instead of a linear assembly line, we're building a stateful, cyclical process where the agent can pause, evaluate its work, and decide to try again.

The core components for this loop are:

1. A conditional edge that inspects the verification results and directs the workflow.
2. A revision node that receives the failed claims and evidence, then attempts to rewrite the article.
3. A counter in our state to prevent infinite loops.

1. The Decision Point: A Conditional Edge

After our verifyClaimsNode runs, the agent's state will contain verified_claims. We need a function to inspect this array and decide the next step. In LangGraph, this logic lives in the function you provide to add_conditional_edges.

Let's define this decision-making function in TypeScript. It will read the verified_claims and revision_number from the state and return a string indicating which node to execute next.

// in your nodes.ts or a new graph.ts file

import { AgentState } from './state';

const MAX_REVISIONS = 2;

export function decideNextStep(state: AgentState): "revise" | "finish" {
  console.log("--- DECIDING NEXT STEP ---");
  const revisionNumber = state.revision_number || 0;

  if (revisionNumber > MAX_REVISIONS) {
    console.log("Maximum revisions reached. Finishing.");
    return "finish";
  }

  const hasIssues = state.verified_claims?.some(
    claim => claim.status === 'Contradicted' || claim.status === 'Unsubstantiated'
  );

  if (hasIssues) {
    console.log("Found claims that are contradicted or unsubstantiated. Routing to revision.");
    return "revise";
  } else {
    console.log("All claims verified. Finishing.");
    return "finish";
  }
}

This function is our guardrail's control unit. It checks for problematic claims and ensures the agent doesn't get stuck trying to fix an unfixable article by limiting the number of revisions.

2. The Action: The reviseArticleNode

If our decideNextStep function returns "revise", the graph will route to a new node we'll create: reviseArticleNode. The purpose of this node is to perform a targeted rewrite of the article.

The key to a successful revision is providing the LLM with the right context. We don't just ask it to "fix the errors." We provide the original text, the specific claims that failed verification, and the evidence that was found.

Now, let's implement the reviseArticleNode. It will construct a detailed prompt and ask the LLM to act as a critical editor, correcting its own work.

// in your nodes.ts file
import { OpenAI } from 'openai';
import { AgentState } from './state';
import { ClaimVerificationResult } from './types/verification';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

// Helper to format the feedback for the prompt
function formatVerificationFeedback(verified_claims: ClaimVerificationResult[]): string {
  const issues = verified_claims.filter(
    claim => claim.status !== 'Supported'
  );

  if (issues.length === 0) {
    return "No issues found.";
  }

  return issues.map(issue => `
    - Claim: "${issue.claim}"
    - Status: ${issue.status}
    - Reasoning: ${issue.reasoning || 'N/A'}
    - Evidence Found: ${issue.evidence || 'N/A'}
  `).join('\n');
}

export async function reviseArticleNode(state: AgentState): Promise<Partial<AgentState>> {
  console.log("--- REVISING ARTICLE ---");

  const feedback = formatVerificationFeedback(state.verified_claims || []);

  const system_message = `
    You are an editor tasked with revising a draft article based on a fact-checking report.
    You will be given the original article and a list of claims that were found to be 'Contradicted' or 'Unsubstantiated'.
    Your task is to rewrite the article to correct these inaccuracies.
    - Use the provided 'Reasoning' and 'Evidence' to make corrections.
    - If a claim is 'Unsubstantiated', rephrase it to be less assertive or remove it if it is not critical.
    - If a claim is 'Contradicted', you MUST correct it or remove it.
    - Do NOT change the parts of the article that are not related to the flagged claims.
    - Ensure the revised article maintains a consistent tone and style.
    - Output ONLY the full text of the revised article.
  `;

  const user_message = `
    **Original Article:**
    ---
    ${state.article}
    ---

    **Fact-Checking Report (Issues to Address):**
    ---
    ${feedback}
    ---

    Please provide the revised article.
  `;

  const response = await openai.chat.completions.create({
    model: "gpt-4-turbo",
    messages: [
      { role: "system", content: system_message },
      { role: "user", content: user_message }
    ],
  });

  const revisedArticle = response.choices[0].message.content || state.article || "";
  const currentRevision = state.revision_number || 0;

  console.log("Article has been revised.");
  return { article: revisedArticle, revision_number: currentRevision + 1 };
}

This node is a powerful example of using an LLM for a sophisticated reasoning task. It's not just generating text; it's performing a targeted edit based on structured feedback.

3. Integrating the Loop in LangGraph

Now we can wire everything together in our LangGraph definition. We'll add the new node and the conditional edge that directs traffic based on the output of our verifyClaimsNode.

// in your main graph definition file

import { StateGraph, END } from "@langchain/langgraph";
import { AgentState } from "./state";
import {
  // ... your other nodes
  extractClaimsNode,
  verifyClaimsNode,
  reviseArticleNode,
  decideNextStep,
  humanApprovalNode // Assuming you have a final approval node
} from "./nodes";

// ... (StateGraph and other node definitions)

const workflow = new StateGraph<AgentState>({
  channels: { /* your state schema */ }
});

// Add nodes
workflow.addNode("extract_claims", extractClaimsNode);
workflow.addNode("verify_claims", verifyClaimsNode);
workflow.addNode("revise_article", reviseArticleNode);
workflow.addNode("human_approval", humanApprovalNode);
// ... add other nodes

// Define edges
workflow.setEntryPoint("your_content_generation_node");
// ... edges leading to extract_claims

workflow.addEdge("extract_claims", "verify_claims");

// This is the crucial part: the conditional edge
workflow.addConditionalEdges(
  "verify_claims",
  decideNextStep, // The function that makes the routing decision
  {
    revise: "revise_article", // If it returns "revise", go to revise_article
    finish: "human_approval"  // If it returns "finish", go to human_approval
  }
);

// This creates the loop
workflow.addEdge("revise_article", "extract_claims");

workflow.addEdge("human_approval", END);

const app = workflow.compile();

Notice the final edge: workflow.addEdge("revise_article", "extract_claims");. This is what creates the loop. After an article is revised, it is sent back to the extract_claims node to be re-evaluated. The cycle continues until decideNextStep returns "finish" or the MAX_REVISIONS limit is hit.

This complete loop—Generate → Extract → Verify → Decide → Revise → (repeat)—is a robust pattern for improving the reliability of generated content.

Conclusion

Today you have successfully integrated your verification pipeline as a true agentic guardrail. By adding a conditional edge and a revision node, you have given your system the ability to self-correct, a significant step towards creating autonomous and trustworthy content.

Key Takeaways:

• Guardrails are active processes: They don't just report issues; they trigger actions.
• Conditional edges are the brains of a LangGraph loop: They enable stateful, dynamic routing based on the results of previous nodes.
• The "Reflexion" pattern is key to self-correction: Providing an agent with feedback on its own output (the claims, reasoning, and evidence) allows it to perform targeted, effective revisions.
• Always have an exit strategy: A simple counter (revision_number) is essential to prevent infinite loops and ensure the process terminates.

Preview of the next lesson:
The revision loop we built is a powerful form of a retry strategy. In the next lesson, "Implement fallback and retry strategies when output quality falls below a defined threshold," we will expand on this concept. We'll explore how to apply similar loops to other quality failures (like brand voice or clarity scores from Module 7, Lesson 1) and discuss more advanced error handling mechanisms to make our entire agentic workflow more resilient.