Recommended resources
The freely available resources this lesson will direct you to.
Hello! Let's dive into the next layer of our defense-in-depth strategy for building robust agentic systems.
In the last lesson, we explored prompt-based guardrails, which use natural language to guide an agent's semantic behavior. These are powerful for setting tone, aligning with principles, and shaping nuanced responses. However, they are inherently probabilistic. For production-grade systems, especially those you'll be delivering to clients, "probably safe" isn't sufficient. We need verifiable certainty.
This lesson focuses on achieving that certainty. Your learning outcome is to design programmatic validation on agent outputs to filter or reject responses that violate safety constraints. We will move from the "soft" rules of prompting to the "hard" rules of code, building a deterministic layer of security and reliability that you can prove is working.
1. The Need for Determinism: From Suggestions to Contracts
Think of a prompt-based guardrail as a well-written policy memo to an employee. It's a strong guideline. A programmatic guardrail, on the other hand, is like the lock on a secure server room door—it's a non-negotiable, physical constraint.
In software engineering, we rely on contracts like API schemas and type systems to ensure components interact reliably. When an LLM's output is unstructured text, it breaks this contract, leading to brittleness and unpredictability. Programmatic validation is how we restore it.
The image below shows several categories of such guardrails. We will focus on implementing validators for Logic and Functionality (e.g., JSON format) and Content Validation (e.g., source context verifier).
2. The Foundation: Structured Output Validation with Pydantic
The most fundamental programmatic validation is enforcing a specific output structure. If you expect an agent to provide user data, you need it in a predictable format, not a free-form sentence that might change tomorrow.
A powerful approach in the Python ecosystem is to use the Pydantic library. It allows you to define data models that serve as strict schemas for LLM outputs. This is analogous to defining a class or a struct to enforce a data contract.
Let's watch a video that explains why this is so critical and how to implement it.
Pydantic is all you need: Jason Liu
The video 'Pydantic is all you need' by Jason Liu clearly articulates the problem of unreliable LLM outputs and presents Pydantic as a robust solution for enforcing structure.
Please watch from the beginning to 07:11. Pay close attention to: The initial problem of parsing unstructured strings and inconsistent JSON (00:00 - 03:00). How Pydantic models provide validation and type safety (03:00 - 05:10). How the instructor library uses these Pydantic models as a response_model in an OpenAI API call, guaranteeing that the output will conform to your schema (05:10 - 07:11).
This response_model approach is a powerful design pattern. It shifts the responsibility of correctly formatting the output to the model (which is fine-tuned for this) and gives your program a guaranteed, validated, and type-safe object to work with.
3. Implementing Advanced Validation Logic
Pydantic's utility extends beyond simple structure. You can embed complex validation logic directly into the data models to enforce specific business rules and safety constraints.
Custom Validators for Business Rules
Imagine you have an agent that recommends discounts. You can add a validator to ensure the discount is never above a certain threshold. The library allows you to define custom validation functions that check the content of the fields.
More powerfully, you can even use another LLM call within a validator to check for semantic properties.
Pydantic is all you need: Jason Liu
Let's continue with the same video, which now demonstrates how to create custom validators, including using an LLM for validation itself.
Watch from 07:11 to 09:02. Notice two key patterns: Standard Validators: A simple function can check a value and raise a ValidationError. LLM-based Validators: The llm_validator shows how you can ask a model 'is this statement objectionable?' as a validation step. This blurs the line between programmatic and prompt-based but is executed as a deterministic check within your code. Automatic Retries: The max_retries parameter in instructor is a crucial pattern for production. If validation fails, it automatically re-prompts the LLM with the error, asking it to fix its own mistake. This builds a self-correcting loop.
Programmatic Grounding to Prevent Hallucination
One of the most significant safety constraints is ensuring an agent's claims are grounded in provided source material. We can enforce this programmatically. The video concludes with a brilliant example of this.
Pydantic is all you need: Jason Liu
This final segment from the 'Pydantic' video showcases a sophisticated validation technique to combat hallucination.
Watch from 14:51 to 16:08. The key design is a validator that iterates through the 'facts' an LLM extracts and programmatically verifies that each 'substring_quote' literally exists in the source text. If it doesn't, the fact is discarded. This is a powerful, verifiable way to enforce grounding.
This pattern is far more reliable than simply prompting an agent "don't hallucinate." You are creating a system that makes ungrounded claims structurally impossible to output.
4. Architectural Patterns for Programmatic Validation
Validation isn't just about checking a final output; it can be woven into the very architecture of your agentic system. In the previous lesson, we discussed architectural patterns for security. Let's revisit some of them through the lens of programmatic validation.
Design Patterns to Secure LLM Agents In Action
The article 'Design Patterns to Secure LLM Agents In Action' provides several examples where programmatic validation is a core part of the security design.
Please read the following sections, focusing on where programmatic checks occur: Pattern 3: LLM Map-Reduce: In the 'map' phase, the code forces the LLM output into JSON and then validates it against a strict Pydantic schema. Any resume that fails this validation is discarded. This programmatic check filters bad data before the next stage. Pattern 5: Code-Then-Execute: Read about the 'provenance tracking' system. The policy that checks if input data comes from more than one untrusted source is a programmatic safety constraint. It validates the origin of data, not just its content. Best Practices: Structured Output Enforcement: This section explicitly calls out schema validation as a 'critical security control,' not just a reliability measure. This reinforces the core topic of our lesson.
These patterns show that programmatic validation is a versatile tool that can be applied at different stages of an agent's workflow: filtering inputs, checking data flow between steps, and validating final outputs.
5. From Validation to Verifiable Governance
For the high-stakes, enterprise-grade systems you aim to build, simply having validation isn't enough. You need to be able to prove to a client that your system is verifiably safe and compliant. This is where formal methods come in, providing a conceptual framework to ground our validation practices.
The diagram below shows an architecture where validators are first-class citizens, integrated into the core processing flow.
A formal approach, such as the one described in the "Architecting Agentic Communities" paper, allows us to specify these constraints in a provable way.
Architecting Agentic Communities using Design Patterns
This paper introduces a formal method (ODP-EL) for designing and verifying agentic systems, which is highly relevant for your background and goals. We will focus on how it formalizes the concept of programmatic constraints.
Please read the following sections to understand how practical validation maps to formal governance: Section 6.3.1, 'Normative constraints': Focus on the concept of 'Embargo tokens.' An embargo is the formal equivalent of a programmatic prohibition. The example embargo(auto_enroll, ALL_AI) is a hard-coded rule preventing AI from performing a specific action. Section 6.4.2, 'Normative Constraints (Policies)': Notice the policy embargo(final_decision, ALL_AI_AGENTS). This is a programmatic validation rule that rejects any workflow where an AI tries to make the final decision, enforcing the Human-in-the-Loop pattern we've discussed. Section 6.5, 'Formal Properties and Runtime Verification': This is the crucial part. It shows how these formal constructs allow you to prove properties like 'Prohibition Property: “AI cannot auto-enroll patients”'. This moves your assurance from testing-based to verification-based.
By framing your validation logic within such a formal system, you can offer clients something incredibly valuable: not just a system that works, but a system that is provably safe and compliant with its design specifications.
Conclusion
This lesson has armed you with the principles and patterns to design deterministic, verifiable safety constraints for your agentic systems. We've moved beyond merely asking an agent to behave and are now architecting systems where misbehavior is structurally impossible.
Key Takeaways:
- Programmatic validation provides deterministic safety, acting as a necessary counterpart to probabilistic, prompt-based guardrails.
- Structured output is the foundation. Using tools like Pydantic to enforce schemas is the first step toward building reliable systems.
- Validation enforces safety constraints. Custom validators can programmatically check for everything from business rule violations (e.g., discount limits) to factual grounding (e.g., source-checking quotes).
- Architectural patterns embed validation. Designs like Map-Reduce and provenance tracking use programmatic checks as core components to ensure security and data integrity.
- Formal methods enable provable safety. For enterprise clients, being able to formally verify that your system adheres to safety properties like "AI cannot make final decisions" is a powerful differentiator.
In our next lesson, we will look at a specific, practical application of these concepts by exploring how to integrate an external content moderation layer to screen agent inputs and outputs. This will involve using specialized third-party APIs that act as a pre-built, sophisticated programmatic validator for content safety.