Optimizing Input Data for AI Comprehension

Introduction

In our last lesson, we focused on diagnostics, establishing a framework for identifying information gaps when an agent produces unreliable outputs. We learned that most agent failures are "context failures," stemming from missing knowledge, flawed context assembly, or undocumented business rules. This process tells us what information is missing.

This lesson addresses the next logical question: once we have the right information, how should we present it to the agent? The way we format input data is a crucial, yet often overlooked, layer of agent design. This directly addresses our learning outcome: Structure input data formats to maximize agent comprehension and output consistency.

From your computer science background, you're familiar with data serialization—the process of translating data structures into a format that can be stored or transmitted. We'll be applying a similar concept here, exploring how to "serialize" information for an LLM to optimize for comprehension, reliability, and even cost.

1. The Core Problem: Why Format Matters

At a high level, every piece of information we put into an agent's context window—user queries, retrieved documents, tool outputs—competes for the model's limited attention and token budget. An unstructured wall of text forces the model to expend effort parsing the input, increasing the risk of misinterpretation. Structuring the input offloads this parsing burden, freeing up the model's reasoning capabilities for the actual task.

However, structure often comes at a cost. Let's start by examining the token efficiency of different formats.

As the video shows, the verbosity of formats like JSON—with its brackets, quotes, and keys—can more than double the token count for the same semantic content. This has direct implications for both API costs and how much information you can fit into the context window. This tension between structural clarity, token efficiency, and human readability is central to designing effective input formats.

2. A Comparative Analysis of Common Formats

For most agentic workflows, the choice of input format boils down to three primary contenders: raw/plain text, Markdown, and XML. Each has distinct advantages and is suited for different stages of development and deployment.

A systematic, evidence-based approach is the best way to understand their trade-offs.

The video's findings provide a clear set of guidelines:

• XML (<tag>): The winner for production systems. The explicit, unambiguous tags help the model clearly distinguish between different parts of the prompt (e.g., instructions vs. user data vs. retrieved examples). This is especially crucial for long, complex prompts where the risk of "context clash" is high. This aligns with recommendations from model providers like Anthropic.
• Markdown (#, *): The best choice for development and readability. Its lightweight syntax makes prompts easy for humans to write, read, and debug. While its performance is strong, it can be slightly less consistent than XML for highly complex instructions.
• Raw/Plain Text: Surprisingly effective for top-tier models on simple tasks, but it lacks the structural anchors needed for reliable performance in complex, multi-part prompts that are common in agentic systems.

This empirical evidence is supported by formal academic research on entity serialization.

The key takeaway from the paper is that both encoder models and LLMs "have a preference for serialization schemes that are close to the corpus they were pretrained on." Since web data (full of XML/HTML and Markdown) forms a massive part of this corpus, these formats are more "native" to the model than bespoke formats with special tokens like [COL] and [VAL].

3. Practical Application: Structuring Different Inputs

Let's translate this theory into practice. In a typical agentic workflow, you'll be handling several types of input data, and each can be structured for clarity. The guiding principle is to unambiguously label every piece of information.

A. Demarcating Context Sources

The most critical use of formatting is to create a clear separation between your instructions, user input, and data retrieved via RAG or tools. XML is exceptionally well-suited for this.

Consider a prompt where an agent needs to answer a user's question using a retrieved document.

Poor (Unstructured):

You are a helpful assistant. Answer the user's question based on the following text.

User question: What is the return policy?

All items can be returned within 30 days of purchase for a full refund, provided they are in their original condition. Sale items are final.

Good (XML-Structured):

You are a helpful assistant. Your task is to answer the user's question using the provided context.

<user_question>
What is the return policy?
</user_question>

<retrieved_document source="faq.pdf">
All items can be returned within 30 days of purchase for a full refund, provided they are in their original condition. Sale items are final.
</retrieved_document>

The XML version leaves no ambiguity. The model knows exactly which part is the query and which part is the evidence, preventing it from confusing the two.

B. Structuring Tabular Data

Tables are a common form of structured data that can be particularly challenging for LLMs. Simply pasting a CSV or a poorly formatted table can lead to misinterpretation. For such specific, highly structured data types, you can design a custom serialization format.

This approach of creating a domain-specific "micro-language" for your data is a powerful design pattern. By consistently representing tables in this format, you teach the model how to parse them reliably.

For example, a simple user table:

Name	Role
Alice	Admin
Bob	User

Could be serialized as:
<CAP> User Roles </CAP> <R> <C> Name </C> <C> Role </C> </R> <R> <C> Alice </C> <C> Admin </C> </R> <R> <C> Bob </C> <C> User </C> </R>

This is far more explicit than a simple Markdown table and more robust for complex automated workflows.

4. Advanced Topic: Token-Efficient Formats

While XML is great for performance and Markdown for readability, what if your primary constraint is cost, especially when passing large amounts of structured data? This brings us back to formats like TOON.

Think of this as a compression step in your context assembly pipeline. For use cases involving large batches of flat, structured data (e.g., feeding hundreds of product records into an agent for categorization), this approach can lead to significant cost savings (30-60% as claimed) and, in some cases, even improved retrieval accuracy. It's a specialist tool, but a valuable one for your toolkit when designing for scale.

Conclusion

In this lesson, we established that the structure of input data is a critical factor for agent reliability and performance. We've moved beyond simply feeding raw text to the model and have developed a more sophisticated understanding of data serialization for LLMs.

Key Takeaways:

• Serialization Matters: The format of your input data directly impacts model comprehension, output consistency, and operational cost.
• Choose the Right Format for the Job:
  • Markdown: Ideal for development due to its high readability.
  • XML: The standard for production due to its explicitness, which improves performance on complex tasks.
  • JSON: A common standard for data interchange, but can be token-inefficient as a direct LLM input format.
• Label Everything: Use structural tags (like XML) to clearly demarcate different sources of information within your prompt (e.g., <instructions>, <user_query>, <retrieved_data>).
• Design for Your Data: For highly structured or domain-specific data like tables, consider creating a custom, consistent serialization format to improve parsing reliability.
• Optimize for Cost: For large-scale data processing, specialized token-efficient formats like TOON can act as a "compression" layer to reduce costs.

Preview of the Next Lesson

We now have a handle on how to structure the dynamic information we feed to an agent. The next logical step is to formalize the static instructions that govern the agent's core identity and behavior.

In the next lesson, we will begin Module 4 by learning how to write system instructions that define an agent's role, tone, boundaries, and decision-making rules. This is where we'll design the agent's "personality" and the fundamental principles it must follow, regardless of the user's input or the data it retrieves.