LLM Architecture: RAG, Agents, and Tools

Hello! Welcome to the final lesson of your course on Large Language Models and AI.

Throughout this journey, we've moved from the building blocks of neural networks to the intricacies of building sophisticated AI applications. In our last lesson, we implemented topical guardrails, a crucial component for ensuring an application's safety and focus. Today, we will zoom out and put all the pieces together.

This capstone lesson addresses the final learning outcome: Create a high-level component diagram for an LLM application combining RAG, agents, and tools. As a software engineer, thinking in terms of system architecture is second nature. We will apply that skill to synthesize everything we've learned—RAG, agents, tools, and security—into a cohesive blueprint for a modern AI application.

1. The AI Technology Stack: A Mental Model

Before diving into a detailed diagram, it's helpful to have a high-level framework. A modern AI application can be viewed as a stack of five distinct layers, each with a specific responsibility.

To introduce this concept, let's watch a short segment from a video by IBM Technology.

This 5-layer model (Infrastructure, Model, Data, Orchestration, Application) gives us a structured way to organize the components we've discussed throughout this course. Now, let's map the concepts you've learned onto this stack.

• Application Layer: The user-facing part (e.g., a chat interface).
• Orchestration Layer: This is the "brain" of the operation. The AI Agent lives here, using frameworks like LangChain or LangGraph to manage planning, tool use, and memory. The guardrails we discussed previously also operate at this layer, inspecting inputs and outputs.
• Data Layer: This is the domain of Retrieval-Augmented Generation (RAG). It includes the knowledge base, the vector database, and the retrieval pipeline (chunking, embedding, searching).
• Model Layer: This contains the core AI models: the LLM for reasoning, the embedding model for converting text to vectors, and optionally, a reranker model.
• Infrastructure Layer: The hardware (CPUs, GPUs) and cloud services that run everything.

2. Core Components of an Agentic RAG System

Let's focus on the Orchestration, Data, and Model layers, which form the core of our application logic. An "Agentic RAG" system combines the autonomous decision-making of an agent with the knowledge-retrieval power of RAG.

The article from the NVIDIA Developer Blog provides a clear breakdown of the components required to build such a system.

As you've read, the agent acts as a dynamic controller. Instead of a fixed RAG pipeline that always runs, the ReAct agent first reasons whether it needs external information. If it does, it acts by invoking the RAG pipeline as a tool. This is the fundamental difference between a simple RAG chain and a more powerful Agentic RAG system.

3. A Catalog of Architectural Patterns

Just as in traditional software engineering, there isn't one single architecture for LLM applications. Instead, there are several established patterns, each with its own trade-offs.

The following article provides a superb typology of seven common RAG architectural patterns, ranging from the very simple to the highly complex. This gives you a set of design options to consider when building your own systems.

These patterns are your architectural toolkit. Notice how they progressively add complexity to solve specific problems:

• Naive RAG is the baseline.
• Retrieve-and-rerank adds a quality control step.
• Hybrid RAG improves retrieval for specific keywords.
• Agentic RAG (Router-type) introduces decision-making, allowing the system to choose between different data sources or tools. This is a crucial pattern for combining RAG with other capabilities.

4. Tracing a Request Through the System

Let's trace a user query to see how these components interact in an Agentic RAG (Router-type) architecture.

Scenario: A user asks, "What were the main takeaways from our Q4 earnings call, and what is our current stock price?"

1. Application Layer: The user submits the query through a web-based chat interface.
2. Orchestration Layer (Agent):
   • The query is received by the Agent.
   • An Input Guardrail checks for moderation issues.
   • The Agent (powered by an LLM) uses its reasoning ability (the "Re" in ReAct) to form a plan:
     1. The first part of the query ("Q4 earnings call") requires searching the internal knowledge base.
     2. The second part ("current stock price") requires a real-time data lookup.
   • The Agent decides to act.
3. Action 1: Use the RAG Tool
   • The Agent invokes its knowledge_base_retriever tool with the query "Q4 earnings call takeaways."
   • Data Layer (RAG):
     • The query is converted to an embedding using the Embedding Model.
     • The system performs a similarity search against the Vector Database containing the company's documents.
     • The top N relevant document chunks are retrieved.
     • A Reranker Model re-orders these chunks for maximum relevance.
   • The most relevant context from the earnings call transcript is returned to the Agent.
4. Action 2: Use the API Tool
   • The Agent now invokes its stock_price_api tool with the company's ticker symbol.
   • Tool/Integration Layer: The tool makes an API call to an external financial data service.
   • The current stock price is returned to the Agent.
5. Generation Step:
   • The Agent now has all the pieces: the original query, the context from the earnings call, and the current stock price.
   • It synthesizes this information into a single, comprehensive prompt for the LLM.
   • The LLM generates a final, human-readable answer.
6. Final Checks:
   • An Output Guardrail checks the generated response for any policy violations (e.g., ensuring it doesn't give financial advice).
   • The final, vetted response is sent back to the Application Layer and displayed to the user.

Your Turn: Design an LLM Application

Now it's time to apply these concepts. Your task is to create a high-level component diagram for the following scenario. You can sketch it on paper or use a simple diagramming tool. The goal is to identify the components and their connections, not to create a beautiful drawing.

Scenario:
You are designing an AI assistant for a team of doctors. The assistant must perform two primary functions:

1. Answer questions about patient history by securely searching a database of anonymized Electronic Health Records (EHR).
2. Provide summaries of the latest medical research by searching public sources like PubMed.

The system must never provide a diagnosis; it should only retrieve and summarize information.

Task:
Create a high-level component diagram for this system. Your diagram should include and label:

• The main user interface.
• An Agent that acts as a router/orchestrator.
• At least two distinct tools the agent can use (one for EHR, one for PubMed).
• The components of the RAG pipeline for searching the EHR data (e.g., Vector DB).
• The core LLM(s).
• A Guardrail to prevent diagnoses.

Draw arrows to indicate the flow of information for a query like: "Summarize recent research on metformin and check if patient ID 4815162342 has a history of diabetes."

---

Conclusion and Course Wrap-Up

Congratulations on completing the final lesson! Today, we've seen how all the individual topics from this course come together to form a complete, production-grade LLM application. By thinking in terms of architectural layers and design patterns, you can move from experimenting with prompts to engineering robust, scalable, and reliable AI systems.

Key Takeaways:

• Modern AI applications can be structured using a layered stack: Application, Orchestration, Data, Model, and Infrastructure.
• The Agent is the central orchestrator, using reasoning to dynamically select and use Tools.
• RAG is a powerful tool in the agent's toolkit, providing access to external knowledge stored in a Vector Database.
• Guardrails are critical components in the orchestration layer that enforce safety and application-specific rules.
• Choosing the right architectural pattern (e.g., Router-Agent, Multi-Agent) is a key design decision based on task complexity and performance requirements.

To leave you with a final summary of how these pieces integrate, please watch the short concluding segment of this helpful video.

You began this journey by learning about individual neurons and have progressed all the way to designing complex, autonomous agentic systems. You now have the conceptual and practical foundation to explore, build, and innovate in the exciting field of applied AI. Well done