RAG-Powered Agents for Knowledge Base Q&A

Hello! Welcome back.

In our last lesson, you built your first conversational agent using LangChain. You saw how a framework can orchestrate the agent's reasoning loop, allowing it to use tools and remember conversation history. However, the tools we used were simple, stateless functions.

A key goal for many AI applications is to have an agent that can act as an expert on a specific, private knowledge base—be it internal company documentation, a complex technical manual, or a library of legal documents. In this lesson, we will bridge the gap between a general-purpose agent and a domain-specific expert.

Our learning outcome is to integrate a RAG system into an agent to answer questions from a knowledge base. You will learn how to equip your agent with a powerful new tool: the ability to search through documents and use the retrieved information to form its answers.

From RAG to "Agentic RAG"

You're already familiar with the Retrieval-Augmented Generation (RAG) pattern, where external documents are retrieved to augment an LLM's prompt. "Agentic RAG" takes this a step further. Instead of having a system that always performs retrieval, we build an agent that decides when to perform retrieval.

The agent is given the RAG system as one of its available tools. It can then reason about a user's query and choose the best course of action:

• Should I use my internal knowledge to answer this?
• Should I use a web search tool for up-to-date information?
• Should I use my special RAG tool to search the private knowledge base?

This decision-making capability is what makes an agentic approach so powerful. The diagram below contrasts a standard RAG pipeline with an Agentic RAG system. Today, we're building the latter.

Building an Agent with a RAG Tool

We'll now walk through the process of building an agent that has access to both a general web search tool and a RAG tool for a specific document. This will give you a clear, practical example of how the agent makes its choice.

The following video provides a concise, step-by-step guide. We'll watch it in segments to focus on the key concepts.

1. Setting Up the Agent and a Basic Tool

First, let's quickly review how an agent is set up and given a simple tool. This part of the video creates an agent with access to a web search tool (Tavily). This will serve as a baseline to compare against the RAG tool we'll add next.

At this point, you have an agent that can answer questions about current events by searching the internet. Now, let's give it specialized knowledge.

2. Creating a RAG Tool for a Specific Knowledge Base

This is the core of our lesson. We will now create a RAG pipeline and, crucially, wrap it into a tool that the agent can use.

The process involves these familiar RAG steps:

1. Load: Fetch a document from a URL.
2. Split: Break the document into smaller chunks.
3. Embed & Store: Convert chunks to vectors and store them in a vector store.
4. Create a Retriever: Build a retriever object that can search the vector store.

The final, and most important, step is to convert this retriever into a Tool. LangChain provides a convenient helper function for this: create_retriever_tool.

The video demonstrates the complete process. By providing a clear and specific description for the retriever tool (e.g., "Use this tool when searching for information about LangChain Expression Language"), you guide the agent to make the correct decision.

Putting It All Together: A Complete Example

Let's consolidate everything into a single, runnable script. The following code creates an agent with two distinct tools:

1. lcel_retriever_tool: A RAG tool that searches a document about the LangChain Expression Language (LCEL).
2. get_word_length: A simple Python function to count letters in a word.

This setup forces the agent to choose between querying its specialized knowledge base or performing a simple, unrelated task.

You will need to install faiss-cpu for the in-memory vector store: pip install faiss-cpu.

import os
from dotenv import load_dotenv

from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_community.document_loaders import WebBaseLoader
from langchain_community.vectorstores import FAISS
from langchain_text_splitters import RecursiveCharacterTextSplitter

from langchain.tools.retriever import create_retriever_tool
from langchain.tools import tool
from langchain import hub
from langchain.agents import create_openai_functions_agent, AgentExecutor




# 1. Load environment variables
load_dotenv()




# 2. Set up the LLM
llm = ChatOpenAI(model="gpt-3.5-turbo-1106", temperature=0)




# 3. Create the RAG tool for LCEL
# Load a document
loader = WebBaseLoader("https://python.langchain.com/docs/expression_language/")
docs = loader.load()




# Split the document into chunks
text_splitter = RecursiveCharacterTextSplitter()
documents = text_splitter.split_documents(docs)




# Create embeddings and a vector store
embeddings = OpenAIEmbeddings()
vector = FAISS.from_documents(documents, embeddings)




# Create a retriever
retriever = vector.as_retriever()




# Wrap the retriever in a tool
lcel_retriever_tool = create_retriever_tool(
    retriever,
    "langchain_lcel_retriever",
    "Use this tool when searching for information about LangChain Expression Language (LCEL).",
)




# 4. Create a second, simple tool
@tool
def get_word_length(word: str) -> int:
    """Returns the length of a word."""
    return len(word)




# 5. Create the Agent
# Pull a standard prompt
prompt = hub.pull("hwchase17/openai-functions-agent")




# Define the list of tools
tools = [lcel_retriever_tool, get_word_length]




# Create the agent
agent = create_openai_functions_agent(llm, tools, prompt)




# Create the agent executor
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)




# 6. Run the Agent
print("--- Asking a question that should use the RAG tool ---")
agent_executor.invoke({"input": "What is LangChain Expression Language?"})

print("\n--- Asking a question that should use the simple tool ---")
agent_executor.invoke({"input": "How many letters are in the word 'engineering'?"})

To run this code:

1. Make sure your .env file contains your OPENAI_API_KEY.
2. Ensure you have the necessary packages: langchain, langchain-openai, python-dotenv, beautifulsoup4, faiss-cpu.
3. Save the code as a Python file and execute it.

When you run the script, observe the verbose=True output. You'll see the agent's thought process. For the first question, it will decide to invoke the langchain_lcel_retriever tool. For the second, it will invoke get_word_length. This clearly demonstrates the agent's ability to reason and select the appropriate tool for the job.

Conclusion

In this lesson, you have successfully empowered your agent with a dedicated knowledge base. By integrating a RAG system as a tool, you've moved from a general-purpose agent to one that can be a specialized expert, capable of reasoning about when to tap into its unique knowledge.

Key Takeaways:

• Agentic RAG: This pattern involves giving an agent a RAG system as a tool, allowing it to decide when to query a knowledge base versus using other tools or its internal knowledge.
• create_retriever_tool: This LangChain function is the most direct way to make a document retriever available to an agent.
• The Power of Description: The description you provide for a tool is paramount. It acts as the primary guidance for the agent's LLM brain, telling it what the tool is for and when it should be used.
• Tool Selection: An agent can be equipped with multiple tools, including RAG-based and standard function-based tools, and it will use the LLM's reasoning capabilities to select the correct one for a given user request.

Our agent can now select the right tool for a single-step task. But what happens when a user's request is more complex and requires multiple actions in a sequence?

In our next lesson, Implement an agent that uses a planner to decompose a complex request into a sequence of steps, we will explore how agents can tackle more sophisticated problems by first creating a plan and then executing it step-by-step.