Building a RAG App with Streamlit

1. A Tale of Two Architectures

When connecting a Streamlit UI to a machine learning backend like your RAG pipeline, there are two common approaches. Given your background in web development with frameworks like Django and React.js, it's worth understanding the trade-offs.

• Approach 1: Direct Import
  The simplest way is to import your RAG function directly into your app.py script and call it. This is fast for small projects and prototypes. However, it creates a monolithic application where the UI and the ML logic are tightly coupled. This can make the app harder to maintain, scale, and can sometimes lead to performance issues with Streamlit if the model is large.

• Approach 2: Decoupled API
  A more robust and professional approach is to separate the front-end (Streamlit) from the back-end (RAG pipeline). The backend logic is wrapped in a web API (using a framework like FastAPI), and the Streamlit app communicates with it by making HTTP requests.

This diagram illustrates the decoupled architecture. The user interacts with the Streamlit UI, which sends a query to the RAG Query Engine (our backend API). The API handles retrieval and generation, then returns the final answer to the UI.

For this project, we will use the decoupled API approach. It aligns with modern software engineering practices you're familiar with, makes your application more scalable, and directly prepares you for the upcoming lessons on model deployment using FastAPI and Docker.

The following video provides an excellent overview of this architecture, explaining how a Streamlit front-end can communicate with a separate API backend.

2. Building the Brain's API with FastAPI

Before we can modify the Streamlit app, we need to expose our RAG pipeline as an API endpoint. As per your course goals, we'll use FastAPI. Your experience with Express.js and Django will make these concepts feel very familiar. We will create a simple server that listens for a user's query and returns the AI Tutor's response.

Let's organize our project. Your directory should look something like this:

/ai_tutor_project
|-- app.py             # Your Streamlit frontend
|-- api.py             # Our new FastAPI backend
|-- rag_pipeline.py    # The file containing your RAG logic
|-- vector_db/         # Your ChromaDB vector store
|-- .env               # Your environment variables

Create a new file called api.py. This will house our backend server. Here is the basic structure:

# api.py

import uvicorn
from fastapi import FastAPI
from pydantic import BaseModel
from typing import List

# Assume your RAG logic is in a function like this in a separate file
# from rag_pipeline import get_rag_response 

# --- Placeholder for your actual RAG function ---
# In a real scenario, you would import this from another module.
# For now, we'll define a simple placeholder to test the API structure.
def get_rag_response(prompt: str, history: List[dict]) -> str:
    print(f"Received prompt: {prompt}")
    print(f"Received history: {history}")
    # In a real implementation, you would pass the prompt and history
    # to your LangChain chain's .invoke() or .stream() method.
    return f"This is the AI Tutor's response to: '{prompt}'"
# -----------------------------------------------

# Pydantic models for request and response data validation
class QueryInput(BaseModel):
    prompt: str
    history: List[dict]

class QueryOutput(BaseModel):
    answer: str

# Initialize the FastAPI app
app = FastAPI(
    title="AI Tutor API",
    description="Endpoints for the AI SQL Tutor RAG pipeline."
)

@app.post("/tutor", response_model=QueryOutput)
async def ask_tutor(query: QueryInput) -> QueryOutput:
    """
    Receives a user's prompt and chat history, gets a response
    from the RAG pipeline, and returns the answer.
    """
    answer = get_rag_response(prompt=query.prompt, history=query.history)
    return {"answer": answer}

# This allows running the server directly for testing
if __name__ == "__main__":
    uvicorn.run(app, host="0.0.0.0", port=8000)

To run this server:
Open your terminal in the project directory and run:
uvicorn api:app --reload

You should see output indicating the server is running on http://127.0.0.1:8000. The --reload flag ensures the server restarts automatically when you save changes to the file.

3. Connecting the UI to the API

Now, we'll modify app.py to call our new /tutor endpoint instead of using the simple "echo" placeholder. We'll use the popular requests library to make the HTTP POST request.

This pattern—using a utility function in the Streamlit app to handle the API call—is a common best practice. The following article provides a clear, text-based example of this structure.

The video below demonstrates this integration live, showing how the Streamlit app receives user input, sends it to the backend, and displays the result.

4. Your Task: Wire Up the Application

It's time to connect the pieces. You will have two services running locally: the FastAPI server and the Streamlit app.

Part 1: Finalize the Backend (api.py)

1. In your api.py file, replace the placeholder get_rag_response function with an import and a call to the actual RAG chain you built in previous lessons.
2. Ensure your RAG chain function can accept the user's prompt and chat history to handle follow-up questions correctly.

Part 2: Update the Frontend (app.py)

Modify your existing app.py from the last lesson.

1. Import requests: Add import requests at the top of your file.
2. Define the API URL: Add a constant for your API endpoint, API_URL = "http://127.0.0.1:8000/tutor".
3. Replace Placeholder Logic: Find the if prompt := st.chat_input(...) block. You will replace the placeholder response logic with the following steps:
   • After displaying the user's message, show a spinner to indicate work is being done (e.g., with st.spinner("Cody is thinking...")).
   • Prepare the JSON payload for the API request. It needs a prompt and the current history from st.session_state.messages.
   • Use a try...except block for robust error handling.
   • Inside the try block, make the POST request: response = requests.post(API_URL, json=payload).
   • Check if the request was successful (response.raise_for_status() is a good way to do this).
   • Parse the JSON response to get the answer: answer = response.json()["answer"].
   • Display the assistant's answer and append it to the session state history.
   • In the except block, handle potential connection errors by displaying an error message in the chat.

Here's a template for the new logic inside your if prompt... block:

# In app.py, inside the 'if prompt := st.chat_input(...):' block

# ... (code to display user message and append to history) ...

with st.chat_message("assistant"):
    with st.spinner("Cody is thinking..."):
        try:
            # Prepare the payload for the API
            payload = {
                "prompt": prompt,
                # Send all but the last message (the new user prompt) as history
                "history": st.session_state.messages[:-1] 
            }
            
            # Call the FastAPI endpoint
            response = requests.post(API_URL, json=payload, timeout=120)
            response.raise_for_status()  # Raise an exception for bad status codes
            
            # Extract the answer from the response
            answer = response.json()["answer"]
            
            # Display and store the assistant's response
            st.markdown(answer)
            st.session_state.messages.append({"role": "assistant", "content": answer})

        except requests.exceptions.RequestException as e:
            error_message = f"An error occurred: {e}"
            st.error(error_message)
            st.session_state.messages.append({"role": "assistant", "content": error_message})

Part 3: Run the Full System

1. Open two separate terminals in your project directory.
2. In Terminal 1, start the backend: uvicorn api:app --reload
3. In Terminal 2, start the frontend: streamlit run app.py
4. Your browser should open with the Streamlit app. Test it by asking a question that requires knowledge from your documents. You should see the "thinking" spinner, and then the actual, context-aware response from your RAG pipeline will appear!

Conclusion

Excellent work! You have now successfully integrated your RAG pipeline with the Streamlit front-end, creating a complete, end-to-end conversational AI application. By choosing a decoupled API architecture, you've built a system that is not only functional but also scalable and maintainable.

Key Takeaways:

• Decoupled Architecture: Separating the UI (Streamlit) from the ML logic (FastAPI) is a professional standard that improves scalability and maintainability.
• FastAPI for ML Serving: FastAPI provides a simple yet powerful way to serve machine learning models and pipelines as robust API endpoints.
• Streamlit as an API Consumer: A Streamlit app can easily interact with any backend API using the requests library, allowing you to build user-friendly interfaces for complex systems.

Preview of the Next Lesson:

Your AI Tutor is now fully functional on your local machine. The final step in making this a true portfolio project is deployment. In the next lesson, "Containerize and Deploy the Chatbot Application," we will use Docker to package your FastAPI backend and your Streamlit frontend into portable containers. This will prepare you to deploy your application to the cloud, making it accessible to anyone, anywhere. Your experience with Docker and Kubernetes will be a great asset in this final stage of the project.