Feedback for Model Improvement

1. The Feedback Loop: Turning Users into Collaborators

The core idea behind collecting feedback is to create a continuous cycle of improvement for your AI model. This is often called a "feedback loop."

Collecting user feedback isn't just about finding bugs; it's a strategic part of MLOps. The article "Collecting user feedback on ML in Streamlit" highlights several key benefits.

As the article explains, user feedback helps you:

• Improve Model Performance: It provides real-world data on where your model is succeeding or failing, which is invaluable for fine-tuning.
• Enhance User Experience: It helps you understand user pain points and preferences.
• Increase Engagement: Users who feel heard are more likely to trust and use your application.
• Enable Continuous Improvement: Feedback is the fuel for ongoing model monitoring and retraining, preventing performance degradation over time.

Now, let's see how to put this theory into practice.

2. Frontend: Capturing Feedback in Streamlit

In a previous lesson, you learned to build interactive UIs with Streamlit. Now, we'll use a specific Streamlit component, st.feedback, which is designed for exactly our purpose.

The Streamlit documentation provides a clear, step-by-step tutorial on building a chat application that collects feedback. We will adapt this for our needs.

Let's synthesize what's in the documentation with a complete, runnable example. This app simulates a model's response and allows the user to give a thumbs-up or thumbs-down.

Example: A Simple Feedback App in Streamlit

import streamlit as st
import time
import requests
import json




# --- Backend API Endpoint (run this in a separate terminal with FastAPI) ---
# We'll define the backend logic in the next section. For now, assume it exists.
FEEDBACK_API_URL = "http://127.0.0.1:8000/feedback"




# --- Streamlit App ---




# Function to simulate an LLM response
def get_model_response(prompt):
    response = f'This is a simulated response to your prompt: "{prompt}"'
    for char in response:
        yield char
        time.sleep(0.02)




# Callback function to save feedback to session_state and send to backend
def save_feedback(response_index, feedback_value):



    # Save to session state
    st.session_state.history[response_index]["feedback"] = feedback_value
    



    # Prepare data to send to the backend
    feedback_data = {
        "prompt": st.session_state.history[response_index-1]["content"],
        "response": st.session_state.history[response_index]["content"],
        "feedback": "positive" if feedback_value == "thumbsup" else "negative",
        "model_version": "v1.0-simulated"
    }
    



    # Send feedback to the backend API
    try:
        res = requests.post(FEEDBACK_API_URL, json=feedback_data)
        if res.status_code == 201:
            st.toast("Feedback submitted successfully!", icon="✅")
        else:
            st.toast(f"Error submitting feedback: {res.text}", icon="❌")
    except requests.exceptions.ConnectionError:
        st.error("Could not connect to the feedback API. Is the backend running?")




# Initialize chat history
if "history" not in st.session_state:
    st.session_state.history = []

st.title("AI with Feedback")




# Display chat messages from history
for i, message in enumerate(st.session_state.history):
    with st.chat_message(message["role"]):
        st.write(message["content"])



        # Add feedback widget for assistant messages
        if message["role"] == "assistant":
            feedback = message.get("feedback")
            st.feedback(
                "thumbs",
                key=f"feedback_{i}",
                disabled=feedback is not None, # Disable if feedback already given
                on_change=save_feedback,
                args=(i, st.session_state[f"feedback_{i}"]),
            )




# Accept user input
if prompt := st.chat_input("Ask the AI something..."):



    # Add user message to history
    st.session_state.history.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.write(prompt)




    # Generate and display assistant response
    with st.chat_message("assistant"):
        response_content = st.write_stream(get_model_response(prompt))
        



        # Add the new response to history BEFORE creating the feedback widget
        response_index = len(st.session_state.history)
        st.session_state.history.append({"role": "assistant", "content": response_content})




        # Add the feedback widget for the new response
        st.feedback(
            "thumbs",
            key=f"feedback_{response_index}",
            on_change=save_feedback,
            args=(response_index, st.session_state[f"feedback_{response_index}"])
        )

This code establishes the frontend part of our feedback loop. When a user clicks a thumb, the save_feedback callback is triggered, which not only updates the UI but also sends the feedback to a backend API.

While st.feedback is excellent, for more complex scenarios you might explore libraries like Trubrics, which offers more advanced components.

3. Backend: Persisting Feedback for Analysis

The feedback collected in Streamlit's session state is temporary. To make it useful for model improvement, we must store it persistently. This is where your FastAPI skills come in.

We need an API endpoint that accepts the feedback data and saves it. A simple approach is to save it to a structured file like a JSON or CSV. A more robust solution would use a database.

Here is a simple FastAPI backend that receives the feedback from our Streamlit app and appends it to a JSON file.

# feedback_api.py

from fastapi import FastAPI, HTTPException, status
from pydantic import BaseModel
import json
from datetime import datetime

app = FastAPI()




# Pydantic model for incoming feedback data
class FeedbackPayload(BaseModel):
    prompt: str
    response: str
    feedback: str  # e.g., "positive" or "negative"
    model_version: str

FEEDBACK_FILE = "feedback_log.json"

@app.post("/feedback", status_code=status.HTTP_201_CREATED)
def store_feedback(payload: FeedbackPayload):
    """
    Receives feedback from the frontend and stores it.
    """



    # Structure the data to be logged
    log_entry = {
        "timestamp": datetime.utcnow().isoformat(),
        "prompt": payload.prompt,
        "response": payload.response,
        "feedback": payload.feedback,
        "model_version": payload.model_version
    }

    try:



        # Read existing data and append new entry
        try:
            with open(FEEDBACK_FILE, "r") as f:
                logs = json.load(f)
        except (FileNotFoundError, json.JSONDecodeError):
            logs = []
        
        logs.append(log_entry)
        



        # Write back to the file
        with open(FEEDBACK_FILE, "w") as f:
            json.dump(logs, f, indent=2)
            
    except Exception as e:
        raise HTTPException(
            status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
            detail=f"Failed to write feedback: {e}"
        )
        
    return {"message": "Feedback stored successfully", "data": log_entry}

To run this system:

1. Save the FastAPI code as feedback_api.py and run it: uvicorn feedback_api:app --reload.
2. Save the Streamlit code as app.py and run it in a separate terminal: streamlit run app.py.

Now, when you submit feedback in the Streamlit app, a new entry will be added to feedback_log.json. This log file becomes your new dataset for analysis and future model fine-tuning.

4. Advanced Concept: Active Feedback with LangGraph

So far, we've focused on collecting feedback for future model improvement. But what if feedback could improve the result right now? This concept is often called Human-in-the-Loop (HITL), and it's particularly powerful in generative AI.

The idea is to treat human feedback not just as a label, but as an instruction to retry or refine. LangGraph, a library for building stateful, multi-agent applications, is perfect for this.

This video demonstrates how to build a system where a user can provide corrective feedback to an AI, which then generates a better response in the same session.

This HITL approach creates a collaborative session between the user and the AI, where the final output is a joint effort. It's a powerful pattern for tasks like writing assistance, code generation, and complex problem-solving.

Conclusion

You have now learned how to create a complete feedback loop in your AI application. This is a critical skill that bridges the gap between a prototype and a production-ready system that can adapt and improve.

Key Takeaways:

• Feedback is essential: It's the primary mechanism for monitoring model performance, identifying weaknesses, and gathering data for continuous improvement.
• Frontend implementation is straightforward: With tools like Streamlit's st.feedback, you can build intuitive UIs for feedback collection with minimal code.
• Backend persistence is key: You must design a system (e.g., a FastAPI endpoint and a database or log file) to store feedback in a structured format, linking it to the original prompt, response, and model version.
• Active feedback is the next frontier: With frameworks like LangGraph, you can build Human-in-the-Loop systems where user feedback immediately refines AI outputs, creating a more interactive and collaborative experience.

Preview of the Next Lesson:

Now that your application is interacting more deeply with users—accepting prompts and feedback—security becomes paramount. In the next lesson, we will "Apply security best practices: input validation and prompt injection prevention," ensuring your AI application is not only smart but also safe and robust against malicious inputs.