Graceful Handling of Empty Retrieval Results

The "No Context" Failure Mode

Your current pipeline is effective when the retriever successfully finds relevant document chunks. But what happens if it doesn't? Imagine a user asks a question that is completely unrelated to the documents in your vector store.

1. Retrieval: The retriever.retrieve(query) call returns an empty list of nodes.
2. Augmentation: Your create_augmented_prompt function is called with an empty context. The resulting prompt sent to the LLM would contain your system instructions and the user's question, but no "Evidence" section.
3. Generation: The LLM receives this prompt and is asked to answer a question without any provided context, despite instructions to use only the provided context.

In this situation, the LLM's behavior can be unpredictable. It might:

• Hallucinate: Invent an answer based on its pre-trained knowledge, directly violating your "constrained generation" instructions.
• Refuse: Correctly identify the lack of context and respond with "I cannot answer this question based on the information provided."
• Give a Vague Answer: Provide a generic, unhelpful response.

None of these outcomes are ideal. Hallucinations destroy user trust, while a simple refusal lacks clarity about why the answer couldn't be found. We need an explicit, reliable way to handle this.

Strategy 1: Prompt Engineering and Programmatic Checks

The most straightforward way to handle this edge case is to combine a simple programmatic check with smart prompt engineering.

The "I Don't Know" Prompt

You can explicitly instruct the LLM on how to behave when the context is insufficient. This reinforces the constrained generation you learned about previously. By adding a rule to your system prompt, you can guide the model to a desired, predictable outcome.

The video below demonstrates this simple but powerful technique. The presenter modifies the system prompt to include the instruction, "if you don't know the answer, just say I don't know," and then tests it with an out-of-scope query.

This is an excellent first line of defense. A well-crafted prompt can often handle cases where the retrieved context is sparse or irrelevant.

The Code-Level Fallback

While good prompting helps, a more robust solution is to catch the failure at the code level, before even calling the LLM. You can modify your main pipeline function to check if the retrieval step returned any documents. If not, you can bypass the augmentation and generation steps entirely and return a canned response.

Here's how you could modify the query_rag_pipeline function from the last lesson:

def query_rag_pipeline(query: str) -> dict:
    """
    Orchestrates the full RAG pipeline, with a fallback for no-retrieval cases.
    """
    # 1. RETRIEVE: Fetch relevant context chunks
    retrieved_nodes = retriever.retrieve(query)

    # --- FALLBACK LOGIC ---
    if not retrieved_nodes:
        answer = "I'm sorry, but I couldn't find any relevant information in the knowledge base to answer your question."
        return {"answer": answer, "sources": []}
    # ----------------------

    # 2. AUGMENT: Construct the augmented prompt
    augmented_prompt = create_augmented_prompt(retrieved_nodes, query)

    # 3. GENERATE: Get the final response from the LLM
    answer = generate_response(augmented_prompt)
    
    sources = [node.metadata.get("file_name") for node in retrieved_nodes]

    return {"answer": answer, "sources": list(set(sources))}

# --- Example Usage ---
# user_query = "What is the capital of Mars?" 
# result = query_rag_pipeline(user_query)
# print(result["answer"]) 
# >> I'm sorry, but I couldn't find any relevant information...

This approach is simple, efficient, and gives the user clear feedback. It prevents unnecessary LLM calls and avoids the risk of hallucination entirely when no context is found.

Strategy 2: Corrective RAG (CRAG) with a Fallback

Giving up is one option, but what if we could correct the retrieval failure? This is the core idea behind more advanced RAG architectures. Instead of just stopping, the system attempts to find information from an alternative source.

Corrective RAG (CRAG) is a powerful pattern designed for this. It introduces an evaluation step after retrieval. If the retrieved documents are deemed irrelevant or insufficient, CRAG triggers a corrective action, such as performing a web search to find the missing information.

This diagram shows the CRAG architecture. After retrieval, an evaluator assesses the documents. If they are incorrect or ambiguous, it can trigger knowledge searching (e.g., a web search) to find better context before generation.

This makes the system far more versatile. It can rely on its primary, trusted knowledge base, but it also has a fallback to the vast, open knowledge of the internet for questions that fall outside that scope. This image below places CRAG within a family of RAG architectures.

This image compares four RAG architectures. Notice how Corrective RAG (c) introduces a decision step with an NLI model to classify retrieved documents as Correct, Ambiguous, or Incorrect, with a web search triggered for the latter two.

The following video provides an excellent, in-depth walkthrough of building a CRAG system that uses web search as its fallback knowledge base. It's a perfect practical guide for a software engineer looking to implement a more sophisticated solution.

For a supplementary text-based resource that covers the same ideas, the following article provides a concise overview and code snippets.

Implementing a CRAG-like system is a significant step up in complexity, but it results in a much more capable and resilient application. It's a clear example of how modern AI systems are built with multiple layers of logic, evaluation, and fallback mechanisms.

Conclusion

In this lesson, you've learned how to handle one of the most common failure modes in a RAG system: failed retrieval. Making your application robust to this edge case is essential for building user trust and creating a reliable product.

Here are your key takeaways:

• The "No Context" problem is critical. An unhandled retrieval failure can lead to hallucinations or unhelpful responses.
• Simple Fallbacks are effective. A combination of programmatic checks ( if not retrieved_nodes: ) and explicit prompt instructions ( "If you don't know, say 'I don't know'" ) provides a strong, simple baseline for robustness.
• Corrective RAG (CRAG) offers a powerful alternative. Instead of just failing, you can build a system that evaluates retrieval quality and falls back to an alternative knowledge source, like a web search, to find the answer.

You have now completed the core module on assembling a RAG pipeline. You've built a system from scratch and made it robust to common failures. But how good is it, really? In the next module, Measuring Success: RAG Evaluation, you will learn how to answer that question. We will dive into the critical metrics and methodologies for assessing the quality of both your retrieval and generation steps.