Offline Indexing and Online Question-Answering Pipelines

Welcome back. Last time, we distinguished RAG from a standalone LLM, fine-tuning, and document search. The key idea was that a RAG application retrieves source material for a particular question and supplies it to an LLM as evidence. But that raises a practical question: how did the material become searchable, and what happens between receiving a question and returning an answer?

This lesson separates those activities into two connected pipelines. You will be able to diagram offline indexing, which prepares your Markdown and Python files for search, and online question answering, which uses that prepared index when someone asks a question. This distinction will give you a clear way to explain the architecture in your project presentation.

---

One system, two different moments

The diagram below is the central picture for this lesson. Read its upper lane as work done to prepare source files, and its lower lane as work done for a user’s question.

Notice the shared datastore/index. It is the handoff between the two lanes. Indexing puts searchable records there; question answering looks them up. The LLM does not need to read every repository file each time a user asks something.

KodeKloud’s RAG Explained For Beginners offers a short explanation of the lower lane. Watch it for the distinction between retrieving material, placing it in the prompt, and generating an answer—not for its particular implementation choices.

There are two useful meanings of “pipeline” here. Indexing and answering describe when work happens: before a question versus in response to one. Retrieval, augmentation, and generation describe major activities within the answering pipeline. They are not competing diagrams of RAG.

“Offline” also does not mean that indexing must run without a network connection. It means indexing is not normally repeated from scratch for every question. A development script might run it just before you test a question; a deployed system might refresh its index on a schedule or after file changes. In either case, the logical boundary remains the same: prepare the searchable collection first, then query it.

---

Lane one: preparing Markdown and Python files

Suppose your repository contains src/loader.py, which declares supported file extensions, and docs/ingestion.md, which explains how to refresh the collection after an edit. Before a RAG system can retrieve the relevant lines, it needs a searchable representation of those files.

Here is a diagram in stages for your project’s indexing lane. Each row’s output becomes material for the next stage; the final row creates the index used by the answering lane.

Indexing stage	Receives	Produces
Load	Selected .md and .py files	Their text and source information, such as paths
Split	Loaded text	Smaller passages, or chunks, associated with their sources
Embed	Chunk text	A numeric vector for each chunk
Store/index	Vectors, chunk text or a way to recover it, and source information	Searchable records in an index

For example, a Markdown section explaining ingestion might become one chunk. A Python function or a small group of related lines might become another. The exact splitting rules matter greatly, but for now the important point is that the unit retrieved later is usually a chunk, not an entire file.

An embedding is a numeric representation used for similarity search. During indexing, the embedder processes each chunk so that the index can later compare it with a representation of the user’s question. The stored record must preserve a connection between the vector and its original passage. Otherwise, a search might identify a relevant vector but have no source text to give the LLM—or no path and line range with which to cite it.

Conceptually, one indexed record might contain:

Chunk text:   SUPPORTED_SUFFIXES = {".md", ".py"}
Source:       src/loader.py
Line range:   4–4
Embedding:    [numeric values representing the chunk]

This is an illustrative record, not a required database format or a claim about your actual files. Its four fields show why indexing is more than “put vectors in a database”: the system also needs to recover what the match says and where it came from.

Oracle’s Select AI with Retrieval Augmented Generation (RAG) describes a concrete product implementation of both lanes. Read only the short workflow descriptions. Its object storage, database, and model configuration are Oracle-specific; the preparation-and-querying pattern is the part to carry over to your vendor-neutral project design.

The Oracle example automates parts of preparation and index updating. In your own project, do not assume an edited file is reflected in answers merely because the source file changed. The indexing workflow must process that change and update the searchable records. Until then, the answering lane can retrieve stale material.

---

Lane two: answering one question

Now imagine someone asks:

Which file types does this project ingest, and what should I do after changing a source file?

The answering lane begins with the question, not with a fresh pass through every Markdown and Python file. Its stages can be diagrammed separately:

Answering stage	Receives	Produces
Represent the query	The user’s question	A query vector suitable for searching the existing index
Retrieve	Query vector and index	Relevant stored chunks, including their text and source information
Assemble context	Question and retrieved chunks	A prompt containing the question and selected source excerpts
Generate	Assembled prompt	An LLM response
Present with citations	Response and source information from the retrieved chunks	An answer whose source references can be checked

For vector retrieval to be meaningful, the question and indexed chunks need compatible embeddings—in a standard setup, they are made with the same embedding configuration. The query vector is used to find candidate passages. The LLM needs the retrieved passage text, not merely the query vector or a list of similarity scores.

Suppose retrieval finds two illustrative chunks: the suffix declaration in src/loader.py and the refresh instruction in docs/ingestion.md. Context assembly supplies the LLM with the original question and those excerpts. A grounded answer could then state that the shown loader accepts .md and .py files and that the documentation instructs users to run python ingest.py after an edit, with references to the respective source lines. As in the previous lesson, those filenames and lines are examples, not references to your repository.

Follow the question through the two-lane image once more. In its lower lane, the query participates in two places: it is represented for retrieval, and the original question is supplied to the LLM alongside the retrieved chunks. This matters. Similarity search helps select evidence, but the generator still needs to know what the user actually asked.

The image stops at “Answer.” Your project’s desired output goes further: it needs source citations. The indexing lane must preserve source locations, retrieval must carry them along with the chunks, and answer handling must attach references that actually support the claims. The LLM producing a plausible sentence is not, by itself, proof that the sentence is supported.

---

Where the lanes meet—and where failures start

The two lanes meet at the index, not inside the LLM. Indexing writes searchable records. Answering reads relevant records and places their text in a prompt. This boundary helps you diagnose a bad answer without treating “the AI” as one opaque component.

If an edited Markdown instruction is absent from the index, even excellent retrieval cannot find the new wording. If the correct Python chunk is indexed but retrieval selects unrelated chunks, the problem is in the answering lane before generation. If the correct excerpts reach the prompt but the answer makes an unsupported claim, the problem is in how the evidence is used or presented. We will learn to inspect these failures in detail later; for now, the two-lane diagram tells you where to look first.

It also prevents three common misunderstandings:

• “We embed the entire repository when each question arrives.” In this design, repository chunks are embedded during indexing. Each new question receives its own query representation for search.
• “The vector database answers the question.” It returns candidate source material. The generator composes the answer from the supplied question and context.
• “The LLM knows our latest files.” It sees the excerpts the application supplies for this request. Those excerpts reflect recent changes only if indexing has kept up.

A concise explanation for your presentation would be:

“We first load our Markdown and Python files, split them into source-linked chunks, embed those chunks, and store searchable records. When a user asks a question, we search that existing index, supply the retrieved text and the question to the LLM, and return an answer with references to the source chunks.”

That description deliberately names both moments of work. It also makes a testable claim about citations: source information must survive the entire journey.

---

Key takeaways

A RAG system has an offline indexing lane that prepares source files as searchable, source-linked chunks and an online answering lane that retrieves selected chunks for a particular question. The index connects them. Retrieval finds evidence; context assembly gives that evidence and the question to the LLM; citation handling makes the resulting claims checkable.

Next, we will follow one Markdown or Python file more closely through those lanes—from its source text to a retrieved excerpt, an answer, and a citation.