RAG vs. LLMs, Fine-Tuning, and Document Search

Welcome back. In the previous lesson, we saw how a RAG application can retrieve excerpts from Markdown and Python files, supply them to an LLM, and produce an answer whose claims can be checked against those excerpts. The important point was that retrieving a source is not the same as using it faithfully.

This lesson puts RAG alongside three approaches that can look similar from the outside: asking an LLM directly, fine-tuning a model, and searching documents. The distinction matters for your project because it explains why you are building a retrieval pipeline rather than simply writing a better prompt or training a model on the repository.

Four ways to respond to the same question

Imagine a fictional repository containing this line in src/loader.py:

SUPPORTED_SUFFIXES = {".md", ".py"}

Its docs/ingestion.md also says, “After changing a source file, run python ingest.py to refresh the searchable collection.” A colleague asks:

Which file types does this project ingest, and what should I do after changing one?

All four approaches below could be used to help the colleague, but they obtain and present information differently.

The dividing line is not whether an LLM is involved. Both standalone generation and RAG use one. It is whether the application fetches relevant external material for this question and gives that material to the model as evidence.

Watch IBM Technology’s comparison for a first pass at RAG, fine-tuning, and prompt engineering. We will then add ordinary document search, which is essential to understanding where RAG begins.

One qualification will matter throughout your project: the video illustrates RAG retrieval with vector embeddings. Vector search is common in RAG, but it is not part of the definition. A RAG application can also retrieve through keyword search or combine both methods. What makes it RAG is the use of retrieved material to augment generation.

The following image is a quick visual comparison of three improvement strategies. Read its descriptions as typical uses, not promises: fine-tuning does not automatically deliver high accuracy, and RAG does not make the underlying model inherently “smarter.”

Standalone generation versus RAG: does the model have the files?

If you ask a standalone LLM the repository question, it might reply, “The loader probably supports Markdown, Python, JSON, and PDF.” That sounds plausible for an ingestion tool, but nothing in the question tells the model which formats this loader accepts. Without access to the repository, it cannot verify the guess.

A more carefully worded prompt might say, “Do not guess; state when you do not know.” That is useful prompt engineering: changing instructions to guide behavior without changing model parameters. It may produce the appropriately cautious answer, “I cannot determine the supported file types without seeing the project.” But clearer instructions alone cannot reveal the contents of src/loader.py. You could paste the relevant code into the prompt yourself; then the model would have evidence, though the application would not yet be retrieving it automatically.

In RAG, the application looks up relevant project material when the question arrives. Suppose it supplies the suffix declaration and the sentence from docs/ingestion.md. The model now has grounds to say that the shown loader accepts .md and .py files, and that the documentation instructs users to run python ingest.py after a change. It has not learned these facts permanently. They are available in the context of this answer.

That difference gives RAG its main advantage for your project. Your Markdown and Python files are particular to your repository and may change during the three-week build. A model’s general training is not a reliable record of them. RAG can use a refreshed searchable collection without retraining the LLM—but “refreshed” is doing real work here. If the files change and the collection is not updated, retrieval may still return old content.

RAG also carries work that a standalone prompt does not: preparing the files for search, maintaining that collection, fetching appropriate excerpts for each question, and checking that the generated answer uses them correctly. It reduces reliance on unsupported guesses; it does not eliminate them.

Fine-tuning versus RAG: changing the model or supplying evidence

Fine-tuning starts with an existing model and performs additional training, changing its parameters. The training material often consists of examples that demonstrate a desired task or response pattern. For instance, examples might teach a model to consistently turn technical notes into concise release summaries.

RAG takes a different route. It leaves the generator’s parameters unchanged and supplies selected source content when the model answers a question. For the repository question, that might be the current suffix declaration and the current ingestion instructions.

This distinction is especially important when the desired answer depends on a precise, changing fact. You could train a model on examples involving SUPPORTED_SUFFIXES, but that would not make its parameters an inspectable copy of src/loader.py. If a future edit adds another suffix, the fine-tuned model does not automatically learn the edit. Nor does fine-tuning, by itself, provide a path and line range that supports a particular answer.

Fine-tuning can still be valuable. If a model repeatedly struggles with a specialized response format or task despite adequate instructions and evidence, additional training may help it perform that task more consistently. It can also be combined with RAG: a fine-tuned generator could answer using retrieved project excerpts. But training a model is not a substitute for looking up current source code when the question is, “What does the code say now?”

For your initial build, that suggests a practical priority rather than a universal rule: make the repository searchable and the answers checkable before considering model training. Fine-tuning introduces its own data preparation, evaluation, and maintenance burden. Your immediate requirement is to answer from Markdown and Python files.

Ordinary document search versus RAG: finding evidence or composing an answer

Document search already does an indispensable part of the job. Given “Which file types does this project ingest?”, a search tool might return src/loader.py with the SUPPORTED_SUFFIXES line highlighted. Given the second half of the question, it might return the relevant paragraph in docs/ingestion.md. The user reads those results and draws the conclusion.

RAG uses retrieval too, but goes one step further: it places selected results in the LLM’s context and asks for an answer based on them. In the fictional example, a well-grounded RAG response might read:

The shown loader accepts .md and .py files [src/loader.py, line 4]. After changing a source file, the documentation says to run python ingest.py to refresh the searchable collection [docs/ingestion.md, line 8].

Those citations are illustrative; they are not references to your actual repository. They show the additional obligation that comes with generation: the application should retain enough source information for a reader to verify each claim.

It would be a mistake to distinguish search from RAG by saying “search matches keywords; RAG understands meaning.” Search can be lexical, semantic, or both, and RAG can use any of those retrieval methods. Lexical search looks for terms such as the exact Python identifier SUPPORTED_SUFFIXES. Semantic search aims to find related content even when the question uses different wording, such as “Which extensions can the loader read?” Both can be useful in a code-and-documentation repository. We will study their trade-offs later.

The more durable distinction concerns the output:

• Search primarily gives the user source results to inspect. It may be the safest and clearest option when an exact line of code is all they need.
• RAG uses source results to produce a composed answer. This is helpful when the question spans multiple excerpts, but the composed answer can misread a result, omit a caveat, or attach an unhelpful citation.

Think about a question such as, “What formats are supported, and what does the documentation tell me to do after an edit?” Search may return two separate hits. RAG can connect them in one response, provided it retrieves both and does not claim more than either source establishes. If retrieval finds only the loader line, a trustworthy RAG answer should address the formats and acknowledge that its supplied evidence does not establish the post-edit procedure.

Search is therefore not a competitor that RAG replaces. It is a capability a RAG system depends on. The added generation step makes the result more convenient, while introducing a new responsibility to preserve the meaning of the sources.

Choosing and explaining the approach for this project

The choice becomes clearer if you ask what problem you are trying to solve.

For a general question that does not depend on your files—“What is a Python module?”—a standalone LLM may be sufficient. Better prompt instructions can help specify the depth or format of that answer.

For “Show me the declaration of SUPPORTED_SUFFIXES,” ordinary search may be preferable. The exact code is the result; a generated paraphrase could add little.

For “How do the loader’s accepted file types relate to the documented ingestion procedure?”, RAG has a clearer role. It can retrieve evidence from both Python and Markdown, bring the excerpts together, and answer with citations. That is the kind of source-spanning question your planned system is meant to handle.

If instead your enduring problem were that a model could see the right evidence but consistently failed at a specialized task or required output style, you might investigate fine-tuning. You would still need retrieval if responses must reflect current repository facts.

These approaches can coexist. Your RAG system will need good retrieval and a clear prompt telling the generator how to use retrieved excerpts. It does not need a fine-tuned model to qualify as RAG. Conversely, putting a vector database next to an LLM does not, on its own, make answers grounded: relevant source text must reach the generator, and its claims must remain supportable.

For a presentation, a concise distinction is:

“Search finds relevant passages; RAG uses retrieved passages to generate an answer. Unlike fine-tuning, it does not train the LLM on our repository. Unlike a standalone LLM, it can answer from selected current project files—provided we keep the searchable collection up to date and verify the answer’s citations.”

Key takeaways

Standalone generation relies on the question and the model’s training unless someone supplies additional content. Fine-tuning changes model parameters to improve performance on a task; it does not automatically look up the latest file. Document search finds source material for inspection. RAG builds on retrieval by giving selected material to an LLM so it can compose a source-grounded answer.

For your Markdown-and-Python project, search and generation are complementary, while source freshness and citation quality remain obligations of the surrounding application. Next, we will separate that application into its two major flows: preparing files for retrieval and answering a user’s question from the prepared collection.