Skip to main content
Back to path

Welcome to a Grasp lesson

Create a free account to keep studying and create your own study path.

Lesson 69

Benchmarking Fine-Tuned Models

Recommended resources

Introduction

Welcome back! In our last lesson, we explored the crucial conceptual difference between Supervised Fine-Tuning (SFT) and alignment techniques like RLHF and DPO. You learned that SFT teaches a model to imitate a desired style, while alignment teaches it to prefer certain types of responses over others, making it more helpful and safe.

This raises a critical question: after investing significant effort in fine-tuning or aligning a model, how do we know if it's actually better? Is it just a "feeling," or can we measure the improvement rigorously?

This lesson addresses that question directly. Our objective is to evaluate a fine-tuned model against its base model using both automated benchmarks and custom evaluation sets. Mastering this skill is non-negotiable for building production-grade LLM systems, as it allows you to quantify model improvements, prevent performance regressions, and make data-driven decisions about which model to deploy.

The Core Principle: Relative Evaluation

The single most important principle in model evaluation is that performance is relative. A score in isolation means very little. The real goal is to measure the delta—the change in performance—between your fine-tuned model and its original base model.

Think of this like a scientific experiment:

  • The base model is your "control" group.
  • The fine-tuned model is your "treatment" group.

Your evaluation should aim to answer two fundamental questions:

  1. Capability Gain: Did the model improve on the target task(s) we fine-tuned it for?
  2. Capability Regression (Catastrophic Forgetting): Did the model's performance degrade on general tasks it was previously good at?

A successful fine-tuning process yields a model that shows significant gains on the target task without substantial regressions elsewhere. Let's look at a visual example of what a successful "capability gain" looks like.

Base Model vs. Fine-tuned Model Response Comparison
This image shows the response of a base GPT-3 model versus a fine-tuned version to the prompt, "Tell me how to fine-tune a model." The base model provides a list of related questions, failing the instruction. The fine-tuned model provides a clear, structured, and helpful guide, demonstrating a clear capability gain on its target task.

To measure these changes, we use two primary methods: standardized benchmarks and custom evaluations.

1. Automated Benchmarks: Standardized Capability Testing

Automated benchmarks are standardized tests designed to measure a model's core capabilities across a range of general knowledge and reasoning domains. Think of them as the SATs or GREs for LLMs. They provide a consistent, reproducible way to compare your model against its base version and the broader landscape of other models.

Common benchmarks include:

  • MMLU (Massive Multitask Language Understanding): Tests knowledge across 57 subjects like history, math, and law.
  • GSM8K (Grade School Math): Measures multi-step mathematical reasoning.
  • HumanEval: Assesses the ability to generate functionally correct Python code.
  • ARC (AI2 Reasoning Challenge): Focuses on science questions.

Running Automated Benchmarks

To learn what these benchmarks tell us and how to run them, let's turn to a practical video guide. It demonstrates how to use the popular lm-evaluation-harness library, a standard tool for this task.

What Do LLM Benchmarks Actually Tell Us? (+ How to Run Your Own)

The video 'What Do LLM Benchmarks Actually Tell Us?' by Adam Lucek provides an excellent overview of what benchmarks are, how to interpret their results, and a step-by-step guide to running one.

Please watch from the beginning to 24:45. Pay close attention to: What benchmarks measure (0:00 - 9:08): Understand the purpose of benchmarks like MMLU and the concept of few-shot vs. zero-shot evaluation. How to compare models (9:08 - 12:48): Notice how the presenter compares his fine-tuned models against each other to quantify improvement. Practical implementation (14:33 - 24:45): Follow the walkthrough of setting up and running the ARC benchmark. This shows how you'd practically compare your fine-tuned model against the public base model results.

Strengths and Limitations

  • Strengths: Benchmarks are crucial for measuring general competence and detecting major regressions. A significant drop in a score like MMLU after fine-tuning could indicate you've damaged the model's core knowledge.
  • Limitations: High scores on academic benchmarks do not guarantee strong performance on your specific, real-world application. This is a classic example of Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure."

This is why custom evaluations are indispensable.

2. Custom Evaluations: Testing What Actually Matters

While benchmarks act as general "unit tests," custom evaluations are your specific "integration tests." They are designed by you to measure performance on the exact tasks your model will face in production.

For an elite engineer, designing and implementing a robust custom evaluation suite is a defining skill. It ensures you are optimizing for true business value, not just abstract benchmark scores.

Hugging Face LLM Course: Custom Evaluation

Let's read the 'Custom Evaluation' section from the Hugging Face LLM Course. It outlines a strategic approach to developing a tailored evaluation plan.

Read the section titled 'Custom Evaluation.' Focus on the four-step strategy for developing a comprehensive approach that starts with standard benchmarks but moves to custom datasets that reflect your actual use case.

As the article highlights, creating a custom evaluation involves two main steps: building a dataset and then scoring the model's outputs.

A. Building Your Custom Evaluation Set

Your custom evaluation set should be a curated collection of prompts that represent the core challenges of your application. Sources include:

  • Real user queries from logs or beta testing.
  • Hand-crafted prompts that target known edge cases or failure modes.
  • "Golden set" examples that represent ideal interactions.

B. Scoring Custom Outputs: The "LLM-as-a-Judge" Pattern

For generative tasks, simple string-matching metrics like BLEU or ROUGE are often misleading. They penalize responses that are semantically correct but stylistically different from a single reference answer.

While human evaluation is the gold standard, it's slow and expensive. A powerful and scalable alternative is the LLM-as-a-Judge pattern. The idea is to use a highly capable model (e.g., GPT-4, Claude 3 Opus) to evaluate the outputs of your fine-tuned and base models against a predefined rubric.

This image neatly illustrates how custom, rubric-based evaluation fits into the broader landscape.

Comparison of LLM Evaluation Approaches
This diagram contrasts conventional benchmarks with coarse-grained and fine-grained evaluation. Our focus on custom evaluation with a user-defined rubric (like Relevance, Coherence) is a form of fine-grained evaluation, allowing for nuanced, task-specific assessment.

Let's explore a practical workflow for implementing this pattern.

Best Practices When Evaluating Fine-Tuned LLMs

The article 'Best practices when evaluating fine-tuned LLMs' provides a fantastic, code-backed example of building an LLM-as-a-judge workflow. It clearly explains what to measure and how to implement it.

Read the following sections: 'What is LLM evaluation?': Pay attention to the distinction between LLM evaluation and RAG evaluation. 'Evaluation Techniques': Note why quantitative metrics like BLEU/ROUGE are not suitable for our use case. 'How we evaluate our LLM-Twin Model': This is the core section. Study the evaluation prompt template and the definitions of Relevance, Cohesiveness, and Conciseness. This provides a concrete rubric for our judge.

The workflow is as follows:

  1. Define a Rubric: Choose qualitative criteria that matter for your task. The article's examples—Relevance, Coherence, and Conciseness—are an excellent starting point.
  2. Create an Evaluation Prompt: Craft a detailed prompt that instructs the "judge" model to score an output based on your rubric and provide a justification.
  3. Generate Responses: For each prompt in your custom evaluation set, generate a response from both the base model and your fine-tuned model.
  4. Execute Judgment: Send each (prompt, generated_response) pair to the judge LLM.
  5. Compare Scores: Aggregate the scores for both models. You can now quantitatively compare them on the criteria that you defined as important. For example, you might find your fine-tuned model's average Relevance score is 4.5/5, while the base model's is 3.2/5—a clear, measurable improvement.

Best Practices and Biases for LLM-as-a-Judge

Using an LLM as a judge is powerful but not foolproof. They are susceptible to biases. As a top-tier engineer, you must be aware of these and mitigate them.

Stanford CME295 Transformers & LLMs | Autumn 2025 | Lecture 8 - LLM Evaluation

The Stanford lecture on LLM Evaluation provides a rigorous overview of the biases inherent in the LLM-as-a-judge approach and best practices for mitigating them.

Watch from 38:31 to 52:38. This segment is dense but critical. Focus on understanding: Key Biases (38:31 - 46:54): Position bias, verbosity bias, and self-enhancement bias. Learn the simple remedies for each. Best Practices (46:54 - 52:38): Absorb the advice on crisp guidelines, binary scales, rationale before score, and using a low temperature for reproducibility.

Key mitigation strategies include swapping the order of responses to check for position bias and using a stronger, third-party model as the judge to avoid self-enhancement bias.

Test your understanding!

You've fine-tuned Mistral-7B on your company's internal documentation to create a helpful Q&A bot. You need to design an evaluation to prove its value to stakeholders before deployment.

Describe a two-part evaluation plan:

  1. Which standard benchmark would you run and why is it important in this context?
  2. Design a custom evaluation using the LLM-as-a-judge pattern. Specify:
    • One example prompt for your custom evaluation set.
    • The three criteria you would use in your rubric for the judge.
    • Which model you would use as the judge and why.
Show answer

Here is a robust evaluation plan:

  1. Standard Benchmark: I would run the MMLU (Massive Multitask Language Understanding) benchmark. While it doesn't test the specific domain of my company's documentation, it's crucial for detecting catastrophic forgetting. I need to ensure that in teaching the model about our internal knowledge, I haven't significantly damaged its general reasoning and world knowledge. A large drop in the MMLU score from the base Mistral-7B would be a major red flag.

  2. Custom Evaluation (LLM-as-a-Judge):

    • Example Prompt: "What is the process for requesting a new software license, and what is the expected SLA?" (This tests retrieval of specific, factual information from the fine-tuning data).
    • Rubric Criteria:
      • Factual Accuracy: Does the response correctly state the process and SLA as per the internal documentation? (Score 1-5)
      • Completeness: Does the response include all necessary steps and details, or does it omit critical information? (Score 1-5)
      • Actionability: Is the response presented clearly and simply, making it easy for an employee to follow the instructions? (Score 1-5)
    • Judge Model: I would use GPT-4o or Claude 3 Opus. These are powerful, state-of-the-art models from different organizations than the model being tested (Mistral). Using a stronger, external model as the judge minimizes "self-enhancement bias" and provides a more objective assessment of quality.

Conclusion

You are now equipped with a comprehensive framework for evaluating fine-tuned models. You know that evaluation is not an afterthought but a critical, integrated part of the model development lifecycle.

Key Takeaways:

  • Evaluation is relative: Always compare your fine-tuned model against its base model to measure the true impact.
  • Use a two-pronged approach:
    1. Automated Benchmarks (e.g., MMLU, HumanEval) serve as standardized "unit tests" to check for broad capability gains and regressions.
    2. Custom Evaluation Sets act as targeted "integration tests" to measure performance on what truly matters for your specific application.
  • LLM-as-a-Judge is a powerful tool: It provides a scalable and nuanced way to score performance on custom tasks, but you must be aware of and mitigate its inherent biases (position, verbosity, etc.).

By combining these methods, you can move beyond subjective impressions and rigorously prove that your fine-tuned model is not just different, but demonstrably better.

What's Next?

So far, we've focused on improving a model by refining its internal knowledge and behaviors through fine-tuning. But what if the knowledge required is too vast, too new, or changes too frequently to be baked into the model's weights?

In our next lesson, we will dive into Retrieval-Augmented Generation (RAG), a powerful architecture that connects LLMs to external, real-time knowledge sources, mitigating hallucinations and overcoming knowledge cutoffs.