Introduction
Welcome to the next module in your learning journey! In the previous module, we explored how models like CodeBERT can "understand" code by converting it into numerical embeddings. We then used these embeddings to train traditional machine learning models for tasks like classifying buggy code. That approach is powerful but often requires a labeled dataset and a dedicated training phase (either training a classifier from scratch or fine-tuning the embedding model).
Today, we shift to a different, more flexible paradigm: Large Language Models (LLMs). These models are so large and have been trained on such vast amounts of text and code that they can perform a wide variety of tasks without any specific training, simply by following instructions. This ability is called in-context learning.
This lesson directly addresses the learning outcome: Describe zero-shot, one-shot, and few-shot prompting techniques for large language models. We will explore the fundamental methods for "prompting"—or instructing—an LLM. Mastering these techniques is the first step toward building powerful AI-driven QA tools, such as the automatic test case generator you're interested in.
The Art of the Prompt
Interacting with an LLM is less like training a model and more like giving instructions to a very knowledgeable, very literal assistant. The instruction you provide is called a prompt. Prompt engineering is the practice of designing effective prompts to get the desired output.
A well-structured prompt typically contains a few key elements:
- Instruction: A clear command telling the model what to do (e.g., "Summarize this text," "Write a unit test").
- Context: Background information the model might need to complete the task.
- Examples: Demonstrations of the task, showing the model the expected input/output format.
The way you combine these elements, particularly the number of examples you provide, defines the prompting technique. Let's dive into the main categories.
Prompt Engineering 101: Understanding Zero-Shot, One-Shot, and ...
To start, let's get a formal introduction to these concepts from the article 'Prompt Engineering 101' by Codecademy. It clearly lays out the building blocks of prompts and the different prompting techniques.
Please read the introduction and the sections 'What is zero-shot prompting?', 'What is one-shot prompting?', and 'What is few-shot prompting?'. Focus on how each technique is defined by the number of examples it includes. The examples in the article are for sentiment analysis, which is a good starting point for understanding the general principle.
Zero-Shot Prompting: Just Ask
As you just read, zero-shot prompting is the simplest technique. You give the model an instruction without providing any examples of how to do it. You rely entirely on the knowledge the model gained during its initial training.
Structure:
- Instruction + Your Input
When to use it: For straightforward tasks that are common knowledge or were likely well-represented in the model's training data.
Example in a QA Context:
Imagine you want a basic Python function for a test setup.
Prompt:
Write a Python function that creates a temporary file, writes a string to it, and returns the file path.
The LLM can likely generate this code perfectly without any examples because this is a standard programming task.
However, zero-shot prompting can fail if the instruction is ambiguous. As the Codecademy article showed with sentiment analysis, simply saying "Classify the following sentences" is too vague. You need to be specific.
Poor Prompt:Review this function.
Better Zero-Shot Prompt:You are a senior software quality engineer. Review the following Python function for potential race conditions and suggest improvements.
Adding a persona ("You are a senior software quality engineer") and specifying the task ("review for potential race conditions") makes the zero-shot prompt much more effective.
One-Shot Prompting: Show One, Do One
When your task is a bit more specialized or you want the output in a specific format, providing a single example can work wonders. This is one-shot prompting.
Structure:
- Instruction + Example Input/Output + Your Input
When to use it: When you need to guide the model on the desired style, format, or a slightly novel task.
Example in a QA Context:
Suppose you're migrating tests from JUnit (Java) to pytest (Python) and want the LLM to help.
Prompt:
Translate the following unit test from JUnit to pytest.
Example:
--Java--
import static org.junit.Assert.assertEquals;
import org.junit.Test;
public class StringUtilsTest {
@Test
public void testReverse() {
assertEquals("olleh", new StringUtils().reverse("hello"));
}
}
--Python--
import pytest
def test_reverse():
assert string_utils.reverse("hello") == "olleh"
---
Translate this:
--Java--
import static org.junit.Assert.assertTrue;
import org.junit.Test;
public class CalculatorTest {
@Test
public void testIsPositive() {
assertTrue(new Calculator().isPositive(10));
}
}
--Python--
The single example clarifies the expected syntax (e.g., assert statements) and structure, guiding the model to produce a much more reliable translation.
Few-Shot Prompting: Learning Patterns on the Fly
For more complex or novel tasks, the model may need to see several examples to understand the underlying pattern. This is few-shot prompting. You provide a "few" examples (typically 2-5) to demonstrate the task. This is the most powerful form of in-context learning.
Structure:
- Instruction + Example 1 + Example 2 + ... + Your Input
When to use it: For complex tasks, domain-specific problems, or when you need a very specific, structured output (like JSON).
Example in a QA Context:
This is directly applicable to one of your project ideas: generating test cases. Let's say you want to generate a unit test given a function signature and a docstring.
Prompt:
Generate a simple pytest unit test for the given Python function.
--Function--
def add(a: int, b: int) -> int:
"""Returns the sum of two integers."""
--Test--
def test_add():
assert add(2, 3) == 5
assert add(-1, 1) == 0
---
--Function--
def is_even(n: int) -> bool:
"""Returns True if the number is even, False otherwise."""
--Test--
def test_is_even():
assert is_even(2) is True
assert is_even(3) is False
assert is_even(0) is True
---
--Function--
def get_first_char(s: str) -> str:
"""Returns the first character of a string."""
--Test--
By providing two distinct examples, the model learns the pattern: create a function named test_*, use assert statements, and cover a couple of basic cases. This enables it to generate a relevant test for the get_first_char function.
At a Glance: Comparing the Techniques
These three techniques form a spectrum, trading simplicity for power.
The following table, adapted from the article "Few-Shot Prompting" on DigitalOcean, provides a clear, side-by-side comparison.
| Prompting Type | Definition | Example | When to Use |
|---|---|---|---|
| Zero-Shot Prompting | This method involves giving the model direct instruction without providing specific examples for the task. | "Translate this sentence from French to English: ‘Bonjour le monde’." | Best for quick, common tasks where you’re confident the model knows how to handle them. |
| One-Shot Prompting | The model is presented with one input-output example before it tackles the actual request. | "Translate: ‘Salut’ → ‘Hello’. Now translate: ‘Bonjour’ → ?" | Ideal when the model could benefit from a specific format or context, but the task is fairly simple. |
| Few-Shot Prompting | The model is shown a few examples of input-output pairs to learn the pattern from the context. | You provide a handful of short translations, and then ask the model to translate a new sentence. | Useful for more complex or novel tasks where the model benefits from multiple demonstrations to infer the pattern. |
Source: Adapted from DigitalOcean, "Few-Shot Prompting: Techniques, Examples, and Best Practices".
A key mechanism that makes one-shot and few-shot prompting work is what's known as in-context learning. The model isn't being retrained or updated. It simply uses the examples within the context of the single prompt to figure out the pattern and apply it to your new input.
Conclusion
In this lesson, you've learned the fundamental techniques for communicating with Large Language Models. These methods are the building blocks for nearly all applications of modern LLMs.
Key Takeaways:
- Prompting is how we instruct LLMs to perform tasks.
- Zero-shot prompting relies on the model's pre-existing knowledge and works for simple, well-defined tasks. The key is to be specific in your instructions.
- One-shot prompting provides a single example to guide the model's output format and style.
- Few-shot prompting provides multiple examples, allowing the model to learn complex patterns "on the fly" through in-context learning. This is a powerful technique for domain-specific tasks like test case generation.
Preview of the Next Lesson:
Theory is great, but your goal is to build. In the next lesson, "Call an LLM API to generate unit tests from function signatures and docstrings," you'll move from theory to practice. You will write Python code to send these prompts to an actual LLM and get a response, taking your first concrete step towards building an automated test generation tool.