Structuring Data for Supervised Fine-Tuning

Hello! Welcome back to our journey into practical fine-tuning.

In the previous lesson, we explored how to fine-tune large language models efficiently using Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA. We learned that LoRA allows us to adapt massive models with minimal computational cost by training only a tiny fraction of new parameters.

Now that we have the right technique, we need the right "fuel" for the training process. This brings us to today's crucial topic. Our learning outcome is to format a dataset into a structured format suitable for supervised fine-tuning. We'll move from raw information to the precise, organized examples an LLM needs to learn a new skill.

The Need for Structure in Fine-Tuning Data

Unlike pre-training, where a model learns language patterns from a vast, unstructured corpus of text, supervised fine-tuning (SFT) is a much more deliberate process. The goal is to teach the model how to behave—how to follow instructions, adopt a certain persona, or respond in a specific style.

To do this, we can't just feed it raw documents. We need to provide clear, structured examples that explicitly show the model: "Given this prompt, produce this ideal response."

A great resource for understanding the different ways to structure this data is the following article from DigitalOcean.

As you read, you'll see two formats are dominant for modern instruction-following models:

1. Instruction-Style: This format is excellent for single-turn, task-oriented interactions. It typically consists of three parts:

   • instruction: The task you want the model to perform.
   • input (optional): Additional context for the task.
   • output: The desired response.

   The famous Stanford Alpaca dataset popularized this format. Here is a visual example of a single data point.

   
   This image shows a typical instruction-style data sample. The model is given an instruction and an (optional) input, and it is trained to generate the corresponding response.

2. Chat-Style: This format is designed for multi-turn conversational interactions. It uses a list of messages, each with a role (like system, user, or assistant) and content. This mirrors the back-and-forth of a real conversation and is ideal for building chatbots or assistants.

For the rest of this lesson, we'll focus on how to turn these structured JSON objects into the final format that the model actually sees during training.

From Structured Data to a Prompt Template

The raw JSON data isn't directly fed into the model. It first needs to be assembled into a single string using a prompt template. This template provides a consistent structure that helps the model distinguish between instructions, inputs, and the expected response.

A very common and influential example is the prompt template used for the Alpaca dataset.

The Alpaca prompt template, as shown in the video, looks like this:

Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.




### Instruction:
{instruction}




### Input:
{input}




### Response:
{output}

If the input field is empty, that part of the template is simply omitted. This simple string formatting is a powerful way to prepare data. You can write a Python function that takes a dictionary (one row of your dataset) and returns this formatted string.

A More Modern Approach: Chat Templates

While manually creating prompt strings works, the modern Hugging Face ecosystem provides a more standardized and robust method: chat templates.

Base models (like Llama 3 or Mistral) are not trained with a specific prompt format. However, their instruction-tuned variants (like Llama 3 Instruct) are. A chat template is a model-specific recipe stored with the tokenizer that defines how a list of messages should be formatted into a single string, complete with special tokens to denote the start and end of turns and roles.

Using a chat template ensures you're formatting the data in the exact way the model was trained to expect it, which is crucial for getting the best performance.

As the article explains, a chat-style dataset might be stored in a format like ShareGPT (a list of conversations), but the tokenizer.apply_chat_template() function handles the conversion to the final string format required by the model. This is the preferred method when working with models from the Hugging Face Hub.

Putting It Into Practice: Code Examples

Let's look at how you would programmatically format a dataset. The core task is to create a function that transforms each raw data sample into a single text string.

The DigitalOcean article you reviewed earlier contains excellent, self-contained Python examples for this.

The most common and efficient way to apply such a formatting function to an entire dataset is by using the .map() method from the Hugging Face datasets library, as shown in the mlabonne/FineTome-100k example.

# Example from the Hugging Face blog post ([LINK](https://huggingface.co/blog/mlabonne/sft-llama3))
def apply_template(examples):
    messages = examples["conversations"]



    # Here, tokenizer.apply_chat_template does the formatting work
    text = [tokenizer.apply_chat_template(message, tokenize=False, add_generation_prompt=False) for message in messages]
    return {"text": text}




# Apply the function to the entire dataset in batches
dataset = dataset.map(apply_template, batched=True)

Finally, it's worth noting a practical detail: the standard file format for storing these structured datasets is JSON Lines (JSONL), where each line in the file is a complete JSON object representing one training example. This format is easy to parse and is supported by virtually all fine-tuning scripts and libraries.

Conclusion

You've now learned the essential principles and practices for preparing a dataset for supervised fine-tuning. This data preparation step is just as important as the training algorithm itself—high-quality, consistently formatted data is the foundation of a successful fine-tuning project.

Key Takeaways:

• Fine-tuning requires structured data that pairs a prompt (instruction/question) with a desired response.
• The two main data structures are instruction-style (e.g., Alpaca format) for single-turn tasks and chat-style (with roles) for conversations.
• Prompt templates are used to convert this structured data into a single string that the model will process.
• Modern workflows use chat templates via tokenizer.apply_chat_template() to ensure the formatting matches what the model expects.
• You can apply formatting programmatically using a function and the .map() method from the datasets library.
• The standard file format for these datasets is JSONL.

We have successfully transformed our raw information into a clean, formatted dataset ready for the model. But before we can start training, we need to be able to measure how well the training is working.

In the next lesson, we will address this by learning how to create a training and validation split for a fine-tuning dataset. This will allow us to train the model on one part of the data and validate its performance on another, unseen part.