In our previous lessons, we explored the "engines" of modern AI—foundation models and their most common variant, Large Language Models (LLMs). We established that these models are powerful but general-purpose. To make them excel at specific tasks, they need to be taught. This brings us to a fundamental concept that underpins all of machine learning: the data used for teaching.
This lesson will focus on defining training data and explaining its critical importance. For a product manager, understanding the role and quality of data is paramount, as it directly impacts the capabilities and reliability of any AI-powered feature you aim to develop. Think of training data as the curriculum and textbooks from which your AI model learns.
What is Training Data?
At its core, training data is simply the information—the set of examples—fed to a machine learning model to teach it how to perform a task. Instead of writing explicit, rule-based instructions (e.g., "if an email contains the word 'free money', mark it as spam"), we provide the model with a large number of examples and let it learn the underlying patterns itself.
The following article from IBM provides a concise and clear definition of this concept.
This article offers a foundational definition of training data and its role in machine learning.
Please read the first three paragraphs, starting from the heading "What is training data?". Focus on the core idea that algorithms are useless without data, much like a student who hasn't studied for a test.
The "Garbage In, Garbage Out" Principle
The importance of training data cannot be overstated. The performance, accuracy, and fairness of any machine learning model are directly dependent on the data it was trained on. This is often summarized by the phrase "garbage in, garbage out." If a model is trained on flawed, biased, or irrelevant data, it will produce flawed, biased, or irrelevant results.
There is no better illustration of this than a real-world example from Amazon. The following video begins by recounting a story of how a machine learning model for recruiting showed a bias against female candidates. This wasn't because the AI was intentionally "sexist," but because of a fundamental flaw in its training data.
How is data prepared for machine learning?
This video from AltexSoft uses a compelling real-world case to demonstrate the critical importance of data quality and relevance.
First, watch the introduction from the beginning to understand the Amazon recruiting tool story. Then, skip to the section that explains the "garbage in, garbage out" principle with a practical example about forecasting turkey sales, from the good old principle. This will highlight that data must not only be clean but also adequate for the specific problem you are trying to solve.
As you can see, the quality, quantity, and relevance of the data are crucial. Your background in process and data analysis is highly relevant here; just as you'd ensure data integrity for a business report, data scientists must rigorously prepare data for model training.
The Building Blocks: Features and Labels
So, what does this data actually look like? In many machine learning scenarios, particularly for tasks like classification or prediction, the training data consists of two key components:
- Features: These are the input variables or attributes that describe the data. In the Amazon example, features would be the words and phrases found in a candidate's resume (e.g., years of experience, skills listed, university attended).
- Labels: This is the "answer" or the target outcome you want the model to predict. In the Amazon case, the label would have been a score indicating whether the candidate was a good hire.
The model's job during training is to learn the relationship between the features and the label. The article below provides a very clear explanation using a house price prediction example.
Machine Learning for Beginners: A Simple Guide
This article clearly defines the concepts of features and labels, which are the fundamental components of a training dataset.
Please read the section under the heading "Features and Labels". This will solidify your understanding of these core terms.
For your goal of customizing a Bedrock model, this concept translates directly to the prompt-completion pairs we will discuss later. The "prompt" you provide is the feature, and the ideal "completion" you write is the label. The model learns to generate completions that match the style and content of your labeled examples.
Data Preparation: Splitting the Dataset
One final, critical concept is that you never use all of your data for training. A common pitfall in machine learning is overfitting, where a model essentially "memorizes" the training data but fails to perform well on new, unseen data. From your project management perspective, this is like building a product that works perfectly in a controlled test environment but fails in the hands of real users.
To prevent this, the dataset is typically split into at least two, and often three, parts:
- Training Set: The largest portion (e.g., 70-80%), used to actually train the model.
- Validation Set: A smaller portion used to tune the model's settings and check its performance during the training process. This helps ensure it's learning general patterns, not just memorizing.
- Test Set: A final, held-back portion that the model has never seen. It's used only at the very end to give an unbiased evaluation of the final model's performance.
The concept of splitting data is crucial for building reliable models, and the distinction between training and validation data will be very important when we get to the Bedrock fine-tuning workflow.
Conclusion
In this lesson, we have defined training data and explored its central role in machine learning. Without high-quality, relevant data, even the most sophisticated algorithm will fail.
Here are the key takeaways:
- Training data is the foundation: It's the collection of examples used to teach a machine learning model to recognize patterns and make predictions.
- Quality is paramount: The "garbage in, garbage out" principle dictates that a model's performance is a direct reflection of the data it was trained on. Biased or irrelevant data leads to biased or useless models.
- Data has structure: For many tasks, data is structured into features (inputs) and labels (desired outputs).
- Preparation is key: Raw data is rarely used as-is. It must be prepared, which includes splitting it into training, validation, and test sets to build a reliable and generalizable model.
Now that you understand what LLMs are and the data they learn from, we can connect these ideas. Our next lesson will introduce the concept of model 'fine-tuning', which is the specific process of taking a pre-trained foundation model and further training it with a smaller, specialized dataset to adapt it for your unique tasks.