Skip to main content
Back to path

Welcome to a Grasp lesson

Create a free account to keep studying and create your own study path.

Lesson 44

Creating Datasets with the Create Samples Tool

Recommended resources

Hello! Welcome to our next lesson on predictive analytics.

In our last session, we discussed the "golden rule" of feature scaling: you must always split your data into training and test sets before applying scaling transformations to prevent data leakage. This naturally leads to the question: how do we perform this split?

Today, we will answer that question directly. Your learning outcome is to use the Create Samples tool to create training, validation, and holdout datasets. This is a fundamental step in any predictive modeling project, and mastering it is essential for building reliable models and for your Alteryx Advanced Certification.

We will cover the "why" behind data partitioning—the distinct roles of training, validation, and holdout data—and then dive into the "how" by learning to configure the Create Samples tool in Alteryx.

1. The Three Datasets: Train, Validate, and Test

Before we build a predictive model, we must split our dataset into at least two, and often three, distinct subsets. Each subset serves a unique and critical purpose in the model development lifecycle.

Data Partitioning for Predictive Modeling
This chart illustrates the standard partition of a dataset. The **Training Data** is used to build the model, the **Validation Data** is used to tune and evaluate different model versions, and the **Holdout (or Test) Data** provides a final, unbiased assessment of the chosen model's performance.

Let's break down the role of each:

  • Training Set: This is typically the largest portion of your data (e.g., 60-80%). The model "learns" from this data by identifying patterns and relationships between your input features and the target variable. The model's internal parameters are fitted using only this dataset.

  • Validation Set: This portion (e.g., 10-20%) is used to tune the model. For example, you might try several versions of a model with different settings (called hyperparameters). You would test each version against the validation set to see which one performs best on data it wasn't trained on. This helps you select the best model configuration and prevent a problem called overfitting, where a model learns the training data too well, including its noise, and fails to generalize to new data.

  • Holdout (or Test) Set: This final piece of data (e.g., 10-20%) is the "final exam" for your chosen model. It has been kept completely separate throughout the training and validation process. You use it only once at the very end to get a final, unbiased estimate of how your model is likely to perform on new, unseen data.

This separation is the cornerstone of building models that are not just accurate on paper but are genuinely useful and reliable in the real world.

2. The Create Samples Tool

In Alteryx, the primary tool for partitioning your data is the Create Samples tool, found in the Preparation tool category. It allows you to randomly split your records into two or three streams based on the percentages you define.

Let's dive into the official documentation to understand how to configure it.

Create Samples Tool - Alteryx Help

The Alteryx Help page for the Create Samples tool provides a direct and clear explanation of its configuration and outputs. Let's review it.

Please read the entire page. It is very short. Pay close attention to: The definitions of Estimation sample percent and Validation sample percent. The purpose of the Random seed. The three output anchors: E, V, and H.

As the documentation explains, the tool is straightforward. Here are the key takeaways from the reading:

  • Outputs: The tool has three output anchors:

    • E for Estimation: This is your training dataset.
    • V for Validation: This is your validation dataset.
    • H for Holdout: This stream contains any remaining records, serving as your test dataset.

  • Configuration:

    • You set the percentage for the Estimation (Training) set and the Validation set.
    • The Holdout (Test) set is automatically created from whatever percentage is left over. For example, if you set Estimation to 70% and Validation to 20%, the Holdout set will automatically contain the remaining 10% of your data.
    • If you only need a two-way split (e.g., train and test), you can set Estimation to 80% and Validation to 20%. The Holdout anchor will simply be empty.

3. A Practical Example and The Importance of Reproducibility

Now, let's see how this tool fits into a larger predictive workflow.

Alteryx Designer: An Introduction to Predictive Analytics - Billigence

This article from Billigence walks through a basic predictive analysis. It provides a great real-world example of how the Create Samples tool is used after initial data cleansing.

In the article, scroll down to the section that begins "To wrap up the data preparation steps...". Read from there until you see the heading "Predictive Possibilities with Alteryx". You will see a demonstration of a 70-20-10 split using the Create Samples tool.

The example in the article implements a common 70% training, 20% validation, and 10% holdout split. This workflow perfectly illustrates where the Create Samples tool sits: right after general data cleaning and preparation, and just before you start feeding the data into any modeling tools.

The Random Seed and Reproducibility

One of the most important settings in the Create Samples tool is the Random seed. The tool randomly assigns rows to the different output samples. If you run the workflow again, you will get a different random sample.

This is often not desirable. For your results to be reproducible and comparable, you need the exact same split every time you run the workflow. By setting the Random seed to a specific integer (e.g., the default of 1), you ensure that the random sampling process starts from the same point each time, resulting in an identical split of your data.

Best Practice: Always set a specific random seed in the Create Samples tool to ensure that your workflow produces consistent and reproducible results. This is crucial for debugging, sharing your work, and comparing different modeling approaches fairly.

Conclusion

You have now learned how to perform one of the most fundamental tasks in the predictive modeling lifecycle. By correctly partitioning your data, you create the necessary foundation for training, validating, and honestly evaluating your models.

Key Takeaways:

  • Predictive modeling requires splitting data into Training, Validation, and Holdout (Test) sets.
  • The Training set is for teaching the model, the Validation set is for tuning it, and the Holdout set is for the final, unbiased performance evaluation.
  • In Alteryx, you use the Create Samples tool to perform this split.
  • The E, V, and H output anchors correspond to the Estimation (Training), Validation, and Holdout sets.
  • Setting a Random seed is critical for ensuring your data split is reproducible.

Preview of the Next Lesson:

We've covered the "what" and "how" of data partitioning. In our next lesson, we will delve deeper into the "why" by exploring the importance of data partitioning for unbiased model validation. We will formally define concepts like overfitting and see why this strict separation of data is the only way to build a model you can truly trust.