Feature Engineering from Existing Data

1. The Landscape of Feature Construction

Feature construction, also called feature generation, is about making the data we already have more useful. Instead of just cleaning or scaling features, we're adding new columns to our dataset that explicitly represent patterns or relationships that a model might otherwise struggle to find on its own.

This can involve anything from simple arithmetic to complex text processing. The image below provides a great high-level map of the different kinds of features we can create.

Let's begin with a video that introduces the concept and provides some foundational examples.

2. Creating Features from Numerical and Domain-Specific Data

One of the most straightforward ways to construct features is by combining or transforming existing numerical and categorical columns. Your intuition about the data and the problem domain plays a huge role here.

Common techniques include:

• Interaction Features: Creating new features by performing mathematical operations between two or more columns (e.g., sum, ratio, product). For example, in a loan application dataset, a debt_to_income_ratio is far more predictive than just debt and income alone.
• Binning (Discretization): Grouping a continuous numerical feature into a set of discrete categories or "bins." For example, converting a specific age into an age_group like 'Child', 'Adult', or 'Senior'.
• Domain-Specific Extraction: Using your knowledge of how the data is structured to pull out meaningful substrings or codes. For instance, if a product ID FD-101 always starts with a two-letter category code, you can extract that code (FD) into a new 'product_category' feature.

The following video provides concrete examples for all these techniques.

3. Harnessing the Power of Time: Datetime Features

Raw date and time columns (e.g., 2023-10-26 14:35:01) are often not directly useful to machine learning models. However, they contain a wealth of information that can be extracted to capture trends, seasonality, and cyclical patterns.

Common datetime features include:

• Time Components: Year, month, day, day of the week, hour, minute, quarter, week of the year.
• Cyclical Features: Often, features like 'day of the week' or 'month' are cyclical (e.g., Sunday is next to Monday; December is next to January). We can represent this relationship using sine and cosine transformations, a topic we'll explore in the next lesson.
• Boolean Flags: is_weekend, is_holiday, is_month_start. These can capture specific behavioral patterns.
• Time Differences: Calculating elapsed time between events, such as a customer's age_at_purchase or days_since_last_login.

The next video segment provides a comprehensive walkthrough of datetime feature engineering in Pandas.

For a quick reference, the article "7 Pandas Tricks for Time-Series Feature Engineering" is a great resource. Specifically, sections 1. Extracting Datetime Components and 5. Measuring Time Between Events provide concise code snippets for the techniques shown in the video.

4. Extracting Signals from Unstructured Text

Your background in NLP will serve you well here. As you know, models require numerical input, so raw text must be converted into a feature vector. While complex methods like BERT embeddings (which we'll touch on later in the course) provide state-of-the-art performance, you can often get great results with simpler, more direct features.

Some simple but effective text features include:

• Text Statistics: Length of the text, word count, sentence count, average word length.
• Keyword Counts: Counting the occurrences of specific, domain-relevant words.
• TF-IDF (Term Frequency-Inverse Document Frequency): This is a classic technique that creates a numerical feature for each word in your vocabulary, weighted by how important that word is to a specific document relative to the entire corpus of documents.

Let's focus on TF-IDF as a foundational technique for text feature engineering.

5. Putting It All Together: A Practical Example

Let's consolidate these ideas with a Python script. Here, we'll take a small, synthetic dataset and apply several of the feature construction techniques we've discussed.

import pandas as pd
import numpy as np




# 1. Create a sample DataFrame
data = {
    'transaction_id': range(5),
    'product_sku': ['SHIRT-BL-M', 'PANT-BK-L', 'SHOE-WT-10', 'SHIRT-RD-M', 'ACC-SLV-OS'],
    'purchase_timestamp': pd.to_datetime(['2023-01-14 10:00:00', '2023-01-14 19:30:00', 
                                      '2023-01-15 12:00:00', '2023-01-16 09:15:00', '2023-01-16 15:45:00']),
    'unit_price': [25.00, 55.00, 80.00, 28.00, 15.00],
    'quantity': [2, 1, 1, 3, 5]
}
df = pd.DataFrame(data)
print("--- Original Data ---")
print(df)




# 2. Construct new features
# Numerical interaction feature
df['total_sale'] = df['unit_price'] * df['quantity']




# Datetime features
df['purchase_day_of_week'] = df['purchase_timestamp'].dt.day_name()
df['purchase_hour'] = df['purchase_timestamp'].dt.hour
df['is_weekend'] = df['purchase_timestamp'].dt.weekday.isin([5, 6])




# Text/Domain-specific features from 'product_sku'
# You can use .str.split() for this
df['product_category'] = df['product_sku'].str.split('-').str[0]
df['product_color'] = df['product_sku'].str.split('-').str[1]

print("\n--- Data with Constructed Features ---")
print(df[['transaction_id', 'total_sale', 'purchase_day_of_week', 'purchase_hour', 'is_weekend', 'product_category', 'product_color']])

In this example, we created:

• A numerical feature (total_sale) by multiplying two existing columns.
• Several datetime features to capture when a purchase was made.
• Two categorical features by parsing a structured string (product_sku), demonstrating the use of domain knowledge.

This is the essence of practical feature construction.

Conclusion

In this lesson, you moved beyond preparing data to actively creating it. Feature construction is where data science becomes a creative process, allowing you to inject domain knowledge and uncover hidden signals for your model.

Key Takeaways:

• Feature construction is the process of creating new features from existing ones to improve model performance.
• Numerical features can be combined through mathematical operations (ratios, sums) or binned into categories.
• Datetime features are extremely valuable for capturing seasonality, trends, and cycles. Use the .dt accessor in Pandas to extract components like day of the week, hour, and month.
• Text features can range from simple statistics (like word count) to sophisticated representations like TF-IDF, which weights words by their importance.
• Domain knowledge is your most powerful tool. Understanding your data's structure and meaning will guide you toward creating the most impactful features.

In our next lesson, we will dive deeper into a specific and powerful type of feature construction: creating polynomial features and interaction terms. This automated technique is excellent for capturing non-linear relationships in your data.