Generating Sequences with `itertools`

1. Combinatorial Sequences: itertools.product

A frequent task is to generate all possible combinations of items from several different sets. This is known as the Cartesian product. For example, if you have a t-shirt available in sizes ['S', 'M', 'L'] and colors ['Red', 'Blue'], the Cartesian product gives you all six possible variations: ('S', 'Red'), ('S', 'Blue'), ('M', 'Red'), and so on.

Instead of writing complex nested loops, you can use itertools.product.

import itertools

sizes = ['S', 'M', 'L']
colors = ['Red', 'Blue']




# The "Java-ish" way with nested loops
variants_loop = []
for size in sizes:
    for color in colors:
        variants_loop.append((size, color))




# The Pythonic way with itertools.product
variants_itertools = list(itertools.product(sizes, colors))

print(variants_itertools)



# Output: [('S', 'Red'), ('S', 'Blue'), ('M', 'Red'), ('M', 'Blue'), ('L', 'Red'), ('L', 'Blue')]

assert variants_loop == variants_itertools

itertools.product is not only more concise but also returns a lazy iterator, making it highly memory-efficient for large inputs.

Combinatorial Cousins: permutations and combinations

itertools provides a suite of tools for combinatorial generation. While product is the most general, it's crucial to know its cousins and when to use them.

The following video from the mCoding channel provides a clear, visual distinction between product, permutations, and combinations.

Here's a quick summary to solidify the concepts:

For detailed definitions and examples, the official Python documentation is your definitive reference.

2. Windowed Sequences: itertools.pairwise

Often, you need to process an item and its immediate successor simultaneously. This is a "sliding window" of size two. Common use cases include calculating differences between consecutive values in a time series or comparing adjacent elements.

Python 3.10 introduced itertools.pairwise for exactly this purpose. It elegantly replaces clumsy indexing logic.

Bad (Manual Indexing):

data = [10, 15, 12, 18, 20]
for i in range(len(data) - 1):
    current, next_item = data[i], data[i+1]
    print(f"Comparing {current} and {next_item}")

Good (itertools.pairwise):

import itertools

data = [10, 15, 12, 18, 20]
for current, next_item in itertools.pairwise(data):
    print(f"Comparing {current} and {next_item}")

This image shows a practical example of using pairwise to calculate the differences between consecutive numbers.

The mCoding video also offers a quick, effective explanation.

Beyond Pairs: The General sliding_window Recipe

What if you need a window of size 3 or more? The itertools documentation provides a standard "recipe" for a general-purpose sliding_window. This is a fantastic example of the "batteries included" philosophy: even if a function isn't built-in, the module provides the blocks to build it cleanly.

Here is that recipe for your reference. Understanding it shows how core itertools functions can be composed.

from collections import deque
from itertools import islice

def sliding_window(iterable, n):



    # sliding_window('ABCDEFG', 4) -> ABCD BCDE CDEF DEFG
    it = iter(iterable)
    window = deque(islice(it, n), maxlen=n)
    if len(window) == n:
        yield tuple(window)
    for x in it:
        window.append(x)
        yield tuple(window)




# Example usage
data = 'ABCDEFG'
print(list(sliding_window(data, 4)))



# Output: [('A', 'B', 'C', 'D'), ('B', 'C', 'D', 'E'), ('C', 'D', 'E', 'F'), ('D', 'E', 'F', 'G')]

3. Chunked Sequences: itertools.batched

Another fundamental pattern is processing an iterable in fixed-size, non-overlapping chunks. This is essential for tasks like:

• Sending batch requests to a rate-limited API.
• Inserting records into a database in batches.
• Processing a large file chunk by chunk.

New in Python 3.12, itertools.batched provides a canonical, efficient way to do this.

Here is a simple example:

import itertools

data = range(10)  # 0, 1, 2, ..., 9



# Process in chunks of 3
for batch in itertools.batched(data, 3):
    print(batch)



# Output:
# (0, 1, 2)
# (3, 4, 5)
# (6, 7, 8)
# (9,)  <-- Note the last batch can be smaller

The following video provides a focused introduction to itertools.batched.

For a deeper dive into practical applications, the mathspp.com blog post is an excellent resource. It covers several real-world scenarios where batched is invaluable.

Exercise: Multi-faceted Data Processing

Let's apply these new tools. This exercise has three independent parts, each targeting one of the functions you've just learned.

import itertools




# --- Part 1: Generate Test Cases with `product` ---
print("--- Part 1: Test Case Generation ---")
user_roles = ["guest", "editor", "admin"]
actions = ["view", "edit", "comment", "delete"]




# TODO: Generate all possible (role, action) pairs to test permissions.
# Your goal is to produce a list of tuples like [('guest', 'view'), ('guest', 'edit'), ...]
all_permission_tests = list(itertools.product(user_roles, actions))




# Verification for Part 1
print(f"Generated {len(all_permission_tests)} permission test cases.")
assert len(all_permission_tests) == 12
assert ('admin', 'delete') in all_permission_tests
print("✅ Part 1 is correct!\n")





# --- Part 2: Batch API Requests with `batched` ---
print("--- Part 2: API Batching ---")



# A long list of item IDs to fetch from an API
item_ids = range(1, 26) # IDs from 1 to 25
BATCH_SIZE = 10




# TODO: Use itertools.batched to group the item_ids into chunks of BATCH_SIZE.
# This simulates preparing data for a batch API call.
batched_ids = list(itertools.batched(item_ids, BATCH_SIZE))




# Verification for Part 2
print(f"Item IDs chunked into {len(batched_ids)} batches:")
for i, batch in enumerate(batched_ids):
    print(f"  Batch {i+1}: {batch}")
assert len(batched_ids) == 3
assert batched_ids[0] == (1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
assert batched_ids[2] == (21, 22, 23, 24, 25)
print("✅ Part 2 is correct!\n")





# --- Part 3: Analyze Price Changes with `pairwise` ---
print("--- Part 3: Stock Price Analysis ---")



# Daily closing prices for a stock
prices = [100, 105, 102, 110, 108, 115, 112]
days = range(len(prices))




# TODO: Use itertools.pairwise to find the biggest single-day price increase.
# Create pairs of consecutive prices and find the maximum difference.
price_changes = [next_price - current_price for current_price, next_price in itertools.pairwise(prices)]
max_increase = max(price_changes)




# Verification for Part 3
print(f"Daily price changes: {price_changes}")
print(f"Maximum single-day increase: {max_increase}")



# The change from 102 to 110 is +8, the largest increase.
assert max_increase == 8
print("✅ Part 3 is correct!")

Conclusion

You have now added three more powerful tools from itertools to your arsenal, each replacing a complex looping pattern with a single, expressive function call.

Key Takeaways:

• itertools.product: Your go-to for generating combinatorial sequences (Cartesian products), perfect for creating test cases or exploring parameter spaces.
• itertools.pairwise: The idiomatic way to create a sliding window of size two, ideal for comparing adjacent elements in a sequence. For larger windows, you can use the sliding_window recipe from the official docs.
• itertools.batched: A modern and efficient tool (Python 3.12+) for creating non-overlapping chunks, essential for batch processing tasks like API calls and database operations.

In the next lesson, we will step back from processing iterables and focus on storing data. You will learn to master Python's core containers, choosing between list, tuple, set, and dict based on semantics and performance, ensuring you always use the right tool for the job.