Skip to main content

Welcome to a Grasp lesson

Create a free account to keep studying and create your own study path.

Lesson 34

Chaining CTEs for Complex Data Transformations

Recommended resources

Hello! Welcome to the next lesson in our journey with SQL.

In our last lesson, you learned how to rewrite a query with a complex subquery into a more readable format using a single Common Table Expression (CTE). We saw that by naming a logical step and defining it at the top of the query, we make our code's intent much clearer.

Today, we'll build directly on that foundation. The learning outcome for this lesson is to chain multiple CTEs to break down a multi-step data transformation. This technique is the key to tackling genuinely complex analytical problems in a clean, logical, and maintainable way. For someone with your interest in systems programming and computational thinking, you can think of this as building a small data processing pipeline directly within a single SQL query, where the output of one step becomes the input for the next.

From One CTE to a Chain

In the real world, many data questions can't be answered in a single step. You often need to:

  1. Filter data to an initial set.
  2. Aggregate or enrich that set.
  3. Perform a final calculation on the aggregated results.

Chaining CTEs allows your SQL code to mirror this logical, step-by-step process. Instead of creating a messy "subquery within a subquery," you define a sequence of named, temporary tables.

Cascading CTEs vs. Nested Subqueries Comparison
This image provides a great mental model. Chaining CTEs is like assembling puzzle pieces—each piece is a clear, logical step (`active_customers`, `latest_orders`) that connects to build a larger picture. In contrast, nested subqueries are like a messy stack of papers, making it hard to see the logic and debug issues.

The syntax is straightforward. You use the WITH keyword only once, and then separate each subsequent CTE definition with a comma.

WITH
  cte_1 AS (
    -- First logical step
    SELECT ... FROM some_table
  ), -- Note the comma
  cte_2 AS (
    -- Second logical step, which can use the result of cte_1
    SELECT ... FROM cte_1
  )
-- Final query that can use either cte_1 or cte_2
SELECT ... FROM cte_2;

Building a Multi-Step Query

Let's see this in action. A common task is to analyze data from different perspectives and then combine those perspectives. For example, we might want to find all "chicken" items on a menu and also find the most expensive item in each menu category, and then see how those two sets of information relate.

Watch this video to see how two CTEs can be defined and then joined together in the final query.

SQL WITH Clause | Clearly Explained | CTEs vs Subqueries vs Temp Tables | Recursive CTEs

This video from Maven Analytics demonstrates the syntax for defining multiple CTEs and then using them together.

Watch the segment from 05:35 to 08:25. Pay close attention to the syntax: the WITH keyword is used only once, and a comma separates the first CTE (MP) from the second (CI). The final SELECT statement then treats MP and CI just like regular tables, joining them to produce the final result.


Chaining CTEs Sequentially

The previous example showed two largely independent CTEs. The real power of chaining comes when one CTE directly processes the output of the one before it. This creates a true logical sequence.

A classic example of this is calculating an "average of averages"—for instance, finding the average salary for each department, and then calculating the average of those departmental averages. A simple AVG(salary) across the whole company won't give you the right answer if departments are of different sizes. You have to do it in two steps.

This next video provides a perfect walkthrough of this exact problem.

54 How to use multiple CTE in SQL Server

Now let's look at a true 'chain' where the second CTE directly uses the results of the first. This video from Learn SSIS walks through calculating an average of averages.

Watch from 02:24 to 07:19. First CTE (02:24-03:54): Observe how the Department_average CTE is created to get the average salary per department. Chaining the Second CTE (03:54-05:26): Notice how a comma is used, and the second CTE, overall_average, is defined. Crucially, its FROM clause refers to the first CTE (Department_average). This is the chain. Final Result (05:26-07:19): The video explains why this multi-step process gives a different, more logically correct answer than just averaging all salaries directly. This highlights the importance of breaking down the logic.

Test your understanding!

Imagine you have two tables: customers (customer_id, name) and orders (order_id, customer_id, order_amount).

Your task is to find the average order amount for only your "VIP" customers, where a VIP customer is defined as someone whose total spending is over $1000.

How would you structure this using two chained CTEs? Describe the purpose of each CTE and the final SELECT statement.

Show answer

You would use a two-step approach:

  1. CTE 1: vip_customers

    • This CTE's purpose is to identify the VIP customers.
    • It would GROUP BY customer_id on the orders table and use a HAVING SUM(order_amount) > 1000 clause to filter for only those customers who meet the VIP criteria. The output would be a list of customer_ids.
  2. CTE 2: vip_orders

    • This CTE's purpose is to gather all orders belonging to the VIP customers.
    • It would JOIN the orders table with the vip_customers CTE on customer_id. The output would be the full order details for only VIPs.
  3. Final SELECT Statement

    • The final query would calculate the average of order_amount from the vip_orders CTE.

Here is the code:

WITH vip_customers AS (
    -- Step 1: Find customer IDs who have spent more than $1000
    SELECT customer_id
    FROM orders
    GROUP BY customer_id
    HAVING SUM(order_amount) > 1000
),
vip_orders AS (
    -- Step 2: Get all orders placed by those VIP customers
    SELECT o.order_amount
    FROM orders AS o
    JOIN vip_customers AS v
      ON o.customer_id = v.customer_id
)
-- Final Step: Calculate the average order amount from the VIP orders
SELECT AVG(order_amount) AS avg_vip_order_amount
FROM vip_orders;

A Practical Data Pipeline Example

Let's look at one more example that resembles a real-world task you might get in a technical support or junior developer role: analyzing log data or user activity. Here, we'll see how a series of CTEs can be used to take raw data, parse it, create multiple aggregations, and then join those aggregations for a final report.

This article provides a step-by-step construction of a query with four chained CTEs.

Using Multiple Common Table Expressions In One SQL ...

This article from Bennadel.com walks through a complex analysis task—analyzing a list of email addresses—by breaking it down into four distinct logical steps, each implemented as a CTE.

Read through the article. Don't worry about the specific VALUES ROW() syntax, which is just a way to create sample data. Focus on the logical flow: emails CTE: Represents the raw data. deconstructed CTE: Processes emails to extract the domain. This is the first link in the chain. stats and aggregated CTEs: Both process deconstructed to create two different summaries (a count and a JSON list). Final SELECT: Joins the two summary CTEs to produce the final report. Notice how each CTE has a single, clear purpose. This modularity is exactly what makes complex queries manageable.

For a final, concise set of examples, you can also look over this next article. It reinforces the patterns we've discussed today with practical scenarios.

Mastering SQL for Data Engineers, Part 3: Common Table ...

This article provides more excellent, concise examples of chaining CTEs for tasks like finding active customers and then getting their latest order dates.

Quickly review 'Example 2: Chaining CTEs for Order Stats' and 'Example 6: Customer Activity Summary'. These are great patterns for multi-step analysis that are very common in professional settings.


Conclusion

You've now learned how to connect CTEs into a logical chain, a technique that dramatically expands the complexity of problems you can solve with clean, readable SQL. By breaking down a large problem into a sequence of smaller, verifiable steps, you make your code easier to write, debug, and share with your team.

Key Takeaways:

  • You can define multiple CTEs in a single query by using the WITH keyword once and separating each CTE with a comma.
  • CTEs can be "chained" by having a later CTE query the results of a preceding one.
  • This approach allows you to build multi-step data transformation pipelines within a single SQL statement, making your logic explicit and easy to follow from top to bottom.
  • This is a fundamental technique for writing professional, production-quality SQL.

Preview of the Next Lesson:
Now that you know how to build queries with both single and chained CTEs, our next lesson will be a practical workshop. We'll focus on the essential skill of refactoring: taking existing, messy queries that use nested subqueries (the kind you might find in a legacy codebase) and transforming them into clean, maintainable CTE-based structures. This is a critical, job-ready skill that demonstrates your ability to not only write good code but also to improve existing code.