Adjusting for Simpson's Paradox

Welcome to the final lesson of our module on causal inference. In our previous sessions, we've built a powerful toolkit for thinking causally. We've used Directed Acyclic Graphs (DAGs) to map out relationships, distinguished between confounding and collider bias, and established the back-door criterion as our guide for identifying which variables to control. You now know that you must adjust for confounders to block spurious paths, and why you must never condition on colliders to avoid creating them.

Today, we bring all these concepts together to address the core learning outcome for this lesson: to solve a numerical exercise where you will apply a formal adjustment strategy to observational data exhibiting Simpson's paradox, thereby estimating the true causal effect. This is the practical culmination of our work, moving from identifying problems to actively solving them. We will take a classic case of confounding and, using a precise formula, reveal the underlying causal truth hidden by misleading aggregate statistics.

From Identification to Estimation: The Adjustment Formula

We've seen how Simpson's paradox arises from a confounding variable. A lingerie style might appear superior overall, only for the trend to reverse when we stratify by a factor like model body type. The causal DAG for this is a classic confounding structure: Style ← Body Type → Appeal.

A visual representation of Simpson's Paradox. The plot on the left shows an overall negative trend in the aggregated data. The plot on the right reveals that when the data is stratified into two groups (blue and red), the trend within each group is positive, reversing the conclusion from the aggregated view.

The back-door criterion tells us we must adjust for Body Type. But how, precisely? The answer lies in the adjustment formula. This formula allows us to simulate an intervention—what would happen if we forced every model to try a specific style, regardless of their body type? This is the meaning of the do()-operator.

For an exposure X (e.g., Lingerie Style), an outcome Y (e.g., Appeal Score), and a confounding variable Z (e.g., Body Type), the formula is:

P (Y = y ∣ d o (X = x)) = z \sum P (Y = y ∣ X = x, Z = z) P (Z = z)

Let's break this down intuitively for our lingerie design context:

$P (Y = y ∣ X = x, Z = z)$ : This is the stratum-specific effect. We calculate the appeal of a lingerie style (X=x) within a specific group of models, for example, only those with a 'Curvy' body type (Z=z). We do this for every group.
$P (Z = z)$ : This is the prevalence of that group in the overall population. For example, what proportion of all models we work with have a 'Curvy' body type?
$\sum_{z} \dots$ : We then calculate a weighted average of the stratum-specific effects, where the weights are the prevalences of each group.

This process creates a synthetic population where the lingerie style and model body type are no longer associated, giving us a clear, unconfounded view of the style's true causal effect on appeal.

The following resources provide a clear overview of the paradox and the formal adjustment procedure.

21.2 Simpson's Paradox | A Guide on Data Analysis - Bookdown

This reading gives a concise definition of Simpson's Paradox and explains its underlying mechanism.

First, read the introductory section, "Simpson's Paradox", to refresh the concept. Then, read the section titled "Why Does This Happen?", which clearly explains how a confounding variable drives the paradox through uneven distribution.

[PDF] A Brief Introduction to Causal Inference and Causal Diagrams

These presentation slides give the formal definition of the adjustment formula and walk through a numerical example.

First, focus on the slide titled "The Backdoor Criterion". This slide presents the exact adjustment formula we will be using. Next, review the slides under the heading "Simpson's Paradox". This section provides a detailed, step-by-step numerical application of the adjustment formula to resolve the paradox, which will serve as a model for our own exercise.

Your Turn: Correcting a Misleading Trend

Now it's time to apply this method to solve a practical problem. Your statistical expertise will be key to navigating the calculations and interpreting the results correctly.

Scenario:
You are analyzing data from recent test shoots to compare two lingerie styles:

Style A: A delicate lace bralette.
Style B: A structured balconette bra.

The outcome is Appeal, which is rated as either "High" or "Low". You notice that Style A appears to be more successful overall. However, you suspect that Model Body Type ('Curvy' or 'Slender') might be a confounding factor, as stylists may have preferentially assigned styles based on body type.

The Data:
A total of 400 models participated in the test shoots. Here is the aggregated data:

Style	Total Models	High Appeal	Low Appeal	Overall Success Rate
Style A	200	172	28	86%
Style B	200	162	38	81%

From this table, Style A appears to be the winner.

Now, here is the same data stratified by Model Body Type:

For 'Curvy' Models:

Style	Total Models	High Appeal	Success Rate
Style A	20	10	50%
Style B	80	48	60%

For 'Slender' Models:

Style	Total Models	High Appeal	Success Rate
Style A	180	162	90%
Style B	120	114	95%

Within each subgroup, Style B is clearly superior. This is a classic case of Simpson's Paradox.

Your Task:

Confirm the distribution of body types in the overall population of 400 models. Calculate $P (Body Type = ’Curvy’)$ and $P (Body Type = ’Slender’)$ .
Using the adjustment formula, calculate the true causal effect of each style on achieving "High Appeal". That is, compute:
- $P (Appeal = ’High’ ∣ d o (Style = A))$
- $P (Appeal = ’High’ ∣ d o (Style = B))$
Based on your adjusted calculations, which style is causally superior? Explain why the initial, aggregated result was misleading.

Take some time to work through the calculations before revealing the solution.

Click for the step-by-step solution.

1. Calculate Population Proportions for the Confounder (Z)

First, we determine the overall proportion of each body type in our total population of 400 models.

Total 'Curvy' models = 20 (testing Style A) + 80 (testing Style B) = 100
Total 'Slender' models = 180 (testing Style A) + 120 (testing Style B) = 300

The proportions (prevalences) are:

$P (Z = ’Curvy’) = \frac{100}{400} = 0.25$
$P (Z = ’Slender’) = \frac{300}{400} = 0.75$

2. Apply the Adjustment Formula

The formula is $P (Y ∣ d o (X)) = \sum_{z} P (Y ∣ X, Z = z) P (Z = z)$ .

For Style A:
We need to calculate $P (Appeal=’High’ ∣ d o (Style=A))$ .

P (Y = High ∣ d o (X = A)) = P (Y = High ∣ X = A, Z = Curvy) P (Z = Curvy) + P (Y = High ∣ X = A, Z = Slender) P (Z = Slender)

Plugging in the values from our tables:

$P (Y = High ∣ X = A, Z = Curvy) = 0.50$
$P (Y = High ∣ X = A, Z = Slender) = 0.90$
$P (Z = Curvy) = 0.25$
$P (Z = Slender) = 0.75$

So, the calculation is:

(0.50 \times 0.25) + (0.90 \times 0.75) = 0.125 + 0.675 = 0.80

The adjusted success rate for Style A is 80%.

For Style B:
Now we calculate $P (Appeal=’High’ ∣ d o (Style=B))$ .

P (Y = High ∣ d o (X = B)) = P (Y = High ∣ X = B, Z = Curvy) P (Z = Curvy) + P (Y = High ∣ X = B, Z = Slender) P (Z = Slender)

Plugging in the values:

$P (Y = High ∣ X = B, Z = Curvy) = 0.60$
$P (Y = High ∣ X = B, Z = Slender) = 0.95$

The calculation is:

(0.60 \times 0.25) + (0.95 \times 0.75) = 0.150 + 0.7125 = 0.8625

The adjusted success rate for Style B is 86.25%.

3. Conclusion

After adjustment, the causal effects are:

Style A: 80% success rate.
Style B: 86.25% success rate.

The true causal effect shows that Style B is superior.

The initial aggregated result was misleading because the comparison was unfair. Style A's overall average was inflated because it was disproportionately tested on 'Slender' models (180/200 = 90% of its tests), a group that generally produces high appeal scores regardless of style. Conversely, Style B was heavily tested on 'Curvy' models (80/200 = 40% of its tests), a group where achieving a 'High Appeal' rating is more challenging. The adjustment procedure corrects for this imbalance, giving us an estimate of what would happen if both styles were tested on a representative sample of models.

Conclusion

This lesson completes our module on causal inference by providing the critical tool for resolving confounding. You have moved from theory to practice, using the adjustment formula to correct a misleading statistical association and uncover the true causal relationship.

Key Takeaways:

Simpson's Paradox is a manifestation of confounding, where an aggregate trend is reversed within subgroups.
A causal DAG allows us to identify the confounder that needs to be controlled for.
The adjustment formula is the mechanism for this control. It estimates the causal effect of an intervention by calculating a weighted average of the stratum-specific effects, using the confounder's population prevalence as the weights.
This procedure provides an unbiased estimate of a style's effectiveness, as if it were tested in a randomized controlled trial.

You are now equipped to not only spot causal fallacies like confounding and collider bias in your data but also to formally correct for them.

In our next module, we will shift from causal questions ("what is the effect?") to predictive ones ("what is the likely outcome?"). We'll begin our journey into classification models by exploring the Naive Bayes classifier, a powerful tool for predicting categorical outcomes like whether a new lingerie design will be 'photoshoot-worthy'.

Recommended resources

From Identification to Estimation: The Adjustment Formula

Your Turn: Correcting a Misleading Trend

Conclusion