CMH Test: Conditional Independence

1. The Problem: Why Not Just Pool Data or Test Separately?

Imagine you've conducted a study on a new treatment and have data from several different clinics. You have a series of 2x2 tables, one for each clinic, showing treatment vs. outcome.

You want to know if the treatment is effective overall. You have two intuitive but flawed options:

1. Pool all the data: You could collapse all the tables into one large 2x2 table and run a single chi-square test.
   • The Flaw: As we saw in the last lesson, if the baseline risk or the mix of patients differs by clinic (i.e., the clinic is a confounder), this can lead to Simpson's Paradox, giving you a completely wrong answer.
2. Test each table separately: You could run a chi-square or Fisher's exact test on each clinic's table.
   • The Flaw: Each individual clinic may have a small sample size. This results in low statistical power, meaning you might fail to detect a real, consistent treatment effect because the evidence in any single clinic is too weak.

The Cochran-Mantel-Haenszel test was designed to solve exactly this dilemma.

2. The Purpose of the CMH Test: Testing Conditional Independence

The CMH test allows us to test for an association between two variables (e.g., Treatment and Outcome) while controlling for a third variable (e.g., Clinic). It does this by asking a specific question: Are the exposure and outcome variables conditionally independent, given the third variable?

To understand this, let's read a brief but important definition.

As the reading states, "conditional independence" means there is no association within any of the strata. In the context of a 2x2 table, this is equivalent to the odds ratio being 1 in every stratum.

This leads us to the formal hypotheses for the CMH test:

• Null Hypothesis ($H_0$): The exposure and outcome are conditionally independent. This means that after controlling for the confounding variable, there is no association between exposure and outcome. In terms of odds ratios, this is equivalent to the common odds ratio across all strata being equal to 1.
  $H_0:{\text{OR}}_1={\text{OR}}_2=\cdots ={\text{OR}}_K=1$
• Alternative Hypothesis ($H_a$): The exposure and outcome are not conditionally independent. There is an overall association, meaning the common odds ratio is not equal to 1.

The purpose of the CMH test is therefore to evaluate this null hypothesis by pooling evidence across all strata without incorrectly collapsing them.

3. How the CMH Test Works

The CMH test achieves its goal by combining information from each partial table in a way that increases statistical power while avoiding bias. Let's look at a powerful example that demonstrates this.

The passive smoking example highlights the core utility of the CMH test:

• It increases power: By combining the evidence from Japan, the UK, and the US, the test was able to detect an association that was too weak to be found in any single country's data.
• It controls for confounding: The analysis correctly keeps the countries separate in the calculation, acknowledging that the baseline rates of lung cancer and the prevalence of smoking might differ across countries. A simple pooled analysis would have been inappropriate.

The logic of the test statistic, as shown in the reading, is to sum the deviations of the observed counts from what would be expected under the null hypothesis within each stratum. A large, consistent deviation across strata provides strong evidence against the null hypothesis of no association.

4. An Important Assumption: Homogeneity of Association

The CMH test is most appropriate and its results are most easily interpreted when the strength of the association is similar across the different strata. This is called the assumption of homogeneous association.

• Homogeneous Association: The odds ratio is approximately the same in all strata (${\text{OR}}_1\approx {\text{OR}}_2\approx \cdots \approx {\text{OR}}_K$).
• Heterogeneous Association (or Effect Modification): The odds ratio is substantially different across strata. For example, a treatment might have a strong positive effect in young patients ($\text{OR}=5.0$) but a weak or even negative effect in old patients ($\text{OR}=0.9$).

If significant effect modification is present, the CMH test is still valid for testing the null hypothesis of conditional independence ($H_0:\text{all ORs}=1$). However, reporting a single "common odds ratio" can be misleading because it averages out important differences. In such cases, it's better to describe the association separately for each stratum.

Most statistical software that performs a CMH test also provides a companion test, often the Breslow-Day test, to check the assumption of homogeneity.

Let's look at a sample output to see how all these pieces fit together.

In this example output, the CMH "General Association" test (p-value < 0.001) tells us to reject the null hypothesis of conditional independence. The Breslow-Day test (p-value = 0.536) tells us we have no reason to doubt the assumption of homogeneous odds ratios, making it appropriate to report the estimated common odds ratio.

Conclusion

In this lesson, we established the critical role of the Cochran-Mantel-Haenszel test in categorical data analysis. It provides a formal method for hypothesis testing in the presence of a potential confounder.

Key Takeaways:

• The purpose of the CMH test is to test for conditional independence between two categorical variables, while controlling for a third stratifying variable.
• It solves the problems of both pooling data (risk of Simpson's Paradox) and analyzing strata separately (low statistical power).
• The null hypothesis ($H_0$) is that there is no association within any stratum, meaning the common odds ratio across all strata is 1.
• The test works by combining evidence of association across all strata to produce a single test statistic and p-value.
• The test is most meaningful when the association is homogeneous (similar) across strata, an assumption that can be checked with tests like the Breslow-Day test.

Preview of the Next Lesson:

The CMH test answers the question: "Is there an association, after controlling for confounding?" The next logical question is: "How strong is that association?" In the next lesson, we will focus on the Mantel-Haenszel estimate of the common odds ratio and how to formally test for the homogeneity of odds ratios, allowing us to decide whether it's appropriate to report this single summary measure.