Sensitivity Analysis for Unmeasured Confounding

The Challenge of Unseen Influences

In an ideal randomized controlled trial, treatment assignment is independent of all other variables, both measured and unmeasured. Our N-of-1 crossover design incorporates randomization, but the complex human dynamics we are studying mean that unmeasured confounders are a persistent concern.

Consider the causal structure of a typical observational finding. We observe a relationship between an exposure (E) and a disease or outcome (D), having adjusted for a set of measured confounders. The concern is the presence of one or more unmeasured confounders (U) that influence both the exposure and the outcome.

The critical question of sensitivity analysis is to determine the required strength of the associations along the confounding pathway (E ← U → D) to explain away the observed association between E and D. Historically, many sensitivity analysis techniques were difficult to use, requiring complex calculations and numerous subjective assumptions about the nature of the unmeasured confounder.

More recent developments have provided tools that are both powerful and easy to interpret. These methods allow us to assess the potential impact of unmeasured confounding without making restrictive assumptions, such as the confounder being a single binary variable.

Quantifying Robustness: The E-value

The E-value provides an elegant solution to the problem of quantifying robustness. It is a single number that summarizes how prone your finding is to being overturned by an unmeasured confounder. The foundational paper by VanderWeele and Ding provides a clear introduction to this concept.

As the paper explains, the E-value for a risk ratio (RR) > 1 is calculated as:

The interpretation is direct: For an unmeasured confounder to explain away the observed risk ratio, it would need to be associated with both the exposure and the outcome by a risk ratio of at least the E-value, conditional on the measured covariates. Weaker confounding could not account for the observed effect.

A crucial part of good practice is to report the E-value not only for the point estimate but also for the limit of the confidence interval (CI) that is closest to the null. An effect might seem robust based on its point estimate, but if the CI is wide and its edge is near the null, a much weaker confounder could render the finding statistically non-significant.

Applying the E-value to Our Triad Experiment

Let's ground this in our vee-triad scenario. Imagine our N-of-1 crossover experiment on living arrangements yields a primary result. After adjusting for measured confounders (e.g., work stress, time of year), we find that the "living together" arrangement (exposure) increases the pivot's relationship satisfaction (outcome) by a risk ratio of RR = 2.4 compared to the baseline "living apart" arrangement. The 95% confidence interval is (1.6, 3.6).

Let's calculate the E-values for this finding.

1. E-value for the point estimate (RR = 2.4):

   $$E-valu{e}_{est}=2.4+\sqrt{2.4\times (2.4-1)}=2.4+\sqrt{2.4\times 1.4}=2.4+\sqrt{3.36}\approx 2.4+1.83=4.23$$

   Interpretation: To explain away our 2.4-fold increase in satisfaction, an unmeasured confounder would need to be associated with both the living arrangement and satisfaction by a risk ratio of 4.23-fold each.

2. E-value for the confidence interval (lower limit = 1.6):

   $$E-valu{e}_{CI}=1.6+\sqrt{1.6\times (1.6-1)}=1.6+\sqrt{1.6\times 0.6}=1.6+\sqrt{0.96}\approx 1.6+0.98=2.58$$

   Interpretation: To shift the lower confidence limit to the null value of 1.0 (i.e., to make the result no longer statistically significant), an unmeasured confounder would need associations of 2.58-fold with both exposure and outcome.

This second E-value is often the more important one. It tells us the threshold of confounding required to challenge the statistical evidence for our conclusion.

The relationship between the required confounder-exposure and confounder-outcome associations is visualized as a curve. The E-value represents the point on this curve where the two associations are equal.

E-values for Other Effect Measures

Our example used a risk ratio, which is the most direct application. However, your linear regression model from the previous lesson produced a coefficient, which is a difference in means. The VanderWeele and Ding paper describes how to compute approximate E-values for other effect measures.

For a continuous outcome, the common approach is to convert the regression coefficient into a standardized mean difference ($d$) by dividing by the outcome's standard deviation. You can then convert $d$ to an approximate RR using the formula in the table and proceed with the E-value calculation. There are also R packages, such as EValue, that can perform these conversions and calculations for you automatically.

Context is Everything: Interpreting the E-value

An E-value of 2.58 is not inherently large or small. Its meaning comes from context. This is where your substantive expertise and knowledge of the causal system become critical. To decide if a 2.58-fold confounding effect is plausible, you can use benchmarking.

The idea is simple: calculate the associations for the confounders you did measure. In our triad example, suppose we measured "household financial stress" and found it was associated with satisfaction with a risk ratio of 1.7 and with the living arrangement with a risk ratio of 1.5. Our E-value for the confidence interval (2.58) is substantially larger than either of these observed confounding strengths.

This allows for a powerful concluding statement: "For an unmeasured confounder to explain away our finding, it would need to be considerably stronger than any of the confounders we did measure and control for." This doesn't prove the effect is real, but it substantially increases our confidence in the result.

Conclusion

This lesson moved us from estimating an effect to critically evaluating its robustness. By embracing the near-certainty of unmeasured confounding, we can add a necessary layer of rigor to our causal claims.

Key Takeaways:

• Assuming no unmeasured confounding is often unrealistic. Sensitivity analysis is a necessary step to assess the robustness of causal claims from observational data.
• The E-value is the minimum strength of association an unmeasured confounder must have with both the exposure and outcome to nullify the observed effect.
• It is crucial to calculate the E-value for both the point estimate and the confidence interval limit closest to the null, as the latter quantifies the bar for challenging the statistical significance of the finding.
• The interpretation of an E-value is context-dependent. Benchmarking against the strength of measured confounders provides a tangible scale for judging its plausibility.

In our analysis so far, we have focused on the sensitivity of our effect estimates to unmeasured variables. However, the estimates themselves are derived from a specific structural causal model (our DAG). What if the entire causal structure is misspecified? In the next lesson, we will broaden our scope to evaluate the robustness of causal conclusions to alternative structural model specifications, exploring how our results might change if we assume a different web of causal relationships.