Bias in Unconditional Logistic Regression for Matched Designs

Hello! Welcome to the tenth module of our course, where we'll be tackling the analysis of matched data.

In all our previous discussions, from chi-square tests to the various forms of logistic regression, we've operated under a critical assumption: that all our observations are independent of one another. However, many powerful study designs, such as pre-test/post-test studies, matched case-control studies, or longitudinal panel studies, intentionally violate this assumption by collecting paired or clustered data.

This lesson addresses a foundational question: what happens if we ignore this dependency and apply the standard logistic regression models we've already learned? As you'll see, doing so can lead to significantly flawed conclusions. Our goal today is to understand precisely why standard (unconditional) logistic regression produces biased estimates in matched study designs.

What are Matched Designs?

First, let's clarify what we mean by "matched" or "clustered" data. These are designs where observations are grouped together because they share certain characteristics.

Case-Control Studies: A person with a disease (a "case") is matched with one or more people without the disease ("controls") who are similar in terms of key characteristics like age, sex, or smoking status. The goal of matching is to control for the confounding effects of these variables.
Panel Data (or Longitudinal Data): The same individuals are measured repeatedly over time. For example, a person's symptoms might be recorded every week under different treatment conditions. Here, the "group" or "cluster" is the individual, and their repeated measurements are not independent.

In both scenarios, the matching process creates sets of observations that are related to each other. A case and their matched control are not independent; they are linked by design. Similarly, your health status this week is not independent of your health status last week.

The Problem: Two Flawed Approaches with Standard Logistic Regression

Suppose we have a matched case-control study and want to assess the effect of an exposure on the outcome. How might we analyze this with standard logistic regression? There are two intuitive, but ultimately flawed, approaches.

Flawed Approach 1: Ignore the Matching

The simplest approach is to "break the matches" and treat all the data as if they came from a simple random sample. You would just run a logistic regression of the outcome on the exposure, ignoring the fact that individuals were paired up.

This is a critical mistake because it undoes the very purpose of matching. The matching was performed to control for confounding. By ignoring it in the analysis, you reintroduce that confounding, which leads to biased estimates of the exposure's effect. The resulting odds ratio is typically biased towards the null, meaning you might underestimate the true effect or miss it entirely.

Flawed Approach 2: The (Unconditional) Fixed Effects Model

A more thoughtful approach would be to try and account for the matching. Since each matched pair (or individual in a panel study) forms a unique group or "stratum," we could try to control for this by adding a dummy variable for each and every pair.

Our logistic regression model would look something like this:

logit (P (Y = 1)) = α_{i} + β X

Here, $X$ is our predictor of interest, $β$ is its coefficient, and $α_{i}$ is a separate intercept term for each matched pair $i$ . This is often called a fixed effects logistic regression. While this seems like a direct way to control for all unique factors within a pair, it fails spectacularly in logistic regression due to something called the Incidental Parameters Problem.

The Incidental Parameters Problem (IPP)

The fixed effects approach fails because of a fundamental issue with Maximum Likelihood Estimation (MLE), the statistical engine that fits logistic regression models.

The video below gives an excellent explanation of why the fixed effects strategies that work for linear regression are not valid for non-linear models like logistic regression.

Conditional fixed-effects logistic model

To understand why this fixed effects approach fails, we need to learn about the 'Incidental Parameters Problem'. This video explains the issue clearly and contrasts it with linear models, with which you may be more familiar.

Please watch the first part of the video, from the beginning to 06:19. Focus on understanding why the strategies used for fixed effects in linear models (like dummy variables) do not work for non-linear models like logistic regression. The key concept is that the number of parameters grows with the number of groups, which violates a core assumption of Maximum Likelihood Estimation.

As the video explained, the core issue is this:

The parameters of interest are the $β$ coefficients. The intercepts for each pair, the $α_{i}$ , are "incidental parameters"—we need to account for them, but we don't care about their specific values.
MLE provides consistent estimates, meaning the estimates converge to the true values as the sample size grows. However, this property relies on the number of parameters being fixed.
In a matched study, if we add more data, we are usually adding more pairs. Each new pair brings with it a new incidental parameter, $α_{i}$ . The number of parameters grows along with the sample size.
Because each $α_{i}$ is estimated from a very small amount of data (e.g., just two people in a matched pair), its estimate is very imprecise. When you have hundreds or thousands of these imprecise estimates in your model, the error "contaminates" the estimation of your main parameter, $β$ , causing it to be biased.

The Dire Consequences of the IPP

So, what does this bias look like in practice? The following reading describes the consequences very clearly.

Regression models for 'case-control' and matched studies

The video gave us the theoretical reason for the problem. Now, let's look at a resource that describes the practical consequences in a very direct way.

Please read the section titled 'Options Consequences' (you'll find it on page 7 of the PDF, under the heading 'Matched Case-Control Studies'). Focus on points 'a' and 'c'. Point 'a' confirms the bias from ignoring matching, and point 'c' describes the bias from using the dummy variable (fixed effects) approach. Pay close attention to the specific claim about what happens in matched pairs (M=1).

This resource provides two crucial takeaways:

Ignoring matching (breaking the pairs) leads to an odds ratio that is biased towards the null. You might wrongly conclude an exposure has no effect.
Using a fixed effects model (a dummy for each pair) leads to an odds ratio that is biased away from the null. For a 1-to-1 matched pair study, this method estimates a log-odds ratio that is double the correct value. This means the estimated odds ratio is the square of the true odds ratio—a massive overestimation of the effect!

Visualizing the Bias

Seeing the poor performance of these methods in a simulation can make the abstract concept of bias very concrete. The following blog post does an excellent job of visualizing the failure of both flawed approaches.

Logistic Regression and the Incidental Parameters Problem

It can be hard to believe how poorly these standard methods perform without seeing it in action. This blog post provides a great simulation that visualizes the problem.

Please read the following parts: The introduction and the section 'Incidental Parameters Problem' to get another clear, concise definition of the IPP. The section 'Visualizing the Distribution of Estimates'. Look closely at the plots. Notice how the 'Naive estimator' (which ignores matching) and the 'dummy fixed effect estimator' perform, especially for short panel lengths. A matched-pair study is the shortest possible panel (length = 2). The short concluding section 'A couple of final thoughts'. This gives a clear, practical takeaway.

The simulation plots from the post drive the point home:

The naive estimator (ignoring matching) is inconsistent; it converges to the wrong answer.
The dummy fixed effect estimator performs very poorly in "short panels" (like the matched pairs in our studies). It is severely biased and should not be used.

Conclusion

Today we established a critical principle for your data analysis toolkit: standard (unconditional) logistic regression is not appropriate for matched or clustered data. Attempting to use it will lead to biased results, regardless of whether you ignore the matching or try to control for it with dummy variables.

Key Takeaways:

Matched designs (e.g., case-control, panel data) violate the independence assumption of standard regression models.
Ignoring the matching structure reintroduces confounding and typically biases the effect estimate towards the null (underestimation).
Controlling for matching with dummy variables in a standard logistic regression (an unconditional fixed effects model) triggers the Incidental Parameters Problem.
The IPP causes the effect estimate to be biased, often severely overestimating the effect. In 1-to-1 matched studies, the odds ratio can be squared.

This brings us to an important question: if these methods don't work, what does? The answer lies in a different kind of model, one designed specifically to handle the Incidental Parameters Problem.

Preview of the Next Lesson:

In our next lesson, we will introduce the solution: Conditional Logistic Regression. We will explore how this technique cleverly modifies the estimation process to eliminate the problematic incidental parameters, allowing for unbiased estimates of effects in matched data. We will also see how the classic McNemar test for paired data is actually just a simple, special case of a conditional logistic regression model.

Recommended resources