Poisson Regression and Log-Linear Models: A Unified Approach

1. A Quick Refresher: The Poisson Regression Model

Before we build the bridge to log-linear models, let's quickly recall the three components of a Poisson regression model, which we've designed specifically for count data.

As the video explained, a Poisson regression model is a GLM with:

1. Random Component: The response variable (the counts) is assumed to follow a Poisson distribution.
2. Systematic Component: A linear predictor, which is a linear combination of the predictor variables (e.g., ${\beta }_0+{\beta }_1{X}_1+\dots$).
3. Link Function: A log link, which connects the expected value of the response to the linear predictor: $\log (\mathbb{E}[Y])=\log (\mu )={\beta }_0+{\beta }_1{X}_1+\dots$.

Keep these three components in mind as we now connect them to log-linear models.

2. The Core Connection: Fitting Log-Linear Models with Poisson Regression

Here is the central idea that unites these two concepts:

To fit a log-linear model to a contingency table, we treat the cell counts as the response variable in a Poisson regression. The categorical variables that define the rows and columns of the table become the predictor variables.

This works because the fundamental assumption of a log-linear model is that the count in each cell, $n_{ij}$, is an independent observation from a Poisson distribution with a mean ${\mu }_{ij}$.

Let's break this down. Consider a simple 2x2 contingency table:

To analyze this with a GLM, we first restructure the data from a wide table format into a "long" format:

Now, look what happens when we fit a Poisson regression to this long-format data, with Count as the response:
glm(Count ~ VarA + VarB, family = poisson(link="log"))

The underlying model for this is:

This is precisely the log-linear model of independence, just written in regression notation!

Similarly, if we fit the saturated model using Poisson regression:
glm(Count ~ VarA * VarB, family = poisson(link="log"))

The underlying model includes an interaction term:

This corresponds exactly to the saturated log-linear model:

The following reading from Penn State's STAT 504 course makes this connection explicit and shows how it's done in both R and SAS.

3. The Power of the Connection: Analyzing Multi-way Tables

The true power of this framework becomes apparent when we move beyond simple 2x2 tables to multi-way tables (e.g., 2x2x3). While a single chi-square test can tell you if there's some association present, it can't describe the complex relationships (like conditional independence or homogeneous association) that might exist.

By using Poisson regression, we can fit and compare various log-linear models to test specific hypotheses about these complex associations.

For a three-way table with variables A, B, and C, we can test different models by simply changing the formula in our glm() call:

• Complete Independence (A, B, C): glm(Count ~ A + B + C, family=poisson)
• Conditional Independence of A and B given C ((AC, BC)): glm(Count ~ A*C + B*C, family=poisson)
• Homogeneous Association (AB, AC, BC): glm(Count ~ A*B + A*C + B*C, family=poisson)
• Saturated Model (ABC): glm(Count ~ A*B*C, family=poisson)

This provides a unified and highly flexible toolkit for dissecting the structure of contingency tables.

This approach transforms contingency table analysis from a series of specific, isolated tests into a comprehensive modeling activity, which is much more aligned with the modern data science workflow.

Conclusion

In this lesson, we solidified the final link in the chain connecting classical contingency table analysis to the modern GLM framework. We saw that the conceptual log-linear models we've been discussing are not just theoretical—they are practically fitted using Poisson regression.

Key Takeaways:

• Log-linear models and Poisson regression are intrinsically connected. A log-linear model is a Poisson regression model applied to contingency table counts.
• The core idea is to treat the cell counts as the response variable and the categorical variables defining the table as the predictor variables.
• The fundamental assumption is that cell counts are independent observations from a Poisson distribution.
• The log-linear model equation (e.g., $\log ({\mu }_{ij})=\lambda +{\lambda }_i^A+{\lambda }_j^B$) is precisely what is estimated by a glm() function with family=poisson and a log link.
• This connection provides a powerful, unified, and flexible method for analyzing complex, multi-way contingency tables by fitting and comparing different regression models.

Preview of the Next Lesson:
This concludes the theoretical part of our module on log-linear models. In our final lesson, we will put all these pieces together in a practical application. We will take a multi-way contingency table, use Poisson regression to fit various log-linear models, and interpret the results to understand the association structure within the data.