Understanding the Three Components of GLMs

The Anatomy of a Generalized Linear Model

A GLM is not a single model, but a flexible framework defined by three key architectural decisions. Let's start with a video that gives a clear, conceptual overview of these three parts.

As the video explained, any GLM can be described by these three components. Now, let's examine each one in more detail. For a more formal definition of these concepts, you can refer to the following resource.

1. The Random Component

The first piece of a GLM is the random component. This is simply the assumed probability distribution of the response variable $Y$.

• What it is: Instead of being restricted to the Normal (Gaussian) distribution as in ordinary linear regression, GLMs allow you to choose a distribution from the exponential family that better matches the nature of your data.
• Why it matters: This directly solves the "non-normality" problem.
  • For binary data (e.g., success/failure), you would choose the Binomial distribution.
  • For count data (e.g., number of events), you would choose the Poisson distribution.
• The takeaway: The random component defines the underlying probability model for the data and its variance structure. The choice of distribution is guided by the type of response variable you are modeling.

You can see examples of this in the "Introduction to Generalized Linear Models" PDF on slide 19, which matches dichotomous data to the Binomial distribution and count data to the Poisson distribution.

2. The Systematic Component (Linear Predictor)

The second piece is the systematic component, also known as the linear predictor. This part should feel very familiar from your previous experience with linear models.

• What it is: It is a linear combination of the explanatory variables ($X_1,X_2,\dots$). We typically denote the linear predictor by the Greek letter eta ($\eta$). $$\eta ={\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+\cdots +{\beta }_k{X}_k$$
• Why it matters: This is the "linear" part of a "Generalized Linear Model." It retains the simple, additive structure of linear regression, making the model parameters (${\beta }_i$) interpretable in a straightforward way.
• The takeaway: The systematic component specifies how your predictor variables are linearly related to the (transformed) mean of the response.

3. The Link Function

The third and most innovative piece is the link function. It serves as the bridge connecting the random component and the systematic component.

• What it is: The link function, denoted by $g()$, is a function that connects the expected value (mean) of the response variable, $\mu =E(Y)$, to the linear predictor, $\eta$. $$g(\mu )=\eta$$ or, written out fully: $$g(E(Y))={\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+\cdots +{\beta }_k{X}_k$$
• Why it matters: This solves the problem of nonsensical predictions. The link function transforms the mean of the response to a scale where a linear relationship is plausible and where the values are not constrained.
  • For a binary response, the mean $\mu$ is a probability $p$ between 0 and 1. The link function maps this [0, 1] interval onto the entire real number line (-$\infty$, +$\infty$), so the linear predictor $\eta$ can take any value.
  • For a count response, the mean $\mu$ must be positive. The link function maps the positive real numbers (0, +$\infty$) onto the entire real number line.
• The takeaway: The link function ensures that the model's predictions are kept within the natural range of the response variable.

Important Distinction: As mentioned in the video, a link function is not the same as transforming the response variable $Y$. A transformation (e.g., log(Y)) alters the data itself. A link function transforms the expected value of $Y$, leaving the original data untouched. This is a subtle but crucial difference.

Canonical Links: The Default Choice

For each distribution in the exponential family, there is one special link function called the canonical link. This is the link that is "natural" for that distribution and often leads to models with desirable statistical properties. For your exam, you should know the canonical links for the most common distributions.

The next video provides a slightly more formal explanation of the link function and introduces these canonical links.

Here is a summary of the key distribution-link pairings you must know. This table combines information from the video and the "Introduction to Generalized Linear Models" PDF (slide 25).

Ordinary linear regression is just a GLM with a Normal random component and an identity link function!

Putting It All Together

The following images provide excellent visual summaries that will be very helpful for your exam revision. They act as "decision guides" for choosing the right components based on your data.

Conclusion

In this lesson, we dissected the Generalized Linear Model into its three core components. You now know how these parts work in concert to create a flexible and powerful modeling framework.

Key Takeaways:

• A GLM is defined by three components:
  1. Random Component: The probability distribution for the response variable $Y$ (e.g., Normal, Binomial, Poisson). It's chosen to match the nature of the data.
  2. Systematic Component: The linear predictor ($\eta ={\beta }_0+{\beta }_1{X}_1+\dots$), which is a linear function of the predictors.
  3. Link Function: A function $g()$ that connects the mean of the response ($\mu$) to the linear predictor ($g(\mu )=\eta$).
• The link function is crucial for ensuring that the model produces sensible predictions (e.g., probabilities between 0 and 1, positive counts).
• For each distribution, there is a canonical link that is considered the default choice, such as the identity link for Normal, log for Poisson, and logit for Binomial.

You've learned to identify the building blocks of a GLM. In the next lesson, we will focus on the practical skill of selecting the appropriate distribution and link function for a GLM based on a description of a response variable, which is a common type of exam question.