Canonical Links for GLMs

What Makes a Link "Canonical"?

In the previous lesson, we saw that any distribution in the exponential family has a natural parameter, denoted by $\theta$. This parameter is directly related to the distribution's mean, $\mu$ (or $\pi$ for probabilities, $\lambda$ for rates).

A link function, $g(\mu )$, connects the mean of our response variable to the systematic component (the linear predictor, $\eta ={\beta }_0+{\beta }_1{X}_1+\dots$).

A link function is called canonical when it is the specific function that makes the linear predictor equal to the natural parameter.

Canonical Link: The link function $g(\mu )$ for which $g(\mu )=\theta$. This means the linear predictor directly models the natural parameter of the distribution: $\eta =\theta$.

This is the most direct and mathematically simplest relationship we can specify between the model's predictors and the parameters of the response distribution.

The following video provides a clear explanation of this concept and walks through the canonical links for our distributions of interest.

The Intuition Behind the "Canonical" Choice

Why is this particular link special? The term "canonical" here suggests a fundamental or standard choice. Think of it as the most "organic" or straightforward connection for a given distribution.

To formalize this a bit, let's look at a clear definition.

By choosing the canonical link, we set $\eta =\theta$. This choice has some desirable statistical properties that often lead to more stable and efficient model estimation, although we won't dive into the mathematical proofs.

For a more intuitive perspective, the selected answer in the following discussion explains this well.

In essence, while you can use other links (which we'll discuss in the next lesson), the canonical link is the default starting point because it represents the most direct mapping between your predictors and the core parameter of your chosen distribution.

Identifying Canonical Links for Key Distributions

Let's now explicitly identify the canonical links for the three distributions at the heart of most introductory GLM applications. As you saw in the Stats4Everyone video, this involves recalling the natural parameter $\theta$ for each distribution that we found in the last lesson.

1. Normal (Gaussian) Distribution

This is the distribution underlying ordinary linear regression.

• Mean Parameter: $\mu$
• Natural Parameter: $\theta =\mu$
• Canonical Link: Since the natural parameter is the mean itself, the function that transforms the mean to the natural parameter is simply $g(\mu )=\mu$. This is the identity link.
• Model Form: $\mu ={\beta }_0+{\beta }_1{X}_1+\dots$

2. Poisson Distribution

This distribution is used for modeling count data.

• Mean Parameter: $\lambda$ (the rate)
• Natural Parameter: $\theta =\log (\lambda )$
• Canonical Link: The function that transforms the mean ($\lambda$) to the natural parameter is $g(\lambda )=\log (\lambda )$. This is the log link.
• Model Form: $\log (\lambda )={\beta }_0+{\beta }_1{X}_1+\dots$

3. Binomial Distribution

This distribution is used for modeling binary outcomes or proportions.

• Mean Parameter: $\pi$ (the probability of success)
• Natural Parameter: $\theta =\log \left(\frac{\pi }{1-\pi }\right)$
• Canonical Link: The function that transforms the mean ($\pi$) to the natural parameter is $g(\pi )=\log \left(\frac{\pi }{1-\pi }\right)$. This is the logit link.
• Model Form: $\log \left(\frac{\pi }{1-\pi }\right)={\beta }_0+{\beta }_1{X}_1+\dots$

The following table provides a clear summary of these relationships, which are the cornerstone of the most common GLMs.

Conclusion

In this lesson, we put a formal definition to the concept of the canonical link function and explored the intuition behind it. You now have a solid grasp of what a canonical link is and can identify the specific ones used for the Normal, Poisson, and Binomial distributions.

Key Takeaways:

• A canonical link function is the specific link $g(\mu )$ that equates the linear predictor ($\eta$) to the natural parameter ($\theta$) of the response distribution in its exponential family form.
• This choice is considered "canonical" or "organic" because it creates the simplest, most direct mathematical relationship between the model's predictors and the distribution's parameters, often leading to desirable statistical properties.
• The three key canonical links to remember are:
  • Identity Link for the Normal distribution ($g(\mu )=\mu$).
  • Log Link for the Poisson distribution ($g(\lambda )=\log (\lambda )$).
  • Logit Link for the Binomial distribution ($g(\pi )=\log (\frac{\pi }{1-\pi })$).

Preview of the Next Lesson:

The canonical link is the standard, but it's not the only option. What if the relationship in our data doesn't quite fit the one suggested by the canonical link? In our next lesson, we will distinguish between canonical and non-canonical link functions. We'll discuss why you might choose a non-canonical link like the probit or complementary log-log link for binomial data and what the practical trade-offs are.