Link Functions and Canonical Links

The Purpose of the Link Function

As we briefly covered, the link function, $g()$, is the crucial bridge connecting the mean of the response variable, $\mu =E(Y)$, to the linear predictor, $\eta$.

So, what is its purpose? The link function solves two main problems simultaneously:

1. The Constraint Problem: Response variables often have natural constraints. For instance, probabilities ($\mu$ from a Binomial model) must be between 0 and 1, and counts ($\mu$ from a Poisson model) must be positive. However, the linear predictor $\eta$ can produce any value from $-\infty$ to $+\infty$. The link function mathematically transforms the constrained scale of $\mu$ to the unconstrained scale of $\eta$, ensuring that when we invert the process to get a prediction, it will always be a valid, sensible value.
2. The Linearity Problem: The relationship between the predictors and the raw mean of the response is often not linear. The link function provides a scale on which this relationship is linear. It "bends the line," as the video from our last lesson described, allowing us to model non-linear relationships while still using a linear equation at the core.

To formalize these ideas and see how they are expressed, let's turn to a concise reading.

Canonical Links: The "Natural" Choice

While several different link functions might be mathematically possible for a given distribution, each distribution in the exponential family has one special link called the canonical link.

• Definition: The canonical link is the function that transforms the mean, $\mu$, to the distribution's natural parameter, $\theta$. (We will explore the exponential family and natural parameters in more detail in the next module, but for now, you can think of it as the most "natural" mathematical pairing).
• Why it's important: Canonical links are the default in most statistical software and often have desirable statistical properties, such as ensuring the concavity of the log-likelihood function, which simplifies parameter estimation.

For your exam, you must be able to instantly pair the common distributions with their canonical links. Let's review them one by one.

1. Normal Distribution (Continuous Data)

• Random Component: Normal distribution.
• Mean: The mean $\mu$ can be any real number ($-\infty ,\infty$).
• Canonical Link: Identity
  • Equation: $g(\mu )=\mu$
  • Full Model: $\mu ={\beta }_0+{\beta }_1{X}_1+\cdots +{\beta }_k{X}_k$
  • Purpose: No transformation is needed because the range of $\mu$ is already unconstrained, just like the linear predictor $\eta$. This special case of a GLM is simply the ordinary linear regression model.

2. Poisson Distribution (Count Data)

• Random Component: Poisson distribution.
• Mean: The mean count $\mu$ must be positive ($\mu >0$).
• Canonical Link: Log
  • Equation: $g(\mu )=\ln (\mu )$
  • Full Model: $\ln (\mu )={\beta }_0+{\beta }_1{X}_1+\cdots +{\beta }_k{X}_k$
  • Purpose: The natural log function maps the domain of positive numbers ($0,\infty$) to the entire real number line ($-\infty ,\infty$). This guarantees that any predicted mean, $\hat{\mu }=e^{\eta }$, will always be positive.

3. Binomial Distribution (Binary/Proportion Data)

• Random Component: Binomial distribution.
• Mean: The mean probability $\mu$ (often written as $\pi$) must be between 0 and 1 ($0<\mu <1$).
• Canonical Link: Logit
  • Equation: $g(\mu )=\ln \left(\frac{\mu }{1-\mu }\right)$
  • Full Model: $\ln \left(\frac{\mu }{1-\mu }\right)={\beta }_0+{\beta }_1{X}_1+\cdots +{\beta }_k{X}_k$
  • Purpose: The logit function, which calculates the log-odds, maps the domain of (0, 1) to the entire real number line. This ensures that any predicted probability, $\hat{\mu }=\frac{e^{\eta }}{1+e^{\eta }}$, will always be between 0 and 1.

The following video provides a concise review of these canonical links and why they are chosen.

This table summarizes what you need to know for the exam.

Visual Decision Guides

Since your exam is application-focused, having a clear decision-making process is vital. The following flowcharts are excellent tools to help you quickly determine the right link function based on the type of data you're analyzing.

Here is a guide for when your outcome is a binary variable.

And here is a guide for when you are modeling count data.

Quick Practice

Let's test your ability to apply these rules. For each scenario below, identify the appropriate probability distribution for the response variable and its corresponding canonical link function.

1. A study investigates whether a bird a researcher observes is male or female (a binary outcome).
2. An urban planner models the number of cars that pass through an intersection per hour.
3. An agronomist models the height of corn stalks (a continuous measurement) based on fertilizer type.

Conclusion

In this lesson, we focused entirely on the role and identity of the link function. You are now equipped with the knowledge to handle exam questions that require you to specify the correct GLM components.

Key Takeaways:

• Purpose of the Link Function: It connects the mean of the response ($\mu$) to the linear predictor ($\eta$) by (1) transforming the constrained range of $\mu$ to the unconstrained range of $\eta$, and (2) creating a scale where the relationship is linear.
• Canonical Links: These are the "natural" or default links for a given distribution.
• The Essential Pairings:
  • Normal distribution $\to$ Identity link
  • Poisson distribution $\to$ Log link
  • Binomial distribution $\to$ Logit link

You have now solidified your understanding of the three core GLM components. In the next lesson, we will put this knowledge into practice by focusing on the complete decision-making process: given a description of a response variable, you will select the most appropriate distribution and link function for a GLM, preparing you for a common type of exam question.