Choosing Distributions and Link Functions for GLMs

The Two-Step Decision Framework

Choosing the right GLM specification is a two-step process that starts with your response variable, $Y$:

1. Choose the Family (Distribution): This decision is driven by the fundamental nature of your response variable. Is it continuous, a count, or binary? This determines the random component of your GLM.
2. Choose the Link Function: This decision is driven by your scientific question. Are you interested in modeling differences, ratios, or odds? This determines the link function, which defines the relationship between the mean and the predictors.

Let's break down each step.

Step 1: Choose the Distribution Based on the Nature of Y

The first question you should always ask is: "What kind of data is my outcome variable, Y?" The answer will point you to the correct probability distribution, or "family," for your model.

The flowchart below provides an excellent visual guide for this decision.

Let's summarize the key paths in this flowchart:

• Is Y continuous?
  • If it's symmetric and can take any real value (like temperature or standardized test scores), use the Normal (Gaussian) distribution.
  • If it's strictly positive and right-skewed (like reaction time or total cost), the Gamma distribution is a strong candidate.
• Is Y a count?
  • If it's a count of events (e.g., number of emails received, number of accidents), start with the Poisson distribution.
  • If the variance of the counts is noticeably larger than the mean (a condition called "overdispersion"), the Negative Binomial distribution is more appropriate.
• Is Y binary?
  • If it represents one of two outcomes (e.g., success/failure, yes/no, alive/dead), use the Binomial distribution.

To see these choices in action with practical examples, please watch the following video segment. It directly addresses when and why you would choose each of these common models.

Step 2: Choose the Link Function Based on the Scientific Question

Once you've selected the distribution family, the next step is to choose the link function. As we've discussed, the link function connects the mean of Y, $\mu =E[Y]$, to the linear predictor, $\eta$.

While the canonical link is the default, the choice of link really depends on what you want to measure. The following resource provides an outstanding guide for this.

Here's a summary of the core idea from that reading:

• Want to model differences? Use the identity link.
  • $g(\mu )=\mu$
  • The effect of a predictor is additive. For example, a one-unit increase in X changes the mean of Y by $\beta$ units.
  • Used for: Mean difference, Risk difference.
• Want to model ratios? Use the log link.
  • $g(\mu )=\ln (\mu )$
  • The effect is multiplicative. For example, a one-unit increase in X multiplies the mean of Y by $e^{\beta }$.
  • Used for: Risk Ratio, Rate Ratio, Count Ratio.
• Want to model odds? Use the logit link.
  • $g(\mu )=\ln \left(\frac{\mu }{1-\mu }\right)$
  • The effect is on the log-odds. A one-unit increase in X multiplies the odds of success by $e^{\beta }$.
  • Used for: Odds Ratio (this is the standard for logistic regression).

Putting It All Together: A Summary Table

The table below, also found in one of your resources, provides a fantastic summary that connects the distribution, its typical use, and the most common link functions. This is a great reference to have in your mind for the exam.

Practice Scenarios

Let's apply this two-step framework. For each scenario, identify the most appropriate distribution (family) and link function.

Scenario 1: A city's public health department wants to model the number of influenza cases reported per week. They note that in some weeks the numbers are very high, and the variance of their data is much larger than the average number of cases. They want to know how factors like public transit usage and school holidays affect the rate of new cases.

Scenario 2: A marketing team wants to predict whether a website visitor will click the "Buy Now" button (a yes/no outcome). They want to understand how the visitor's source (e.g., social media, direct search) influences the odds of making a purchase.

Scenario 3: A materials scientist is modeling the time (in hours) until a certain polymer component fails under stress. Time is a continuous, positive-only variable, and the failure times are heavily skewed to the right (many fail early, a few last a long time). The goal is to compare the mean time-to-failure between different manufacturing processes.

Conclusion

You now have a robust framework for one of the most fundamental tasks in applied modeling: selecting the right GLM for a given problem.

Key Takeaways:

• It's a two-step process: First, determine the distribution family based on the nature of your response variable Y (e.g., Normal for symmetric continuous, Poisson for counts, Binomial for binary).
• Second, select the link function based on the scientific question you want to answer (e.g., identity for differences, log for ratios, logit for odds).
• Having this decision strategy ready will allow you to quickly deconstruct exam questions and choose the correct modeling approach.

In the next module, we will dive into the Exponential Family of Distributions. This will provide the mathematical underpinning for why all these different distributions can be handled within the single, unified framework of GLMs. We will then move on to the practical mechanics of how a computer actually estimates the parameters for these models.