Negative Binomial Distribution: Exponential Family Form

The Negative Binomial (NB2) Distribution

Like many distributions, the Negative Binomial has several different parameterizations. For the purpose of GLMs, the most useful one is the NB2 parameterization, which is defined by its mean $\mu$ and a dispersion parameter. We will use the parameterization from the exam-style problem you've seen, where the dispersion parameter is denoted by $\gamma$. The size parameter, often called $r$ or $k$, is related to $\gamma$ by $r=1/\gamma$.

The PMF of a Negative Binomial random variable $Y$ with mean $\mu$ and dispersion $\gamma$ is:

This formula looks intimidating, but our goal is to show it belongs to the exponential family. The method is the same as before: algebraically manipulate it to fit the standard form.

Converting the Negative Binomial PMF

The algebraic conversion of the Negative Binomial PMF is very similar to what you've done for other distributions. It involves taking the logarithm, grouping terms, and identifying the components.

To see this process demonstrated, the following video provides a clear, step-by-step walkthrough. A quick heads-up: the video uses a slightly different but common parameterization of the Negative Binomial PMF. Don't worry about the difference in the starting formula—the algebraic strategy and logic for identifying the components are exactly the same.

The NB2 Components in Exponential Family Form

As you saw in the video, the core of the conversion is to express the PMF in the form:

For the standard NB2 parameterization with mean $\mu$ and dispersion $\gamma =1/r$, if we perform the same algebraic steps, we arrive at the following components. For this conversion, we treat the dispersion parameter $\gamma$ (and thus $r$) as a known constant. This makes the Negative Binomial a member of the exponential family.

1. Natural Parameter ($\theta$): This is defined by the term that multiplies $y$ in the log-PMF.

   $$\theta =\ln \left(\frac{\gamma \mu }{1+\gamma \mu }\right)$$

   This is the canonical parameter for the Negative Binomial distribution.

2. Cumulant Function ($b(\theta )$): This function is derived from the parts of the PMF that depend on $\mu$ but not on $y$. After substituting $\mu$ in terms of $\theta$, we get:

   $$b(\theta )=-r\ln (1-e^{\theta })=-\frac{1}{\gamma }\ln (1-e^{\theta })$$

3. Dispersion Function ($a(\varphi )$): Since we are treating the dispersion parameter $\varphi =\gamma$ as known for this derivation, the function $a(\varphi )$ is simply:

   $$a(\varphi )=1$$

4. Normalizing Function ($c(y,\varphi )$): These are the remaining terms, which depend on the data $y$ and the dispersion parameter $\varphi =\gamma$.

   $$c(y,\varphi )=\ln \left(\frac{\Gamma (y+1/\gamma )}{\Gamma (y+1)\Gamma (1/\gamma )}\right)$$

This set of components precisely defines the Negative Binomial (NB2) distribution within the exponential family framework.

Connecting to the Exam-Style Problem

Let's look at the problem sheet you'll be studying. Part (a) asks you to do exactly what we just did.

The question provides the NB2 PMF and asks you to show it is in the exponential family, identifying the components. The components we just derived are the solution to part (a) of this problem. Understanding this derivation is a direct answer to a potential exam question.

Conclusion

In this lesson, you've completed your survey of key distributions in the exponential family by tackling the Negative Binomial (NB2) distribution. You learned how to convert its PMF into the standard form and identified its key components.

Key Takeaways:

• The Negative Binomial (NB2) distribution is essential for modeling overdispersed count data.
• Its PMF can be converted to the exponential family form: $\exp \left(y\theta -b(\theta )+c(y,\varphi )\right)$, assuming the dispersion parameter $\varphi =\gamma$ is known.
• The components for the NB2 distribution are:
  • Natural Parameter: $\theta =\ln (\frac{\gamma \mu }{1+\gamma \mu })$
  • Cumulant Function: $b(\theta )=-(1/\gamma )\ln (1-e^{\theta })$
  • Dispersion Function: $a(\varphi )=1$ (as $\varphi$ is considered fixed)

You have now successfully converted all the key distributions for your exam—Binomial, Poisson, Normal, Gamma, and Negative Binomial—into the exponential family form. This algebraic skill provides the theoretical foundation for how GLMs work.

In the next lesson, we will shift from derivation to application. You'll learn a practical decision-making process for choosing the most appropriate distribution for a response variable based on its characteristics (e.g., mean, variance, domain). This is a critical skill for tackling application-based exam questions.