Slutsky's Theorem: Combining Convergences

Statement of the Theorem

At its core, Slutsky's theorem addresses the limiting behavior of combinations of random sequences. Let's say we have one sequence of random variables, $X_n$, that converges in distribution to a random variable $X$. We also have another sequence, $Y_n$, that converges in probability to a constant, $c$. What can we say about the distribution of $X_n+Y_n$ or $X_n{Y}_n$?

Slutsky's theorem provides the answer.

Theorem (Slutsky's Theorem):
If $X_n\stackrel{d}{\to }X$ and $Y_n\stackrel{p}{\to }c$, where $c$ is a constant, then:

1. Addition: $X_n+Y_n\stackrel{d}{\to }X+c$
2. Multiplication: $X_n{Y}_n\stackrel{d}{\to }cX$
3. Division: $X_n/Y_n\stackrel{d}{\to }X/c$, provided $c\ne 0$.

A common and important special case is when the limit $X$ is a random variable and another sequence $A_n\stackrel{p}{\to }a$. Then $A_n{X}_n+Y_n\stackrel{d}{\to }aX+c$.

The following video provides a clear statement of the theorem and immediately moves to a classic application.

It is crucial to distinguish Slutsky's theorem from the Continuous Mapping Theorem (CMT).

• CMT: Deals with applying a single continuous function to a single convergent sequence. If $X_n\stackrel{d}{\to }X$, then $g(X_n)\stackrel{d}{\to }g(X)$.
• Slutsky's Theorem: Deals with applying arithmetic operations between two different sequences that have different modes of convergence ($\stackrel{d}{\to }$ and $\stackrel{p}{\to }$).

Canonical Application: The Asymptotic Distribution of the t-statistic

Perhaps the most common and intuitive application of Slutsky's theorem is in establishing the asymptotic distribution of the Student's t-statistic.

Let $X_1,\dots ,X_n$ be i.i.d. random variables with mean $\mu$ and finite variance ${\sigma }^2$. The Central Limit Theorem tells us about the standardized sample mean:
$Z_n=\frac{{\bar{X}}_n-\mu }{\sigma /\sqrt{n}}=\sqrt{n}\frac{({\bar{X}}_n-\mu )}{\sigma }\stackrel{d}{\to }N(0,1)$
In practice, $\sigma$ is almost always unknown. We replace it with a consistent estimator, the sample standard deviation $S_n$, to form the t-statistic:
$t_n=\frac{{\bar{X}}_n-\mu }{S_n/\sqrt{n}}=\sqrt{n}\frac{({\bar{X}}_n-\mu )}{S_n}$
How do we find the limiting distribution of $t_n$? We can rewrite it as a product:
$t_n=\left(\sqrt{n}\frac{({\bar{X}}_n-\mu )}{\sigma }\right)\times \left(\frac{\sigma }{S_n}\right)$

Let's analyze the two components:

1. First term: As we just stated from the CLT, $\sqrt{n}\frac{({\bar{X}}_n-\mu )}{\sigma }\stackrel{d}{\to }N(0,1)$. This is our $X_n$.
2. Second term: By the Weak Law of Large Numbers, the sample variance $S_n^2$ is a consistent estimator for the population variance ${\sigma }^2$. That is, $S_n^2\stackrel{p}{\to }{\sigma }^2$. Since $g(x)=\sqrt{x}$ is a continuous function (for $x>0$), the Continuous Mapping Theorem implies $S_n=\sqrt{S_n^2}\stackrel{p}{\to }\sqrt{{\sigma }^2}=\sigma$. Applying the CMT again with $h(y)=\sigma /y$, we get $\frac{\sigma }{S_n}\stackrel{p}{\to }\frac{\sigma }{\sigma }=1$. This is our $Y_n$ converging to the constant $c=1$.

We now have exactly the setup for Slutsky's theorem:
$t_n=\underset{X_n\stackrel{d}{\to }N(0,1)}{\underbrace{\left(\sqrt{n}\frac{({\bar{X}}_n-\mu )}{\sigma }\right)}}\times \underset{Y_n\stackrel{p}{\to }1}{\underbrace{\left(\frac{\sigma }{S_n}\right)}}$
By the multiplication rule of Slutsky's theorem, the product converges in distribution to the product of the limits:
$t_n\stackrel{d}{\to }N(0,1)\times 1=N(0,1)$
This justifies the use of the standard normal distribution for inference on the mean with large samples, even when the population variance is unknown.

The following resources walk through this exact argument.

Generalization: Studentization

The principle we used for the t-statistic is known more generally as Studentization. It refers to the process of dividing a statistic by a consistent estimate of its scale (standard deviation). Slutsky's theorem is the theoretical backbone that guarantees this process works asymptotically.

In general, if you have a statistic $T_n$ such that under a null hypothesis:
$\frac{\sqrt{n}(T_n-{\theta }_0)}{\sigma ({\theta }_0,\nu )}\stackrel{d}{\to }Z$
where $Z$ is some limiting distribution (e.g., $N(0,1)$) and $\sigma$ is a scale parameter that may depend on ${\theta }_0$ and other nuisance parameters $\nu$. If you can find a consistent estimator ${\hat{\sigma }}_n\stackrel{p}{\to }\sigma ({\theta }_0,\nu )$, then Slutsky's theorem ensures that the "Studentized" statistic has the same limiting distribution:
$\frac{\sqrt{n}(T_n-{\theta }_0)}{{\hat{\sigma }}_n}=\underset{\stackrel{d}{\to }Z}{\underbrace{\left(\frac{\sqrt{n}(T_n-{\theta }_0)}{\sigma ({\theta }_0,\nu )}\right)}}\times \underset{\stackrel{p}{\to }1}{\underbrace{\left(\frac{\sigma ({\theta }_0,\nu )}{{\hat{\sigma }}_n}\right)}}\stackrel{d}{\to }Z$
This is a powerful and frequently used technique for constructing test statistics and confidence intervals when nuisance parameters are present.

A More Advanced Application: Asymptotic Normality of the MLE

Slutsky's theorem is also the crucial final step in one of the most celebrated results of statistical theory: the asymptotic normality of the Maximum Likelihood Estimator (MLE).

The proof, which you've seen in the context of your physics research in other forms (e.g., Laplace's method for integrals), involves a Taylor expansion of the log-likelihood's derivative (the score function) around the true parameter ${\theta }_0$. This leads to the following approximate relationship for the MLE ${\hat{\theta }}_n$:
$\sqrt{n}({\hat{\theta }}_n-{\theta }_0)\approx {\left(-\frac{1}{n}{\sum }_{i=1}^n\frac{{\partial }^2\log f(X_i;{\theta }_0)}{\partial {\theta }^2}\right)}^{-1}\left(\frac{1}{\sqrt{n}}{\sum }_{i=1}^n\frac{\partial \log f(X_i;{\theta }_0)}{\partial \theta }\right)$
Let's analyze the two main components on the right-hand side under standard regularity conditions:

1. The scaled score vector: This is a sum of i.i.d. random variables (the scores), each with mean zero. By the CLT, it converges in distribution to a normal distribution:
   $\frac{1}{\sqrt{n}}\sum \frac{\partial \log f}{\partial \theta }\stackrel{d}{\to }N(0,i({\theta }_0))$
   where $i({\theta }_0)$ is the Fisher information.

2. The average Hessian matrix: This is an average of i.i.d. random variables. By the Law of Large Numbers, it converges in probability to its expectation:
   $-\frac{1}{n}\sum \frac{{\partial }^2\log f}{\partial {\theta }^2}\stackrel{p}{\to }-E\left[\frac{{\partial }^2\log f}{\partial {\theta }^2}\right]=i({\theta }_0)$

The expression for $\sqrt{n}({\hat{\theta }}_n-{\theta }_0)$ is the product of the inverse of the second term and the first term. Slutsky's theorem allows us to combine these limits: the first term converges in distribution, while the inverse of the second term converges in probability to $i({\theta }_0{)}^{-1}$. The result is that $\sqrt{n}({\hat{\theta }}_n-{\theta }_0)$ converges in distribution to $N(0,i({\theta }_0{)}^{-1})$.

This next reading shows exactly where "Slutsky's lemma" is invoked to complete this foundational proof.

Visual Example: The F-statistic

As a final example, consider the F-statistic used in linear regression to test joint hypotheses. Asymptotically, a scaled version of this statistic converges to a Chi-squared distribution.

This provides another clear example of the theorem's product rule in action, similar to the t-statistic but leading to a different limiting distribution.

Conclusion

In this lesson, we demystified Slutsky's theorem and established its central role in asymptotic statistics. It provides the formal rules for manipulating sequences of random variables that converge in different modes.

Key Takeaways:

• Slutsky's Theorem allows you to find the limit in distribution of arithmetic combinations of random variables, where one sequence converges in distribution ($\stackrel{d}{\to }$) and the other converges in probability ($\stackrel{p}{\to }$) to a constant.
• The most common application is Studentization, where an unknown variance parameter in a test statistic is replaced by a consistent estimate. The asymptotic normality of the t-statistic is the canonical example.
• The theorem is a critical component in proving fundamental results, such as the asymptotic normality of the Maximum Likelihood Estimator, where it combines a CLT result for the score with a WLLN result for the Hessian.

Preview of the Next Lesson:

In our discussion of the MLE, we introduced the Fisher Information, $i(\theta )$, as both the variance of the score and the expected negative Hessian. This quantity is central to understanding the limits of statistical estimation. In the next module, we will begin by exploring the concept of sufficiency. We will define what makes a statistic "sufficient" for a parameter, which means it captures all the information about that parameter contained in the sample. This will lay the groundwork for a deeper investigation into Fisher information and the theory of optimal estimation.