Deriving the One-Sample T-Test Statistic

1. From Z-statistic to t-statistic: An Intuitive Leap

Let's begin by building an intuition for why a new distribution is necessary. The Z-statistic has only one source of randomness: the sample mean, $\bar{X}$. The population parameters $\mu$ and $\sigma$, and the sample size $n$, are all constants.

Now, consider the new statistic we form by substituting the sample standard deviation $s$ for $\sigma$:
$t=\frac{\bar{X}-\mu }{s/\sqrt{n}}$
Here, we have two sources of randomness: $\bar{X}$ and $s$. The value of $s$ will vary from one sample to the next. This additional uncertainty, especially when our sample size $n$ is small, means that our new statistic will have more variability than a standard normal variable. Its distribution will be wider and have "heavier" tails.

The following video provides an excellent overview of this concept, introduces the historical context of the t-distribution, and sets the stage for our derivation.

As the video explains, the distribution of this new statistic is the Student's t-distribution. It is characterized by a parameter called "degrees of freedom," which is related to the sample size. As the sample size $n$ increases, our estimate $s$ becomes more reliable and closer to $\sigma$. Consequently, the t-distribution converges to the standard normal distribution.

2. Formal Definition of the t-Distribution

Before we can prove that our statistic follows a t-distribution, we need to formally define it. This definition is based on two well-known distributions: the standard normal and the chi-squared (${\chi }^2$) distribution.

A random variable $T$ is said to follow a Student's t-distribution with $\nu$ (the Greek letter 'nu') degrees of freedom if it can be constructed as follows:
$T=\frac{Z}{\sqrt{U/\nu }}$
where:

1. $Z$ is a standard normal random variable, $Z\sim N(0,1)$.
2. $U$ is a chi-squared random variable with $\nu$ degrees of freedom, $U\sim {\chi }^2(\nu )$.
3. $Z$ and $U$ are independent.

This abstract definition is the mathematical key to our derivation. The following short video states this definition concisely.

3. Deriving the One-Sample t-statistic

Our goal now is to show that the statistic $t=\frac{\bar{X}-\mu }{s/\sqrt{n}}$ perfectly matches the formal definition of a t-distributed random variable. Our derivation will proceed in three steps:

1. Construct the standard normal part ($Z$) from the numerator.
2. Construct the independent chi-squared part ($U$) from the denominator.
3. Combine them algebraically to show they form the required ratio.

Throughout this derivation, we assume we have a random sample $X_1,\dots ,X_n$ drawn from a normal distribution $N(\mu ,{\sigma }^2)$.

Step 1: Constructing the Standard Normal Numerator

This step is straightforward and should be familiar. We know that the sample mean $\bar{X}$ of $n$ i.i.d. random variables from $N(\mu ,{\sigma }^2)$ is also normally distributed:
$\bar{X}\sim N(\mu ,{\sigma }^2/n)$
Standardizing $\bar{X}$ gives us a standard normal random variable, which will be our $Z$:
$Z=\frac{\bar{X}-\mu }{\sigma /\sqrt{n}}\sim N(0,1)$

Step 2: Constructing the Independent Chi-Squared Denominator

This is the most involved part of the proof. It rests on a cornerstone result in mathematical statistics known as Cochran's Theorem. For a random sample from a normal distribution, the theorem implies two crucial facts:

1. The sample mean $\bar{X}$ and the sample variance $s^2=\frac{1}{n-1}\sum (X_i-\bar{X}{)}^2$ are independent random variables.
2. The quantity $\frac{(n-1)s^2}{{\sigma }^2}$ follows a chi-squared distribution with $n-1$ degrees of freedom.
   $\frac{(n-1)s^2}{{\sigma }^2}\sim {\chi }^2(n-1)$

Proving these facts requires a bit of linear algebra and manipulation of moment-generating functions, which should align well with your mathematical background. The following resource provides the necessary proofs.

So, we have found our chi-squared variable: $U=\frac{(n-1)s^2}{{\sigma }^2}$, with $\nu =n-1$ degrees of freedom. We have also established that it is independent of our $Z$ variable (since $Z$ is a function of $\bar{X}$, which is independent of $s^2$).

Step 3: Assembling the t-statistic

Now we have all the components. We just need to perform some algebra to show that $t=\frac{\bar{X}-\mu }{s/\sqrt{n}}$ can be arranged into the required $Z/\sqrt{U/\nu }$ form.

The following resource provides a very clean and direct presentation of this final algebraic step.

Let's walk through that algebra here as well. We start with our statistic:
$t=\frac{\bar{X}-\mu }{s/\sqrt{n}}$
Divide the numerator and the denominator by the constant $\sigma /\sqrt{n}$:
$t=\frac{(\bar{X}-\mu )/(\sigma /\sqrt{n})}{(s/\sqrt{n})/(\sigma /\sqrt{n})}=\frac{\frac{\bar{X}-\mu }{\sigma /\sqrt{n}}}{\frac{s}{\sigma }}$
We immediately recognize the numerator as our $Z\sim N(0,1)$.

Now, let's rewrite the denominator:
$\frac{s}{\sigma }=\sqrt{\frac{s^2}{{\sigma }^2}}$
To get this into the form $\sqrt{U/\nu }$, we multiply the inside of the square root by $\frac{n-1}{n-1}$:
$\sqrt{\frac{s^2}{{\sigma }^2}}=\sqrt{\frac{(n-1)s^2}{{\sigma }^2(n-1)}}=\sqrt{\frac{\frac{(n-1)s^2}{{\sigma }^2}}{n-1}}$
We recognize the numerator inside the square root as our $U\sim {\chi }^2(n-1)$ and the denominator as our degrees of freedom $\nu =n-1$.

Substituting everything back, we get:
$t=\frac{Z}{\sqrt{U/\nu }}$
This expression exactly matches the formal definition of a Student's t-distribution with $n-1$ degrees of freedom. This completes the derivation.

Conclusion

In this lesson, you have rigorously derived one of the most fundamental tools in statistics. By starting with the intuitive problem of an unknown population variance, you have navigated the mathematical details to prove why the one-sample t-test statistic follows the distribution that bears Student's name.

Key Takeaways:

• When the population variance ${\sigma }^2$ is unknown, we use the sample variance $s^2$ as an estimate.
• Substituting $s$ for $\sigma$ in the Z-statistic introduces additional randomness, leading to a new statistic: $t=\frac{\bar{X}-\mu }{s/\sqrt{n}}$.
• This t-statistic follows a Student's t-distribution with $\nu =n-1$ degrees of freedom, which has heavier tails than the standard normal distribution for small $n$.
• The derivation rests on Cochran's Theorem, which establishes that for a normal sample, $\bar{X}$ is independent of $s^2$ and that $\frac{(n-1)s^2}{{\sigma }^2}\sim {\chi }^2(n-1)$.
• The proof is completed by algebraically arranging the t-statistic into the ratio of a standard normal variable and the square root of an independent, scaled chi-squared variable.

Preview of the Next Lesson:

You have now built a solid theoretical foundation for this module. You've explored hypothesis testing frameworks, understood error trade-offs, and derived the crucial t-statistic. In the next lesson, we will put all this knowledge into practice. You will tackle the final project for this module: Perform an A/B test analysis on a sample dataset, from hypothesis formulation to p-value interpretation, using Python's SciPy library. This will be a hands-on application of the t-test you've just derived.