Understanding Distributions for Statistical Testing

The Student's t-Distribution: Handling Small Samples

In the last lesson, we discussed the Normal distribution, which is central to statistics. However, its use in testing often assumes we know the population's standard deviation ($\sigma$). In reality, we almost never do. We usually have to estimate it using the sample standard deviation ($s$).

When our sample size is large (typically n > 30), this estimation is reliable enough that we can still use the normal distribution (via a z-statistic). But what happens when our sample is small? Using the sample standard deviation introduces extra uncertainty. The t-distribution is designed to handle precisely this situation.

To understand the motivation and core conditions for using the t-distribution, let's watch this video from Gate Smashers. It's geared towards a GATE audience and explains the concepts clearly.

Key Properties of the t-Distribution

Let's summarize the crucial points from the video:

1. When to use it: The t-distribution is used for inference about a population mean when:

   • The sample size is small (conventionally n < 30).
   • The population standard deviation ($\sigma$) is unknown.
   • The underlying population is assumed to be normally distributed.

2. Shape and "Fatter Tails": Like the normal distribution, the t-distribution is bell-shaped and symmetric around zero. However, it has fatter tails. This means it assigns more probability to outcomes that are far from the mean, accounting for the increased uncertainty from a small sample.

This image provides a perfect visual comparison:

3. Degrees of Freedom (df): The t-distribution is not a single distribution but a family of them. A specific t-distribution is defined by one parameter: degrees of freedom (df). For a single sample, df = n - 1, where n is the sample size.

   • As df (and thus the sample size n) increases, the t-distribution gets closer and closer to the standard normal distribution. By the time n is around 30, they are virtually identical.

4. The t-statistic: The formula for the test statistic looks very similar to the z-statistic, but it uses the sample standard deviation ($s$) instead of the population standard deviation ($\sigma$).
   $t=\frac{\bar{x}-\mu }{s/\sqrt{n}}$
   Where $\bar{x}$ is the sample mean, $\mu$ is the population mean, $s$ is the sample standard deviation, and $n$ is the sample size.

The Chi-Squared (${\chi }^2$) Distribution: Testing Frequencies and Fit

The Chi-squared distribution has a very different origin and purpose. It is not typically used for testing population means. Instead, it forms the basis for tests involving categorical data, such as:

• Goodness-of-Fit tests: Does our observed data fit a particular theoretical distribution?
• Tests of Independence: Are two categorical variables (e.g., "gender" and "voting preference") related or independent?

Let's watch this Khan Academy video to understand what the Chi-squared distribution fundamentally is.

Key Properties of the Chi-Squared Distribution

Here are the essential takeaways:

1. Fundamental Definition: A Chi-squared variable with $k$ degrees of freedom is the sum of the squares of $k$ independent standard normal random variables.
   ${\chi }_k^2=Z_1^2+Z_2^2+\cdots +Z_k^2$
   Where each $Z_i$ is a random variable from a standard normal distribution. Because it's a sum of squares, a ${\chi }^2$ value can never be negative.

2. Shape: The ${\chi }^2$ distribution is not symmetric. It is skewed to the right. Its exact shape depends entirely on its degrees of freedom.

3. Degrees of Freedom (df): As with the t-distribution, the ${\chi }^2$ distribution is a family of curves defined by degrees of freedom ($k$).

   • For small df, the distribution is heavily skewed.
   • As df increases, the distribution becomes more spread out and more symmetric, eventually approaching a normal distribution.

This image clearly illustrates how the shape changes with the degrees of freedom.

4. Key Properties for GATE:
   • Mean: $\mu =k$ (the degrees of freedom)
   • Variance: ${\sigma }^2=2k$ (twice the degrees of freedom)

Summary for Quick Revision

For a consolidated text-based summary that you can use for revision, the following resource provides a clear and concise overview of both distributions.

As an optional exercise to test your recognition, you can try the multiple-choice questions in the AnalystNotes resource [ID: [LINK](https://analystnotes.com/cfa-study-notes-students-t-distribution-chi-square-and-f-distributions.html)]. Focus on questions 3, 6, and 8 which are directly related to today's topic.

Conclusion

In this lesson, we added two vital statistical distributions to your toolkit, focusing on their practical purpose.

Key Takeaways:

• t-distribution: A bell-shaped, symmetric distribution used for inference on means when the sample size is small (n < 30) and the population standard deviation is unknown. It has "fatter tails" than the normal distribution to account for greater uncertainty. It is defined by degrees of freedom ($df=n-1$).
• Chi-squared (${\chi }^2$) distribution: A right-skewed distribution used for tests involving categorical data (goodness-of-fit, independence). It is formed by the sum of squared standard normal variables. It is also defined by degrees of freedom ($k$).

Preview of the Next Lesson:
We've now seen how sample size plays a critical role in choosing a distribution (t vs. normal). The next lesson will explore the powerful theory that underpins this: the Central Limit Theorem (CLT). The CLT explains why the normal distribution is so ubiquitous in statistics and how it allows us to make inferences about a population from a sample, which is the foundation of all hypothesis testing.