Understanding and Applying T-Tests for Process Mean Comparison

The Core Logic of Hypothesis Testing with t-Tests

Before we dive into the specific types of t-tests, it's essential to understand what they all have in common. At its heart, a t-test calculates a single number—the t-statistic (or t-value)—which is essentially a signal-to-noise ratio.

• The Signal: The difference between your sample mean and the mean you are comparing it to (the null hypothesis).
• The Noise: The variability or "spread" within your sample data (measured by the standard error).

A large t-statistic indicates that the "signal" (the difference) is large compared to the "noise" (the random variation), suggesting the difference is real. But how large is large enough? That's where the p-value comes in.

The p-value is the probability of observing a t-statistic as extreme as, or more extreme than, the one calculated from your data, assuming the null hypothesis is true.

• A small p-value (typically < 0.05) means your observed result is very surprising if the null hypothesis were true. This leads you to reject the null hypothesis and conclude the difference is statistically significant.
• A large p-value (≥ 0.05) means your result is not surprising. You fail to reject the null hypothesis, concluding that the observed difference could be due to random chance.

Let's watch a video that provides a great overview of the different t-tests and the key concepts of the t-value and p-value.

Now, let's apply this logic to the three main types of t-tests you'll use in manufacturing projects.

1. The One-Sample t-Test: Checking Against a Standard

You use a one-sample t-test when you want to compare the mean of a single group against a known target, specification, or historical value.

Manufacturing Scenario:
A CNC machine is set up to cut a specific part to a length of 125 mm. After a maintenance event, you want to verify if the machine is still calibrated correctly. You take a random sample of 15 parts and measure their lengths.

The Question: Is the average length of the parts produced by this machine significantly different from the 125 mm target?

Steps:

1. State the Hypotheses:
   • Null Hypothesis (H₀): The true mean length (μ) is 125 mm. (μ = 125)
   • Alternative Hypothesis (H₁): The true mean length is not 125 mm. (μ ≠ 125). This is a two-tailed test because we care about deviations in either direction (too long or too short).
2. Check Assumptions:
   • The data (length) is continuous.
   • The sample is random.
   • The data is approximately normally distributed (which you would check with a probability plot).
3. Perform the Test: Using statistical software, you input your sample data and the target value (125). The software calculates the t-statistic and the p-value.
4. Interpret the Result:
   • If p < 0.05, you reject H₀. You have statistical evidence that the machine is no longer calibrated to the 125 mm target.
   • If p ≥ 0.05, you fail to reject H₀. You do not have enough evidence to say the machine is off-target.

The image below provides a great visual summary of this process.

2. The Two-Sample t-Test: Comparing Two Independent Groups

This is one of the most common tests in Six Sigma. You use it to compare the means of two separate, independent groups.

Manufacturing Scenario:
You are evaluating two different suppliers (A and B) for a critical raw material. Your CTQ is the material's purity. You take a sample of 10 batches from Supplier A and 10 batches from Supplier B and measure the purity of each.

The Question: Is there a statistically significant difference in the average purity of material from Supplier A versus Supplier B?

Steps:

1. State the Hypotheses:
   • Null Hypothesis (H₀): The mean purity of Supplier A is equal to the mean purity of Supplier B. (μ_A = μ_B)
   • Alternative Hypothesis (H₁): The mean purity is different. (μ_A ≠ μ_B)
2. Check Assumptions:
   • The two groups are independent.
   • Purity is continuous and normally distributed for both groups.
   • The variances of the two groups are roughly equal. (Statistical software often runs a test, like Levene's Test, to check this and can adjust the t-test calculation if they are unequal).
3. Perform the Test and Interpret: As before, you use software to find the p-value and compare it to 0.05.

Let's read through a detailed, practical example from a Six Sigma perspective.

3. The Paired t-Test: Before-and-After Comparisons

The paired t-test is used when you have two sets of measurements that are related to each other. This typically occurs in a "before-and-after" scenario on the same set of items or subjects.

The power of this test comes from a simple trick: instead of analyzing two groups, you calculate the difference for each pair and then perform a one-sample t-test on those differences.

Manufacturing Scenario:
To reduce energy consumption, your team installs new, energy-efficient motors on 12 large mixing machines. You have the average daily kWh consumption data for each specific machine before the change and after the change.

The Question: Did the new motors significantly reduce the average energy consumption?

Steps:

1. Calculate the Differences: For each of the 12 machines, calculate the difference: Energy_Before - Energy_After. This gives you a single set of 12 "improvement" values.
2. State the Hypotheses (on the differences):
   • Null Hypothesis (H₀): The true mean of the differences is zero. (μ_diff = 0)
   • Alternative Hypothesis (H₁): The true mean of the differences is greater than zero. (μ_diff > 0). This is a one-tailed test because we are specifically testing for a reduction in consumption (i.e., a positive improvement).
3. Perform a One-Sample t-Test on the Differences: Test if the mean of your 12 difference values is significantly different from 0.
4. Interpret the Result: If p < 0.05, you reject H₀. You have strong evidence that the new motors achieved a statistically significant reduction in energy consumption.

The following image shows exactly how you would structure this analysis, for instance, in a spreadsheet.

Practical Considerations and Common Pitfalls

Mastering t-tests also means knowing their limitations.

The point about statistical vs. practical significance is vital for an IE. A t-test might show a statistically significant reduction in cycle time of 0.1 seconds (p < 0.05). But if implementing that change costs $50,000, is a 0.1-second improvement practically significant? Probably not. Always use your process knowledge and business acumen to interpret the results.

For further practice, the document "Chapter 2: Statistical Tests, Confidence Intervals and Comparative..." (LINK) provides numerous worked examples and exercises with detailed solutions (starting on page 60). It's an excellent resource if you wish to solidify your understanding of the calculations.

Conclusion

In this lesson, you have learned the practical steps to perform and interpret the three fundamental types of t-tests. This moves you from simply identifying potential causes to statistically verifying them.

Key Takeaways:

• A t-test measures the difference between means (the signal) relative to the variation in the data (the noise).
• The p-value tells you the probability of seeing your result by chance if there were no real difference. A p-value less than your significance level (e.g., 0.05) indicates a statistically significant result.
• One-Sample t-Test: Compares the mean of one group to a fixed target.
• Two-Sample t-Test: Compares the means of two independent groups.
• Paired t-Test: Compares the means of two related measurements (e.g., before-and-after) by running a one-sample test on the differences.
• Always consider practical significance in addition to statistical significance.

Preview of the Next Lesson:

T-tests are powerful for comparing one or two means. But what happens when you need to compare three or more? For instance, how would you determine if there is a significant difference in performance across three different shifts, four machines, or five suppliers? Trying to use multiple t-tests for this is inefficient and increases the risk of error. In our next lesson, we will learn the correct tool for this job: Analysis of Variance (ANOVA).