Understanding Regression Output

1. A Roadmap for Interpreting Regression Output

When you run a regression analysis in Excel, it generates a SUMMARY OUTPUT that can look intimidating. It's best to think of it in three distinct sections, each answering a different question.

Our process will be:

1. ANOVA: Check if the model is significant overall.
2. Regression Statistics: Assess how well the model fits the data.
3. Coefficients: Determine which specific drivers are significant.

Let's begin with the most fundamental question: is our model even useful?

2. Is the Model Statistically Significant? The ANOVA Table

The ANOVA (Analysis of Variance) table tests the overall significance of your model. It answers the question: "Is my model, with all its drivers, better at predicting the outcome than simply guessing the average every time?"

The most important number here is Significance F.

• Significance F: This is the p-value for the entire model. It tests the null hypothesis that all of your independent variables together have no effect on the dependent variable (i.e., all their coefficients are zero).

The Rule of Thumb: If Significance F < 0.05, you can reject the null hypothesis. This means your model is statistically significant, and your independent variables collectively have a real relationship with the dependent variable.

If Significance F is high (e.g., > 0.05), your model is likely not useful, and you shouldn't proceed with interpreting its other stats.

The following reading provides a great summary of the ANOVA section and the other components we will discuss.

3. How Well Does the Model Fit? Regression Statistics

Once you've confirmed your model is significant (Significance F < 0.05), the next question is about its explanatory power. The Regression Statistics table helps with this. The key metric here is R-squared.

• R-Square ($R^2$): This value tells you the proportion of the variance in the dependent variable that is explained by your model. It ranges from 0 to 1 (or 0% to 100%). An $R^2$ of 0.75 means that 75% of the variation in your sales, for example, can be explained by the drivers in your model.
• Adjusted R-Square: This is a modified version of $R^2$ that accounts for the number of independent variables in your model. It's a more honest measure of model fit, as it penalizes you for adding "useless" drivers that don't improve the model. For multiple regression, you should almost always use Adjusted R-Square.

Let's get the formal definitions from our reading.

The "Good R-squared" Trap

A common question is, "What's a good Adjusted R-squared?" It's tempting to think higher is always better, but the reality is far more nuanced. An experienced analyst knows that the answer is: "It depends."

In some fields like physics, you might expect $R^2$ values over 95%. In finance or social sciences, where human behavior introduces a lot of noise, an $R^2$ of 40% might be very insightful.

This next reading from a Duke University professor is a classic on the topic and provides the kind of critical perspective valued in high-level finance roles.

Key takeaways from this critical perspective:

• There is no universal "good" R-squared value. It depends on your industry, your data, and your goals.
• A very high $R^2$ (e.g., 97%) can be a red flag, often indicating you are regressing two variables that share a common trend (like auto sales and personal income over time), not a true causal relationship. This is a common issue with time-series data.
• A low $R^2$ can still be very valuable if it reveals a weak but statistically significant signal in very noisy data.
• The standard error of the regression (also in the Regression Statistics table) is often a more practical measure of model accuracy than R-squared because it's in the same units as your dependent variable. A lower standard error means more precise predictions.

4. Which Drivers are Significant? The Coefficients Table

You've confirmed the model is valid overall and assessed its fit. Now, it's time to zoom in on the individual drivers in the Coefficients table. In the last lesson, we used the Coefficients column. Now, we focus on the P-value.

• P-value: For each independent variable, the p-value tests the null hypothesis that its coefficient is zero. In practical terms, it tells you the probability that the driver has no real effect on the outcome, and the relationship you're seeing is just due to random chance.

The Rule of Thumb: If a driver's p-value < 0.05, you can reject the null hypothesis and conclude that the driver is statistically significant. If the p-value > 0.05, the driver is likely not significant, and you should consider removing it from your model.

Removing insignificant variables makes your model simpler, more robust, and easier to explain to business partners.

5. Case Study: Interpreting an Example Output

Let's apply everything we've learned to a practical example. Imagine you've run a regression to predict a student's final score based on hours studied and the number of prep_exams taken. You get the following output:

Here is how an analyst would interpret this:

1. Check Overall Significance (ANOVA): Look at Significance F. The value is 2.95E-06, which is 0.00000295. This is far less than 0.05.

   • Conclusion: The model is statistically significant and has predictive value.

2. Assess Overall Fit (Regression Statistics): Look at Adjusted R-Square. The value is 0.703.

   • Conclusion: The model explains approximately 70.3% of the variance in student scores. This is a reasonably strong fit. The Standard Error is 10.3, meaning individual predictions are, on average, off by about 10.3 points.

3. Evaluate Individual Drivers (Coefficients): Look at the P-value for each variable.

   • Intercept: P-value is 0.0001. Significant.
   • hours: P-value is 0.000006. Highly significant.
   • prep_exams: P-value is 0.52. This is much greater than 0.05.
   • Conclusion: The number of hours studied is a significant predictor of the final score. However, prep_exams is not a statistically significant predictor.

4. Action & Final Equation: Based on this analysis, the prep_exams variable should be removed from the model. You would then re-run the regression using only hours as the independent variable. The coefficient for hours (5.56) is meaningful, while the coefficient for prep_exams (-0.60) is not reliable because its p-value is too high.

Conclusion

You now have the complete toolkit to not only build a regression model but to critically evaluate it like a professional analyst. This ability to separate signal from noise is what distinguishes a simple number-cruncher from a strategic financial partner.

Key Takeaways:

• Always start by checking Significance F. If it's not < 0.05, your model is not statistically valid.
• Use Adjusted R-Square to understand the model's explanatory power, but remember that a "good" value is context-dependent.
• Scrutinize the p-value of each driver. Keep the ones with p-values < 0.05 and discard the rest.
• Building a regression model is an iterative process. Your first model is rarely your last. Refine it based on your interpretation of the output.

In our next lesson, we will take our validated regression equation and learn how to formally integrate it into a larger, driver-based financial forecast model in Excel, preparing it for scenario analysis and real-world application.