Skip to main content
Back to path

Welcome to a Grasp lesson

Create a free account to keep studying and create your own study path.

Lesson 7

Choosing the Best Fit: Adjusted R-squared and RSE

Recommended resources

Welcome back. In the previous lesson, we established why Adjusted R-squared is the essential tool for comparing regression models with different numbers of predictors, as it penalizes the inclusion of useless variables.

Today, we will complete your model comparison toolkit. Your exam won't just ask you to pick the model with the highest Adjusted R-squared; it will expect a more nuanced justification. This lesson focuses on combining Adjusted R-squared with the Residual Standard Error (RSE) to make a robust and defensible choice between competing models. Mastering this balance is a hallmark of a strong answer.

The Two Pillars of Model Comparison

When you are given output from several models and asked to choose the "best" one, you need to evaluate them on two key criteria: explanatory power and prediction error.

  1. Adjusted R-squared (): This metric, which we've already covered, measures the proportion of variance in the response variable explained by the model, adjusted for the number of predictors. It is a measure of relative model fit. For your exam, remember: Higher is better.

  2. Residual Standard Error (RSE): This is the other crucial metric. It represents the typical distance that the observed values fall from the regression line. In simpler terms, it's the average size of the prediction error, measured in the original units of your response variable (e.g., dollars, kilograms, etc.). It is a measure of absolute model fit. For your exam, remember: Lower is better.

You can find a concise definition of RSE in this resource.

How to Interpret Regression Output in R

This article provides clear, one-paragraph definitions of the key model fit statistics.

In the section "Assessing Model Fit," read the short definitions provided for Residual standard error, Multiple R-Squared, and Adjusted R-squared. This will reinforce your understanding of what each metric represents.

The Decision Rule: A Balancing Act

The goal is to find a model that is both powerful and precise. Ideally, your chosen model will have the highest Adjusted R-squared and the lowest RSE.

However, you will often face a trade-off. Adding another variable might slightly increase the Adjusted R-squared while only slightly decreasing the RSE. This is where a key statistical concept comes into play: the principle of parsimony.

Principle of Parsimony: All other things being roughly equal, the simpler model (the one with fewer predictors) is preferred. Do not add complexity to a model for trivial gains. A model with one or two powerful predictors is often better than a bloated model with many marginally useful ones, as the latter is more likely to be overfit to the sample data.

Practical Example: Comparing Two Models in R

Let's see how this works in practice. A researcher is trying to predict a Fertility score.

  • model1 uses all available predictors.
  • model2 uses all predictors except one called Examination.

The R output for the key performance metrics of each model is shown below (this is from the glance() function in R's broom package).

Model 1 Output:
adj.r.squared = 0.671, sigma = 7.17

Model 2 Output:
adj.r.squared = 0.671, sigma = 7.17

Here, sigma is the name R uses for the Residual Standard Error (RSE).

Notice that both the Adjusted R-squared and the RSE are identical for both models. model2 achieves the exact same performance as model1 but does so with one fewer variable. According to the principle of parsimony, model2 is the superior choice. It is simpler and just as effective.

Now, let's confirm this interpretation by reading a brief analysis.

Regression Model Accuracy Metrics: R-square, AIC, BIC, ...

This article provides a direct, practical example of comparing two regression models using the exact metrics we are focused on.

In the article, find the section "Comparing regression models performance". Read the author's interpretation in points 1, 2, and 3. Notice how they use Adjusted R-squared (adj.r.squared) and RSE (sigma) together to make a decision, emphasizing the value of a simpler model when performance is equivalent.

How to Interpret (Exam-Style Sentence):

If asked to choose between these two models, your answer should be:

Model 2 is the preferred model. While both models have identical Adjusted R-squared (0.671) and Residual Standard Error (7.17), Model 2 is more parsimonious as it achieves this performance with fewer predictors.

Practice Drill

You are a data analyst for a real estate company. You have built three models to predict house prices (Price, in thousands of dollars).

  • Model A: Predicts Price using Sq_Feet.

    • Adjusted R-squared = 0.68
    • RSE = $58.2k

  • Model B: Predicts Price using Sq_Feet and Num_Bedrooms.

    • Adjusted R-squared = 0.75
    • RSE = $49.1k

  • Model C: Predicts Price using Sq_Feet, Num_Bedrooms, Lot_Size, and Garage_Size.

    • Adjusted R-squared = 0.76
    • RSE = $48.5k

Your Task:

Which model would you recommend to the company? Write a precise, 2-line justification for your choice, balancing model performance with the principle of parsimony.

Solution

Recommended Model: Model B.

Exam-Style Justification:
Model B is the preferred choice. Moving from Model A to Model B yields a substantial increase in Adjusted R-squared (0.68 to 0.75) and a significant decrease in RSE ($58.2k to $49.1k). In contrast, the gains from adding two more predictors in Model C are minimal, making Model B the most effective and parsimonious option.

Conclusion

You now have a complete framework for justifying your choice of a regression model. On your exam, don't just point to one number. Build a robust argument using both metrics.

Key Takeaways for Your Exam:

  • Two Metrics: To compare models, use both Adjusted R-squared (higher is better) and Residual Standard Error (RSE) (lower is better).
  • Look for Balance: The best model strikes an optimal balance between high explanatory power () and low prediction error (RSE).
  • Embrace Parsimony: When faced with very small improvements, always favor the simpler model. A complex model with marginal gains is not worth the risk of overfitting.

In our next lesson, we will shift gears from assessing overall fit to testing for overall significance using the ANOVA table. This will introduce the F-test, a formal hypothesis test that answers the fundamental question: "Is this regression model, as a whole, providing any statistically useful information?"