Correlation in Game Analytics

Understanding Relationships: Direction and Strength

When we look at two variables, we first want to visualize their relationship. The standard tool for this is a scatter plot, where each point represents an individual observation (like a single player or a single match).

A relationship has two key properties:

1. Direction:

   • Positive: As one variable increases, the other tends to increase. (e.g., More time played, higher player level).
   • Negative: As one variable increases, the other tends to decrease. (e.g., More defensive structures built, fewer successful enemy raids).
   • None: The variables show no discernible pattern.

2. Strength: How closely the data points follow a pattern. A tight, clear line indicates a strong relationship; a diffuse, shapeless cloud suggests a weak one.

We quantify this direction and strength using a single number: the correlation coefficient, denoted as $r$. This value always falls between -1 and +1.

To formalize this a bit, we often distinguish between two types of variables.

• Explanatory Variable (X-axis): The variable we think might be causing or explaining the change (e.g., gold spent on units).
• Response Variable (Y-axis): The variable that measures the outcome (e.g., win rate).

Let's watch a short video to solidify these foundational concepts.

Measuring Correlation: Pearson vs. Spearman

Now that we can visually identify a relationship, how do we calculate the $r$ value? There are several methods, but two are particularly useful for game design.

1. Pearson's Correlation Coefficient (r)

This is the most common type of correlation. It measures the strength of a linear relationship between two continuous variables (e.g., numbers that can take any value, like damage, health, or time).

The formula looks complex, but its logic is intuitive. It essentially calculates how consistently the two variables vary together from their respective means.

Let's explore the logic and assumptions behind Pearson correlation.

A key limitation of Pearson correlation is that it only captures linear relationships. If you have a relationship with diminishing returns (e.g., stacking armor gives less and less effective health benefit), Pearson correlation might underestimate its strength.

2. Spearman's Rank Correlation ($\rho$ or $r_s$)

This is where Spearman correlation comes in. Instead of using the raw data values, it first converts them to ranks (1st, 2nd, 3rd, etc.) and then performs a Pearson correlation on those ranks.

This makes it perfect for two scenarios:

1. When the relationship is monotonic but not linear (i.e., it consistently increases or decreases, but not in a straight line).
2. When your data is already in ranks (ordinal data), like player leaderboards, league tiers (Bronze, Silver, Gold), or item rarity.

This method is highly relevant to your goal of modeling player skill, which is often represented by a rank or rating.

Let's see how it works.

Correlation in Practice: From Data to Design Insights

Calculating an $r$ value is only the first step. The real power comes from using it to make design decisions.

Application 1: Identifying Key Metrics for Winning

In a complex game like an RTS or Grand Strategy title, hundreds of variables could contribute to a win. Correlation analysis helps you identify which ones matter most. By correlating various gameplay metrics (like resource income, units lost, technology researched) with the outcome (win/loss), you can discover the strongest predictors of success.

This is exactly what was done in an analysis of professional League of Legends matches.

This paper demonstrates how correlation coefficients aren't just abstract numbers; they become weighting factors in a practical model for predicting outcomes, a technique directly applicable to your goal of creating more rigorous balance frameworks.

Application 2: Validating a Model of Player Skill

You've expressed a strong interest in modeling abstract concepts like player skill. Correlation is a key tool for validating such models. If you build a metric intended to represent skill, you can check its validity by correlating it with an external measure of skill, like player rank or self-reported survey data.

Researchers did this when developing a system to measure player engagement in PlayerUnknown’s Battlegrounds (PUBG).

The Most Important Caveat: Correlation ≠ Causation

This is the single most important principle in statistical analysis. Just because two variables are correlated does not mean one causes the other. There might be a third, unmeasured variable (a lurking variable) causing both.

The classic example is the strong positive correlation between ice cream sales and shark attacks. Does eating ice cream cause shark attacks? No. The lurking variable is hot weather, which causes more people to buy ice cream and more people to go swimming.

In game design, you might find a strong correlation between players who buy a specific cosmetic item and a higher win rate. This doesn't mean the item is "pay-to-win." It's more likely that highly engaged, high-skill players (the lurking variable) are both more likely to win and more likely to buy cosmetic items.

Understanding this distinction is critical to avoid making incorrect balance changes based on spurious correlations.

Conclusion and Next Steps

Today we've added a fundamental technique to your analytical arsenal. You can now go beyond looking at metrics in isolation and begin to map the web of relationships that define your game's dynamics.

Key Takeaways:

• Correlation measures the strength and direction of a relationship between two variables, quantified by the correlation coefficient $r$ (from -1 to +1).
• Scatter plots are the primary tool for visualizing these relationships.
• Pearson correlation is used for linear relationships between continuous variables.
• Spearman correlation is used for ranked data or non-linear, monotonic relationships.
• Correlation is a powerful tool for identifying key performance indicators and validating models of abstract concepts like skill.
• Crucially, correlation does not imply causation. Always be on the lookout for lurking variables before drawing conclusions.

In our next lesson, we will discuss the Law of Large Numbers. This principle helps explain why the theoretical probabilities and expected values we design into our games (like a 20% crit chance) actually manifest as stable, predictable outcomes when observed across thousands or millions of player actions. It's the concept that ties our design intent to real-world player experience.