The Discount Factor: Measuring Player Patience

Introduction

In our last lesson, we discovered the fundamental difference between finitely and infinitely repeated games. You learned that in a finitely repeated Prisoner's Dilemma, the logic of backward induction causes any potential for cooperation to unravel, leading to defection in every round. In contrast, the absence of a final period in an infinitely repeated game creates a "shadow of the future," making cooperation a possibility.

This raises a critical question: how do we evaluate an endless stream of payoffs? If cooperating forever yields a payoff of 3 + 3 + 3 + ..., which is infinity, and being exploited forever yields 0 + 0 + 0 + ..., this is clear. But what if one strategy gives you 4 + 1 + 1 + ... and another gives 3 + 3 + 3 + ...? Both sums still diverge to infinity, making them impossible to compare.

Today's lesson solves this problem. Our learning outcome is to define the discount factor and interpret its role in modeling player patience. You will learn the mathematical tool that allows us to compare infinite payoff streams and understand how it elegantly captures the trade-off between immediate gratification and long-term rewards. This concept is the key that unlocks the analysis of cooperation in all repeated games.

1. The Problem with Infinity and Why We Discount

Before we can analyze strategies in infinitely repeated games, we must first address the mathematical challenge of comparing infinite series of payoffs. A simple sum doesn't work.

The following video explains why we need a more sophisticated approach and introduces the two core justifications for "discounting" future payoffs.

Game Theory 101 (#55): Discount Factors

In this video, William Spaniel clearly articulates the problem with summing infinite payoffs and then presents the two primary reasons why a payoff tomorrow is worth less than the same payoff today.

Watch from the beginning until 03:31. As you watch, focus on: Why simply adding up payoffs in an infinite game leads to a nonsensical comparison. The two key rationales for discounting the future: the time value of consumption and the risk that the game may end.

As the video explained, we resolve the issue of infinite sums by acknowledging that future payoffs are worth less to a player than present ones. This is captured by the discount factor.

2. Defining the Discount Factor

The discount factor is the cornerstone for evaluating payoffs in repeated games.

Definition: The discount factor, denoted by the Greek letter delta ( $δ$ ), is a number such that $0 < δ < 1$ . It represents the weight given to a payoff received one period in the future, relative to a payoff received today.

A payoff of $x$ received today is worth $x$ .
The same payoff $x$ received in the next period is worth $δ x$ in today's terms.
The same payoff $x$ received two periods from now is worth $δ^{2} x$ in today's terms.
...and a payoff $x$ received $t$ periods from now is worth $δ^{t} x$ in today's terms.

This structure allows us to calculate the present value of an entire infinite stream of payoffs $x_{0}, x_{1}, x_{2}, \dots$ as a finite sum:

U = x_{0} + δ x_{1} + δ^{2} x_{2} + δ^{3} x_{3} + \dots = t = 0 \sum \infty δ^{t} x_{t}

Since $δ < 1$ , this sum converges to a finite number, allowing us to meaningfully compare different strategies.

The following video segment formally introduces the symbol $δ$ and demonstrates how it's applied period after period.

Game Theory 101 (#55): Discount Factors

Let's continue with the same video, which now defines the discount factor \delta and shows how the total payoff is constructed.

Watch from 03:25 to 06:33. Note the range of \delta and why the endpoints (0 and 1) are excluded. Pay close attention to how the exponent of \delta increases with each future time period.

The Geometric Series Formula

A particularly useful result, given your mathematics background, is the sum of an infinite geometric series. If a player receives a constant payoff $x$ in every period, the present value is:

U = x + δ x + δ^{2} x + \dots = x (1 + δ + δ^{2} + \dots)

Let $S = 1 + δ + δ^{2} + \dots$ . Then $δ S = δ + δ^{2} + δ^{3} + \dots$ . Subtracting the second equation from the first gives $S - δ S = 1$ , which simplifies to $S (1 - δ) = 1$ , or $S = \frac{1}{1 - δ}$ .

Therefore, the total discounted payoff of receiving $x$ forever is:

U = \frac{x}{1 - δ}

This simple, powerful formula is one you will use constantly when analyzing repeated games. The next video shows a quick application of this formula in the context of a Prisoner's Dilemma.

GTO-5-06: Discounted Repeated Games

This clip from a Game Theory Online lecture shows the geometric series formula in action. Note that the presenter uses the Greek letter beta (\beta) instead of delta (\delta), but the concept is identical.

Watch from 07:54 to 08:20. See how a perpetual stream of payoffs of 3 becomes simply 3 / (1 - \beta).

3. Interpreting the Discount Factor: Patience and Probability

The discount factor $δ$ is more than just a mathematical convenience; it has two deep and related interpretations that are crucial for strategic analysis.

A. Patience

The most common interpretation is that $δ$ represents a player's patience.

A player with a high $δ$ (close to 1) is very patient. They value future payoffs almost as much as present ones. The exponential decay of value is slow.
A player with a low $δ$ (close to 0) is very impatient. They heavily prioritize immediate rewards and disregard the future. The decay of value is rapid.

A player's willingness to cooperate, and thus forgo an immediate temptation to defect, will depend heavily on how patient they are. A patient player is more willing to endure short-term sacrifices for long-term gains.

This graph visually demonstrates the concept. The dashed line ( $δ = 0.9$ ) represents a patient player, whose valuation of the future decays slowly. The solid line ( $δ = 0.75$ ) represents a more impatient player.

Exponential Discounting with Different Discount Factors — This graph shows the value of a future payoff as a function of time (t) for two different discount factors. A higher discount factor ($\delta=0.9$) corresponds to greater patience, as future payoffs retain more value compared to a lower discount factor ($\delta=0.75$).

B. Probability of Continuation

A second, equally valid interpretation is that $δ$ is the probability that the game continues to the next period. After each round, there's a $δ$ chance of playing again and a $1 - δ$ chance the interaction ends forever.

Why does this lead to the same mathematical formula?

The probability of reaching period 1 is $δ$ .
The probability of reaching period 2 is the probability of continuing after period 0 and after period 1, which is $δ \times δ = δ^{2}$ .
The probability of reaching period $t$ is $δ^{t}$ .

The expected payoff from period $t$ is the payoff itself, $x_{t}$ , multiplied by the probability of ever reaching that period, $δ^{t}$ . Summing these expected payoffs gives the exact same formula for present value: $\sum δ^{t} x_{t}$ .

This interpretation is particularly useful because it models many real-world scenarios where the duration of a relationship is uncertain (e.g., business partnerships, international relations).

This dual meaning is one of the most elegant aspects of the discount factor. Whether you think of it as a psychological trait (patience) or a structural feature of the environment (uncertainty), the strategic implications are the same.

4. The Role of $δ$ in Strategic Trade-offs

The discount factor is the critical parameter that determines whether cooperation can be sustained. It mediates the fundamental trade-off in a repeated Prisoner's Dilemma:

Defecting now: Gives a high one-time "temptation" payoff, but may trigger future punishment (e.g., a return to mutual defection forever).
Cooperating now: Gives a moderate "reward" payoff, but keeps the door open to receiving that reward indefinitely.

A player will choose to cooperate only if the present value of the cooperation stream is greater than the present value of defecting and facing the consequences.

The following reading provides a rigorous analysis of this trade-off. It derives the exact condition on $δ$ that makes cooperation possible.

[PDF] Repeated games

This document provides a clear, mathematical derivation of the condition required to sustain cooperation in an infinitely repeated Prisoner's Dilemma. It perfectly illustrates how the discount factor balances the short-term gain from defecting against the long-term loss of cooperation.

Please read the section titled 'Infinitely repeated prisoner’s dilemma' (it's near the end of the document). You can find it by searching for that heading. Focus on the comparison between the payoff stream from cooperating and the payoff stream from defecting. Pay close attention to how the inequality is set up and solved for 'd' (their notation for \delta).

As the document showed, cooperation is sustainable if and only if $δ \geq 1/2$ for that specific game. The key insight is that the required level of patience ( $δ$ ) depends directly on the payoffs of the stage game. If the temptation to defect is higher, or the punishment for defecting is weaker, players will need to be even more patient (have a higher $δ$ ) to resist defecting.

Conclusion

In this lesson, you have been introduced to one of the most important tools in game theory: the discount factor. It is the mechanism that allows us to rigorously analyze strategic interactions that unfold over an indefinite period of time.

Key Takeaways:

The discount factor ( $δ$ ) is a value between 0 and 1 used to calculate the present value of future payoffs, resolving the problem of comparing infinite sums.
The total discounted value of receiving a constant payoff $x$ forever is given by the formula $U = \frac{x}{1 - δ}$ .
$δ$ has two crucial interpretations: as a measure of a player's patience (high $δ$ = patient) and as the probability of continuation to the next period.
The discount factor governs a player's strategic choice by balancing the immediate gain from defection against the discounted value of future cooperation. Cooperation is only possible if players are "sufficiently patient."

We have now established that cooperation is possible and what tool we need to measure the conditions. In our next lesson, we will put this tool to work. We will formally analyze the conditions for sustaining cooperation in an infinitely repeated Prisoner's Dilemma using a grim trigger strategy, deriving the precise threshold for $δ$ that makes cooperation a stable equilibrium.

Recommended resources