Modeling Belief Evolution with Bayesian Updating

Introduction

In our last lesson, we explored Michael Spence's model of costly signaling, establishing how significant, hard-to-fake investments can credibly separate different "types" of partners. We determined the range of costs that make a signal effective, but we did so by relying on the intuitive idea that the couple updates their beliefs after observing the signal. Today, we will formalize this critical process.

This lesson focuses on using Bayesian updating to model precisely how a partner's beliefs about another's preferences or intentions evolve after observing their actions. Given your Master's in Statistics and background in Bayesian Decision Theory, you are already an expert in the mechanics of Bayes' theorem. Our goal today is not to re-learn the formula, but to apply it rigorously within the structure of a signaling game. We will define the concept of a Perfect Bayesian Equilibrium (PBE), which combines players' strategies with a system of beliefs, and see how this framework allows us to mathematically trace the evolution of belief in response to signals, both expected and unexpected.

This iterative process of belief revision is captured perfectly in the diagram below, showing how a "Prior" belief is refined into a "Posterior" after each observed action.

This diagram conceptualizes the core of our lesson: a Bayesian agent observes an opponent's actions (like Cooperate 'C' or Defect 'D') and uses Bayes' theorem to update their internal probability distribution—their beliefs—about the opponent's strategy or type.

Perfect Bayesian Equilibrium: Strategies and Beliefs

In the dynamic games of incomplete information that characterize so many relationship negotiations, we need a solution concept more refined than a simple Nash Equilibrium. The Perfect Bayesian Equilibrium (PBE) provides this by integrating two key components:

Strategies: A strategy for each player, specifying the action they will take at every information set where they might have to move.
Beliefs: A belief for each player, specifying their probability assessment over the nodes within every information set. In simpler terms, at any point where a player is uncertain about the history of the game (e.g., the other player's "type"), they must have a probabilistic belief about where they are.

These two components are linked by two conditions:

Sequential Rationality: Given their beliefs, players' strategies must be optimal at every information set. A player must choose the action that maximizes their expected utility, based on what they currently believe and how they expect others to play from that point forward.
Belief Consistency: Beliefs must be consistent with the strategies. Wherever possible, beliefs are determined by Bayes' rule. This is the heart of our lesson today.

Let's unpack this with a scenario.

Scenario: Interpreting a Potential Partner's Vibe

A couple, you and your partner, are meeting a potential "Bull," Alex, for the first time. You are trying to figure out if his underlying "type" is more Dominant ( $T_{D}$ ) or more Submissive ( $T_{S}$ ). From his dating profile and initial texts, you have a prior belief that there's a 50% chance of either: $P (T_{D}) = 0.5$ and $P (T_{S}) = 0.5$ .

During the conversation, Alex (the Sender) has to choose a communication style. He can be Assertive (Signal A) or Deferential (Signal D). You (the Receiver) observe this signal and must then decide whether to steer the conversation towards a more Direct negotiation of desires or keep it Casual.

This setup is a classic signaling game, which can be visualized on a game tree like the one below.

A generic game tree for a two-type, two-signal signaling game. "Nature" chooses the Sender's type ($t_1$ or $t_2$). The Sender observes their type and sends a message ($m_1$ or $m_2$). The Receiver observes the message (but not the type) and chooses an action ($a_1$ or $a_2$), leading to payoffs for both.

Updating Beliefs on the Equilibrium Path

The "equilibrium path" refers to all the actions and outcomes that have a positive probability of occurring if players follow their equilibrium strategies. On this path, updating beliefs is a direct application of Bayes' rule.

Case 1: A Separating Equilibrium

Suppose a separating equilibrium exists where a Dominant Alex always chooses the Assertive signal, and a Submissive Alex always chooses the Deferential signal.

Strategy: $s (T_{D}) = A$ , $s (T_{S}) = D$

You observe Alex being Assertive (Signal A). What is your new, posterior belief that he is the Dominant type, $P (T_{D} ∣ A)$ ?

Using Bayes' rule:

P (T_{D} ∣ A) = \frac{P ( A ∣ T _{D} ) \cdot P ( T _{D} )}{P ( A )}

Let's break this down:

$P (A ∣ T_{D}) = 1$ , as the Dominant type always plays Assertive in this equilibrium.
$P (T_{D}) = 0.5$ , your prior belief.
$P (A)$ is the total probability of observing Signal A. We find this using the law of total probability: $P (A) = P (A ∣ T_{D}) P (T_{D}) + P (A ∣ T_{S}) P (T_{S})$ In this strategy, $P (A ∣ T_{S}) = 0$ . So, $P (A) = (1) (0.5) + (0) (0.5) = 0.5$ .

Plugging it all back in:

P (T_{D} ∣ A) = \frac{1 \cdot 0.5}{0.5} = 1

Upon observing the Assertive signal, your belief that Alex is Dominant updates from 50% to 100%. The signal has perfectly revealed his type.

The following video provides an excellent, methodical walkthrough of finding a separating equilibrium, paying close attention to how beliefs are updated via Bayes' rule on the path.

Game Theory 101 (#78): Separating Equilibrium

Watch this segment from William Spaniel's "Game Theory 101" series to see Bayesian updating in action within a PBE.

First, familiarize yourself with the four-step method for finding separating equilibria. Then, pay very close attention to the section on posterior beliefs, where the presenter explicitly uses Bayes' rule to show how observing a specific action leads the player to believe the opponent is a certain type with probability 1. Finally, see how this updated belief makes the subsequent choice sequentially rational.

The Challenge: Updating Beliefs Off the Path

What happens when you observe an action that should not happen in equilibrium? This is an "off-the-equilibrium-path" event.

Case 2: A Pooling Equilibrium

Suppose a pooling equilibrium exists where both types of Alex find it optimal to be Deferential.

Strategy: $s (T_{D}) = D$ , $s (T_{S}) = D$

If you observe the Deferential signal (Signal D), your belief does not change. $P (T_{D} ∣ D) = P (T_{D}) = 0.5$ . The signal provides no new information.

But what if you observe Alex being Assertive? According to the equilibrium strategies, this should never happen. If we try to apply Bayes' rule to find $P (T_{D} ∣ A)$ , the denominator $P (A)$ would be zero, which is undefined.

P (A) = P (A ∣ T_{D}) P (T_{D}) + P (A ∣ T_{S}) P (T_{S}) = (0) (0.5) + (0) (0.5) = 0

This is a fundamental challenge. The theory of PBE handles this by stating that for information sets off the equilibrium path, beliefs are not constrained by Bayes' rule. They can be chosen arbitrarily. However, this choice is not trivial. The off-path belief itself is what sustains the equilibrium.

For this pooling equilibrium to hold, the Assertive signal must be unattractive to both types. This is only true if your reaction to the Assertive signal is sufficiently "punishing." Your reaction, in turn, depends on your off-path belief.

If your off-path belief is, "Wow, only a truly Dominant type would go off-script like that," you might become more interested, rewarding the deviation.
If your off-path belief is, "An Assertive signal from someone who should be Deferential? That seems insecure and over-compensating. He must be the Submissive type trying to act tough," you might withdraw, punishing the deviation.

For the pooling equilibrium to be stable, your off-path belief must support a response that deters both types from deviating. For a more formal treatment of this, including the idea of "trembles" that give a theoretical basis for off-path beliefs, the following reading is excellent.

Chapter 16 Dynamic Games with Incomplete Information

This chapter from an MIT course provides the formal definitions underpinning PBE. Given your background, the mathematical precision will be more efficient than a purely descriptive approach.

First, read the definitions of belief assessment and sequential rationality. This formalizes the two core components of PBE. Next, focus on the section defining belief consistency, which explains how Bayes' rule applies to information sets on the equilibrium path. Finally, and most importantly for this part of our discussion, read the section that explains how consistency is handled for off-path information sets by considering "trembling" probabilities. This is the rigorous way to think about what beliefs should be when you observe a zero-probability event.

A Worked Example: Re-framing "Beer and Quiche"

The "Beer and Quiche" game is a classic signaling game that perfectly illustrates these concepts. Let's re-frame it for our context. A new Bull is either "Tough" ( $T_{T}$ , genuinely confident) or a "Wimp" ( $T_{W}$ , secretly insecure), with some prior probabilities. He can signal by ordering a "Beer" or a "Quiche" at the bar with the hotwife. The hotwife then chooses to "Duel" (be challenging/flirtatious) or "Don't" (remain platonic).

The key is that the hotwife's optimal action depends on her belief about the Bull's type, and her belief is formed by observing his drink order. The following reading walks through the PBE calculation, showing how different prior beliefs can lead to either a separating or a pooling equilibrium.

Signaling

This document provides a concise mathematical analysis of the Beer-Quiche game.

First, read the brief introduction to signaling games and the definitions of separating and pooling equilibria. Then, work through Example 1, where the prior probabilities lead to a separating equilibrium. Pay close attention to the formula for µ2(T|IQ), which is the explicit Bayesian calculation of the receiver's belief. Finally, review Example 2, where a change in the prior probabilities leads to a pooling equilibrium, and notice how the off-path beliefs are what make the equilibrium work.

Conclusion

Today we have placed the intuitive idea of belief change onto a solid mathematical foundation. By integrating strategies with a system of beliefs governed by Bayes' rule, the Perfect Bayesian Equilibrium provides a powerful tool for analyzing interactions under uncertainty.

Key Takeaways:

A Perfect Bayesian Equilibrium (PBE) is a set of strategies and beliefs for each player.
Strategies must be sequentially rational given a player's beliefs at every point in the game.
Beliefs must be consistent with the strategies. On the equilibrium path, this means they are derived using Bayes' rule.
Observing an expected action in a separating equilibrium allows for a precise update that can reveal the sender's type with certainty.
Observing an expected action in a pooling equilibrium provides no new information, and posterior beliefs equal prior beliefs.
Off the equilibrium path, Bayes' rule fails. Beliefs can be chosen arbitrarily, but these off-path beliefs are what sustain the equilibrium by making deviation unattractive.

Preview of the Next Lesson:

We have now analyzed costly signals (like a Spence-style investment) and observable actions (like ordering a beer). In both cases, the action itself had some inherent meaning or cost. But what about signals that are costless to send, such as saying "I want this" or "I promise that"? In our next lesson, we will analyze the strategic use of "cheap talk" and determine the specific conditions under which mere words can credibly convey information and align expectations.

Recommended resources