Conditional Expectation and the Conditioning Paradox

Hello! Welcome to the second module of our course.

In the first module, we established the Radon-Nikodym framework, culminating in the key insight that a probability density function (PDF) is a Radon-Nikodym derivative with respect to the Lebesgue measure. Now, we're ready to put this powerful machinery to work. This module focuses on resolving the central "philosophical" problem you identified for your essay: how to make sense of conditioning on an event of probability zero.

Today's lesson addresses the heart of the matter. We will review the formal, measure-theoretic definition of conditional expectation with respect to a σ-algebra, often written as $E[Y|\mathcal{G}]$. Our goal is to understand why this definition, which treats conditional expectation as a random variable, provides a rigorous and unambiguous solution to the conditioning paradox, where elementary definitions fail.

The Problem: Conditioning on the Impossible

From your background in econometrics and statistics, you are deeply familiar with conditional expectation. You've frequently worked with expressions like $E[Y|X=x]$, the expected value of a random variable $Y$ given that another random variable $X$ has taken a specific value $x$. For instance, this could be the expected return of an asset given the market index is at a certain level.

When $X$ is discrete, this is straightforward. We use the familiar formula:
$P(A|B)=\frac{P(A\cap B)}{P(B)}$
But what happens when $X$ is a continuous random variable? The probability that $X$ takes on any single value $x$ is zero. That is, $P(X=x)=0$. The formula above breaks down with a zero in the denominator.

You've navigated this in practice by using probability densities:
$f_{Y|X}(y|x)=\frac{f_{X,Y}(x,y)}{f_X(x)}$
And then computing the expectation by integrating: $E[Y|X=x]=\int y\cdot f_{Y|X}(y|x)dy$. This works beautifully for many standard models. However, it sidesteps the foundational question: what does "given $X=x$" truly mean when that event has zero probability? Relying on densities feels more like a computational recipe than a fundamental definition. The fragility of this approach is revealed when things get complicated, a phenomenon known as the Borel-Kolmogorov paradox.

The Borel-Kolmogorov Paradox: Ambiguity in Conditioning

The paradox demonstrates that if you're not careful, the very idea of conditioning on a zero-probability event can lead to contradictory results. The "answer" you get depends on how you approach the zero-probability limit.

A classic example involves choosing a point uniformly at random on the surface of a sphere. What is the conditional distribution of its location, given that it lies on a specific great circle (e.g., the equator)? Since a great circle is a line with no area, the probability of landing on it is zero.

It turns out that if you think of the great circle as a line of longitude (a meridian), you get one answer for the conditional distribution. If you think of it as a line of latitude (the equator), you get a different answer. Yet, it's the same great circle.

The following paper provides a superb philosophical and mathematical discussion of this paradox. It shows how the problem isn't just a brain-teaser but arises from the standard techniques of conditional PDFs you've used in your applied work.

This paradox reveals a deep truth: the notion of conditioning "given an event" is insufficient when that event has zero measure. The information we need is not just the event itself, but the context or structure within which we are observing it. This is where σ-algebras enter the picture.

The Solution: Conditional Expectation as a Random Variable

The measure-theoretic solution re-frames the entire concept. Instead of thinking of $E[Y|X=x]$ as a number that depends on $x$, we define a new object: $E[Y|\mathcal{G}]$, the conditional expectation of $Y$ given a σ-algebra $\mathcal{G}$.

What is $\mathcal{G}$? In our case, it is the σ-algebra generated by the random variable $X$, denoted $\sigma (X)$. You can think of $\sigma (X)$ as the collection of all questions about $X$ that can be answered with a "yes" or "no" (e.g., "Is $X>5$?", "Is $X$ in the set $[a,b]$?"). It formally represents the information obtained by observing $X$.

The crucial insight is that $E[Y|\sigma (X)]$ is not a number, but a random variable. It is a function of $X$, and its value changes as $X$ changes. The following video provides an excellent and intuitive bridge from the familiar concept to this more abstract one.

This conceptual leap is the key. We are no longer looking for a value conditioned on a single outcome, but a function that tells us the expected value of $Y$ for every possible outcome of $X$.

The Formal Definition

So, what is this random variable $Z=E[Y|\mathcal{G}]$ formally? It is defined not by a direct formula, but by its properties.

Definition: Let $(\Omega ,\mathcal{F},P)$ be a probability space, $\mathcal{G}$ be a sub-σ-algebra of $\mathcal{F}$, and $Y$ be an integrable random variable. The conditional expectation of $Y$ given $\mathcal{G}$, denoted $E[Y|\mathcal{G}]$, is a random variable $Z$ that satisfies two conditions:

1. Measurability: $Z$ is $\mathcal{G}$-measurable. This is the technical way of saying that $Z$ "only depends on the information in $\mathcal{G}$." If $\mathcal{G}=\sigma (X)$, it means $Z$ is a function of $X$.
2. The Defining Property: For every set $A\in \mathcal{G}$, we have $${\int }_{A}ZdP={\int }_{A}YdP$$ This property essentially states that $Z$ has the same average value as $Y$ over any set of outcomes definable by the information in $\mathcal{G}$. It is the "best approximation" of $Y$ using only the information available in $\mathcal{G}$.

The following image provides a beautiful visualization of this "best approximation" idea.

The following video lecture provides the formal statement of this definition and, critically, uses the Radon-Nikodym theorem to prove that such a random variable $Z$ always exists and is unique (almost surely). This provides the explicit link between the abstract definition and the machinery we built in the last module.

How This Resolves the Paradox

The Borel-Kolmogorov paradox arose from the ambiguity of what "given a great circle" means. Kolmogorov's solution, now formalized in the definition of conditional expectation, is that one cannot condition on an isolated null event. One must condition on a sub-σ-algebra.

• In the "surface area solution," we condition on the σ-algebra generated by the latitude-longitude coordinate system.
• In the "arc length solution," we implicitly condition on a different σ-algebra, one that only knows about position along the great circle itself.

The paradox dissolves because the two seemingly contradictory results are actually the correct conditional expectations for two different conditioning σ-algebras. The measure-theoretic definition forces us to be precise about the information structure, and this precision removes the ambiguity.

Let's return to the Rescorla paper to see this conclusion articulated.

Conclusion

In this lesson, we've formally established the measure-theoretic definition of conditional expectation as the rigorous solution to the paradox of conditioning on zero-probability events.

Key Takeaways:

• The elementary definition of conditional probability, $P(A|B)=P(A\cap B)/P(B)$, fails for continuous variables where the conditioning event has probability zero. This leads to ambiguities and paradoxes like Borel-Kolmogorov.
• The measure-theoretic solution is to define conditional expectation not as a number, but as a random variable: $E[Y|\mathcal{G}]$.
• This random variable is $\mathcal{G}$-measurable (it only uses information in $\mathcal{G}$) and satisfies ${\int }_{A}E[Y|\mathcal{G}]dP={\int }_{A}YdP$ for all sets $A$ in the conditioning σ-algebra $\mathcal{G}$.
• This framework resolves the paradox by forcing us to be explicit about the conditioning σ-algebra ($\mathcal{G}$, or $\sigma (X)$), which represents the structure of the available information. Different information structures can, and should, lead to different conditional expectations.

We have now defined the object $E[Y|\sigma (X)]$ and seen that its existence is guaranteed by the Radon-Nikodym theorem. In our next lesson, we will make this connection even more concrete. We will see exactly how the Radon-Nikodym theorem provides the mechanism for constructing this conditional expectation, and we will verify that for well-behaved cases like the bivariate normal distribution, this rigorous process recovers the familiar formula $f(y|x)=f(x,y)/f_X(x)$ that you've used in your econometric work.