Sigma-Algebras: Defining Valid Events

Hello! Welcome to the first lesson in our course on measure theory and its connection to statistics and econometrics.

Introduction

In this lesson, we will tackle the first core concept of modern probability theory: the σ-algebra. Our goal is to define what a σ-algebra is and, more importantly, to understand its crucial role in defining the set of "valid" events in a probability model.

This is the first step in building the formal framework of a probability space. Understanding this concept is essential because it addresses a subtle but profound problem that arises when dealing with continuous or uncountably infinite outcomes—a common scenario in the statistical and financial models you've worked with.

You're familiar with assigning probabilities to outcomes, but you may have taken for granted which collections of outcomes (or "events") can be assigned a probability. We'll see that without a rigorous foundation, paradoxes can emerge that undermine the consistency of probability theory. This lesson provides the first piece of the solution.

1. The Problem: Why Can't We Assign a Probability to Every Set?

In your studies of probability, you've worked with both discrete sample spaces (like the outcomes of a dice roll) and continuous ones (like the price of a stock, which can be any positive real number).

For a finite sample space, $\Omega$, things are simple. If $\Omega =\{H,T\}$ for a coin flip, the set of all possible events is just the power set of $\Omega$, which is $\mathcal{P}(\Omega )=\{\varnothing ,\{H\},\{T\},\{H,T\}\}$. We can consistently assign a probability to every single one of these four events.

However, for an uncountable sample space, like the interval $[0,1]$, this approach breaks down.

• If we assign a positive probability to each individual point, the total probability would sum to infinity.
• If we assign zero probability to every point, the probability of the whole interval $[0,1]$ would seem to be zero, not one.

More formally, mathematicians in the early 20th century discovered the existence of so-called "non-measurable sets". These are bizarre, pathological sets of points for which concepts like "length," "area," or "volume" cannot be defined in a consistent way. The famous Banach-Tarski paradox is a mind-bending example of this.

To get a feel for why we can't simply define a measure (like length or probability) on every subset of the real numbers, please watch the following segment.

The key takeaway is that to build a consistent theory of probability, we cannot allow any arbitrary subset of our sample space to be an "event." We need to be more selective. We need a well-behaved collection of subsets to which we can safely assign probabilities. This collection is what we call a σ-algebra.

2. The Definition of a σ-algebra

A σ-algebra is a collection of subsets of the sample space $\Omega$ that satisfies a few simple, yet powerful, closure properties. These properties ensure that if we start with some basic events, we can combine them in logical ways (using "and," "or," "not") and the resulting sets will also be valid events.

Let $\Omega$ be a sample space. A collection of subsets of $\Omega$, denoted by $\mathcal{F}$, is called a σ-algebra (or σ-field) if it satisfies the following three axioms:

1. The entire sample space is included: $\Omega \in \mathcal{F}$.
2. Closure under complementation: If a set $A$ is in $\mathcal{F}$, then its complement, $A^c=\Omega \setminus A$, must also be in $\mathcal{F}$.
3. Closure under countable unions: If we have a countable sequence of sets $A_1,A_2,A_3,\dots$ that are all in $\mathcal{F}$, then their union, ${\bigcup }_{i=1}^{\infty }{A}_i$, must also be in $\mathcal{F}$.

Any subset of $\Omega$ that is an element of a specific σ-algebra $\mathcal{F}$ is called a measurable set (with respect to $\mathcal{F}$). In the context of probability, these are our "events."

To see a concise presentation of this definition, please watch the following short video.

From these three axioms, we can also deduce a few other important properties:

• The empty set $\varnothing$ is in $\mathcal{F}$ (since $\Omega \in \mathcal{F}$ and $\mathcal{F}$ is closed under complements, ${\Omega }^c=\varnothing$ must be in $\mathcal{F}$).
• $\mathcal{F}$ is closed under countable intersections. This follows from De Morgan's laws: ${\bigcap }_{i=1}^{\infty }{A}_i={\left({\bigcup }_{i=1}^{\infty }{A}_i^c\right)}^c$. Since each $A_i^c$ is in $\mathcal{F}$, their countable union is in $\mathcal{F}$, and the complement of that union is also in $\mathcal{F}$.

3. The Intuition: Why These Axioms?

The axioms of a σ-algebra aren't arbitrary. They correspond directly to the logical operations we want to perform on events. Given your background in programming and interest in logic, you might find it helpful to think of a σ-algebra as the structure needed to make probabilistic reasoning logically consistent.

Here's a summary of that powerful analogy:

Logical Operation	Set Operation	σ-algebra Axiom & Justification
Tautology (always true)	The entire sample space $\Omega$	$\Omega \in \mathcal{F}$. The event "something happens" must be an event we can assign a probability to (specifically, probability 1).
Negation (NOT A)	The complement $A^c$	Closure under complements. If we can ask about the probability of an event $A$, we must also be able to ask about the probability of $A$ not happening.
Disjunction (A OR B)	The union $A\cup B$	Closure under (countable) unions. If we can ask about individual events, we must be able to ask about the event that at least one of them occurs.

The requirement for countable unions is what makes a σ-algebra powerful enough for advanced topics. It allows us to handle limits and sequences of events, which is fundamental for defining concepts in time series analysis and stochastic processes, such as the probability of a stock price ever crossing a certain threshold.

4. Examples of σ-algebras

Let's make this more concrete with a few examples.

Based on that reading and our definition, here are the key examples:

1. The Trivial σ-algebra: For any $\Omega$, the collection $\mathcal{F}=\{\varnothing ,\Omega \}$ is the smallest possible σ-algebra. It represents a state of minimal information: we can only ask about the probability of the "impossible event" ($\varnothing$) or the "certain event" ($\Omega$).

2. The Power Set: For any $\Omega$, the collection of all subsets, $\mathcal{P}(\Omega )$, is the largest possible σ-algebra. As we've discussed, this is the standard choice for finite or countable sample spaces, where non-measurable sets are not an issue.

3. A Custom σ-algebra: Consider the sample space of a dice roll, $\Omega =\{1,2,3,4,5,6\}$. Let's say we are only interested in whether the outcome is "even" or "odd". The corresponding σ-algebra would be:
   $\mathcal{F}=\{\varnothing ,\{1,3,5\},\{2,4,6\},\{1,2,3,4,5,6\}\}$
   You can verify that this collection satisfies all three axioms. The subset $\{1,2\}$, for example, is not in this σ-algebra, meaning the event "the roll is a 1 or a 2" is not a "valid" event in this specific model. This illustrates how the σ-algebra defines the "granularity" of the information we are modeling.

4. The Borel σ-algebra: This is the most important σ-algebra for statistics and econometrics. For the sample space $\Omega =\mathbb{R}$ (the real numbers), we can't use the power set. Instead, we use the Borel σ-algebra, denoted $\mathcal{B}(\mathbb{R})$. It is defined as the smallest σ-algebra that contains all open intervals $(a,b)$.
   While its full structure is complex, the key idea is that it is "big enough" to contain all the sets you would ever care about in practice—intervals (open, closed, half-open), single points, and countable unions/intersections of them—but "small enough" to exclude the pathological non-measurable sets. When you work with a continuous random variable (like a stock return modeled by a Normal distribution), you are implicitly using the Borel σ-algebra as your set of events.

Conclusion

Let's summarize the key takeaways from this lesson:

• Due to mathematical paradoxes, we cannot consistently assign a probability (or "measure") to every subset of an uncountable sample space.
• A σ-algebra, $\mathcal{F}$, is a specially chosen collection of subsets of the sample space $\Omega$ that serves as the set of "valid" events.
• It is defined by three axioms: it must contain $\Omega$, and it must be closed under complementation and countable unions.
• These axioms are motivated by the need for logical consistency: if we can reason about events, we should be able to reason about their opposites ("not") and combinations ("or").
• The σ-algebra forms a crucial component of a probability space, which is the triple $(\Omega ,\mathcal{F},P)$. It acts as the domain for the probability measure $P$.

You now understand the "what" and "why" of the $\mathcal{F}$ in the probability space triple. You've seen that it's not just a piece of abstract formalism; it's a necessary "patch" that makes probability theory rigorous and consistent, especially when dealing with the continuous phenomena prevalent in economics and finance.

Preview of the Next Lesson:
In the next lesson, we will complete the picture by formally defining the full probability space $(\Omega ,\mathcal{F},P)$. We will discuss the properties of the probability measure $P$ and see how these three components work together to form the bedrock of modern probability theory.