Sample Spaces and Events

Welcome to a new module in our statistical journey. So far, we have focused on describing and transforming data that has already been collected. We've learned to summarize its central tendency, spread, shape, and even to standardize it onto a universal scale using Z-scores. Now, we pivot from the world of certainty—describing what is—to the world of uncertainty—quantifying the likelihood of what might be.

This marks our entry into the realm of probability, the mathematical language of chance. Probability is the bedrock upon which inferential statistics is built, and it is fundamental to understanding and creating machine learning models. In this lesson, we will start with the most basic building blocks. You will learn how to formally define and identify the sample space (the set of all possibilities) and events (the specific outcomes of interest) for any given random experiment.

From Experiment to Outcome

The starting point for any probability problem is a random experiment. This is any action or process whose outcome is uncertain before it occurs. Examples include flipping a coin, rolling a die, or observing the number of users who log into a server in the next minute.

The set of all possible outcomes of a random experiment is called the sample space, often denoted by $S$ or $Ω$ (Omega). Each individual possible outcome is an element of this set.

An event is any collection of outcomes we are interested in—in formal terms, an event is a subset of the sample space.

Let's begin by reading the formal definitions for these core terms.

3.1: Sample Spaces, Events, and Their Probabilities

This text from Statistics LibreTexts provides the essential vocabulary for our lesson.

Please read the initial section titled Sample Spaces and Events. Focus on the definitions of a random experiment, sample space, and event. Then, review Example 1 (tossing a single coin) and Example 2 (rolling a single die) to see these definitions applied to simple, concrete cases.

This relationship between outcomes, events, and the sample space can be visualized as a hierarchy. An outcome is the most granular result. An event can be one or more outcomes. The sample space contains all possible outcomes.

This diagram illustrates that an outcome is a single result, an event is a subset of possible outcomes, and the sample space encompasses all possible outcomes of an experiment.

The Language of Sets in Probability

As your computer science background suggests, the language of set theory is the perfect tool for formalizing these ideas. The sample space $Ω$ is the universal set for our experiment, and any event $A$ is simply a subset, $A \subseteq Ω$ .

The following video does an excellent job of framing probability through the lens of set theory, which will be a very natural perspective for you.

Set Theory in Probability: Sample Spaces and Events

This video by Steve Brunton explicitly connects probability concepts to the formal language of set theory.

Please watch these segments: Introduction: Watch the first minute (Introduction) to understand why set theory is necessary for a rigorous discussion of probability. Sample Space: Pay close attention to the definition of the sample space as the set of all possible outcomes. Events as Subsets: From this point, the instructor works through a concrete example of three coin flips, enumerating the sample space and then defining events as specific subsets of that space.

Enumerating the Sample Space

For simple experiments, listing the sample space is straightforward.

One coin flip: $S = {Heads, Tails}$
One die roll: $S = {1, 2, 3, 4, 5, 6}$

When experiments have multiple stages, the number of outcomes grows quickly. For example, consider rolling two standard six-sided dice. The outcome for each die is independent, and we often represent the outcome as an ordered pair, like (Die 1, Die 2). How many possible outcomes are there?

The sample space consists of all possible pairings. Since Die 1 has 6 outcomes and Die 2 has 6 outcomes, the total number of combined outcomes is $6 \times 6 = 36$ .

A grid showing the complete sample space for rolling two dice. Each cell represents one of the 36 possible outcomes, from (1, 1) to (6, 6).

Using this sample space, we can define various events. For instance:

Event A: The sum of the dice is 5.
This corresponds to the subset of outcomes: $A = {(1, 4), (2, 3), (3, 2), (4, 1)}$ .
Event B: Both dice show the same number (doubles).
This corresponds to the subset: $B = {(1, 1), (2, 2), (3, 3), (4, 4), (5, 5), (6, 6)}$ .
Event C: The first die is a 1.
This corresponds to the subset: $C = {(1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (1, 6)}$ .

Systematic Listing with Tree Diagrams

For experiments with sequential stages, a tree diagram is an invaluable tool for systematically enumerating the sample space. This approach should feel familiar, as it's analogous to traversing tree data structures in computer science. Each path from the root to a leaf node represents a unique outcome.

The following video provides a clear introduction to defining sample spaces and events, and it demonstrates the use of a tree diagram.

Introduction to probability // Events, Sample Space, Formula, Independence

Dr. Trefor Bazett gives a very clear and visual introduction to the core concepts.

Watch from the beginning to this point. He introduces the idea of a sample space and event with a coin-flipping example and then shows how a tree diagram helps organize the outcomes systematically. Continue watching for another minute (until here) for the formal definitions of sample space S and event E as a subset of S.

A Computer Science Perspective

These concepts translate directly to scenarios in computing. Let's consider a practical example from the lecture notes of an engineering course, which defines these concepts with a level of formality you'll appreciate.

Probability Theory Basics - Course Websites

These lecture notes from the University of Illinois ECE department frame probability in a way that is highly relevant to engineering and computer science.

First, read the definitions under the heading "Basic Concepts". Note the distinction between discrete and continuous sample spaces. Next, read the definition of an event on the following slide. Finally, study Example 1 under the heading "Combinatorial Problems". This example defines the sample space for two successive executions of an if-then-else statement and identifies events within it. This is a perfect illustration of how these concepts apply to program behavior.

In the if-then-else example, the random experiment is observing the execution path. The sample space $S = {(s1,s1), (s1,s2), (s2,s1), (s2,s2)}$ represents the complete set of possible execution paths over two runs. An event like "Statement s1 is executed at least once" is the subset $E_{1} = {(s1,s1), (s1,s2), (s2,s1)}$ . This framework allows us to reason about the probabilistic behavior of software systems.

Conclusion

In this lesson, we laid the essential groundwork for our study of probability. We have moved from describing collected data to building a formal structure for analyzing uncertain phenomena.

Key Takeaways:

A random experiment is a process with an uncertain outcome.
The sample space ( $S$ ) is the set of all possible outcomes of an experiment.
An outcome is a single element of the sample space.
An event ( $E$ ) is a subset of the sample space ( $E \subseteq S$ ), representing a collection of outcomes we are interested in.
Tools like grids and tree diagrams provide systematic ways to enumerate the sample space for complex, multi-stage experiments.

Now that we can precisely define the universe of possibilities (the sample space) and the outcomes of interest (events), we are ready for the next logical step: assigning a numerical value to an event that quantifies its likelihood. In our next lesson, we will do just that by learning to calculate basic probabilities.

Recommended resources