Basic Probability Rules

1. The Language of Probability: Sample Spaces and Events

Before we can calculate any probabilities, we need a formal way to describe random experiments and their outcomes. The two most important concepts are the sample space and the event space.

• Sample Space (S): The set of all possible outcomes of an experiment.
• Event Space (E): A subset of the sample space that corresponds to an event we are interested in.

To get a clear understanding of these terms, please watch the following segment from a Stanford University lecture on probability for computer scientists. The instructor uses simple examples like coin flips and dice rolls to make these abstract definitions concrete.

As you saw, defining these spaces is the first step in solving any probability problem. For example, when rolling a single six-sided die:

• The sample space is S = {1, 2, 3, 4, 5, 6}.
• If we're interested in the event "rolling an even number," the event space is E = {2, 4, 6}.

2. The First Rule: Equally Likely Outcomes

Now that we can define our spaces, how do we calculate the probability of an event? The simplest and most fundamental rule applies when all outcomes in the sample space are equally likely.

In this special but common case, the probability of an event E is the ratio of the number of outcomes in the event space to the number of outcomes in the sample space.

Let's continue with the Stanford lecture, where this rule is introduced along with some important axioms of probability. Pay close attention to the lottery example, as it's a great illustration of a common pitfall: applying this rule when outcomes are not equally likely.

3. Applying the Rule: Counting is Key

The rule P(E) = |E| / |S| transforms probability problems into counting problems. Your task becomes:

1. Define a sample space where all outcomes are equally likely.
2. Count the total number of outcomes (|S|).
3. Count the number of outcomes that satisfy your event (|E|).
4. Calculate the ratio.

The next part of the video provides an excellent, detailed walkthrough of this process for the experiment of rolling two dice. Since you have a background in Python, you'll appreciate that the instructor even writes a short simulation to empirically verify the calculated probability.

The key insight here is subtle but powerful: sometimes you have to be clever in how you define your sample space. By treating the two dice as distinct (e.g., a red die and a green die), you create a sample space of 36 equally likely outcomes, like (1, 1), (1, 2), ..., (6, 6). If you had incorrectly defined the sample space as just the possible sums {2, 3, ..., 12}, the outcomes would not be equally likely, and our rule would fail.

4. More Advanced Counting: Combinations

Let's tackle a more complex problem. This next example involves drawing toys from a box and introduces the idea of using combinations (a concept from combinatorics often written as "n choose k" or $\left(\genfrac{}{}{0ex}{}{n}{k}\right)$) to count the outcomes. This problem reinforces the strategy of making items theoretically "distinct" to ensure an equally likely sample space.

The two successful approaches in the video (ordered and unordered) highlight a deep point: as long as you are consistent in how you count for your event space and your sample space, you can arrive at the right answer. The unordered approach, using combinations, is often more direct.

For reference, the formula for combinations (choosing k items from a set of n where order doesn't matter) is:
$\left(\genfrac{}{}{0ex}{}{n}{k}\right)=\frac{n!}{k!(n-k)!}$
Your Python skills might come in handy here, as libraries like scipy.special.comb can calculate this for you.

5. Combining Probabilities: The Addition Rule

So far, we've calculated the probability of a single event. What if we want to know the probability of event A or event B happening? For this, we use the General Rule of Addition.

The probability of A or B is the sum of their individual probabilities, minus the probability of their intersection (the part where they overlap).

$P(A\cup B)=P(A)+P(B)-P(A\cap B)$

The notation ∪ means "union" (or), and ∩ means "intersection" (and).

If two events are mutually exclusive (they cannot happen at the same time, so their intersection is empty), the rule simplifies to P(A ∪ B) = P(A) + P(B).

The following resource provides a clear textual summary and then a worked example of applying these rules.

Conclusion

In this lesson, we laid the essential groundwork for our study of probability. You've moved from the world of deterministic calculus to the world of uncertainty, and you now have the tools to begin quantifying it.

Key Takeaways:

• Probability problems start with defining a sample space (all possible outcomes) and an event space (the outcomes you are interested in).
• When all outcomes in the sample space are equally likely, the probability of an event is simply the ratio of the size of the event space to the size of the sample space: P(E) = |E| / |S|.
• Applying this rule often turns probability problems into counting problems. A key strategy is to define your sample space in a way that makes outcomes distinct and equally likely.
• The Addition Rule (P(A ∪ B) = P(A) + P(B) - P(A ∩ B)) allows you to calculate the probability that at least one of two events will occur.

Preview of the Next Lesson:
Today, we focused on counting total and favorable outcomes. In the next lesson, we will introduce a pivotal concept: conditional probability. We'll ask questions like, "What is the probability of event A, given that event B has already happened?" This will lead us directly to Bayes' Theorem, a cornerstone of modern data science and machine learning.