Probability: The Addition Rule

Welcome back! In our last session, we mastered the classical definition of probability, learning how to calculate the likelihood of a single event by comparing favorable outcomes to the total sample space. We ended with a question: what happens when we're interested in the probability of one event or another? This lesson directly addresses that question.

Today, we will explore the addition rule of probability. This is a fundamental tool that allows us to calculate the probability of the union of two events—that is, the probability that at least one of them occurs. By the end of this lesson, you will be able to apply this rule to solve problems involving both overlapping and non-overlapping events. This concept is a crucial building block for understanding more complex probabilistic models you'll encounter in data science.

The Intuition: Avoiding the Double Count

Let's say we draw one card from a standard 52-card deck. What is the probability of drawing a King or a Heart?

You might first think to simply add the probabilities:

The probability of drawing a King is $P (King) = 4/52$ .
The probability of drawing a Heart is $P (Heart) = 13/52$ .

If we add them, we get $4/52 + 13/52 = 17/52$ . But is this correct? We've overlooked something important: the King of Hearts. This single card is both a King and a Heart. By adding the two probabilities, we've counted it twice.

This "double-counting" problem is the key challenge in calculating the probability of a union of events. The addition rule provides the formal mechanism to correct for it. The following video offers an excellent, step-by-step visual explanation of this idea.

Addition rule for probability | Probability and Statistics | Khan Academy

This Khan Academy video uses a simple, concrete example involving colored shapes in a bag to build the intuition behind the addition rule from the ground up.

Watch from the beginning until the end of the main derivation. As you watch, pay close attention to these three steps: How Sal calculates the probability of picking a "cube" and the probability of picking a "yellow" object separately. How he identifies the "yellow cube" as the overlap or intersection of these two events. Most importantly, how he shows that to find the probability of "yellow or cube," you must add the individual probabilities and then subtract the probability of the overlap to avoid counting it twice.

The General Addition Rule

As the video demonstrates, the principle for finding the probability of event A or event B is to sum their individual probabilities and subtract the probability of their intersection (the part they have in common). This is known as the General Addition Rule of Probability.

In set notation, "A or B" is the union ( $A \cup B$ ) and "A and B" is the intersection ( $A \cap B$ ). The rule is formally stated as:

P (A \cup B) = P (A) + P (B) - P (A \cap B)

Venn diagrams are the perfect tool for visualizing this relationship. The diagram below clearly shows the two events, A and B, their intersection, and why subtracting it is necessary.

This Venn diagram illustrates the addition rule. For non-mutually exclusive events (left), the probability of the union is the sum of the individual probabilities minus their overlapping intersection. For mutually exclusive events (right), there is no overlap, so the probability of the union is simply the sum of the individual probabilities.

To solidify your understanding of this visual logic, let's turn to a brief reading that explains the rule using Venn diagrams.

Probability of Unions | CK-12 Foundation

This article from the CK-12 Foundation provides a concise explanation of the addition rule, focusing on the visual intuition.

Read the initial section of the lesson, starting from the top down to the end of the paragraph that introduces the formula. Focus on the explanation that begins here. Notice how it explicitly states that you must subtract the shared outcomes to avoid counting them twice.

A Special Case: Mutually Exclusive Events

What if two events have no overlap? For example, when rolling a single die, what is the probability of rolling a 1 or a 6? An outcome cannot be both a 1 and a 6 at the same time. These events are called mutually exclusive (or disjoint).

For mutually exclusive events, their intersection is empty, meaning the probability of both occurring simultaneously is zero.

P (A \cap B) = 0

In this special case, the addition rule simplifies beautifully:

P (A \cup B) = P (A) + P (B)

This is shown on the right side of the "Addition Rules for Probability" diagram above. The two circles do not overlap, so we can simply add their areas (probabilities) together.

Let's reinforce this concept with another short reading.

Probability of Unions | CK-12 Foundation

This section of the same CK-12 article explains how the addition rule simplifies for disjoint events.

Please read the section under the heading "Understanding Probability". It directly addresses why the subtraction term disappears for mutually exclusive events.

Putting the Rule into Practice

Now, let's apply the addition rule to solve a few problems. The following resource provides several clear, practical examples. Since your background is in Computer Science, you'll notice some examples use data presented in tables (or DataFrames), a common scenario in data analysis.

Seven Detailed Examples Using The Addition Rule

This guide from the University of Illinois provides several worked-out examples of the addition rule. We will focus on three that illustrate the key concepts we've discussed.

Start with Example 1: Milk Tea. This is a straightforward application of the general addition rule where the probabilities are given. Notice how P(\text{popping boba and lychee jelly}) is subtracted. You can find it here. Next, look at Example 2: A Lottery. This is a perfect illustration of mutually exclusive events. Since Dhiraj and Emily cannot both win, P(\text{Dhiraj and Emily}) = 0, and the rule simplifies. Read the problem and solution here. Finally, examine Example 7: Is a Hotdog a Sandwich?. This problem requires you to first calculate the probabilities from the raw counts in the survey data, just as we did in the last lesson, before applying the general addition rule. This is a very common workflow in data science. Study the problem and its solution here.

Conclusion

In this lesson, we've added a powerful tool to our probabilistic toolkit: the addition rule. You now understand how to calculate the probability of the union of two events, $P (A \cup B)$ .

Key Takeaways:

The union of events A and B represents the outcome that at least one of the events occurs.
The General Addition Rule states that $P (A \cup B) = P (A) + P (B) - P (A \cap B)$ .
The subtraction of $P (A \cap B)$ is crucial to correct for double-counting the outcomes present in both events.
For mutually exclusive (disjoint) events, which cannot occur simultaneously, $P (A \cap B) = 0$ . The rule simplifies to $P (A \cup B) = P (A) + P (B)$ .

We've now covered how to calculate $P (A)$ and $P (A \cup B)$ . However, in many of our examples, the probability of the intersection, $P (A \cap B)$ , was either given or was zero. In the next lesson, we will focus directly on the intersection of events. We'll introduce the concepts of conditional probability and independence, which will lead us to the multiplication rule—the primary method for calculating $P (A \cap B)$ .

Recommended resources

The Intuition: Avoiding the Double Count

The General Addition Rule

A Special Case: Mutually Exclusive Events

Putting the Rule into Practice

Conclusion