Probability Rules: Addition, Multiplication, and Complement

1. The Building Blocks: Sample Spaces and Events

Before we can apply any rules, we need to be precise about what we're calculating the probability of. The most common mistake is to think of probability in terms of single outcomes. The key insight is that probability operates on events, which are sets of outcomes.

Let's build this crucial intuition from the ground up.

To formalize this, all of probability is built on three simple, common-sense rules known as Kolmogorov's Axioms. Given your preference for a theory-first approach, understanding these axioms provides a solid, logical foundation for everything that follows.

In summary:

• Sample Space ($\Omega$): The set of all possible outcomes of an experiment.
• Event (A, B, ...): A subset of the sample space. It can be one outcome or many.
• Probability P(A): A number between 0 and 1 that measures the likelihood of event A occurring. It's calculated as the number of outcomes in event A divided by the total number of outcomes in the sample space (for equally likely outcomes).

2. The Complement and Addition Rules: "NOT" and "OR"

Now that we have our building blocks, we can start combining events.

The Complement Rule (NOT)

The complement rule is the simplest. If you know the probability of an event happening, this rule gives you the probability of it not happening. The complement of event A is written as $A^c$ or A'.

The rule is:
$P(A^c)=1-P(A)$
This works because an event either happens or it doesn't; the two possibilities must sum to 1 (100%).

This "shortcut" of calculating the opposite and subtracting from 1 is a common problem-solving strategy in probability.

The Addition Rule (OR)

What if we want to know the probability of event A or event B occurring? This is the probability of their union, written as $P(A\cup B)$. A naive approach would be to just add their probabilities: $P(A)+P(B)$. However, this can lead to an error.

As the video explains, the general addition rule is:
$P(A\cup B)=P(A)+P(B)-P(A\cap B)$
Here, $P(A\cap B)$ is the probability of the intersection of A and B—that is, the probability of both events happening together. We subtract it to correct for having counted it twice.

A special case arises when two events are mutually exclusive, meaning they cannot happen at the same time (their intersection is empty, so $P(A\cap B)=0$). In this case, the rule simplifies to:
$P(A\cup B)=P(A)+P(B)\text{(for mutually exclusive events)}$

3. The Multiplication Rule: "AND" and Independence

Now, let's focus on the intersection itself: the probability of event A and event B occurring, written $P(A\cap B)$. To calculate this, we need to introduce one of the most important concepts in probability: independence.

Two events are independent if the outcome of one has no effect on the outcome of the other. For example, flipping a coin and rolling a die are independent events.

When two events A and B are independent, the multiplication rule is straightforward:
$P(A\cap B)=P(A)\times P(B)\text{(for independent events)}$

When events are not independent (i.e., they are dependent), this simple multiplication rule doesn't work. We will need a more powerful tool called conditional probability to handle those cases, which is the focus of our next lesson.

4. Applying the Rules: A Worked Example

Theory is essential, but application is the goal. Let's see how these rules are used in practice to solve a problem step-by-step.

This example solidifies the process:

1. Identify what you know from the problem statement ($P(A)$, $P(B)$, etc.).
2. Determine which probability you need to find ("and" $\to$ intersection, "or" $\to$ union).
3. Select the correct formula (Addition Rule, Multiplication Rule).
4. Plug in the numbers and calculate the result.

Now, try one yourself.

Practice Problem:
You roll a fair six-sided die. Let event A be "rolling a number divisible by 2" and event B be "rolling a number divisible by 3". What is the probability of rolling a number divisible by 2 or 3?

Solution Walkthrough:

1. Identify Probabilities:
   • Sample Space $\Omega =\{1,2,3,4,5,6\}$.
   • Event A (divisible by 2) = $\{2,4,6\}$. So, $P(A)=3/6=1/2$.
   • Event B (divisible by 3) = $\{3,6\}$. So, $P(B)=2/6=1/3$.
2. Determine the Goal: We need $P(A\text{ or }B)$, which is $P(A\cup B)$.
3. Select the Formula: The events are not mutually exclusive because the outcome '6' is in both. So we must use the general addition rule: $P(A\cup B)=P(A)+P(B)-P(A\cap B)$.
4. Find the Intersection: The intersection $A\cap B$ is "a number divisible by both 2 and 3," which is $\{6\}$. So, $P(A\cap B)=1/6$.
5. Calculate:
   $P(A\cup B)=\frac{1}{2}+\frac{1}{3}-\frac{1}{6}=\frac{3}{6}+\frac{2}{6}-\frac{1}{6}=\frac{4}{6}=\frac{2}{3}$

You can find more examples and practice problems in the "Sample Questions" section of the GeeksforGeeks article on Probability Rules.

Conclusion

In this lesson, you have learned the fundamental grammar of probability. You moved from the abstract idea of "chance" to a formal system for calculating it.

Key Takeaways:

• Probability operates on events, which are sets of outcomes from a sample space.
• The Complement Rule provides a shortcut for finding the probability of an event not happening: $P(A^c)=1-P(A)$.
• The Addition Rule calculates the probability of event A or B happening, correcting for any overlap: $P(A\cup B)=P(A)+P(B)-P(A\cap B)$.
• The Multiplication Rule calculates the probability of event A and B happening, which for independent events simplifies to: $P(A\cap B)=P(A)\times P(B)$.

This cheat sheet provides a great visual summary of all the rules we've discussed.

Preview of the Next Lesson:
We've focused on independent events, but in many real-world AI scenarios, events are dependent. For example, the probability of it raining tomorrow is dependent on whether it is cloudy today. In the next lesson, we will explore conditional probability to handle these dependencies. This will lead us directly to Bayes' Theorem, a cornerstone of machine learning that allows a system to update its beliefs as it receives new evidence.