Conditional Probability and Bayes' Theorem

1. Describing Data: A Quick Look at Statistics

Before we dive into updating probabilities, let's briefly cover the fundamental tools for summarizing a set of data or a probability distribution. You have already encountered the concepts of Mean (Expectation) and Variance.

• Mean ($\mu$): The average value. For a set of $n$ data points $x_1,x_2,\dots ,x_n$, it's $\mu =\frac{1}{n}{\sum }_{i=1}^n{x}_i$. For a probability distribution, it's the expected value $E[X]$.
• Median: The middle value of a sorted dataset. If $n$ is odd, it's the middle element. If $n$ is even, it's the average of the two middle elements. The median is less sensitive to extreme outliers than the mean.
• Mode: The value that appears most frequently in a dataset.
• Variance (${\sigma }^2$): Measures the average squared deviation from the mean, indicating the "spread" of the data. For a dataset, it's ${\sigma }^2=\frac{1}{n}{\sum }_{i=1}^n(x_i-\mu {)}^2$.
• Standard Deviation ($\sigma$): The square root of the variance, $\sigma =\sqrt{{\sigma }^2}$. It is expressed in the same units as the data, making it more intuitive to interpret than variance.

For GATE, you should be familiar with these definitions, but the main focus of this topic is on the application of probability rules, which we will cover next.

2. Conditional Probability: Probability with New Information

The core of advanced probability lies in answering the question: "How does the likelihood of an event change if I know something else has already happened?" This is the essence of conditional probability.

The probability of event $A$ occurring, given that event $B$ has already occurred, is written as $P(A|B)$ and is defined as:
$P(A|B)=\frac{P(A\cap B)}{P(B)}$
where $P(A\cap B)$ is the probability that both $A$ and $B$ occur, and we assume $P(B)>0$.

Think of it as narrowing your sample space. Instead of considering all possible outcomes, you are only considering the outcomes where $B$ is true. Within that new, smaller world, what is the chance that $A$ also happens?

This definition leads to the Multiplication Rule, a very useful tool for finding the probability of an intersection:
$P(A\cap B)=P(A|B)P(B)=P(B|A)P(A)$

A crucial related concept is the Law of Total Probability. It allows you to find the total probability of an event $A$ by considering its probability across a set of mutually exclusive and exhaustive events $B_1,B_2,\dots ,B_n$ that partition the sample space.
$P(A)={\sum }_{i=1}^{n}P(A\cap B_i)={\sum }_{i=1}^{n}P(A|B_i)P(B_i)$
This is often visualized using a tree diagram, where you sum the probabilities of all paths leading to event $A$.

3. Bayes' Theorem: Inverting Conditional Probabilities

Bayes' theorem is one of the most elegant and impactful results in probability. It provides a way to "invert" conditional probabilities. Often, we know the probability of an effect given a cause, $P(\text{Effect}|\text{Cause})$, but what we really want to find is the probability of the cause given the effect, $P(\text{Cause}|\text{Effect})$.

Derivation and Formula

Starting from the two forms of the multiplication rule:
$P(A\cap B)=P(A|B)P(B)$
$P(A\cap B)=P(B|A)P(A)$

Setting them equal gives $P(B|A)P(A)=P(A|B)P(B)$. Rearranging for $P(B|A)$ gives us Bayes' Theorem:
$P(B|A)=\frac{P(A|B)P(B)}{P(A)}$

Let's break down the terms in a typical diagnostic scenario:

• $B$: The event that a patient has a disease (the "cause").
• $A$: The event that the patient's test is positive (the "effect").
• $P(B|A)$ (Posterior): The probability the patient has the disease given a positive test. This is what we want to find.
• $P(B)$ (Prior): The initial probability that a patient has the disease, based on general population statistics.
• $P(A|B)$ (Likelihood): The probability of a positive test given the patient has the disease (the test's sensitivity).
• $P(A)$ (Evidence): The overall probability of getting a positive test, regardless of whether the patient has the disease. We calculate this using the Law of Total Probability: $P(A)=P(A|B)P(B)+P(A|\neg B)P(\neg B)$.

So, the full form of Bayes' theorem for a simple two-case scenario (B or not B) is:
$P(B|A)=\frac{P(A|B)P(B)}{P(A|B)P(B)+P(A|\neg B)P(\neg B)}$

The following video provides an excellent walkthrough of these concepts with solved examples, which is the best way to master the application for GATE.

4. A Structured Approach to Solving GATE Problems

To consolidate what you've learned, let's walk through a typical GATE-style problem. The key is to be systematic.

Problem: A factory has two machines, M1 and M2. M1 produces 40% of the total bolts, and M2 produces 60%. Of the bolts produced by M1, 2% are defective. Of the bolts produced by M2, 5% are defective. A bolt is drawn at random from the total output and is found to be defective. What is the probability that it was produced by machine M2?

Solution Strategy:

1. Define Events:

   • Let $M_1$ be the event that the bolt is from machine M1.
   • Let $M_2$ be the event that the bolt is from machine M2.
   • Let $D$ be the event that the bolt is defective.

2. List Given Probabilities (Priors and Likelihoods):

   • $P(M_1)=0.40$ (Prior probability of being from M1)
   • $P(M_2)=0.60$ (Prior probability of being from M2)
   • $P(D|M_1)=0.02$ (Likelihood of being defective given it's from M1)
   • $P(D|M_2)=0.05$ (Likelihood of being defective given it's from M2)

3. State the Goal (Posterior Probability):

   • We want to find $P(M_2|D)$ – the probability it's from M2 given that it's defective.

4. Apply Bayes' Theorem:
   $P(M_2|D)=\frac{P(D|M_2)P(M_2)}{P(D)}$

5. Calculate the Denominator (Total Probability of Defect):

   • Using the Law of Total Probability:
     $P(D)=P(D|M_1)P(M_1)+P(D|M_2)P(M_2)$
     $P(D)=(0.02)(0.40)+(0.05)(0.60)$
     $P(D)=0.008+0.030=0.038$

6. Calculate the Final Answer:
   $P(M_2|D)=\frac{(0.05)(0.60)}{0.038}=\frac{0.030}{0.038}=\frac{30}{38}=\frac{15}{19}$

So, the probability that the defective bolt came from machine M2 is approximately 78.9%. Notice how our initial belief that a bolt is from M2 ($P(M_2)=60\%$) was updated to a higher probability ($78.9\%$) once we gained the new evidence that the bolt was defective. This is because M2 has a higher defect rate.

Conclusion

This lesson concludes our journey through the Engineering Mathematics module. You've now covered the essential concepts of conditional probability and Bayes' theorem, which are critical for reasoning under uncertainty.

Key Takeaways:

• Conditional Probability $P(A|B)=P(A\cap B)/P(B)$ tells you how information about event $B$ changes the probability of event $A$.
• The Law of Total Probability is your tool for finding the overall probability of an event by summing its probabilities across different scenarios.
• Bayes' Theorem, $P(B|A)=P(A|B)P(B)/P(A)$, allows you to formally update your initial beliefs (priors) with new evidence to arrive at a revised belief (posterior).
• For GATE problems, a systematic approach is key: define events, list known probabilities, state the goal, and apply the appropriate formula (usually Bayes' theorem using the Law of Total Probability for the denominator).

You have now completed Module 3. We are about to shift gears significantly. Our next lesson will be the start of Module 4: Digital Logic, a core topic in Computer Science. We will begin with Topic 4.1: Boolean Algebra, the mathematical foundation for designing all digital circuits.