Modeling with Discrete Probability Distributions

1. The Bernoulli Distribution: A Single Trial

A Bernoulli trial is the simplest possible random experiment: a single trial with only two possible outcomes. We typically label these outcomes as "success" (represented by 1) and "failure" (represented by 0).

Think of it as a single coin flip, a single free-throw attempt, or checking if a single manufactured part is defective.

The key parameter for a Bernoulli distribution is $p$, the probability of success. The probability of failure is then $q=1-p$.

To get a formal introduction, please watch the first part of the following video.

As the video explained, the key characteristics are:

• Random Variable $X$: Number of successes, can only be 0 or 1.
• Parameter: $p$ (probability of success).
• PMF: $P(X=1)=p$ and $P(X=0)=1-p$.
• Expected Value: $E(X)=\mu =p$
• Variance: $Var(X)={\sigma }^2=p(1-p)=pq$

Simulating a Bernoulli Trial in Python

Given your interest in combining theory with code, let's see how to simulate this. We can use the numpy library, which is a cornerstone of data science in Python.

The code np.random.choice(['success','failure'], p=(0.9, 0.1)) is a straightforward way to simulate one trial where the probability of success is 0.9.

2. The Binomial Distribution: A Sequence of Trials

What happens when we perform more than one Bernoulli trial? For example, instead of one coin flip, we flip a coin 10 times. The Binomial distribution helps us model the number of successes in a fixed number of independent Bernoulli trials.

For a scenario to be modeled by a Binomial distribution, four conditions must be met:

1. Fixed number of trials (n): We know exactly how many times the experiment is repeated (e.g., 10 coin flips).
2. Two possible outcomes: Each trial is a success or a failure.
3. Independent trials: The outcome of one trial does not affect the outcome of another.
4. Constant probability of success (p): The probability of success is the same for every trial.

Understanding the Binomial Formula

The formula for the Binomial PMF can look intimidating, so let's build an intuition for it first.

Now that you have the intuition, let's formalize it. The probability of getting exactly $k$ successes in $n$ trials is given by the Binomial PMF:

$P(X=k)=\left(\genfrac{}{}{0ex}{}{n}{k}\right)p^k(1-p{)}^{n-k}$

Where:

• $\left(\genfrac{}{}{0ex}{}{n}{k}\right)=\frac{n!}{k!(n-k)!}$ is the binomial coefficient. It counts the number of ways to choose $k$ successes from $n$ trials.
• $p^k$ is the probability of getting $k$ successes.
• $(1-p{)}^{n-k}$ is the probability of getting $n-k$ failures.

The following video segment provides a concise formal explanation and connects it back to the mean and variance.

This leads to the beautifully simple formulas for the mean and variance of a Binomial distribution:

• Expected Value: $E(X)=\mu =np$
• Variance: $Var(X)={\sigma }^2=np(1-p)=npq$

Notice how these are just the mean and variance of a single Bernoulli trial, scaled by $n$. This is because the trials are independent!

Simulating a Binomial Distribution in Python

We can easily simulate binomial experiments in Python.

This function allows you to simulate thousands of binomial experiments in an instant, which is very useful for checking your understanding or modeling complex systems.

3. The Poisson Distribution: Events over an Interval

Our final distribution for today is the Poisson distribution. While Binomial counts successes in a set number of trials, Poisson models the number of events occurring in a fixed interval of time or space.

Examples include:

• The number of emails you receive in an hour.
• The number of customers arriving at a store in a day.
• The number of typos on a page of a book.

The key parameter is $\lambda$ (lambda), which is the average rate of events in that interval.

The core properties of the Poisson distribution are:

• Random Variable $X$: Number of events, can be $0,1,2,...$ (theoretically infinite).
• Parameter: $\lambda$ (average number of events per interval).
• PMF: $P(X=k)=\frac{e^{-\lambda }{\lambda }^k}{k!}$, where $k$ is the number of events.
• Expected Value: $E(X)=\mu =\lambda$
• Variance: $Var(X)={\sigma }^2=\lambda$

The Link Between Binomial and Poisson

The Poisson distribution can be seen as an approximation of the Binomial distribution when the number of trials $n$ is very large and the probability of success $p$ is very small. In this case, $\lambda \approx np$.

This connection is very useful and is a common topic in probability theory.

Conclusion

Excellent work today! We've covered three essential discrete distributions that form a major part of the probability syllabus for the GATE exam.

Key Takeaways:

Here is a summary table to help you keep these distributions straight:

Understanding when to use each model is just as important as knowing the formulas. Always check the conditions of the scenario you are analyzing.

Preview of the Next Lesson:

Having covered the most important discrete distributions, we will now move to the continuous world. In our next lesson, we will model scenarios using Uniform, Exponential, and Normal distributions. The Normal distribution, in particular, is arguably the most important distribution in all of statistics and is fundamental to many machine learning concepts.