Probability and Expectation with Standard Distributions

1. The Language of Randomness: Variables and Distributions

Before diving into specific distributions, let's establish a common vocabulary. A random variable, usually denoted by a capital letter like $X$, is a variable whose value is a numerical outcome of a random phenomenon. These variables come in two main flavors.

• Discrete Random Variables: These can only take on a countable number of distinct values. Think of the number of heads in three coin flips (0, 1, 2, or 3) or the number of defective items in a batch. We describe their behavior using a Probability Mass Function (PMF), $P(X=x)$, which gives the probability of each specific outcome. A key property is that the sum of all probabilities must equal 1: ${\sum }_{i}P(X=x_i)=1$.

• Continuous Random Variables: These can take any value within a given range. Examples include the height of a person or the time until a component fails. Since the probability of hitting any exact value is zero, we use a Probability Density Function (PDF), $f(x)$. Here's the connection to our last lesson: the probability that $X$ falls within an interval $[a,b]$ is found by integrating the PDF over that interval: $P(a\le X\le b)={\int }_a^{b}f(x)dx$. The total area under the PDF curve must equal 1: ${\int }_{-\infty }^{\infty }f(x)dx=1$.

Two other crucial concepts for any distribution are its Expectation (Mean) and Variance.

• Expectation $E[X]$, or $\mu$, is the long-term average value of the random variable.
• Variance $Var(X)$, or ${\sigma }^2$, measures the spread or dispersion of the values around the mean. The standard deviation, $\sigma$, is the square root of the variance.

2. Discrete Distributions: Counting Events

For GATE, the two most important discrete distributions are Binomial and Poisson. They both deal with counting the number of "successes" or events.

Let's summarize the key takeaways you should memorize for the exam.

Binomial Distribution

• When to use: Use it when you have a fixed number of independent trials ($n$), each with only two possible outcomes (e.g., success/failure, pass/fail), and the probability of success ($p$) is constant for every trial.
• Probability Mass Function (PMF): The probability of getting exactly $k$ successes in $n$ trials is:
  $P(X=k)=\left(\genfrac{}{}{0ex}{}{n}{k}\right)p^k(1-p{)}^{n-k}$
• Mean (Expectation): $E[X]=np$
• Variance: $Var(X)=np(1-p)$

Poisson Distribution

• When to use: Use it to model the number of times an event occurs in a fixed interval of time or space, when events happen with a known constant mean rate and independently of the time since the last event. Think of "arrivals" per hour or "defects" per square meter.
• Probability Mass Function (PMF): The probability of $k$ events occurring in an interval, where $\lambda$ is the average number of events per interval, is:
  $P(X=k)=\frac{{\lambda }^k{e}^{-\lambda }}{k!}$
• Mean (Expectation): $E[X]=\lambda$
• Variance: $Var(X)=\lambda$
  • GATE Tip: A classic identifier for the Poisson distribution is that its mean equals its variance. If a problem states this, you should immediately think of Poisson.

3. Continuous Distributions: Measuring on a Scale

Now we move to variables that can take any value in a range. Here, your integration skills become essential.

Uniform Distribution

This is the simplest continuous distribution. It models a situation where any outcome within a specific interval $[a,b]$ is equally likely.

• Probability Density Function (PDF):
  $f(x)=\{\begin{array}{ll}\frac{1}{b-a}& \text{for }a\le x\le b\\ 0& \text{otherwise}\end{array}$
  The PDF is just a constant value over the interval, forming a rectangle.
• Probability: Calculating probability is just finding an area: $P(c\le X\le d)={\int }_c^d\frac{1}{b-a}dx=\frac{d-c}{b-a}$.
• Mean: $E[X]=\frac{a+b}{2}$ (the midpoint of the interval)
• Variance: $Var(X)=\frac{(b-a{)}^2}{12}$

Exponential Distribution

This distribution describes the time between events in a Poisson process. For example, if customer arrivals follow a Poisson distribution, the time between consecutive arrivals follows an Exponential distribution.

• Probability Density Function (PDF):
  $f(x)=\{\begin{array}{ll}\lambda e^{-\lambda x}& \text{for }x\ge 0\\ 0& \text{otherwise}\end{array}$
  Here, $\lambda$ is the rate parameter (the same $\lambda$ from the associated Poisson process).
• Mean: $E[X]=1/\lambda$ (If arrivals occur at a rate of 3 per hour, the average time between arrivals is 1/3 of an hour).
• Variance: $Var(X)=1/{\lambda }^2$
• Key Property (Memorylessness): The probability that you have to wait an additional amount of time does not depend on how long you've already been waiting. Formally, $P(X>s+t|X>s)=P(X>t)$. A very useful formula derived from this is the survival function: $P(X>t)=e^{-\lambda t}$.

Normal (Gaussian) Distribution

This is the most important continuous distribution in all of statistics, and a favorite in GATE. It describes many natural phenomena, and the Central Limit Theorem states that the sum of many independent random variables tends towards a normal distribution.

The PDF for a general normal distribution $X\sim N(\mu ,{\sigma }^2)$ is quite complex:
$f(x)=\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{1}{2}{\left(\frac{x-\mu }{\sigma }\right)}^2}$
The good news is you will almost never have to integrate this directly. The entire strategy for solving normal distribution problems revolves around transforming the problem into a simpler, standard form.

The following video is a masterclass in solving GATE-level normal distribution problems. We will walk through it step-by-step.

Conclusion

This lesson has equipped you with the essential knowledge to tackle a wide range of probability questions in GATE. You now understand the five most important distributions and, crucially, how to calculate probabilities and other properties for each.

Key Takeaways:

• Identify the Distribution: The first step is always to read the problem carefully and identify the underlying distribution:
  • Binomial: Fixed trials, two outcomes, constant probability.
  • Poisson: Counting events in a fixed interval.
  • Uniform: All outcomes in a range are equally likely.
  • Exponential: Time between Poisson events.
  • Normal: Bell-shaped curve, often describing measurements, averages, or sums.
• Know the Formulas: For each distribution, you must know the formulas for its PMF/PDF, mean, and variance by heart.
• Master the Z-score: For any Normal distribution problem, the path to the solution is almost always through the Z-score transformation $Z=(X-\mu )/\sigma$.
• Calculus Connection: Remember that for continuous distributions, probability is the definite integral of the PDF, representing the area under the curve.

In our final lesson for this module, we will build upon these concepts to explore Statistics and Bayes' Theorem. We will learn how to describe datasets using measures like mean and variance, and how to update our beliefs (probabilities) in light of new evidence using the powerful Bayes' theorem.