Prior vs. Posterior: Dressing Room Decisions

The Two Critical Questions of a Fitting Session

During a fitting or test shoot, you are constantly gathering evidence and asking two fundamentally different questions, even if you don't phrase them this way.

1. The Question of Evidence Quality: "If this 'Audrey' balconette bra is truly a star performer, what is the chance our photographer will give it a 'strong recommend' in the test shots?" This is written as $P(\text{Evidence}|\text{Style Suits})$. This probability, the likelihood, measures the diagnostic power of your evidence source. A high value means your photographer has a good eye for winning styles.

2. The Question for Decision-Making: "The photographer just gave the 'Audrey' balconette a 'strong recommend'. What is the probability that it is, in fact, a star performer?" This is written as $P(\text{Style Suits}|\text{Evidence})$. This is the posterior probability, and it's what you actually need to decide whether to feature the style, invest in more shots, or move on.

The common and dangerous assumption is that these two probabilities are roughly equal. As you saw in the last lesson, they are not. The logical trap of equating them is so common that it has its own name.

The Prosecutor's Fallacy in the Fitting Room

The "Prosecutor's Fallacy" occurs when we mistake the probability of the evidence given a hypothesis for the probability of the hypothesis given the evidence. It gets its name from courtroom situations where, for example, the probability of finding a DNA match if the defendant is innocent (a very small number) is misinterpreted as the probability the defendant is innocent given the DNA match.

To explore this, we'll look at a short article from the Centre for Evidence-Based Medicine at the University of Oxford that explains this fallacy clearly.

The key takeaway is that the probability of something being true depends not just on the new evidence, but also on how likely it was to be true in the first place—the base rate, or in our terms, the prevalence of "star" lingerie styles.

Visualizing the Impact of Prevalence

The prevalence of "star" styles is the single most important factor driving the difference between $P(\text{Evidence}|\text{Style Suits})$ and $P(\text{Style Suits}|\text{Evidence})$. The following diagram, also from the Oxford CEBM, illustrates this perfectly.

Imagine a "test" for identifying a winning lingerie style. This test could be a stylist's opinion, a photographer's reaction, or even an early engagement metric on a social media teaser. Let's assume the test has:

• Sensitivity = 98%: It correctly identifies 98% of winning styles.
• False Positive Rate = 1%: It incorrectly flags 1% of average styles as winners.

Now, let's see how the results change based on the prevalence of winning styles in the pool you're testing.

Let's break this down for our lingerie context:

• Scenario (a): Prevalence = 0.02%. This is like scouting for brand new, unproven designs. Out of 1,000,000 potential styles, only 200 are "stars".

  • True Positives: The test finds 98% of them: 196.
  • False Positives: The test incorrectly flags 1% of the 999,800 average styles: 9,998.
  • You get a positive test result. What is the chance it's a false alarm? It's $P(\text{Average Style}|\text{Positive Test})=\frac{\text{False Positives}}{\text{All Positives}}=\frac{9,998}{196+9,998}\approx 98\%$. Despite a seemingly accurate test, a positive result is almost certainly wrong!

• Scenario (b): Prevalence = 20%. This is like choosing from a curated collection of already successful silhouettes. Out of 1,000,000 styles, 200,000 are potential "stars".

  • True Positives: The test finds 98% of them: 196,000.
  • False Positives: The test incorrectly flags 1% of the 800,000 average styles: 8,000.
  • Now, what's the chance of a false alarm given a positive test? $P(\text{Average Style}|\text{Positive Test})=\frac{8,000}{196,000+8,000}\approx 3.9\%$. Here, a positive result is highly trustworthy.

The test's intrinsic properties, $P(\text{Positive}|\text{Star})$ and $P(\text{Positive}|\text{Average})$, didn't change. The only thing that changed was the base rate of star styles.

Formalizing the Terminology

Your statistics background makes you familiar with the formal terms for these probabilities. Let's explicitly map them to our lingerie problem, using a helpful resource that defines them clearly.

Here is the mapping to our problem:

• Sensitivity or True Positive Rate: $P(\text{Evidence}|\text{Style Suits})$. This is the likelihood we discussed. It's the probability your test (e.g., the photographer) correctly identifies a star performer.
• Specificity or True Negative Rate: $P(\text{No Evidence}|\text{Style Doesn’t Suit})$. The probability your test correctly dismisses an average style.
• Positive Predictive Value (PPV): $P(\text{Style Suits}|\text{Evidence})$. This is the posterior probability—the one you use for decision-making. It's the probability a style is actually a star, given that your test said it was.

The Prosecutor's Fallacy, in these terms, is the error of confusing Sensitivity with Positive Predictive Value.

Numerical Exercise: Evaluating the Photographer's Opinion

Let's solidify this with a numerical problem. You are evaluating a new "plunge" bra style for the photoshoot.

Your Information:

• Prevalence: From past experience, you know that only about 2% of completely new styles achieve "top-tier" status in your shoots. So, $P(\text{Top Tier})=0.02$.
• Photographer's Judgment: You've analyzed the photographer's feedback on past styles.
  • Sensitivity: They give a positive opinion to 90% of styles that end up being top-tier.
  • Specificity: They correctly give a negative or neutral opinion to 95% of styles that are not top-tier.

Your Task:

1. What is the value of $P(\text{Positive Opinion}|\text{Top Tier})$? In one sentence, explain what this probability represents in this context.
2. Your photographer gives the new "plunge" bra a positive opinion. Calculate the probability that it is actually a top-tier style, i.e., $P(\text{Top Tier}|\text{Positive Opinion})$.
3. Compare the values from #1 and #2. In your own words, explain why a 90% accurate opinion only results in the probability you calculated.

Solution

1. $P(\text{Positive Opinion}|\text{Top Tier})=0.90$. This is the sensitivity of the photographer's judgment. It means that if a bra style is genuinely top-tier, there is a 90% chance the photographer will recognize it and give a positive opinion.

2. To calculate $P(\text{Top Tier}|\text{Positive Opinion})$, we use Bayes' theorem.

   • Prior: $P(\text{Top Tier})=0.02$
   • Likelihood: $P(\text{Positive Opinion}|\text{Top Tier})=0.90$
   • False Positive Rate: This is $1-\text{Specificity}$. So, $P(\text{Positive Opinion}|\neg \text{Top Tier})=1-0.95=0.05$.

   First, we find the total probability of a positive opinion, $P(\text{Positive Opinion})$, using the law of total probability:

   $$P(\text{Positive Opinion})=P(\text{Pos Opinion}|\text{Top Tier})P(\text{Top Tier})+P(\text{Pos Opinion}|\neg \text{Top Tier})P(\neg \text{Top Tier})$$ $$P(\text{Positive Opinion})=(0.90)(0.02)+(0.05)(1-0.02)$$ $$P(\text{Positive Opinion})=0.018+(0.05)(0.98)=0.018+0.049=0.067$$

   Now we can find the posterior (the PPV):

   $$P(\text{Top Tier}|\text{Positive Opinion})=\frac{P(\text{Positive Opinion}|\text{Top Tier})P(\text{Top Tier})}{P(\text{Positive Opinion})}=\frac{0.018}{0.067}\approx 0.2687$$

   So, given the photographer's positive opinion, the probability that the plunge bra is actually top-tier is about 26.9%.

3. Comparison: The photographer's ability to spot a winning style is very high at 90% (sensitivity), but the probability that their praise actually indicates a winning style is only about 27% (PPV). The reason for this huge gap is the low prevalence of top-tier styles (2%). Because 98% of all styles are not top-tier, the 5% false positive rate from the photographer, while seemingly small, generates many more "false alarms" (0.05 * 98 = 4.9) than the 90% sensitivity generates "true hits" on the rare top-tier styles (0.90 * 2 = 1.8). Most of the positive feedback is therefore directed at styles that won't ultimately make the cut.

Conclusion

Today we have formalized the crucial difference between measuring the quality of your evidence and using that evidence to make a decision. This distinction is at the core of sound Bayesian reasoning and is essential for navigating the creative, data-sparse environment of a photoshoot.

Key Takeaways:

• $P(\text{Evidence}|\text{Style Suits})$, or Sensitivity, measures the accuracy of your information source.
• $P(\text{Style Suits}|\text{Evidence})$, or Positive Predictive Value, is the posterior probability you need to make an informed decision.
• Confusing these two is the Prosecutor's Fallacy.
• The prevalence (or prior probability) of the outcome you're looking for is the main driver of the difference between them. When 'star' styles are rare, even a good test will have a low PPV.

In our last lesson, you calculated a posterior probability. Today, you've contextualized it, given it a formal name (PPV), and understood why it behaves the way it does. In our next lesson, we will build on this by calculating both the positive predictive value and the false positive rate for an entire screening process, which will allow you to evaluate and optimize your method for finding those rare winning styles within an imbalanced dataset.