Optimal Stopping: The 1/e Rule

Welcome back, Detective.

Our initial analysis of the "Siren's Silhouette" contest showed that a winning score of 99.8, while impressive, wasn't a statistical unicorn. It fell just one standard deviation above the expected maximum under a simple random scoring model. This means the score itself isn't our smoking gun. However, this conclusion was based on the assumption that all 20 designs were reviewed before a winner was chosen.

The anonymous tipster's message included a cryptic but crucial phrase: "They didn't even look at all the finalists properly." This clue shifts our focus from the outcome of the contest to its process. What if the judges made their decision sequentially, without seeing the full set of options? This presents a classic problem of decision-making under uncertainty, a domain where your operations research background will be particularly insightful.

This leads us to the next phase of our investigation. Today, your objective is to apply the optimal stopping rule (the 1/e rule) to determine the ideal point to stop evaluating candidates and make a selection in a sequential process. We will explore whether the contest's procedure, rather than its outcome, contains the evidence of a pre-planned "setup."

The Core Dilemma: To Stop or to Continue?

Imagine the judges at Aphrodite's Veil, evaluating the 20 finalist lingerie designs one by one. After viewing each design, they must make an irrevocable decision: select it as the winner and end the contest, or reject it forever and move to the next.

If they choose too early, they risk missing a truly spectacular design that comes later. If they wait too long, they might have already passed on the best one. This is a high-stakes balancing act between exploration and exploitation.

The following video from Veritasium provides an excellent overview of this exact type of problem.

Why is this number everywhere?

This video, titled "Why is this number everywhere?", introduces the fundamental challenge of sequential decision-making.

Please watch the segment from the setup of the problem. It frames the choice you face when you can't assess all options at once and must make immediate, final decisions. Notice how the host defines the general strategy: first, you observe and reject some options to learn, and then you switch to an evaluation mode.

This scenario is formally known as the Secretary Problem. Its solution provides a mathematically optimal strategy for maximizing the probability of selecting the single best option from a sequence.

The "Look-Then-Leap" Strategy

The optimal strategy for the secretary problem has an elegant structure called "look-then-leap." It consists of two distinct phases:

The "Look" Phase (Exploration): You review an initial set of candidates (let's say, the first r of them) purely to gather information. You note the best one you've seen so far but commit to rejecting all of them, no matter how good they are. This phase establishes your benchmark.
The "Leap" Phase (Exploitation): After rejecting the first r candidates, you enter the decision-making phase. You now select the very first candidate who is better than the benchmark you established in the "look" phase.

If no candidate in the "leap" phase surpasses the benchmark, you are forced to select the very last candidate to fulfill the requirement of making a choice.

This strategy is detailed in the following article, which you may find provides a clear, practical perspective on the problem.

The secretary problem - Pain is inevitable. Suffering is optional.

This article by Changyao Chen frames the problem and clearly defines the "look-then-leap" strategy.

First, read the introduction in The problem section to solidify the rules. Then, focus on the section describing the strategy, which formalizes the two-phase approach we just discussed.

The crucial question, of course, is what is the optimal value of r? How many designs should the judges have reviewed simply to set their standard?

The Magic Number: 1/e

The mathematical solution to this problem is both beautiful and surprisingly universal. It states that the optimal number of candidates to sample in the "look" phase is $n / e$ , where $n$ is the total number of candidates and $e$ is Euler's number (approximately 2.718). This is often called the 37% Rule, since $1/ e \approx 0.368$ .

The optimal strategy is therefore:
Reject the first 37% of the candidates. Then, hire the first one you see who is better than anyone you saw in that initial 37%.

Remarkably, the probability of selecting the single best candidate using this method also converges to $1/ e$ , or about 37%. This may not sound high, but it is provably the best you can do under these constraints.

The derivation involves maximizing the probability of success, which can be expressed as a sum that approximates an integral. The video you watched earlier provides an excellent walkthrough of this derivation.

Why is this number everywhere?

This segment of the Veritasium video derives the 1/e result using calculus.

Watch the derivation starting from the calculation. The key is seeing how the probability of success, P, is formulated as a function of the stopping point, S, and total options, N. The expression P \approx (S/N) \ln(N/S) is maximized when S/N = 1/e. Given your background, you'll recognize this as a standard optimization problem solved by taking the derivative and setting it to zero.

This strategy has applications far beyond hiring secretaries, extending to dating, apartment hunting, and even parking. Your work as a relationship analyzer might even find this "soulmate algorithm" amusingly relevant.

A humorous take on applying the optimal stopping rule to dating, highlighting the cold logic of the 37% rule. The core idea is to establish a baseline before committing.

Case File: Re-examining the "Siren's Silhouette" Contest

Let's apply this rule to our case.

Total number of finalist designs, $n = 20$ .

According to the 1/e rule, the optimal number of designs to review in the "look" phase is:
$r = \frac{n}{e} = \frac{20}{2.718} \approx 7.35$

Since we can't review a fraction of a design, the judges should have used the first 7 designs to establish their quality benchmark.

The optimal strategy would have been:

Look Phase: Scrutinize designs #1 through #7 and note the highest score among them. Reject all 7.
Leap Phase: Starting with design #8, select the first design that scored higher than the best of the first 7.

This gives us a new hypothesis. The tipster's claim that "they didn't even look at all the finalists properly" might not imply lazy or biased judging. It could describe the dispassionate application of this optimal stopping algorithm. If the winning design was, for example, the 8th one evaluated and it happened to be a star, the contest would have ended right there, leaving the remaining 12 designs unseen.

Let's consider the odds. A random guess would give the judges a $1/20 = 5%$ chance of picking the best design. By using the optimal stopping rule, their chances jump to approximately 37%. For an efficiency expert, this is a compelling improvement.

The graph below visualizes this advantage. For a small number of candidates, the difference isn't dramatic. But as the number of candidates grows, the probability of success with random selection plummets toward zero, while the optimal stopping strategy holds steady at a robust 37%.

This graph compares the success probability of the optimal stopping rule (blue dots) against random selection (green crosses) as the number of candidates (n) increases. The optimal strategy consistently performs around the 1/e mark (red dashed line), proving its superiority and stability.

Conclusion

Our investigation has taken a sharp turn. The clue that initially suggested sloppiness might instead point to a sophisticated, if ruthless, efficiency. We have established a plausible, mathematically-grounded scenario for why the judges might have concluded the contest early.

Your key takeaways from today's analysis are:

Optimal Stopping: For sequential decisions with no recall, the "secretary problem" provides the optimal strategy for maximizing the chance of selecting the best candidate.
The 1/e Rule: The strategy involves a "look-then-leap" approach. You first explore $n / e$ (or ~37%) of the options to set a benchmark, then you select the next option that exceeds this benchmark.
A New Hypothesis: The early conclusion of the "Siren's Silhouette" contest may not have been a sign of a fix, but rather the result of applying this optimal algorithm. The judges may have been playing the odds perfectly.

Preview of the Next Lesson

We have a powerful new theory, but it rests on a key assumption: that the goal was to find the single best design. This strategy accepts a ~63% chance of failure in exchange for the best possible shot at a grand slam. But what if the studio's goal was different? What if they wanted to ensure the winner was good enough, with a low risk of picking a dud? For that, you don't need to find the absolute maximum; you need to estimate the average quality of the candidates. To do that reliably, how many do you need to see?

In our next session, we will investigate this alternative motive by exploring how to calculate the required sample size to estimate a parameter with a specified level of confidence. This will help us determine if a different kind of statistical reasoning could have been at play.