When Should We Stop Looking and Start Choosing?

A version of this article appears in my book, Twisted Logic: Puzzles, Paradoxes, and Big Questions (Chapman & Hall/CRC Press, 2024).

WHEN TO STOP LOOKING AND START CHOOSING

The ‘Secretary Problem’ is a classic scenario in decision-making and probability theory. The ‘Optimal Stopping Problem’ or ‘Secretary Problem’, as it is often called, offers insights into the dilemma of when to stop looking and start choosing. Whether it is finding the right partner, hiring the best assistant, or identifying the ideal place to live, this mathematical problem delivers a powerful solution.

CHOOSING A CAR

Let’s say that you have 20 used cars to choose from, offered to you in a random sequence. You have three minutes to evaluate each. Once you turn one down, there is no returning to it, such is the speed of turnover, but the silver lining is that you are guaranteed the sale of any vehicle you do select. If you come to the end of the line, you must accept whatever remains, even if it happens to be the least desirable. Your decision is guided solely by the relative merits of the vehicles on offer.

BALANCING BETWEEN TOO EARLY AND TOO LATE

There are two significant failures in your quest to find the best vehicle for you—stopping too early and stopping too late. If you stop too early, you might miss out on a better option. Conversely, if you stop too late, you risk passing over the best option while waiting for a better option that might not exist. So, how do you find the right balance?

INTRODUCING THE OPTIMAL STOPPING STRATEGY

Do you have a strategy that is better than random selection?

The Optimal Stopping Problem provides a solution. If there are three cars in the flash sale, optimal stopping strategy suggests rejecting the first option in order to gain more information about the relative merits of those available. If the second option turns out to be worse, you should wait, despite the risk of ending up with the third, which could potentially be the worst of the three. However, if the second option is better, you should accept it immediately, foregoing the possibility that the third might be a better match.

EXTENDING THE STRATEGY: FROM 4 TO 100

With four options, you should reject the first. Again, if the second is better than the first, take that. If not, and the third is better, take that. Otherwise, you must take the fourth and hope for the best. With a hundred options, you should inspect the first 37 and then choose the first after that which is better than the best of the first 37.

This strategy, often referred to as the 37% Rule, is based on the mathematical constant, e (Euler’s number). The value of 1/e is approximately 0.36788% or 36.788%, which rounds up to 37%. Following this rule, you have a 37% chance of finding the best car by employing this strategy.

THE GROUNDWORK

When faced with a choice of n candidates for a job, the challenge lies in deciding when to stop the process of rejection and start the process of selection. The mathematical answer to this, as highlighted before, is to reject the first n/e candidates, where ‘e’ is the base of natural logarithms, approximately 2.7. So, if there are 100 choices, n/e becomes 100/2.7, which is about 37. This strategy effectively breaks the selection process into two phases: the assessment phase and the selection phase.

The resulting principle is, therefore, surprisingly straightforward: reject the first 37% of candidates to gather information about the quality of the pool, then select the next candidate who is better than anyone seen so far.

REAL-WORLD APPLICATIONS

While the Secretary Problem is a simplified and somewhat idealised situation, the 37% Rule can have valuable applications in real-world scenarios:

Job Hiring: Hiring managers can use the 37% rule as a strategic guideline during the candidate evaluation phase.

Home Hunting: The principle is also applicable as a heuristic when looking for a home to buy or rent, especially in a fast-moving msrket.

Online Shopping: This principle can also be useful when shopping online to streamline purchasing decisions. By reviewing a certain portion of available options before making a selection, shoppers can reduce the overwhelming array of choices and enhance their overall shopping efficiency and satisfaction.

CRITIQUES AND LIMITATIONS

While the 37% Rule provides a theoretically optimal solution to the Secretary Problem, it does have certain limitations:

Idealised Assumptions: The problem assumes that options are presented one at a time, in random order, and once rejected, they cannot be recalled.

Risk of Missing Out: Following the 37% Rule means you run the risk of the best option being rejected during the assessment phase.

Difficult to Determine the Total Pool: The problem assumes you know the total number of options upfront.

Emotional Considerations: The rule neglects emotional considerations, personal intuition, and human subjectivity.

ADAPTING THE RULE FOR UNCERTAINTY

The rule can be adapted if there is a probability that your selection of a range of options might opt out or be withdrawn. For example, if there is a 50% chance that the selection might opt out or be withdrawn after selecting it, then the 37% rule can be converted into a 25% rule, reflecting the added uncertainty. There is also a rule-of-thumb for when the aim is to select a good option, if not necessarily the best. Out of 100 options, for example, the square root rule suggests seeing the first ten (the square root of 100) and then selecting the first option of those remaining that is better than the best of those ten.

CONCLUSION: EXPLORATION AND EXPLOITATION

The Secretary Problem teaches us about the balance between exploration (gathering information) and exploitation (making a decision), offering a structured approach to navigating complex choices. Despite limitations, the Secretary Problem and the 37% Rule offer valuable insights into these trade-offs. It also provides a mathematically grounded approach to making complex decisions.