Bayes and the Bobby Smith Problem

A Tennis Tale with a Twist

A version of this article appears in my book, Twisted Logic: Puzzles, Paradoxes, and Big Questions (Chapman & Hall/CRC Press).

THE WORLD OF BOBBY SMITH, A BUDDING TENNIS PRODIGY

Bobby Smith, a young tennis player, faces daunting odds in his journey to professional status. In his world, 1 in 1,000 schoolboy tennis players make it to the professional ranks.

THE TEST: BOBBY’S GATEWAY TO THE ACADEMY

Bobby takes a crucial test to join the prestigious tennis academy, which serves as a breeding ground for future professionals. Though he passes the test and enters the academy, we must consider what this really indicates about his chances of turning pro.

THE FALLACY: MISINTERPRETING PROBABILITIES

It’s crucial not to confuse the probability of Bobby turning pro (given his academy entry) with the inverse, the probability of him entering the academy if destined to turn pro. While all professionals come from the academy, not all academy members become professionals.

THE GATEKEEPER: A SPECIAL TEST

Bobby is given a test designed to gauge the potential of young tennis players, which is used to determine who will have the privilege of becoming a member of the tennis academy, a training and nurturing ground for aspiring professionals. Bobby takes this test with the goal of securing membership.

THE CHALLENGE: OVERCOMING THE ODDS

The test is taken by a thousand of these budding tennis players, including Bobby, all of whom want to enter the academy. Just 5% of those tested will gain entry to the academy and then fail to become professional players.

Graduation from the academy is also a condition of entry to the professional tour in Bobby’s world. As such, we can rule out anyone who does not gain entry to the academy as a future professional player.

BOBBY’S TRIUMPH: ENTERING THE ACADEMY

Fortunately for Bobby, he aces the test and joins the academy. This is a crucial step for him. After all, every professional player in Bobby’s world, as we have noted, is a graduate of the academy.

It might now seem almost certain that he has a bright future ahead in the world of professional tennis. But is this a correct assessment?

Well, it is undeniable that without entrance to the academy there is no way for Bobby to achieve professional status, but he has aced the test and is now a member of the academy. Give the accuracy of the test in sifting talent, can we now look forward with some confidence to his future sporting career?

Well, determining the probability of Bobby becoming a professional tennis player if he scores well enough on the test to gain entry to the academy is a complex matter. It involves factors beyond just his entrance to the academy. Many other elements, such as his dedication, talent, and the competitive environment, play roles in determining his chances. Even so, it does look promising, or does it?

THE FALLACY: AN ILLUSION OF CERTAINTY

Back to the test result, we must be very careful not to confuse the probability of Bobby going on to a professional tennis career given his entrance to the academy with its inverse—the probability that he would enter the academy if he were to go on to attain the professional ranks.

In our example, the probability of his entrance to the academy given that Bobby will make it to professional circles is a sure thing. All future professional players will be graduates of the academy. What we seek to know, however, is something different—it is the probability that Bobby will become a professional player given that he enters the academy. This is a very different question.

Put another way, the fallacy arises from confusing two distinct probabilities:

The probability of a hypothesis being true (Bobby will become a professional tennis player) given some evidence (entrance to the academy).

The probability of the evidence (entrance to the academy) given the hypothesis is true (Bobby will become a professional player).

In simple terms, if we know that Bobby became a professional, he definitely went to the academy. But that’s not what we’re interested in. We want to know the odds of Bobby becoming a professional, given that he got into the academy.

So what is the actual chance that Bobby will become a professional tennis player if he scores well enough on the test to gain entry to the academy?

CALCULATING THE REAL PROBABILITY: BEWARE OF FALSE POSITIVES

When we dig deeper into the data, we uncover some revealing insights. Consider the 5% of students who pass the test and enter the academy but don’t go on to become professional players—they are the ‘false positives’ in our scenario. If we assume 1,000 students take the test, 50 such ‘false positives’ get into the academy.

Add to them the one student who does become a pro (from the original pool of 1,000), and you find that Bobby’s chances of turning pro, even after making it into the academy, are just 1 in 51. This translates to approximately 1.96%.

This will only change if we know some additional information about Bobby.

THE MEDICAL ANALOGY: VIRUS TESTING

Interestingly, this concept aligns with the ‘false positives’ problem in the medical field, particularly in regard to virus testing. Let’s take a group of 1,000 people getting tested for a certain virus. Even with a test accuracy of 95%, about 5% of those tested (50 individuals) will also test positive despite not carrying the virus. On top of these, one individual does have the virus. Thus, if you test positive, the probability of actually carrying the virus is again about 1.96%, unless there is some additional information we need to take into account.

A MATHEMATICAL ASSURANCE: BAYES’ THEOREM

Though we’ve already figured out Bobby’s chances of turning pro, there’s another way to confirm our findings. This alternative method involves Bayes’ theorem. This theorem helps us calculate the updated probability of a hypothesis (in our case, Bobby turning pro) after obtaining new evidence (Bobby entering the academy).

The formula is expressed as follows:

ab/[ab + c (1 − a)]

where

a is the prior probability, i.e. the probability that Bobby will turn pro before we know his test results (0.001, as Bobby is one among 1,000),

b is the probability of Bobby entering the academy if he will turn pro (which is 100%, as all pros in Bobby’s world are academy graduates), and

c is the probability of Bobby entering the academy if he won’t turn pro (which is 50 out of 999, as out of the 999 kids who won’t turn pro, 50 will enter the academy).

By plugging these values into Bayes’ theorem, we confirm that Bobby’s chances of becoming a professional, despite gaining entry to the academy, are not 95% as one might think, but around 1.96%.

CRUNCHING NUMBERS: THE HARD REALITY

To summarise, let’s analyse the situation numerically. Among the 1,000 kids applying for the academy, 50 will be accepted but won’t make it to professional status. One will eventually turn pro. So, out of the 51 kids admitted, only one will become a professional. Therefore, the chance of becoming a professional tennis player if you enter the tennis academy is 1 in 51, or roughly 1.96%, unless there is some additional information that we need to take into account.

THE TWIST: A SUCCESS STORY

Despite the low probability, Bobby turns out to be the exception. He defies the odds and ends up winning the Australian Open under a different name.

CONCLUSION: BEYOND THE NUMBERS

Bobby’s story highlights how statistical probabilities can mislead our intuition. Understanding these concepts is crucial, whether assessing the future of a tennis player or interpreting medical test results. Despite the odds, individuals like Bobby can defy statistics, reminding us that while numbers describe populations, they don’t predetermine individual destinies.