When Should We Believe the Eyewitness?

Bayes and the Taxi Problem

A Version of this article appears in Twisted Logic: Puzzles, Paradoxes, and Big Questions. By Leighton Vaughan Williams. Chapman & Hall/CRC Press. 2024.

THE BASICS OF THE TAXI PROBLEM

Let’s set the stage for our story. We’re in New Brighton, a city with a fleet of 1,000 taxis. Of these, 850 are blue and 150 are green. One day, a taxi is involved in an accident with a pedestrian and leaves the scene. We don’t know the colour of the taxi, and we don’t have any reason to believe that blue or green taxis are more likely to be involved in such incidents.

An independent eyewitness now comes forward. She saw the accident and claims that the taxi was green. To verify the reliability of her account, investigators conduct a series of observation tests designed to recreate the conditions of the incident. These tests reveal that she is correct about the colour of a taxi in similar conditions 80% of the time.

So, what is the likelihood that the taxi involved was actually green?

INITIAL PROBABILITIES AND INTUITIVE ESTIMATES

Your first instinct might be to believe that the chance that the taxi was green is around 80%. This assumption is based on the witness’s track record of identifying the colour of a taxi accurately. However, this conclusion doesn’t consider other crucial information—the overall number of blue and green taxis in the city.

Given the total taxi population, only 15% of them are green (150 out of 1,000), while a substantial 85% are blue. Ignoring this ‘base rate’ of taxi colours leads to a common mistake known as the ‘Base Rate Fallacy’.

APPLYING BAYES’ THEOREM TO THE TAXI PROBLEM

Bayes’ theorem is a method that helps us adjust our initial estimates based on new evidence but allowing for this base rate of the total numbers of blue and green taxis. In this way, it offers a means of updating our initial estimates after taking account of some new evidence.

For our Taxi Problem, the new evidence is the witness statement. The witness says the taxi was green, and we know that there’s an 80% chance that she is correct if the taxi was indeed green (based on her observation test). But there’s also a 20% chance that she would mistakenly say the taxi was green if it were blue.

Bayes’ theorem helps us adjust initial beliefs with new evidence, considering the base rate. Here’s how it works in the Taxi Problem:

Prior Probability: Initially, there’s only a 15% chance (150 out of 1,000 taxis) that the taxi is green.

Conditional Probability of Green Taxi (If Witness Correct): The eyewitness is correct 80% of the time.

Conditional Probability of Green Taxi (If Witness Incorrect): There’s a 20% chance the eyewitness would mistakenly identify a blue taxi as green.

After applying Bayes’ theorem, the adjusted (or ‘posterior’) probability that the taxi is green is just 41%, using the formula: ab/[ab + c (1 − a)].

THE ROLE OF NEW EVIDENCE AND MULTIPLE WITNESSES

What happens if another eyewitness comes forward? Suppose this second witness also reports that the taxi was green and, after a similar set of tests, is found to be correct 90% of the time. Now we should recalculate the probabilities using the same principles of Bayes’ theorem but including the new evidence.

The updated ‘prior’ probability is no longer the original 15%, but the 41% we calculated after hearing from the first witness. After running the numbers again, using Bayes’ formula, the revised probability that the taxi was green increases to 86%.

INTERPRETING WITNESS TESTIMONIES WITH BAYES’ THEOREM

Let’s dive a bit deeper into the implications of these results. Here are some situations that may seem counterintuitive at first, but make sense when we apply Bayes’ theorem:

The 50-50 Witness: Suppose we have a witness who is only right half the time—in other words, they are as likely to be right as they are to be wrong. Our intuition tells us that such a witness is adding no useful information, and Bayes’ theorem agrees. The testimony of such a witness doesn’t change our prior estimate.

The Perfect Witness: Now, imagine a witness who is always right—they have a 100% accuracy rate in identifying the taxi colour. In this case, if they say the taxi was green, then it must have been green. Bayes’ theorem concurs with this conclusion.

The Always-Wrong Witness: What about a witness who always gets the colour wrong? In this case, if they say the taxi is green, then it must have been blue. Bayes’ theorem agrees. We can trust this witness by assuming the opposite of what they say is the truth.

In summary, a 50% accurate witness adds no value to our estimate. A 100% accurate witness’s testimony is definitive. An always-wrong witness inversely confirms the truth.

THE BASE RATE FALLACY AND ITS IMPLICATIONS

The Base Rate Fallacy occurs when we don’t give enough weight to ‘base rate’ information (like the overall number of blue and green taxis) when making probability judgments. This mistake can lead us to overvalue specific evidence (like a single eyewitness account) and undervalue more general information like the ratio of blue to green taxis. Even so, the eyewitness may still be correct.

Again, if someone loves talking about books, we might intuitively guess that they are more likely to work in a bookstore or library than as, say, a nurse. But there are many more nurses than there are librarians or bookstore employees, and many of them love books. So, taking account of the base rate, we may well conclude that it’s more likely that the book enthusiast is a nurse than a bookstore employee or librarian.

AVOIDING THE BASE RATE FALLACY

The Base Rate Fallacy leads us to ignore general information (like the ratio of blue to green taxis or nurses to librarians) in favour of specific evidence (an eyewitness account or specific bit of information). It’s essential to balance specific and general information to avoid skewed judgments.

THE UNVEILING OF THE TRUTH

In the case of the New Brighton Taxi Problem, the mystery was solved when CCTV footage surfaced. The taxi involved was revealed to be yellow, a twist no one expected. Not really—there are no yellow taxis in New Brighton. In fact, both eyewitnesses were correct and the taxi was green.

CONCLUSION: TRUTH AND TESTIMONY

While our story was hypothetical, the principles it illustrates are very real and applicable in a wide variety of situations and circumstances. Bayes’ theorem, base rates, and new evidence are all important parts of the detective’s toolkit.