How important is witness evidence? A Bayesian perspective.

Further and deeper exploration of paradoxes and challenges of intuition and logic can be found in my recently published book, Probability, Choice and Reason.

New Amsterdam has 1,000 taxis. 850 are yellow, 150 are green.  One of these taxis accidentally knocks down a pedestrian and then drives away without stopping. We have no reason to believe that drivers of green taxis are any more or any less likely than drivers of yellow taxis to knock down a pedestrian and drive away. Neither do we have any reason to believe that green or yellow taxis are disproportionately represented in the area of New Amsterdam where the hit and run took place.

There is one witness, however, who did see the event. The witness says the colour of the taxi was green.

The witness is given a rigorous observation test, which recreates as closely as possible the event in question, and her judgment proves correct right 80 per cent of the time. We have no reason to doubt the integrity of the witness.

So what is the probability that the taxi was green?

The intuitive answer is in the region of 80 per cent, as the only evidence is that of the witness, and the test of her powers of observation shows that she is right 80 per cent of the time. That is not the Bayesian approach, however, which is to also consider the evidence in the light of the baseline, or prior, probability that the taxi was green before the witness evidence came to light.

The prior probability can be derived from an identification of the proportion of taxis in New Amsterdam that are green. This is 15 per cent (of the 1,000 taxis, 150 are green).

Now, the (posterior) probability that a hypothesis is true after obtaining new evidence, according to the x,y,z formula of Bayes’ Theorem, is equal to:

xy/[xy+z(1-x)]

x is the prior probability, i.e. the probability that a hypothesis is true before you the new evidence arises.

y is the probability the new evidence would arise if the hypothesis is true.

z is the probability the new evidence would arise if the hypothesis is false.

This is a straightforward calculation.

x = 0.15 (15 per cent of taxis are green)

y = 0.8 (the witness is correct 80 per cent of the time)

z = 0.2 (the witness is wrong 20 per cent of the time)

Inserting these numbers into the formula gives:

Posterior probability = 0.15 x 0.8/ (0.15×0.8 + 0.2×0.85) = 0.12/ (0.12+0.17) = 41%

In other words, the true probability that the taxi that knocked down the pedestrian was green is not 80 per cent (despite the witness evidence) but about half that. The baseline probability is that important.

But Bayesians are not content to leave it that. The next step is to look for further new evidence.

Say, for example, that a new witness appears, totally independent of the other, and is also given the observation test, revealing a reliability score of 90 per cent. Again, we have no reason to doubt the integrity of this witness. What a Bayesian does now is to insert that number (0.9) into the Bayes formula (y=0.9) so that z (the probability that the witness is mistaken) = 0.1.

The new baseline (or prior) probability, x, is no longer 0.15, as it was before the first witness appeared, but 0.41 (the probability incorporating the evidence of the first witness).

New posterior probability = 0.41 x 0.9/ (0.41×0.9 + 0.1×0.59) = 0.369/ (0.369+0.059) = 86.2%

This is also the new baseline probability underpinning any new evidence which might arise.

There are three illustrative cases which bear highlighting.

The first is a scenario where the new witness scores 50 per cent on the observation test. Here is a case where intuition and Bayes’ formula converge. Intuition tells us that a witness who is right only half the time about the colour of the taxi is also wrong half the time, and so any evidence they give is worthless. In terms of the equation, such a witness would be accorded y = 0.5 and z = 0.5.

Putting these values of y and z into the equation leads to the following:

xy/[xy+z(1-x)] BECOMES 0.5x / [0.5x + 0.5 (1-x)]

0.5x / [0.5x + 0.5 (1-x)] = 0.5x / (0.5 + 0.5x – 0.5x) = 0.5x / 0.5 = x

So when x and y both equal 0.5 in regard to new evidence, this evidence has no impact on the probability of the hypothesis being tested being true. The posterior probability (x) equals the prior probability (x).

In other words, when y = z = 0.5, the posterior probability equals the prior probability. In this case, the witness’s evidence can be discounted.

The second illustrative case is where a new witness is 100 per cent reliable about the colour of the taxi. In this case, y =1 and z =0. Intuition tells us that the evidence of such a witness solves the case. If the infallible witness says the taxi was green, it was green. Bayes’ formula agrees. Inserting y = 1, z = 0 into the formula gives:

xy/[xy+z(1-x)] = x / (x + 0) = x/x = 1.

So the new (posterior) probability that the taxi is green = 1.

This leads directly to the third illustrative case. If the new witness scores 0 per cent on the observation test, this indicates that they always identify the wrong colour for the taxi. If they say it is green, it is definitely not green. So the chance (posterior probability) that the taxi is green if they say so is zero. This accords with the formula.

xy/[xy+z(1-x)] = 0 / [0 + (1-x)] = 0

Of course, this is valuable information, as it can be reversed to useful effect. A witness who always identifies a green taxi as yellow and vice-versa, and is 100 per cent consistent in doing so, yields us infallible information simply by reversing their identified colour.

So if the witness says the taxi is yellow, we can now identify the taxi as definitely green. This now converges on the second illustrative case.

Similarly, a witness who is, say, 25 per cent accurate in identifying the colour of the taxi in the observation test also yields us valuable information. By reversing the identified colour, this yields a 75 per cent accuracy score, which can be inserted accordingly into Bayes’ formula to update the probability that the taxi that knocked down the pedestrian was green.

The only observation evidence that is worthless, therefore, is evidence that could have been produced by the flip of a fair coin.

And the conclusion to the case? CCTV evidence was later produced in court which was able to conclusively identify the taxi and the driver. The pedestrian never regained consciousness. The driver told the jury that he panicked when the pedestrian unexpectedly stepped out in front of him, and drove off because he feared he would lose his livelihood. He was completely unaware that the victim had hit his head awkwardly, and had thought at the time that it was a very minor accident.

This was rejected by the jury, who accepted the prosecution’s contention that he had acted with premeditation. They based their decision on their view that a driver who was so motivated would indeed have driven off. The taxi driver in this case did drive off, which was what someone who acted wilfully, deliberately and with premeditation would do. It was all the evidence they needed to reach their unanimous verdict.

James Parker, a 29-year-old long-time resident of New Amsterdam, of previous good character, with no previous convictions or any known motive for the crime, is currently serving a sentence of life in a maximum security prison with no possibility of parole.

Further Reading and Links

https://selectnetworks.net/