Can We Trust the Jury?

A Problem Revisited 

In a Nutshell

The Conviction

Jurors in the UK are not allowed to discuss their deliberations outside of the jury room. As such, the system is not self-correcting in any systematic way. When a jury goes wrong, we don’t know why and can’t learn from it. But sometimes we can apply common sense to guess where they erred. 

A classic case of a British miscarriage of justice (there have been plenty of others) is that of Sally Clark, a lawyer who was convicted of the murder of her two infant sons. It was a verdict simply waiting to happen in a system arguably built for failure. 

The Investigation and Trial: Building a Case on Uncertainty

Initially, the deaths of her children were assumed to be instances of so-called Sudden Infant Death Syndrome (SIDS), until suspicions were raised based on the coincidence of two deaths in the same family. Sally Clark was arrested, charged and brought to trial. 

Statistical Evidence: The Misinterpretation

A key moment in the trial was the production by the prosecution of a witness who traded in statistics that the jury presumably took on trust. The witness, a then respected paediatrician, asserted that the probability of two infants from the same family dying from SIDS was incredibly low, about 1 in 73 million. He compared the odds to backing a real longshot in the Grand National horse race four years in a row and winning each time.  

The Prosecutor’s Fallacy: The Dangerous Conflation of Probabilities

The flaws in the witness’s statistical argument were substantial and sadly very consequential. He had mistakenly assumed that the deaths of Clark’s children were unrelated or ‘independent’ events. There was no reason to assume this. Just for example, there may well be an underlying familial or genetic factor that might contribute to SIDS. This was assumed away. 

More subtly, but certainly no less devastatingly or dangerously, the argument represents a classic misinterpretation of probability known as the ‘Prosecutor’s Fallacy’. This fallacy conflates the probability of observing specific evidence if a hypothesis is true with the probability that the hypothesis is true given that evidence. These are two very different things but easy for a jury to confuse.

The Prosecutor’s Fallacy Explained

The fallacy arises from confusing two different probabilities:

1.       The probability of the evidence given innocence.

2.       The probability of innocence given the evidence.

The Need for Comparative Likelihood Assessment

The Royal Statistical Society emphasised the need to compare the likelihood of the deaths under each hypothesis—the hypothesis of SIDS and the hypothesis of murder.

Prior Probability: Understanding the Likelihood of Guilt before Observing the Evidence

Prior probability, a concept integral to understanding the Prosecutor’s Fallacy, is often overlooked in court proceedings. This term refers to the probability of a hypothesis (in this case, that Sally Clark is a double child killer) being true before any evidence is presented.

Given that she had no history of violence or harm towards her children, or anyone else, or any indication of such a tendency, the prior probability of her being a murderer would be extremely low. This needs to be compared to the likelihood of two cases of SIDS in a family group.

In a letter from the President of the Royal Statistical Society to the Lord Chancellor, Professor Peter Green explained the issue succinctly:

The jury needs to weigh up two competing explanations for the babies’ deaths: SIDS or murder. The fact that two deaths by SIDS is quite unlikely is, taken alone, of little value. Two deaths by murder may well be even more unlikely. What matters is the relative likelihood of the deaths under each explanation, not just how unlikely they are under one explanation.

Put another way, before considering the evidence, the prior probability of Clark being a murderer, given her background and lack of violent history, was extremely low. The prior probability of two SIDS deaths in one family, while rare, was still significantly higher than the prior probability of a mother murdering her two children. The rarity of two SIDS deaths alone doesn’t provide sufficient grounds for a murder conviction.

The Case of Lottie Jones

To illustrate the Prosecutor’s Fallacy, consider the fictional case of Lottie Jones, charged with winning the lottery by cheating. The fallacy occurs when the expert witness equates the low probability of winning the lottery (1 in 45 million) with the probability that a lottery win was achieved unfairly.

As in the Sally Clark case, the prosecution witness in this fictional parody commits the classic ‘Prosecutor’s Fallacy’. He assumes that the probability Lottie is innocent of cheating, given that she won the Lottery, is the same thing as the probability of her winning the Lottery if she is innocent of cheating. The former probability is astronomically higher than the latter unless we have some other indication that Lottie has cheated to win the Lottery. It is a clear example of how it is likely, in any large enough population, that things will happen that are improbable in any particular case. In other words, the 1 in 45 million represents the probability that a Lottery entry at random will win the jackpot, not the probability that a player who has won did so fairly!

Lottie just got very, very lucky just as Sally Clark got very, very unlucky.

The Aftermath: Tragedy and Lessons Learned

Sally Clark’s conviction was quashed in 2003, but she never recovered from her ordeal and sadly died just a few years later. Her story stands as testament to the potential for disastrous consequences when statistics are misunderstood or misrepresented. Even when quashing her conviction, the judgment was based primarily on other evidence, side-lining the role that the proper application of statistics and Bayesian probability should have brought to the case. 

O.J. Simpson: An Alternate Scenario

Even in high-profile cases, such as American former actor and NFL football star O.J. Simpson’s murder trial in the 1990s, this same misinterpretation of statistics is prevalent. Simpson’s defence team argued that it was unlikely Simpson killed his wife because only a small percentage of spousal abuse cases result in the spouse’s death. This argument, though statistically accurate, overlooks the relevant information—the fact that about 1 in 3 murdered women were killed by a spouse or partner. This represents a very clear case of the misuse of the Inverse or Prosecutor’s Fallacy in argumentation before a jury.

Conclusion: The Importance of Statistical Literacy

The importance of statistics in our justice system cannot be overstated. We must recognise the potential for misinterpretation and the potentially devastating results. A concerted effort to promote statistical literacy, particularly within our legal systems, can, if heeded, go a long way in preventing future miscarriages of justice, and rectifying current ones. In truth, however, little if any progress has been made in this regard, and we have a very long way to go!