When Should We Trust the Jury?

Exploring a Courtroom Tragedy

A version of this article appears in Twisted Logic: Puzzles, Paradoxes, and Big Questions. By Leighton Vaughan Williams. Chapman and Hall/CRC Press, 2024.

THE CONVICTION

In the final weeks of the 20th century, a lawyer named Sally Clark was convicted of the murder of her two infant sons. Despite being a woman of good standing with no history of violent behaviour, Clark was swept up in a whirlwind of accusations, trials, and appeals that would besmirch the criminal justice system and cost her dearly.

THE INVESTIGATION AND TRIAL—BUILDING A CASE ON UNCERTAINTY

The deaths of Clark’s two children were initially assumed to be tragic instances of Sudden Infant Death Syndrome (SIDS), a cause of infant mortality that was not well understood even by medical experts. However, the authorities became suspicious of the coincidental deaths, leading to Clark’s eventual trial. As the investigation evolved, it subsequently transpired that numerous pieces of evidence helpful to the defence were withheld from them.

STATISTICAL EVIDENCE—THE MISINTERPRETATION

The prosecution presented a piece of seemingly damning statistical evidence during Clark’s trial. One of their witnesses, a paediatrician, asserted that the probability of two infants from the same family dying from SIDS was incredibly low—approximately 1 in 73 million. He compared the odds to winning a bet on a longshot in the iconic Grand National horse race four years in a row.

THE PROSECUTOR’S FALLACY—THE DANGEROUS CONFLATION OF PROBABILITIES

The flaws in the statistical argument presented at the trial were both substantial and consequential. The paediatrician had mistakenly assumed that the deaths of Clark’s children were unrelated, or ‘independent’ events. This assumption neglects the potential for an underlying familial or genetic factor that might contribute to SIDS.

Moreover, the paediatrician’s argument represents a common misinterpretation of probability known as the ‘Prosecutor’s Fallacy’. This fallacy involves conflating the probability of observing specific evidence if a hypothesis is true, with the probability that the hypothesis is true given that evidence. These are two very different things but easy for a jury of laymen to confuse.

THE PROSECUTOR’S FALLACY EXPLAINED

This fallacy arises from confusing two different probabilities:

The probability of observing specific evidence (in this case, two SIDS deaths) if a hypothesis (Clark’s guilt) is true.

The probability that the hypothesis is true given the observed evidence.

THE NEED FOR COMPARATIVE LIKELIHOOD ASSESSMENT

The Royal Statistical Society emphasised the need to compare the likelihood of the deaths under each hypothesis—SIDS or murder. The rarity of two SIDS deaths alone doesn’t provide sufficient grounds for a murder conviction.

PRIOR PROBABILITY—UNDERSTANDING THE LIKELIHOOD OF GUILT BEFORE THE EVIDENCE

Prior probability—a concept integral to understanding the Prosecutor’s Fallacy—is often overlooked in court proceedings. This term refers to the probability of a hypothesis (in this case, that Sally Clark is a child killer) being true before any evidence is presented.

Given that she had no history of violence or harm towards her children, or anyone else, or any indication of such a tendency, the prior probability of her being a murderer would be extremely low. In fact, the occurrence of two cases of SIDS in a single family is much more common than a mother murdering her two children.

The jury should weigh up the relative likelihood of the two competing explanations for the deaths. Which is more likely? Double infant murder by a mother or double SIDS?

More generally, it is likely in any large enough population that one or more cases of something highly improbable will occur in any particular case.

In a letter from the President of the Royal Statistical Society to the Lord Chancellor, Professor Peter Green explained the issue succinctly:

The jury needs to weigh up two competing explanations for the babies’ deaths: SIDS or murder. The fact that two deaths by SIDS is quite unlikely is, taken alone, of little value. Two deaths by murder may well be even more unlikely. What matters is the relative likelihood of the deaths under each explanation, not just how unlikely they are under one explanation.

Put another way, before considering the evidence, the prior probability of Clark being a murderer, given her background and lack of violent history, was extremely low. The probability of two SIDS deaths in one family, while rare, was still much higher than the likelihood of the mother murdering her two children.

The jury should weigh up the relative likelihood of the two competing explanations for the deaths. Which is more likely? Double infant murder by a mother or double SIDS?

More generally, it is likely in any large enough population that one or more cases of something highly improbable will occur in any particular case.

In a letter from the President of the Royal Statistical Society to the Lord Chancellor, Professor Peter Green explained the issue succinctly:

The jury needs to weigh up two competing explanations for the babies’ deaths: SIDS or murder. The fact that two deaths by SIDS is quite unlikely is, taken alone, of little value. Two deaths by murder may well be even more unlikely. What matters is the relative likelihood of the deaths under each explanation, not just how unlikely they are under one explanation.

Put another way, before considering the evidence, the prior probability of Clark being a murderer, given her background and lack of violent history, was extremely low. The probability of two SIDS deaths in one family, while rare, was still much higher than the likelihood of the mother murdering her two children.

THE NEED FOR COMPARATIVE LIKELIHOOD ASSESSMENT

The Royal Statistical Society emphasised the need to compare the likelihood of the deaths under each hypothesis—SIDS or murder. The rarity of two SIDS deaths alone doesn’t provide sufficient grounds for a murder conviction.

The Fictional Case of Lottie Jones

To illustrate the Prosecutor’s Fallacy, consider the fictional case of Lottie Jones, charged with winning the lottery by cheating. The fallacy occurs when the expert witness equates the low probability of winning the lottery (1 in 45 million) with the probability that a lottery win was achieved unfairly.

As in the Sally Clark case, the prosecution witness in this fictional parody commits the classic ‘Prosecutor’s Fallacy’. He assumes that the probability Lottie is innocent of cheating, given that she won the Lottery, is the same thing as the probability of her winning the Lottery if she is innocent of cheating. The former probability is astronomically higher than the latter unless we have some other indication that Lottie has cheated to win the Lottery. It is a clear example of how it is likely, in any large enough population, that things will happen that are improbable in any particular case. In other words, the 1 in 45 million represents the probability that a Lottery entry at random will win the jackpot, not the probability that a player who has won did so fairly!

Lottie just got very, very lucky just as Sally Clark got very, very unlucky.

THE AFTERMATH—TRAGEDY AND LESSONS LEARNED

Following her acquittal in 2003, Sally Clark never recovered from her ordeal and sadly died just a few years later. Her story stands as testament to the potential for disastrous consequences when statistics are misunderstood or misrepresented.

O.J. SIMPSON—AN ALTERNATE SCENARIO

Even in high-profile cases, such as American former actor and NFL football star O.J. Simpson’s murder trial in the 1990s, this same misinterpretation of statistics is prevalent. Simpson’s defence team argued that it was unlikely Simpson killed his wife because only a small percentage of spousal abuse cases result in the spouse’s death. This argument, though statistically accurate, overlooks the relevant information—the fact that about 1 in 3 murdered women were killed by a spouse or partner. This represents a very clear case of the misuse of the Inverse or Prosecutor’s Fallacy in argumentation before a jury.

CONCLUSION: THE IMPORTANCE OF STATISTICAL LITERACY

The importance of statistics in our justice system cannot be overstated. We must recognise the potential for misinterpretation and the potentially devastating results. A concerted effort to promote statistical literacy, particularly within our legal systems, can hopefully go a long way in preventing future miscarriages of justice.