Much of our thinking is flawed because it is based on faulty intuition. But by using the framework and tools of probability and statistics, we can overcome this to provide solutions to many real-world problems and paradoxes. Further and deeper exploration of paradoxes and challenges of intuition and logic can be found in my recently published book, Probability, Choice and Reason.
When it comes to situations like waiting for a bus, our intuition is often wrong.
Imagine, there’s a bus that arrives every 30 minutes on average and you arrive at the bus stop with no idea when the last bus left. How long can you expect to wait for the next bus? Intuitively, half of 30 minutes sounds right, but you’d be very lucky to wait only 15 minutes.
Say, for example, that half the time the buses arrive at a 20-minute interval and half the time at a 40-minute interval. The overall average is now 30 minutes. From your point of view, however, it is twice as likely that you’ll turn up during the 40 minutes interval than during the 20 minutes interval.
This is true in every case except when the buses arrive at exact 30-minute intervals. As the dispersion around the average increases, so does the amount by which the expected wait time exceeds the average wait. This is the Inspection Paradox, which states that whenever you “inspect” a process, you are likely to find that things take (or last) longer than their “uninspected” average. What seems like the persistence of bad luck is simply the laws of probability and statistics playing out their natural course.
Once made aware of the paradox, it seems to appear all over the place.
For example, let’s say you want to take a survey of the average class size at a college. Say that the college has class sizes of either 10 or 50, and there are equal numbers of each. So the overall average class size is 30. But in selecting a random student, it is five times more likely that he or she will come from a class of 50 students than of 10 students. So for every one student who replies “10” to your enquiry about their class size, there will be five who answer “50”. The average class size thrown up by your survey is nearer 50, therefore, than 30. So the act of inspecting the class sizes significantly increases the average obtained compared to the true, uninspected average. The only circumstance in which the inspected and uninspected average coincides is when every class size is equal.
We can examine the same paradox within the context of what is known as length-based sampling. For example, when digging up potatoes, why does the fork go through the very large one? Why does the network connection break down during download of the largest file? It is not because you were born unlucky but because these outcomes occur for a greater extension of space or time than the average extension of space or time.
Once you know about the Inspection Paradox, the world and our perception of our place in it are never quite the same again.
Another day you line up at the medical practice to be tested for a virus. The test is 99% accurate and you test positive. Now, what is the chance that you have the virus? The intuitive answer is 99%. But is that right? The information we are given relates to the probability of testing positive given that you have the virus. What we want to know, however, is the probability of having the virus given that you test positive. Common intuition conflates these two probabilities, but they are very different. This is an instance of the Inverse or Prosecutor’s Fallacy.
The significance of the test result depends on the probability that you have the virus before taking the test. This is known as the prior probability. Essentially, we have a competition between how rare the virus is (the base rate) and how rarely the test is wrong. Let’s say there is a 1 in 100 chance, based on local prevalence rates, that you have the virus before taking the test. Now, recall that the test is wrong one time in 100. These two probabilities are equal, so the chance that you have the virus when testing positive is 1 in 2, despite the test being 99% accurate. But what if you are showing symptoms of the virus before being tested? In this case, we should update the prior probability to something higher than the prevalence rate in the tested population. The chance you have the virus when you test positive rises accordingly. We can use Bayes’ Theorem to perform the calculations.
In summary, intuition often lets us down. Still, by applying the methods of probability and statistics, we can defy intuition. We can even resolve what might seem to many the greatest mystery of them all – why we seem so often to find ourselves stuck in the slower lane or queue. Intuitively, we were born unlucky. The logical answer to the Slower Lane Puzzle is that it’s exactly where we should expect to be!
When intuition fails, we can always use probability and statistics to look for the real answers.
Leighton Vaughan Williams, Professor of Economics and Finance at Nottingham Business School. Read more in Leighton’s new publication Probability, Choice and Reason.
Would You Switch or Stick?
In a Nutshell
It’s a game show, and you have a chance of winning a brand gleaming new red sports car. All you need to do is choose which of three identical doors leads to the car. They are labelled 1, 2 and 3.
You choose Door 1.
The host, name of Monty Hall, knows where the car is, and opens Door 3, which he knows leads to a goat. He might equally well have opened Door 2, if that concealed the goat. So Door 3 is now open.
No problem – you’d prefer the car!
There are two unopened doors left. Door 1, which you chose, and Door 2. It looks like you now have a 50/50 chance of winning the car.
At this point, Monty offers you a choice, to stick with Door 1 or switch to Door 2. He always makes this offer, so there’s no clue in the offer.
But does it matter what you do – switch or stick?
It certainly looks as though it shouldn’t. There are two doors left and the car must be behind one of them. Surely it’s now 50-50?
It isn’t!
Marilyn vos Savant
The problem became famous when it appeared in Marilyn vos Savant’s column in Parade magazine. She said you should switch.
Switching, she said, gives you a 2 in 3 chance of winning the car. Staying with your original choice gives you only a 1 in 3 chance.
How come?
When you first choose Door 1, the chance that you’ve picked the car is 1 in 3. That means there’s a 2 in 3 chance that the car is behind one of the other two doors.
Monty then opens one of those doors. But because he knows where the car is, he must open a door with a goat behind it.
That’s the key to the puzzle. He is forced to open a door revealing a goat – he can’t reveal the car. Two times in three your original choice is wrong. In those cases Monty has no choice: one of the other doors leads to the car, so he must open the other one, which leads to a goat. One time in three your original choice is right, in which case both the other doors lead to goats and he can open either.
Your original door doesn’t suddenly become more likely to lead to the car when he opens the other door. It still has the same 1 in 3 chance it had when you chose it.
So you should switch, and give yourself a 2/3 chance of the car!
Look at It Another Way
There are only three possibilities.
If the car is behind Door 1, you switch and lose.
If the car is behind Door 2, Monty opens Door 3. You switch to Door 2 and win.
If the car is behind Door 3, Monty opens Door 2. You switch to Door 3 and win.
So switching wins in two of the three possible cases.
But if you’re still not convinced, let’s make the game a bit bigger.
Pick a Card
Instead of three doors, imagine 52 cards lying face down on a table. One of them is the Ace of Spades.
You choose one card. Now, what is the chance you’ve actually picked the Ace of Spades?
It’s 1 in 52.
Now suppose Monty knows where the Ace of Spades is. He turns over 50 of the other 51 cards and every one of them is not the Ace of Spades.
There are now just two cards left.
Your original card had a 1 in 52 chance of being the Ace of Spades. Monty turned over 50 cards he knows are not the Ace of Spades. That doesn’t change the odds about your original choice. It’s still 1/52. If he had no clue where the Ace of Spades was and still turned over 50 cards blind, and only two cards remained, now we are looking at a genuine 50/50. But he knew where it was all along.
In this case, switching wins 51 times out of 52.
So would you switch now?
I would!
The Inspection Paradox Revisited
In a Nutshell
Imagine a bus that arrives, on average, every 20 minutes. Sometimes it comes a little sooner, sometimes a little later. But it’s 20 minutes on average.
You arrive at the bus stop at a random time. How long should you expect to wait?
Ten minutes seems the obvious answer – half way between the average 20 minute interval between buses. If the buses arrived precisely every 20 minutes, that would be the right answer. But in reality they don’t – sometimes they arrive a little early and sometimes a little late.
The Bus Stop
Let’s make the example very simple. Suppose half the intervals between buses are 10 minutes and the other half are 30 minutes.
The average interval is still 20 minutes.
But now when you turn up at the bus stop, you are three times as likely to arrive during one of the 30-minute gaps as during one of the 10-minute gaps. It’s simply a longer time period in which you could pitch up.
If you do arrive during a 30-minute interval, your average wait will be 15 minutes. If you arrive during a 10-minute interval, it will on average be just 5 minutes.
So your expected waiting time isn’t 10 minutes at all. It is:
(3 × 15 + 1 × 5) / 4 = 12.5 minutes.
Something rather strange has happened. The average gap between buses is 20 minutes, yet if you arrive at a random time you should expect to wait 12.5 minutes rather than 10.
You haven’t been unlucky. You’ve encountered the Inspection Paradox.
What’s Going On?
The explanation is actually quite simple.
When we inspect something at random, we are more likely to encounter the things that take up more space or more time. And this doesn’t just apply to buses.
How Big Is Your Class?
Suppose a college tells you that its average class size is 30 students. You check this by stopping students at random and asking how many are in their class. Now, let’s say half the university’s classes contain 10 students and the other half contain 50. The average class size really is 30.
But you aren’t equally likely to come across someone from the classes of 10 and the classes of 50. There are five times as many students available to be interviewed from a class of 50 as from a class of 10.
So for every student answering “10”, we should expect five to answer “50”.
The average class size reported by the students will therefore be:
(5 × 50 + 1 × 10) / 6 = 43.3.
The university says the average class size is 30. The students say it is more than 43.
And both are right!
They are simply answering different questions. The university is averaging across classes. Our survey is effectively averaging across students.
How Long Do Students Spend in the Library?
Here’s another example. Take your clipboard into a university library and ask the students you come across how long they usually stay in the library. Obviously, your sample will contain a disproportionate number of long-stay students, simply because they are more likely to come into contact with you. The short stay students have in large part left the library by the time you arrive and start and finish asking questions – or else they haven’t yet arrived.
It’s the bus stop story all over again, in another guise. Similarly, you might experience longer restaurant waits or supermarket queues more frequently simply because longer waits and queues are around for longer and therefore provide more opportunities for you to encounter them.
Why Do I Always Hit the Biggest Potato?
There’s an even simpler version of the same idea. You’re digging potatoes and accidentally slice through one with your spade. It’s an unusually big potato. Coincidence? Not at all. There’s more chance of the spade striking a bigger potato for the very simple reason it takes up more space. Same idea when your internet connection fails while you’re downloading that monster size file. There’s simply more time in which something can go wrong.
The Lesson
The Inspection Paradox is one of those statistical ideas that does seem rather odd until you see what’s going on. When there’s more time to enter the scene, more space to bump into, we’re more likely to be there. And that has implications which are sometimes counterintuitive. So next time you’ve been standing at a bus stop for 15 minutes waiting for a bus that supposedly comes “every 20 minutes”, don’t immediately conclude that you’ve been unlucky. It may just be the Inspection Paradox.
Then again, the bus might simply be late!
A Problem Revisited
In a Nutshell
The Conviction
Jurors in the UK are not allowed to discuss their deliberations outside of the jury room. As such, the system is not self-correcting in any systematic way. When a jury goes wrong, we don’t know why and can’t learn from it. But sometimes we can apply common sense to guess where they erred.
A classic case of a British miscarriage of justice (there have been plenty of others) is that of Sally Clark, a lawyer who was convicted of the murder of her two infant sons. It was a verdict simply waiting to happen in a system arguably built for failure.
The Investigation and Trial: Building a Case on Uncertainty
Initially, the deaths of her children were assumed to be instances of so-called Sudden Infant Death Syndrome (SIDS), until suspicions were raised based on the coincidence of two deaths in the same family. Sally Clark was arrested, charged and brought to trial.
Statistical Evidence: The Misinterpretation
A key moment in the trial was the production by the prosecution of a witness who traded in statistics that the jury presumably took on trust. The witness, a then respected paediatrician, asserted that the probability of two infants from the same family dying from SIDS was incredibly low, about 1 in 73 million. He compared the odds to backing an 80 to 1 longshot in the Grand National horse race four years in a row and winning each time.
The Prosecutor’s Fallacy: The Dangerous Conflation of Probabilities
The flaws in the witness’s statistical argument were substantial and sadly very consequential. Aside from the apparent calculation error, he had mistakenly assumed that the deaths of Clark’s children were unrelated or ‘independent’ events. There was no reason to assume this. Just for example, there may well be an underlying familial or genetic factor that might contribute to SIDS. This was assumed away.
More subtly, but certainly no less devastatingly or dangerously, the argument represents a classic misinterpretation of probability known as the ‘Prosecutor’s Fallacy’. This fallacy conflates the probability of observing specific evidence if a hypothesis is true with the probability that the hypothesis is true given that evidence. These are two very different things but easy for a jury to confuse.
The Prosecutor’s Fallacy Explained
The fallacy arises from confusing two different probabilities:
1. The probability of the evidence given innocence.
2. The probability of innocence given the evidence.
The Need for Comparative Likelihood Assessment
The Royal Statistical Society emphasised the need to compare the likelihood of the deaths under each hypothesis—the hypothesis of SIDS and the hypothesis of murder.
Prior Probability: Understanding the Likelihood of Guilt before Observing the Evidence
Prior probability, a concept integral to understanding the Prosecutor’s Fallacy, is often overlooked in court proceedings. This term refers to the probability of a hypothesis (in this case, that Sally Clark is a double child killer) being true before any evidence is presented.
Given that she had no history of violence or harm towards her children, or anyone else, or any indication of such a tendency, the prior probability of her being a murderer would be extremely low. This needs to be compared to the likelihood of two cases of SIDS in a family group.
In a letter from the President of the Royal Statistical Society to the Lord Chancellor, Professor Peter Green explained the issue succinctly:
The jury needs to weigh up two competing explanations for the babies’ deaths: SIDS or murder. The fact that two deaths by SIDS is quite unlikely is, taken alone, of little value. Two deaths by murder may well be even more unlikely. What matters is the relative likelihood of the deaths under each explanation, not just how unlikely they are under one explanation.
Put another way, before considering the evidence, the prior probability of Clark being a murderer, given her background and lack of violent history, was extremely low. The prior probability of two SIDS deaths in one family, while rare, was still significantly higher than the prior probability of a mother murdering her two children. The rarity of two SIDS deaths alone doesn’t provide sufficient grounds for a murder conviction.
The Case of Lottie Jones
To illustrate the Prosecutor’s Fallacy, consider the fictional case of Lottie Jones, charged with winning the lottery by cheating. The fallacy occurs when the expert witness equates the low probability of winning the lottery (1 in 45 million) with the probability that a lottery win was achieved unfairly.
As in the Sally Clark case, the prosecution witness in this fictional parody commits the classic ‘Prosecutor’s Fallacy’. He assumes that the probability Lottie is innocent of cheating, given that she won the Lottery, is the same thing as the probability of her winning the Lottery if she is innocent of cheating. The former probability is astronomically higher than the latter unless we have some other indication that Lottie has cheated to win the Lottery. It is a clear example of how it is likely, in any large enough population, that things will happen that are improbable in any particular case. In other words, the 1 in 45 million represents the probability that a Lottery entry at random will win the jackpot, not the probability that a player who has won did so fairly!
Lottie just got very, very lucky just as Sally Clark got very, very unlucky.
The Aftermath: Tragedy and Lessons Learned
Sally Clark’s conviction was quashed in 2003, but she never recovered from her ordeal and sadly died just a few years later. Her story stands as testament to the potential for disastrous consequences when statistics are misunderstood or misrepresented. Even when quashing her conviction, the judgment was based primarily on other evidence, side-lining the role that the proper application of statistics and Bayesian probability should have brought to the case.
O.J. Simpson: An Alternate Scenario
Even in high-profile cases, such as American former actor and NFL football star O.J. Simpson’s murder trial in the 1990s, this same misinterpretation of statistics is prevalent. Simpson’s defence team argued that it was unlikely Simpson killed his wife because only a small percentage of spousal abuse cases result in the spouse’s death. This argument, though statistically accurate, overlooks the relevant information—the fact that about 1 in 3 murdered women were killed by a spouse or partner. This represents a very clear case of the misuse of the Inverse or Prosecutor’s Fallacy in argumentation before a jury.
Conclusion: The Importance of Statistical Literacy
The importance of statistics in our justice system cannot be overstated. We must recognise the potential for misinterpretation and the potentially devastating results. A concerted effort to promote statistical literacy, particularly within our legal systems, can, if heeded, go a long way in preventing future miscarriages of justice, and rectifying current ones. In truth, however, little if any progress has been made in this regard, and we have a very long way to go!
Where are the Aliens?
In a Nutshell
Where Is Everybody?
We are in the 1950s and the world is on the brink of the Space Age. Enrico Fermi, a Nobel Prize-winning physicist known for his work on the Manhattan Project, poses a question during a casual lunch conversation.
“Where is everybody?”
Given the vastness of the universe, why is there no evidence or contact with any extra-terrestrial civilisations?
The Age and Size of the Universe
The age and size of the universe are key aspects of the Fermi Paradox. The universe is approximately 13.8 billion years old, and the Milky Way galaxy, where our solar system resides, is about 13.6 billion years old. This vast timescale implies that if the evolution of life and development of technological civilisations is a common process, there should have been ample time for numerous advanced civilisations to arise. The Milky Way alone is home to an estimated 100 billion to 400 billion stars, many of which may host their own planets. The opportunities for life are in principle enormous. Recent astronomical discoveries make the question even more interesting, including the identification of thousands of exoplanets, many of which are in the habitable zone of their stars.
So where is everybody?
Technological Advancement and Singularity
The concept of technological singularity is the idea of a point where technological growth becomes uncontrollable and irreversible. If other civilisations have reached singularity, leading to exponential growth in their capabilities, why is there no evidence of their existence? If we consider the rapid pace of human technological development, it’s reasonable to think that an extra-terrestrial civilisation, with a head start of even a few thousand years, would have achieved technological feats far beyond our comprehension.
Could it be that the very nature of singularity leads civilisations to evolve in ways that are undetectable to us, or perhaps, that the pursuit of singularity inadvertently leads to self-destruction?
Proposed Solutions
The Zoo Hypothesis is one proposed solution to the Fermi Paradox. It suggests that extra-terrestrial civilisations are aware of our existence but have intentionally chosen not to contact us, perhaps preferring simply to observe us. This could be due to a policy of non-interference, aimed at allowing younger civilisations like ours to develop and evolve independently.
The Great Filter hypothesis proposes that there is a critical barrier or a series of barriers that drastically reduce the probability of intelligent life arising and persisting. The concept of the Great Filter helps explain the lack of observed extra-terrestrial civilisations by suggesting that one or more critical steps in the development of life or civilisation are extremely unlikely or have a high probability of self-destruction.
A related hypothesis is the Rare Earth Hypothesis, which suggests that while simple life forms might be relatively common, more complex, multicellular organisms are exceptionally rare.
The Transcension Hypothesis offers a different perspective on the Fermi Paradox. It proposes that advanced civilisations might not expand outwards into the cosmos but rather inwards, by miniaturising and compressing their technological and informational systems. As a civilisation advances, it might focus on developing virtual realities, advanced simulations, and artificial intelligence rather than pursuing interstellar travel and communication.
The Search Goes On
None of these ideas gives us an answer.
Perhaps extra-terrestrial civilisations are deliberately avoiding us. Perhaps intelligent life is exceptionally rare. Perhaps advanced civilisations tend to destroy themselves. Or perhaps they eventually develop in ways that make them increasingly difficult for us to detect.
Or could it just be that the distances between civilisations are just so vast that they are, and may always be, insuperable in terms of communication, contact, or detection?
For the moment, we simply don’t know.
More than seventy years after Fermi asked the question, it remains a good one.
Where is everybody?
Size Matters
In a Nutshell
Here’s a question.
How many people need to be in a room before it is more likely than not that at least two of them share a birthday?
A hundred? Two hundred?
The answer is 23.
That seems remarkably low. So how can it possibly be right?
Let’s assume, to keep things simple, that every day of the year is equally likely to be somebody’s birthday and forget about 29 February.
Start with two people, Julia and Julian.
Suppose Julia was born on 1 May. The chance that Julian was also born on 1 May is just 1 in 365.
Nothing very surprising there.
Now add a third person. The probability that Julian has a different birthday from Julia is 364/365. The probability that the third person then has a birthday different from both of them is 363/365.
So the probability that all three birthdays are different is:
(364/365) × (363/365).
Add another person and we multiply again, this time by 362/365.
And so it goes on.
The interesting thing is how quickly the probability of a shared birthday begins to rise.
With 5 people, the chance is only 2.7%.
With 10, it’s 11.7%.
With 16, it’s 28.1%.
But with 23 people it reaches 50.7%.
By the time we get to 32 people it’s around 75%, and with 40 it’s close to 90%.
So what’s going on?
How many pairs?
The trick is that we’re not asking whether anyone shares your birthday. We’re asking whether any two people share a birthday.
And that creates many more possibilities than we might at first imagine.
Put 23 people in a room and there aren’t just 23 opportunities for a match. There are 253 different pairs of people.
Julia can be paired with Julian and with everyone else. Julian can then be paired with everyone except Julia, whose pairing we have already counted. And so on.
By the time we have worked through the room, we have 253 different pairs.
Suddenly, a shared birthday doesn’t seem quite so unlikely.
There is another way of looking at it.
There are 365 open boxes and we drop 23 balls into the boxes at random. Now, what’s the chance at least two of the balls end up in the same box? It’s the same problem looked at in a different way, and the solution is identical. The chance is just over 50-50.
What about your birthday?
Now change the question. What’s the chance that somebody in the room shares your particular birthday? With 23 people in the room, the chance now is only about 6%. Even with 366 people, the probability is still only about 63%. And that gets to the heart of the paradox.
We think of 23 people and consider the probability that someone will have a particular birthday. But the question is whether any birthday matches any other birthday.
There are a lot more ways for that to happen.
Now here’s an experiment you can do. At the start of a football match there are 22 players on the pitch – 23 including the referee. Now check their birthdays. It’s actually a bit more likely than not that there will be at least one shared birthday. Size really does matter, but not in the way most people intuitively think!
The Exchange Paradox Revisited
September 14, 2026
The Two Envelopes Paradox
In a Nutshell
Imagine it! Two envelopes on the table in front of you, both unopened, both containing cash. One of the envelopes contains exactly twice as much as the other. Call them envelope 1 and envelope 2.
You are called on to open one of them, just one of them. You do so and withdraw £100 (say five £20 notes).
Now, you can keep the £100 or switch it for whatever is in the other envelope.
The question is simple to state. Should you switch? On the face of it, there does seem to be quite a good case for doing so.
The other envelope must contain either £50 or £200, since one of the envelopes has twice in it compared to the other. So, if the other envelope contains £200, switching wins you a net £100. If it contains £50, you lose only £50.
So you have a chance of gaining £100 against an equal chance of losing £50.
That sounds like a good bet.
So it sounds like you should switch. But hold on a bit.
Something has gone wrong
Suppose you do switch. You now have the other envelope. So why don’t we apply the same logic again and swap envelopes once more? Is this also a good bet? After all, if there’s £50 in it, you have an equal chance of winning £100 and of losing £25 by switching again. After all, the other envelope contains either twice as much or half as much. Switching envelopes back and forth suddenly seems like a money making machine.
Something is clearly wrong.
Start again
Let us suppose the two envelopes contain £100 and £200. If you open the £100 envelope and switch, you will gain £100.
If you open the £200 envelope and switch, you’re down £100. So why switch? Half the time you gain £100 and half the time you lose £100.
So what on earth is going on? Well, when we opened the original envelope and saw £100, we assumed that the other envelope was equally likely to contain £50 or £200.
But was it?
What if we don’t open the envelope?
There is a clever way of seeing the problem.
Choose an envelope but don’t open it for now. Should you switch? There can’t be any reason to do so. You chose randomly between two envelopes and switching simply gives you the one you didn’t choose. There are after all only two envelopes, not three. We imagined £50 in one envelope, £100 in the other, and £200 in the third. In that case, finding £100 in the first envelope means that you can win £100 or lose £50. But that’s in a world of three envelopes. In the real world, there are only two. In our example, they contain either £50 and £100 or £100 and £200, and you don’t know which. They never contain £50 and £100 and £200. And once we have seen £100, we have no reason to assume that £50 and £200 are equally likely unless we know something about how the amounts were chosen.
So should you switch if you find £100? Maybe. If you have some information about how the amounts were selected, £100 might tell you something useful. There could then be a perfectly good reason to switch—or not to.
But if all you know is that one envelope contains twice as much as the other, and there are only two envelopes on the table, not three, there is no general reason to switch.
The strange thing is how convincing the argument for switching seems when you imagine three envelopes. Until you look at the table and see only two!
In a Nutshell
Ok, let’s toss a coin. Why not?
If it lands Heads, World A is created. A world containing a single person, with a black beard.
If it lands Tails, World B is created. This has two people. A black beard and a brown beard.
You wake up in complete darkness. You can’t see the colour of your beard. So what’s the chance you’re in World B?
At first sight, 1/2 seems the obvious answer. Heads and Tails are equally likely and you have nothing else to go on. But hold on a bit — there’s only one person in World A and two in World B.
So perhaps the probability that you are in World B isn’t 1/2 after all. Perhaps it’s 2/3.
This is essentially the intuition behind what philosophers call the Self-Indication Assumption, or SIA. Other things equal, you are more likely to live in the world containing more observers. Seems obvious. But is it really as obvious as it seems? To find out, let’s turn on the light.
Now, what colour is your beard? It’s black! But a black beard is certain in World A and only a 50-50 chance in World B.
The new information therefore favours World A.
Now look at it another way.
Before the light came on, World B was twice as likely because it had two people in it, not one. But only one of the two has a black beard. So the chance that we’re in World A is the same as that we’re in World B.
We are back to 50-50.
Same experiment. Same coin. Same black beard.
Different answers.
How come?
The disagreement comes from what we thought our own existence told us before the light was switched on. Is a world more probable simply because it contains more opportunities for someone like you to exist or to observe it? The Self-Indication Assumption says Yes. The alternative view, called the Self-Sampling Assumption (SSA), says No. We should take things at face value.
And that brings us to a fascinating thought experiment known as the Presumptuous Philosopher.
Imagine you are a physicist considering two theories of the universe. Both on the face of it seem equally plausible.
There is just one difference.
Theory A predicts a universe containing an enormous number of people like us, say a million times more than Theory B. If we apply SIA straightforwardly, the fact that we exist appears therefore to provide extraordinarily strong evidence for Theory A, for the simple reason that there are so many more people like us in that view of the universe. Our presumptuous philosopher declares, therefore, that we need not wait for any more experiments. He can settle the matter right away. How? Simply put, he exists, you exist, we all exist – and that’s much more likely to be the case in the massively populated universe that Theory A envisages.
But suppose now that new evidence emerges from a particle accelerator overwhelmingly favouring Theory B. Could that ever be strong enough, bar virtual certainty, to overcame the enormous advantage Theory A acquired merely by predicting vastly more observers?
I don’t think intuition gives us an easy way out.
Should the fact that I exist really make me favour a universe containing the hugely greater number of observers even if the evidence of the experiments points heavily the other way? SIA suggests that it should. But do we really want to be like the Presumptuous Philosopher who says that experiments make Theory B 100 times more likely than Theory A but let’s ignore that — because there are 1,000 times more people in a universe where Theory A is true.
So which assumption is right, the Self-Indication Assumption (SIA) or the Self-Sampling Assumption (SSA)? A lot hangs on this!
To be honest, I’m simply not sure. But I’m willing to be persuaded.
In a Nutshell
The Vienna Circle
The story has its origins in 1920s Vienna. A group of philosophers, mathematicians and scientists gathered there around a charismatic academic called Moritz Schlick. This so-called ‘Vienna Circle’ believed that philosophy as we knew it was redundant.
Their key idea was the so-called verification principle. A statement is meaningful only if it can be verified by empirical observation or is true by logical definition. But claims that couldn’t be tested against the world or couldn’t be established by logic weren’t merely false but were meaningless.
Enter Ayer
The man who did most to bring these ideas to Britain was a 24-year-old philosopher, Alfred Jules Ayer, known as Freddie. Ayer had visited Vienna and returned to publish Language, Truth and Logic in 1936. It became one of the most influential works of twentieth-century British philosophy.
There was, however, a problem, and it wasn’t going to go away. Can the verification principle be verified? Can we verify it empirically? There is no experiment we can perform that establishes that only empirically verifiable statements are meaningful. Is it true by definition? Again, apparently not.
So what status does the verification principle itself have? By its own criterion, it seems to be meaningless. And the difficulties didn’t end there.
Popper, Wittgenstein and Quine
What distinguishes a genuinely scientific claim, according to Karl Popper, was not that it could be verified but that it could, in principle, be falsified. No number of observations of white swans can finally prove that all swans are white, but a single black swan can prove that they aren’t.
Language, according to Ludwig Wittgenstein, is a toolbox that serves far more purposes than the positivists allowed.
For W.V.O. Quine, there is no clear distinction between statements true by definition and statements established through experience – the positivist position was untenable.
Ayer’s verdict
Ayer was asked in his later years what he regarded as the principal defects of logical positivism. “I suppose”, he replied, “the most important of the defects was that nearly all of it was false.”
But was logical positivism really a complete failure? Well, the verification principle didn’t survive, and the questions the positivists tried to eliminate returned. But philosophy didn’t simply return to where it had been before. The positivists demanded clarity and were suspicious of impressive-sounding claims for which no adequate justification could be given. Modern philosophy retained those instincts even after abandoning the verification principle itself.
Freddie
And what of Freddie?
There is a wonderful story about Ayer attending a party at which he encountered heavyweight boxing champion Mike Tyson in an altercation involving the young Naomi Campbell. Tyson reportedly demanded to know whether Ayer realised who he was.
Ayer was all too aware but replied that he was the former Wykeham Professor of Logic, that they were therefore both pre-eminent in their respective fields, and suggested that they discuss the matter rationally. Wrongfooted by Freddie, Tyson relented and the matter came to an amicable conclusion.
Later in life Ayer also had a near-death experience that briefly caused the lifelong atheist to reconsider some of his assumptions, although he subsequently retreated.
I have my own, rather less philosophical, memory of Freddie Ayer.
One Saturday afternoon I saw him roar past me on the M25, driving at considerable speed and surrounded by a haze of cigarette smoke.
Logical positivism may have run out of road. Freddie Ayer never seemed in much danger of doing so.
In a Nutshell
Here’s a thought experiment with an interesting twist. I won’t give that away just yet.
It goes like this.
On Sunday you go to sleep. Someone in the same room now tosses a coin.
If it lands heads, you are woken the next day and interviewed. If it lands tails, you are woken the next day, asked a question, you go straight back to sleep and then the same happens on the Tuesday. Now, whenever you wake, you don’t know whether it is Monday or Tuesday and you don’t even know whether you have been woken before. It’s all a bit of a haze. In the original version of this problem, you are given a memory-erasing drug but let’s not over-complicate things.
You are Sleeping Beauty.
The question you are asked each time you wake is a simple one:
What probability should you now assign to the coin having landed heads?
You might think the answer is obvious. It was a fair coin when you went to sleep and nothing has happened to change that. Heads and tails each had a probability of 1/2. So it’s still a half — yes?
Halfers and thirders
There are two camps here — halfers and thirders.
The halfers say 1/2. Sleeping Beauty knew before going to sleep that she would certainly be woken at least once, so waking up has apparently given her no new information about the result of the toss. So it’s still a half.
The thirders see things differently.
Whenever Beauty wakes, she could be in one of three situations:
- Heads and it’s Monday.
- Tails and Monday.
- Tails and Tuesday.
Only one of these involves heads. So, the probability of heads should be 1/3.
Both arguments seem reasonable but they can’t both be right.
Would you bet on it?
One way of thinking about the problem is to turn it into a bet. Suppose that every time Sleeping Beauty wakes she is offered odds of 2 to 1 that the coin landed heads. In other words, she stakes £10 and wins £20 if heads occurred. So, if the coin landed heads, she wakes once, places one £10 bet and wins a net £20. If it landed tails, she wakes twice and loses £10 on each occasion. Her total loss is £20.
Since heads and tails are equally likely when the coin was first tossed (we assume the coin isn’t rigged), over time the £20 gains and £20 losses balance one another.
So 2 to 1 looks like fair odds. And odds of 2 to 1 correspond to a probability of 1/3.
That seems a pretty good argument for the thirders.
What has Sleeping Beauty learned?
Before going to sleep on Sunday, Beauty knows that the coin is fair. The probability of heads is therefore 1/2. But she knows in advance that if the coin toss is heads, she will wake just on Monday. If tails, she will wake on Monday and Tuesday.
So the halfer asks: what new information has she actually received that should make her change her mind about the coin?
The thirder has an answer. Beauty may not have learned anything unexpected, but she now knows that she is experiencing one of the awakenings generated by the experiment. Tails generates twice as many such awakenings as heads. From the point of view of a randomly encountered awakening, tails should therefore be twice as likely.
And there, I think, lies the difficulty.
Are we asking about the probability that a fair coin lands heads? Or are we asking about the probability that she is waking up in a timeline generated by the coin landing heads? Remember that she is twice as likely to be waking up on a Tails day (Monday and Tuesday) than a Heads day (just Monday).
The first question seems to point to 1/2.
The second seems to point to 1/3.
Are these really two different questions, or are there two ways of reasoning about the same question? The jury is still very much out.
So what should Sleeping Beauty say when she opens her eyes?
I think I would say 1/3. But that’s because I’m a betting man, and I believe in reason.
Sleeping Beauty, on the other hand, is no fool and she knows that the coin was tossed just once, and it’s a fair coin. Who can blame her if she says 1/2, and sticks with it. And that’s the twist!
Newcomb’s Paradox Revisited: A Simple Thought Experiment
In a Nutshell
Imagine that you are presented with two boxes. The first is transparent and contains $1,000. The second is opaque and contains either $1 million or nothing.
You now have a choice—to take both boxes or to take only the opaque box. Which should you do?
In one sense, the answer seems obvious. Take both. Whatever is inside the opaque box, you will be $1,000 better off. But there’s a complication.
The Predictor
Before you arrived, a very accurate “Predictor” tried to predict what you would do. If it predicted that you would take both boxes, it left the opaque box empty.
If it predicted that you would take only the opaque box, it put $1 million in that box.
The Predictor, it should be noted, has already made its decision—the money is either there or it is not. Nothing you do now can change that. It’s already happened. And the Predictor, as we’ve already mentioned, is usually right.
Now which box, or boxes, do you take? Interestingly, there are two apparently compelling arguments that lead to opposing answers.
This is Newcomb’s Paradox, named after the physicist William Newcomb and popularised by the philosopher Robert Nozick.
The case for taking both boxes
The argument for taking both boxes is wonderfully simple.
The opaque box already contains either $1 million or nothing. Now suppose it contains $1 million. If you take one box, in this case you receive $1 million. If you take both boxes, you receive $1 million plus a handy $1,000 bonus.
Now suppose the opaque box contains nothing. If you take one box, you now receive nothing. If you take both, however, you receive $1,000, the amount in the transparent box.
So whichever state of the world actually exists, taking both boxes leaves you $1,000 better off. At least that’s the logic of the two-boxers.
The case for taking one box
Suppose instead of applying this strict logic, you examine what actually happens to people who make the decision to take one box or two.
As we’ve already highlighted, the Predictor is very accurate. People who take both boxes therefore tend to discover that the opaque box is empty and they leave with just the money in the transparent box, the $1,000.
People who take only the opaque box tend instead to discover it contains $1 million and leave as millionaires.
Based on this evidence, which group would you now rather join?
There’s something that seems odd about a theory of rational choice which tells you to behave like the people who almost invariably end up with $1,000 rather than to behave like those who almost always end up with $1 million.
And yet the two-box argument hasn’t gone away. The money was placed in the box before you made your choice and nothing you do now can change that.
This is what makes Newcomb’s problem a paradox rather than merely a puzzle.
Reason against evidence?
What we have exposed here is a conflict between two approaches to decision-making.
The two-boxer concentrates on causation. Your current choice cannot cause the Predictor to change a decision already made. This points to what is known as causal decision theory.
The one-boxer focuses instead on evidence. Choosing one box is extremely strong evidence that the opaque box contains $1 million, but that’s what we have observed in practice. This is closer to what’s known as evidential decision theory.
Neither side needs to believe in time travel or backward causation.
The one-boxer isn’t claiming that choosing the opaque box magically puts the money there. The claim is simply that, in a world containing such an accurate Predictor, there’s a strong relation between whether you choose one box or two and the contents of the opaque box.
Even so, the Predictor can’t change anything now, so why leave the $1,000 on the table?
What sort of person are you?
There is a way of thinking about the problem which I find particularly interesting.
Picture yourself standing in front of the boxes. You ask yourself, “So what should I do now?”
Given that the Predictor is so good at judging you and forecasting your likely behaviour, based presumably on something about you, perhaps the more important question you should be asking is: “What sort of person, what sort of decision-maker, should I want to be?”
Look at it like this. Suppose you could decide beforehand to become a one-boxer. If the Predictor is convinced of this, it would presumably be aware of this and put $1 million in the opaque box.
That sounds promising.
But there’s a catch.
You can’t simply decide in advance to be a one-boxer and then, when the moment arrives, take both boxes. If that is your plan, you were never really a one-boxer at all. So how do you convince the Predictor that you are indeed a genuine one-boxer? Simply put, to be the sort of person whom the Predictor expects to take one box, you may actually have to take one box.
So which would I take?
I would take one box.
Here’s why.
The two-box argument at first sight seems totally rational. After all, when I make my choice, the contents of the opaque box are already fixed. Taking the transparent box as well as the opaque box can’t possibly remove money from the other one. So, if there are a million dollars there, why not take the extra thousand? The reason is the evidence before your eyes. As you watch a succession of people playing the game, you notice that it’s the one-boxers who are walking away with the $1 million prizes. The two-boxers are left with the contents of the transparent box.
After watching this happen often enough, there comes a point when I stop worrying about why the one-boxers are winning the big money and simply join them.
This is what makes Newcomb’s Paradox so challenging and so fascinating at the same time. Reason appears to tell us to take two boxes. The evidence seems to point to one.
In the end, I guess I would rather be an irrational but rich one-boxer than a perfectly rational but poor two-boxer!
My book, Twisted Logic: Puzzles, Paradoxes, and Big Questions, is now available.
Professor Leighton Vaughan Williams 