People have been wrongly hanged because of it. People have been wrongly punished because of it. And people have suffered unnecessarily because of it. It’s the mistaken belief that the probability of something being true based on seeing the evidence is the same thing as seeing the evidence if something is true. This can have, has had and continues to have, devastating implications. In fact, the probabilities of these two things will often diverge enormously.
To dramatize the problem I will introduce you to the Shakespearean tragedy, Othello. In the play, Othello’s wife Desdemona is set up by the evil Iago, who plants a treasured keepsake that Othello had given her in the home of young Cassio. When Othello comes upon the keepsake, he soon leaps to the mistaken conclusion that Desdemona has been unfaithful to him, with tragic consequences.
Othello made the mistake of believing that the probability that Desdemona was unfaithful given the evidence of the treasured keepsake being found in Cassio’s home was the same probability that the keepsake would have been found in Cassio’s home if Desdemona had been unfaithful to him. Easy mistake. We do it all the time in everyday life, usually with less dramatic implications. More importantly, juries do it all the time, as do practitioners in others fields, like medicine.
Let’s put it another way. What is the chance that someone who has been repeatedly shot in a flat that you rent out would die? Very high. The evidence here is the dead person, the gunshot wounds and the fact that you have access to the flat. The hypothesis is that you are the murderer. Now, the probability we would see that evidence if you ARE the murderer is 100%. But the probability that you are the murderer given that we see that evidence is much lower. There are perhaps many different people who could have committed the murder, even if you are one of them. This seems obvious, and when stated this way it IS obvious, but in real life the problem is usually not stated or understood so clearly, and is often disguised.
This is sometimes referred to as the ‘Prosecutor’s Fallacy.’ It is the fallacy of making out that someone is guilty because the evidence is consistent with their guilt. This is often enough to convict, because this measure is often confused with the probability that the accused is guilty given that the evidence exists. They are totally different things. But even when they are clearly distinguished, the probability we assign to guilt can be seriously over-estimated because of a common cognitive failing known as the prior indifference fallacy. This is the fallacy of believing that the likelihood that something is true rather than false, when we have little prior idea, starts out as 50-50. This is just not so without proper justification but the implications of this belief, which may be implicit, are potentially huge. The prior probability, in the absence of any evidence, is simply not 50-50 unless there is a very good reason to believe that to be so before we see any evidence. Unless we can anchor this properly, all successive evidence-based reasoning will be flawed.
Fortunately for us, there is a rule used by those conversant with the laws of probability which can in fact help determine the actual relation between the truth of a hypothesis and the evidence relating to that hypothesis. The solution it arrives at is very rarely the same as would be arrived at without it. It is called Bayes’ Rule, but not many people know it, or how to apply it. Until more people do, the relationship between truth and justice is likely to remain severely strained.
Further Reading and Links
It shouldn’t be possible for us to exist. But we do. That’s the sort of puzzle I like exploring. So I will. Let’s start with the so-called ‘Cosmological Constant.’ This is an extra term added by Einstein in working out equations in general relativity that describe a non-expanding universe. The need for the cosmological constant is required to explain why a static universe doesn’t collapse in upon itself through the action of gravity. It’s true that the force of gravity is infinitesimally small compared to the electromagnetic force, but it has a lot more influence on the universe because all the positive and negative electrical charges in the universe somehow seem to balance out. Indeed, if there were just a 0.00001 per cent difference in the net positive and negative electrical charges within a body, it would be torn apart and cease to exist. This cosmological constant, therefore, is added to the laws of physics simply to balance the force of gravity contributed by all of the matter in the universe. What it represents is a sort of unobserved “energy” in the vacuum of space which possesses density and pressure, which prevents a static universe from collapsing in upon itself. But we now know from observation that galaxies are actually moving away from us and that the universe is expanding. In fact, the Hubble Space Telescope observations in 1998 of very distant supernovae showed that the Universe is expanding more quickly now than in the past. So the expansion of the Universe has not been slowing due to gravity, but has been accelerating. We also know how much unobserved energy there is because we know how it affects the Universe’s expansion. But how much should there be? We can calculate this using quantum mechanics. The easiest way to picture this is to visualize “empty space” as containing “virtual” particles that continually form and then disappear. This “empty space”, it turns out, “weighs” 1,093 grams per cubic centimetre. Yet the actual figure differs from that predicted by a factor of 10 to the power of 120. The “vacuum energy density” as predicted is simply 10120 times too big. That’s a 1 with 120 zeros after it. So there is something cancelling out all this “dark” energy, to make it 10 to the power of 120 smaller in practice than it should be in theory. Now this is very fortuitous. If the cancellation figure was one power of ten different, 10 to the power of 119, then galaxies could not form, as matter would not be able to condense, so no stars, no planets, no life. So we are faced with the mindboggling fact that the positive and negative contributions to the cosmological constant cancel to 120 digit accuracy, yet fail to cancel beginning at the 121st digit. In fact, the cosmological constant must be zero to within one part in roughly 10120 (and yet be nonzero), or else the universe either would have dispersed too fast for stars and galaxies to have formed, or else would have collapsed upon itself long ago. How likely is this by chance? Essentially, it is the equivalent of tossing a coin and needing to get heads 400 times in a row and achieving it. Go on. Do you feel lucky? Now, that’s just one constant that needs to be just right for galaxies and stars and planets and life to exist. There are quite a few, independent of this, which have to be equally just right. We can talk about those another time, but this I think sets the stage. I’ve heard this called the fine-tuning argument. I’m now rather interested in finding out who or what is composing this very fine tune.
Update at: https://leightonvw.com/2015/08/03/why-is-there-something-rather-than-nothing/
Further Reading and Links
If something might or might not exist, but is unobservable, it is more likely to exist than something which can be observed, with any positive probability, but is not observed. If something might or might not exist, it is more likely to exist if it is less likely to be observed and is not observed than something else which is more likely to be observed, and is not observed.
Further Reading and Links
See application of the VW Possibility Theorem to the Black Raven Paradox. Link: https://leightonvw.com/2014/12/11/seeing-a-blue-tennis-shoe-makes-it-more-likely-that-flamingos-are-pink/
See application of the VW Possibility Theorem to a unified theory of science. Link: https://leightonvw.com/2014/11/28/430/
See application of the VW Possibility Theorem to the Boy-Girl Paradox
https://leightonvw.com/2014/11/27/the-boy-girl-paradox/
When we pose a question, it is usual that we want an answer. Sometimes the answer is clear, because it is defined to be what it is, or else is true as a matter of logic.
For example, what is 2 plus 2? If your number system defines the answer to be 4, then it is 4.
If I roll two standard dice, the highest total I can achieve is 12. So if I am asked what is the probability that I will roll a total of 14, the answer is zero.
For most questions, however, which are not true by definition or logic, there is no sure answer to any question, only various levels of probability.
Similarly, for any set of observations, the rule or set of rules that gave rise to these observations might not be clear. There may be a large number of different explanations which are consistent with the data.
For example, what rule gives rise to the number sequence 1,3,5,7? If we know this, it will help us to predict what the next number in the sequence is likely to be, if there is one.
Two hypotheses spring instantly to mind. It could be: 2n-1, where n is the step in the sequence. So the third step, for example, gives 2×3-1 = 5. If this is the correct rule generating the observations, the next step in the sequence will be 9 (5×2-1).
But it’s possible that the rule generating the number sequence is: 2n-1 + (n-1)(n-2)(n-3)(n-4). So the third step, for example, gives 2×3-1 + (3-1)(3-2)(3-3)(3-4) = 7. In this case, however, the next step in the sequence will be 33.
So if this is all the information we have, we have two different hypotheses about the rule generating the data. How do we decide which is more likely to be true? In general, when we have more than one hypothesis, each of which could be true, how can we decide which one actually is true?
For the answer, we need to turn to some basic principles of scientific enquiry. I list these as Epicurus’ Principle, Occam’s Razor, Bayes’ Theorem, Popperian ‘Falsifiability’; Solomonoff Induction and the Vaughan Williams ‘Possibility Theorem.’
To address these, and how they contribute to a grand unified theory of scientific enquiry, is beyond the scope of this post, but I can at least provide a basic explanation of the terms.
Epicurus’ Principle is the idea that if there are a number of different possible truths, we should keep open the possibility that any of them might be true until we are forced by the evidence to do otherwise. Otherwise stated, it is the maxim that if more than one theory is consistent with the known observations, keep them all.
Occam’s Razor is the idea that the theory which explains all (or the most) and assumes the least is most likely. This is totally consistent with Epicurus’ Principle, with the additional insight that a simpler theory consistent with known observations is more likely to be true.
Bayes’ Theorem is the idea that the likelihood of a hypothesis being true is a combination of the likelihood of it being true before some new evidence arises and the likelihood of the new evidence arising if the hypothesis is true and if the hypothesis is false. Its most critical insight is that the probability of a hypothesis being true given the evidence is a very different thing to the probability of the evidence arising given that the hypothesis is true.
Popperian ‘Falsifiability’ is the idea that a scientific hypothesis should be testable and falsifiable. Otherwise stated, it notes that a single observational event may prove hypotheses wrong, but no finite sequence of events can verify them correct.
Solomonoff induction is the idea that the information contained in the various explanations consistent with known observations can in principle be reduced to binary sequences, and that the shorter the binary sequence the more likely that explanation of the observations is to be true.
The Vaughan Williams ‘Possibility Theorem’ states that: “If something that might or might not exist is unobservable, or is less likely to be observed, it is more likely to exist than if it can be observed (or is more likely to be observed) but is not observed.” This is critical when assessing how the probability of a hypothesis being true might be affected by information which potentially exists and is relevant but is missing because it is for whatever reason unobserved or unobservable.
Combining these principles into a unified framework can help identify the truth based on known and potentially missing observations.
That is the next step.
Further Reading and Links
In 1694, the Bank of England was founded by Act of Parliament, with the original purpose of acting as the Government’s banker and debt-manager. 1694 was also the year that the French Enlightenment writer and philosopher, Francois-Marie Arouet, better known as Voltaire, was born.
It is also the year from which we can be trace the so-called ‘Halloween Effect’. This is the effect seminally confirmed in the leading academic journal, the American Economic Review, in 2002, by researchers Sven Bouman and Ben Jacobsen. In a paper titled, ‘The Halloween Indicator, Sell in May and Go Away: Another Puzzle’, they test the hypothesis that stock market returns tend to be significantly higher in the November-April period than in the May-October period and find it to be true in 36 of the 37 developed and emerging markets studied in their sample between 1973 and 1998, though it can be observed in their UK data back to 1694. They further find the effect is particularly pronounced in the countries of Europe and that it persists over time. The puzzle was especially noteworthy because the anomaly has been widely recognised for years, though not previously rigorously tested, yet it still persists.
Some analysts trace this back to the practice of the landed classes of selling off stocks in May of each year as they headed to their country estates for the summer months, and re-investing later in the year. Times have moved on, but summer vacations and attitudes may have not. That’s a theory, at least, but a strange one to explain modern-day investment strategies if true.
In a bigger follow-up study by Jacobsen now working with finance professor, Cherry Yi Zhang, published in 2012, they looked at 108 countries, using over 55,000 observations, including more than 300 years of UK data. And guess what! They found that the effect is confirmed for 81 of the 108 countries, with the post-Halloween returns out-performing the pre-Halloween period returns by 4.52 per cent on average, and by 6.25 per cent over the past 50 years looked at alone.
Strange but true! The Halloween Indicator seems to be working better than ever.
So let’s say it’s Halloween today. Time to get stuck into the stock market? You decide!
Reading and Links
Boumen, S. and Jacobsen, B. (2002). The Halloween Indicator: “Sell in May and Go Away”: Another Puzzle. American Economic Review, 92, 5. 1618-1635.
Click to access HalloweenIndicator.pdf
https://www.sciencedirect.com/science/article/pii/S1057521910000608
https://www.sciencedirect.com/science/article/pii/S0275531916300034
Calendar Effects. Wikipedia. https://en.wikipedia.org/wiki/Calendar_effect
Market anomaly. Wikipedia. https://en.wikipedia.org/wiki/Market_anomaly
If Mr. Smith wants to SELL me his horse, do I really WANT to buy it? It’s a question as old as markets and horses have existed, but it was for many, many years, one of the unspoken questions of economics.
It is a question Groucho Marx once posed in a slightly different way, when he declared that he refused to join any club which was prepared to accept him as a member.
So how do we solve this paradox? The paradox of the horse that is, not the Groucho paradox.
For most of the history of economics, the answer was quite simple. Simply assume perfect markets and perfect information, so the horse buyer would know everything about the horse, and so would the seller, and in those cases where the horse is worth more to the buyer than the seller, both can strike a mutually beneficial deal. Gains from trade, it’s called.
In the real world, of course, life is not so straightforward, and the person selling the horse is likely to know rather more about it than the potential purchaser. This is called ‘asymmetric information’, and the buyer is facing what is called an ‘adverse selection’ problem, as he has adverse information relative to the seller.
This is a classic case of what economics professor George Akerlof sought to address in 1970, in a seminal paper called ‘The Market for Lemons’.
Akerlof had become intrigued by the limited tools that economists were using in the late 1960s. Unemployment, so went much of the general thinking, was caused by money wages adjusting too slowly to changes in the demand for labour. This was the so-called ‘neo-classical synthesis’ and it assumed classic markets, albeit they could be a bit slow to work.
At the same time, economists had come to doubt that changes in the availability of capital and labour could in themselves explain economic growth. The role of education was called upon as a sort of magic bullet to explain why an economy grew as fast as it did. But that posed a problem for Akerlof. How can we distinguish the impact on productivity of the education itself from the extent to which education simply helped grade people, he asked. The idea here is that more able people will tend on average to seek out more education. So how far does education in itself contribute to growth, and how far does it help simply as a signal and a screen for employers? In the real world, of course, these signals could be useful because employers are like the horse buyers – they know less about the potential employees than the employees know about themselves, the classic adverse selection problem.
Akerlof turned to the market for used cars for the answer, not least because at the time a major factor in the business cycle was the big fluctuation in sales of new cars. He quickly spotted the problem. Just like in the market for horses, the first thing a potential used car buyer is likely to ask is “Why should I WANT to buy that used car if he wants so much to SELL it to me”. The suspicion is that the car is what Americans call a ‘lemon’, a sub-standard pick of the crop. Owners of better quality used cars, called ‘plums’, are much less likely to want to sell.
Now let’s say you’re willing to spend £10,000 on a plum but only £5,000 on a lemon. In such a case, the best price you’d be willing to pay is about £7,500, and only then if you thought there was an equal chance of a lemon and a plum. At this price, though, sellers of the plums will tend to back out, but sellers of the troublesome lemons will be very happy to accept your offer. But as a buyer you know this, so will not be willing to pay £7,500 for what is very likely to be a lemon. The prices that will be offered in this scenario may well spiral down to £5,000 and only the worst used cars will be bought and sold. The bad lemons have effectively driven out the good plums, and buyers will start buying new cars instead of plums. Just as with horses, asymmetric and imperfect information in the used car market has the potential, therefore, to severely compromise its effective operation.
What can be done about this? For the answer we must go back to part of the reason why people seek education, which is to signal personal qualities which might otherwise be difficult to discern. This is part of the wider theory of signalling and screening, and it takes us to another place, on another day.
Further Reading and Links
Akerlof, G. (1970), The Market for Lemons: Quality, Uncertainty and the Market Mechanism. Quarterly Journal of Economics. 84:488-500.
In a game first popularized in the academic literature in the early 1980s, two people are invited separately to play a game, with a monetary prize. The players are acting anonymously, playing the game via a computer terminal. The game, widely known these days as the ‘Ultimatum Game’, involves one of the players, who we will call Jack, being given £50, say, by the experimenter. He must now decide whether or how much he should offer the other player, who we will call Jill. Remember that Jack and Jill don’t know each other, and will remain anonymous to each other. The game is only played once, so there is no comeback from Jill whatever Jack does. There is, however, one consideration for Jack to think about. If Jill turns down the offer, they both walk away empty-handed. So how much should Jack offer Jill? And how much will he? Traditional economic theory about rational behaviour would suggest that Jack, as a profit-maximizing agent, should offer Jill a very small amount, and that Jill should accept this very small amount rather than get nothing. In fact, early experimenters who put real people into this scenario usually found that the amount of money offered lay somewhere between 50-50 and 65-35. Sometimes, nevertheless, the second player was indeed offered only a small amount, and in those cases where this was less than 30% of the prize, usually refused. In other words, when Jill was offered less than £15 of the £50, she usually walked away from the deal, leaving both with nothing. Is this reconcilable with rational economic behaviour? One explanation that is often proposed is that offers of less than 30% or so are considered as desultory, even insulting, and Jill is getting utility (as economists would call it) from punishing Jack. Yet the low offer made by Jack is not in fact a personal insult, and arises as part of the design of the game. Indeed, neither player will ever know who the other person is. It is certainly not profit-maximizing behaviour by Jill. Is there another explanation? One explanation that makes some sense, proposed in the mid-1980s by the distinguished mathematician and game theoretician, Robert Aumann, is that people tend to evolve rules of thumb according to which they behave in their day-to-day lives. One such rule he identifies as “Don’t be a sucker; don’t let people walk all over you.” This might indeed work well as a general rule for Jill to live by, insofar as it helps build up her reputation for people’s future reference. But in this particular situation, turning down £15 does nothing to build up her reputation, because she is anonymous. Aumann’s explanation is that Jill doesn’t think like that. She has built up this rule-of-thumb behavioural code over a lifetime, and will not so easily abandon it a particular context, when the situation is different. This is what we might call ‘bounded rationality’, in that people do not usually consciously maximize in each decision situation, but instead use rules of thumb that work well “on the whole”. So, that leaves a couple of questions. The first is whether Jack is being rational when he offers a small slice of the cake to Jill, and the second is whether he is he being altruistic or self-interested when he offers her a bigger slice? Reference: Robert Aumann, Rationality and Bounded Rationality: The 1986 Nancy L. Schwartz Memorial Lecture.
Derren Brown, the illusionist, is no stranger to the use of the idea of the wisdom of crowds as part of his entertainment package. A few years ago, for example, he selected a group of people and asked them to estimate how many sweets were in a jar.
All conventional ‘wisdom of crowds’ stuff, albeit wrapped as part of a magical mystery tour. His relatively more recent venture into this world of apparent wisdom went down a rather singular avenue, however, as he explained how a group of 24 people could predict the winning Lottery numbers with uncanny accuracy.
The idea in essence was that each of the 24 would make a guess about the number on each ball and the average of each of these guesses would converge on the next set of winning numbers. It appeared to work – but that is the thing about illusionists; they are good at producing illusions.
I will not go into how he did generate the effect of predicting the lottery draw, because there is no point if you already know, and because it would spoil the fun if you don’t. What is sure, however, is that the musings of the crowd had nothing to do with it.
But why not? After all, if the crowd can accurately guess the weight of an ox or the number of jelly beans in a jar, why not the numbers on the lottery balls? The simple answer, of course, is because the lottery balls are drawn randomly. And the thing about random events is that they are unpredictable. This is at the heart of what economists term ‘weak form market efficiency’, i.e. that future movements in market prices cannot be predicted from past movements. In this sense, the series has no memory.
So what is likely to happen if you do get a group of friends around and ask each to guess the number that will appear on each of the balls drawn next Saturday? If you take the average of the guesses about each in turn, my best estimate is that you are likely to end up with a prediction for each ball that is about 30 or less. Why so? Partly this is because people tend to pick birthdays but it’s also because the averaging of a large number of guesses is likely to produce a number somewhere nearer the mid-point of the set of numbers than the extremes.
But if you do use these numbers and just happen to win, you’re likely to be sharing your winnings with a lot of other people who’ve chosen the same numbers as you. The better strategy is to populate your ticket with bigger numbers, which are likely to be less popular.
This strategy won’t alter your chance of winning but it will increase how much you can expect to win if you do win. And that is no illusion!
When asked to list my all-time heroes, the name of William of Ockham (or Occam) is never far from my lips. Born in the late 13th century, in the Surrey village of Ockham, this Franciscan philosopher, theologian and political writer, is generally considered to be one of the major figures in medieval scholarship.
In this regard, he ranks alongside the likes of his fellow theologians Thomas Aquinas and John Duns Scotus in the pantheon of great pre-Renaissance thinkers. Despite the title he earned at Oxford University of Venerabilis Inceptor (‘Worthy Beginner’) it is therefore by his alternative title of Doctor Invincibilis (‘Unconquerable Doctor’) that he comes down to us. Of all his writings, and they are each worthy of separate study, it is for his principle of parsimony in explanation and theory-building that he is best known today. It is a principle that Fox Mulder refers to in an episode of the X-files and that Jodie Foster defers to in ‘Contact’. Indeed, in William Peter Blatty’s novel, Legion (on which ‘The Exorcist III’ is based), the lead character complains that he was not put on earth “to sell William of Occam door to door.” He needn’t have bothered. William of Ockham sells himself well enough without help, through the principle that is known as ‘Occam’s Razor.’
The Razor is perhaps most clearly defined in Encyclopedia Britannica’s Student edition, where it is taken as an admonishment to devise no more explanations than necessary for any given situation. Put another way, it advises that one should opt for explanations in terms of the fewest possible number of causes, factors or variables. The adults’ version of Encyclopedia Britannica puts it more elegantly, but perhaps less clearly, in these terms – ‘Pluritas non est ponenda sine necessitate’ (‘Plurality should not be posited without necessity’). As such, the principle can be interpreted as giving precedence to simplicity; of two competing theories, the simplest explanation of an entity is to be preferred. There are some higher truths, which may be known to us by experience or revelation, and which Ockham witnesses as necessary rather than contingent entities, to which we are not advised to apply the razor. This is a part of Ockham’s trenchant analysis which is often forgotten, but at least need not concern us when considering the theme of today’s article.
So how do modern-day analysts stand on the shoulders of this medieval giant? The best explanation is perhaps by way of example, and for this we need to travel to the Hong Kong racetrack and to the professional gamblers who devise sophisticated forecasting models of the outcomes of the races run at the Sha Tin and Happy Valley tracks. The basic methodology is to identify each individual factor that could possibly predict the outcome. And what do you do then? How do you decide what to include and what not? For the answer I asked a man who has conservatively made tens of millions of dollars at the track from this very approach. As we enjoyed the view from his Sydney penthouse, he summed it up in a sentence. “I apply Occam’s Razor”, he said, “it really is as simple as that!”
Now say that you have a missing aircraft, and the number of explanations are seeming to grow by the minute. What should we do? Apply Occam’s Razor, of course. And see what we get.
I was once told a story about the value of crowd wisdom in turning up buried treasure. The story was that by asking a host of people, each with a little knowledge of ships, sailing and the sea, where a vessel is likely to have sunk in years gone by, it is possible with astonishing accuracy to pinpoint the wreck and the bounty within. Individually, each of those contributing a guess as to the location is limited to their special knowledge, whether of winds or tides or surf or sailors, but the idea is that together their combined wisdom (arrived at by averaging their guesses) could pinpoint the treasure more accurately than a range of other predictive tools. At least that’s the way it was told to me by an economist who was in turn told the story by a physicist friend. To any advocate of the power of prediction markets, this certainly sounds plausible, so I decided to investigate further. Soon I was getting acquainted with the fascinating tale of the submarine USS Scorpion, as related by Mark Rubinstein, Professor of Applied Investment Analysis at the University of California at Berkeley. In a fascinating paper titled, ‘Rational Markets? Yes or No? The Affirmative Case’, he tells of a story related in a book called ‘Blind Man’s Bluff: The Untold Story of American Submarine Espionage’ by Sherry Sontag and Christopher Drew. The book tells how on the afternoon of May 27, 1968, the submarine USS Scorpion was declared missing with all 99 men aboard. It was known that she must be lost at some point below the surface of the Atlantic Ocean within a circle 20 miles wide. This information was of some help, of course, but not enough to determine even five months later where she could actually be found. The Navy had all but given up hope of finding the submarine when John Craven, who was their top deep-water scientist, came up with a plan which pre-dated the explosion of interest in prediction markets by decades. He simply turned to a group of submarine and salvage experts and asked them to bet on the probabilities of what could have happened. Taking an average of their responses, he was able to identify the location of the missing vessel to within a furlong (220 yards) of its actual location. The sub was found. Sontag and Drew also relate the story of how the Navy located a live hydrogen bomb lost by the Air Force. Now this story has got me thinking. Let’s just say we are looking for a missing aircraft. Could prediction markets possibly help? It’s a question I’m currently asking myself. Maybe others should be as well!
Professor Leighton Vaughan Williams 