If you are content to point and shoot with an automatic camera, you will these days usually do just fine. But let’s say you are looking for a bit more, with some manual control over the settings. That’s where most amateur snappers tend to take fright. They think it’s all a bit technical. Actually, it’s not. It’s all essentially about ‘exposure’, which is basically the brightness or darkness of a photo, and this comes down to three settings – aperture, shutter speed and ISO.
The aperture is simply a set of blades which widen and narrow to control how much light enters the camera. Aperture sizes are measured by f-stops, with a high f-stop (say f/18 or f/22) corresponding to the aperture being quite small (less light entering the camera), and a low f-stop (say f/3.5 or f/5.6) meaning that the aperture is bigger (more light). The aperture also controls what is known as the depth of field, which is an indication of how much of the picture is sharp and how much is blurry. So if you want a figure in the foreground to be sharp and the background blurry, you would want a shallow depth of field (low f-stop, wide aperture). If you want the entire field sharp (for example, a mountain range) you are looking for a high f-stop (small aperture). In summary, a wide aperture (low f-stop, say f/5.6) gives you a brighter photo but a shallower depth-of-field, while a small aperture (high f-stop, say f/18) gives you a darker picture but more depth of field.
The shutter speed is simply a measure of how long the shutter is open, so a slow shutter speed (say 1/60 of a second) allows time for more light to enter, producing a brighter picture (but more blur if objects are in motion), and a fast shutter speed (say 1/800 of a second) produces a darker picture but less blur. In summary, a fast shutter speed (say 1/1000) produces a darker picture, but will be susceptible to less blur, while a slow shutter speed (say 1/80) produces a brighter picture but is susceptible to more blur.
The ISO controls the exposure a different way, thrugh software in the camera that makes it extra sensitive to light. In particular, a high ISO (say 1600) will produce a brighter picture than a low ISO (say 200). A high ISO is consistent with more digital noise in the picture, however, which tends to make the photo look a bit more grainy. In summary, you would select a higher ISO for a brighter photo, but that opens you up to a ‘noisier’ (more grainy) picture, while a lower ISO will give you a darker (but less grainy) picture.
The art now is in combining these settings to give the best overall effect. For example, say you want to take a photo with some movement in it, so you decide to select a fast shutter speed (say 1/800). But the picture comes out darker than you’d like as a result. So you try to compensate by opening up the aperture (to say f/3.5), although this reduces the depth-of-field, blurring the background. Still, the blur doesn’t concern you too much as it’s what’s in the foreground that is the subject of the picture. It’s still a bit darker than you’d ideally like, though. Finally, you turn to the ISO setting and increase that to brighten the picture, while being careful to balance this with your desire to avoid too much digital noise.
By manually and independently setting the aperture, the shutter speed and the ISO, you now have the picture pretty much as close as possible to how you wish it to come out. The automatic mode will often do a good enough job, but using the manual settings allows you to have that bit more control over the final product. You can also use ‘aperture priority’ mode, where you set the aperture and the camera automatically sets the shutter speed. Or else use ‘shutter priority’ mode, where you set the shutter speed and the camera sorts out the f-stop.
That’s photography using manual settings in a nutshell. I hope it’s been of some help.
Further and deeper exploration of paradoxes and challenges of intuition and logic can be found in my recently published book, Probability, Choice and Reason.
You win a quiz show and are offered a choice. You are presented with a transparent box containing £10,000 and an opaque box which contains either £100,000 or nothing. Now, you can open the opaque box and take what is inside, or you can open both boxes and take the contents inside both. Which should you choose? Well, if that’s all the information you have, it’s obvious that you should open both boxes. You certainly will not win less than by just opening one of the boxes, but you might win a lot more. So far, so good. But now introduce an additional factor. Before making your decision, you had to undergo a computerised sophisticated psychometric test (a Predictor) which you are now told has been unerring in its prediction of what hundreds of previous contestants would decide. Whenever they chose both boxes, there was nothing inside the opaque box. Whenever they had chosen just the opaque box, however, they found a cool £100,000 inside. When you make your decision the computer’s decision has already been made. The contents of the opaque box have already been placed there. What is happening is that the Predictor informs the game show organisers its prediction of whether a contestant will choose two boxes or one box. Whenever it predicts that the contestant will choose two boxes, no money is placed in the opaque box. Whenever it predicts that the contestant will choose just the opaque box, £100,000 has been deposited in the box.
This is essentially the basis of what is known as Newcomb’s Paradox or Newcomb’s Problem, a thought experiment devised by William Newcomb of the University of California and popularised by philosopher Roberz Nozick in a paper published in 1969.
So what should you do? Open just the opaque box or open both boxes.
In his paper, Nozick writes that “To almost everyone, it is perfectly clear and obvious what should be done. The difficulty is that these people seem to divide almost evenly on the problem with large numbers thinking that the opposing half is just being silly.”
The argument of those who argue for opening both boxes (the so-called ‘two-boxers’) is that the money has already has already been deposited at the time you are asked to make your decision. Taking two boxes can’t change that, so that’s the rational thing to do.
The argument of those who argue for opening just the opaque box (the so-called ‘one-boxers’) is that the psychometric test is either a perfect or near-perfect predictor of what you will do. It has never got it wrong before. Every single previous contestant who has opened two boxes has found the opaque box empty, and every single previous contestant who has opened just the opaque box has won the £100,000. So do what all the evidence tells you is the sensible thing to do and open just the opaque box.
One way of considering the question is to ask whether your choice in some way determines the choice of the Predictor, and thereby the decision as to whether to place the £100,000 in the box. Well, there’s no time-travelling retro-causality involved. The predictor is basically a piece of computer software which bases its prediction on a psychometric test. It just so happens that the test is uncannily accurate in knowing what people will do.
Look at it this way. Bottom line is that you have a free choice, so why not open both boxes? The problem is that if you are the type of person who is a two-boxer, the predictor will have found this out from the super-efficient psychometric test. If you are the type of person, however, who is a one-boxer, the predictor will find that out too.
So it’s not that there is any good reason in itself to open one box rather than two. After all, what you decide now can’t change what is already in the box. But there is a good reason why you should be the type of person who only opens one box. And the best way to be the sort of person who only opens one box is to only open one box. For that reason, the way to win the £100,000 is to agree to open just the opaque box and leave the other box untouched.
But why leave behind that extra £10,000 when the £100,000 which you are about to win is already in the box?
That’s Newcomb’s Paradox. You decide! Are you are a one-boxer or two?
On the 9th of November, 1999, Sally Clark, a 35-year-old solicitor and mother of a young child, was convicted of murdering her first two children. The presiding Judge, Mr. Justice Harrison, declared that “… we do not convict people in these courts on statistics. It would be a terrible day if that were so.” As it turned out, it was indeed a terrible day, for Sally Clark and for the justice system.
The background to the case is that the death of the babies was put down to natural causes, probably SIDS (‘Sudden Infant Death Syndrome’). Later the Home Office pathologist charged with the case became suspicious and Sally Clark was charged with murder and tried at Chester Crown Court. It later transpired that essential evidence in her favour had not been disclosed to the defence, but not before a failed appeal in 2000. At a second Appeal, in 2003, she was set free, and the case is now recognised as a huge miscarriage of justice.
So what went wrong?
A turning point in the trial was the evidence given by a key prosecution witnesses, who argued that assuming Sally Clark was innocent, the probability of a baby dying of SIDS was 1 in 8,543. So the probability of two babies dying was that fraction squared, or 1 in about 73 million. It’s the chance, he argued, “… of backing that long odds outsider at the Grand National … let’s say it’s a 80 to 1 chance, you back the winner last year, then the next year there’s another horse at 80 to 1 and it is still 80 to 1 and you back it again and it wins. Now we’re here in a situation that, you know, to get to these odds of 73 million you’ve got to back that 1 in 80 chance four years running … So it’s the same with these deaths. You have to say two unlikely events have happened and together it’s very, very, very unlikely.”
Perhaps unsurprisingly in face of this interpretation of the evidence, the jury convicted her and she was sentenced to life in prison.
But the evidence was flawed, as anyone with a basic understanding of probability would have been aware. One of the basic laws of probability is that you can only multiply probabilities if those probabilities are independent of each other. This would be true only if the cause of death of the first child was totally independent of the cause of death of the second child. There is no reason to believe this. It assumes no genetic, familial or other innocent link between these sudden deaths at all. That is a basic error of classical probability. The other error is much more sinister, in that it is harder for the layman to detect the flaw in the reasoning. It is known as the ‘Prosecutor’s Fallacy’ and is a well-known problem in the theory of conditional probability, and in particular the application of what is known as Bayesian reasoning, which is discussed in the context of Bayes’ Theorem elsewhere.
The ‘Prosecutor’s Fallacy’ is to conflate the probability of innocence given the available evidence with the probability of the evidence arising given the fact of innocence. In particular, the following propositions are very different:
1. The probability of observing some evidence (the dead children) given that a hypothesis is true (here that Sally Clark is guilty).
2. The probability that a hypothesis is true (here that Sally Clark is guilty) given that we observe some evidence (the dead children).
These are totally different propositions, the probabilities of which can diverge widely.
Notably, the probability of the former proposition is much lower than of the latter.
Indeed, the probability of the children dying given that Sally Clark is a child murderer is effectively 1 (100%). However, the probability that she is a child murderer given that the children have died is a whole different picture.
Critically, we need to consider the prior probability that she would kill both babies, i.e. the probability that she would kill her children, before we are given this evidence of sudden death. This is the concept of ‘prior probability’, which is central to Bayesian reasoning. This prior probability must not be viewed through the lens of the later emerging evidence. It must be established on its own merits and then merged through what is known as Bayes’ Rule with the emerging evidence.
In establishing this prior probability, we need to ask whether there was any other past indication or evidence to suggest that she was a child murderer, as the number of mothers who murder their children is almost vanishingly small. Without such evidence, the prior probability of guilt should correspond to something like the proportion of mothers in the general population who serially kill their children. This prior probability of guilt is close to zero. In order to update the probability of guilt, given the evidence of the dead children, the jury needs to weigh up the relative likelihood of the two competing explanations for the deaths. Which is more likely? Double infant murder by a mother or double SIDS. In fact, double SIDS is hugely more common than double infant murder. That is not a question that the jury, unversed in Bayesian reasoning or conditional probability, seems to have asked themselves.
More generally, it is likely in any large enough population that one or more cases will occur of something which is improbable in any particular case. Out of the entire population, there is a very good chance that some random family will suffer a case of double SIDS. This is no ground to suspect murder, however, unless there was a particular reason why the mother in this particular family was, before the event, likely to turn into a double child killer.
To put it another way, consider the wholly fictional case of Lottie Jones, who is charged with winning the National Lottery by cheating. The prosecution expert gives the following evidence. The probability of winning the Lottery jackpot without cheating, he tells the jury, is 1 in 45 million. Lottie won the Lottery. What’s the chance she could have done so without cheating in some way? So small as to be laughable. The chance is 1 in 45 million. So she must be guilty. Sounds ridiculous put like that, but it is exactly the same sort of reasoning that sent Sally Clark, and sends many other innocent people, to prison in real life.
As in the Sally Clark case, the prosecution witness in this fictional parody committed the classic ‘Prosecutor’s Fallacy’, assuming that the probability that Lottie is innocent of cheating given the evidence (she won the Lottery) was the same thing as the probability of the evidence (she won the Lottery) given that she didn’t cheat. The former is much higher than the latter, unless we have some other indication that Lottie has cheated to win the Lottery. Once again, it is an example of how it is likely that in any large enough population one or more cases will occur of something which is improbable in any particular case. The probability that needed to be established in the Lottie case was the probability that she would win the Lottery before she did. If she is innocent, that probability is 1 in tens of millions. The fact that she did, in fact, win the Lottery does not change that.
Lottie just got very, very lucky. Just as Sally Clark got very, very unlucky.
Sally Clark never recovered from the trauma of losing her children and spending years in prison falsely convicted of killing them. She died on 16th March, 2007, of acute alcohol intoxication.
Further reading
Bayes’ Theorem. The most powerful equation in the world. https://leightonvw.com/2017/03/12/bayes-theorem-the-most-powerful-equation-in-the-world/
This is probably the most important idea in probability. Truth and justice depend on us getting it right. https://leightonvw.com/2014/12/13/this-is-probably-the-most-important-idea-in-probability-truth-and-justice-depends-on-us-getting-it-right/
Further and deeper exploration of paradoxes and challenges of intuition and logic can be found in my recently published book, Probability, Choice and Reason.
The Two Envelopes Problem, also known as the Exchange Paradox, is quite simple to state. You are handed two identical-looking envelopes, one of which, you are informed, contains twice as much money as the other. You are asked to select one of the envelopes. Before opening it, you are given the opportunity, if you wish, to switch it for the other envelope. Once you have decided whether to keep the original envelope or switch to the other envelope, you are allowed to open the envelope and keep the money inside. Should you switch?
Switching does seem like a no-brainer. Note that one of the envelopes (you don’t know which) contains twice as much as the other. So, if one of the envelopes, for example, contains £100, the other envelope will contain either £200 or £50. By switching, it seems, you stand to gain £100 or lose £50, with equal likelihood. So the expected gain from the switch is 1/2 (£100) + 1/2 (-£50) = £50-£25 = £25.
Looked at another way, the expected value of the money in the other envelope = 1/2 (£200) + 1/2 (£50) = £125, compared to £100 from sticking with the original envelope.
More generally, you might reason, if X is the amount of money in the selected envelope, the expected value of the money in the other envelope = 1/2 (2X)+ 1/2 (X/2) = 5/4 X. Since this is greater than X, it seems like a good idea to switch.
Is this right? Should you always switch?
Solution (Spoiler Alert)
If the above logic is correct, then after switching envelopes the amount of money contained in the other envelope can be denoted as Y.
So by switching back, the expected value of the money in the original envelope = 1/2 (2Y) + 1/2 (Y/2) = 5/4Y, which is greater than Y, following the same reasoning as before. So you should switch back.
But following the same logic, you should switch back again, and so on, indefinitely.
This would be a perpetual money-making machine. Something is surely wrong here.
One way to consider the question is to note that the total amount in both envelopes is a constant, A = 3X, with X in one envelope and 2X in the other.
If you select the envelope containing X first, you gain 2X-X = X by switching envelopes.
If you select the envelope containing 2X first, you lose 2X-X = X by switching envelopes. So your expected gain from switching = 1/2 (X) + 1/2 (-X) = 1/2 (X-X) = 0.
Looked at another way, the expected value for the originally selected envelope = 1/2 (2X) + 1/2 X = 3/2 X. The expected value for the envelope you switch to = 1/2 (2X) = 1/2 X = 3/2 X. These amounts are identical, so there is no expected gain (or loss) from switching.
So which is right? This reasoning or the original reasoning. There does not seem a flaw in either. In fact, there is a flaw in the earlier reasoning, which indicated that switching was the better option. So what is the flaw?
The flaw is in the way that the switching argument is framed, and it is contained in the possible amounts that could be found in the two envelopes. As framed in the original argument for switching, the amount could be £100, £200 or £50. More generally, there could be £X, £2X or £1/2 X in the envelopes. But we know that there are only two envelopes, so there can only be two amounts in these envelopes, not three.
You can frame this as £X and £2X or as £1/2 X and £X, but not legitimately as £X, £2X and £1/2 X. By framing it is as two amounts of money, not three, in the two envelopes, you derive the answer that there is no expected gain (or loss) from switching.
If you frame it as £X and £2X, there is a 0.5 chance you will get the envelope with £X, so by switching there is a 0.5 chance you will get the envelope with £2X, i.e. a gain of £X. Similarly, there is a 0.5 chance you selected the envelope with £2X, in which case switching will lose you £x. So the expected gain from switching is 0.5 (£X) + 0.5 (-£X) = £0.
If you frame it as £X and £1/2 X, there is a 0.5 chance you will get the envelope with £X, so by switching there is a 0.5 chance you will get the envelope with £1/2 X, i.e. a loss of £1/2 X. Similarly, there is a 0.5 chance you selected the envelope with £1/2 X, in which case switching will gain you £1/2 X. So the expected gain from switching is 0.5 (-£1/2 X) + 0.5 (£1/2 X) = £0.
There is demonstrably no expected gain (or loss) from switching envelopes.
In order to resolve the paradox, you must label the envelopes before you make your choice, not after. So envelope 1 is labelled, say, A, and envelope 2 is labelled, say, B. A corresponds in advance to, say, £100 and B corresponds in advance to, say, £200, or to £50, but not both. You don’t know which corresponds to which. If you choose one of these envelopes, the envelope marked in advance with the other letter will contain an equal amount more or less than the one you have selected. So there is no advantage (or disadvantage) in switching in terms of expected value. In summary, the clue to resolving the paradox lies in the fact that there are only two envelopes and these contain two amounts of money, not three.
First posed in Scientific American in 1959, the Three Prisoners Problem remains a classic of conditional probability. The problem, or a version of it, is simple to state. There are three prisoners on death row, Adam, Bob and Charlie. They are told that each of them has had their names entered into a hat and the lucky name to be randomly chosen will be pardoned as an act of clemency to celebrate the King’s birthday. The warden knows who has been pardoned, but none of the prisoners do.
Adam asks the warden to name one of the prisoners who will definitely NOT be pardoned. Either way, he agrees that his own fate should not be revealed. If Bob is to be spared, name Charlie as one of the men to be executed. If Charlie is to be spared, name Bob as one of the men to be executed. If it is he, Adam, who is to be pardoned, the warden should just flip a coin and name either Bob or Charlie as one of the men to be executed.
The warden agrees and names Charlie as one of the men going to the gallows.
Given this information, what is the probability that Adam is going to be pardoned, and what is the chance that Bob will instead be pardoned?
Adam reasons that his chance of being spared before the conversation with the warden was 1/3, as there are three prisoners, and only one of these will be pardoned by random lot. Now, though, he reasons that one of either he or Bob is to walk free, as he knows that Charlie is not the lucky one. So now Adam reasons that his chance of being pardoned has risen to 1/2. But is he right?
Solution (Spoiler alert).
Before talking to the warden, Adam correctly concludes that his chance of evading the gallows is 1/3. It is either he, Bob or Charlie who will be released, and each has an equal chance, so each has a 1/3 chance of being pardoned.
When Adam asks the warden to name one of the OTHER men who will be executed, he is asking the warden not to name him either way, whether he is to be pardoned or not. The warden (as we are told in the question) selects which of the other men to name by flipping a coin. Now, Adam gains no new information about his fate. The information he does gain is about the fate of Bob and Charlie. By naming Charlie as the condemned man, the warden is ruling out the chance that Charlie is to be pardoned.
So Adam now knows the chance that Charlie will be spared has decreased from a 1/3 chance before the warden revealed this information to a zero chance after he reveals it.
But his own chance of being spared remains unchanged, because the warden was not able to reveal any new information relevant to his own fate. New information is a requirement for changing the probability that something will happen or not. So his probability of being pardoned remains at 1/3.
The new information he does have is that Charlie is not the lucky man, so the chance that Bob gets lucky is 2/3.
Put another way, how is it possible that Adam and Bob heard received the same information but their odds of surviving are so different? It is because, when the warden made his selection, he would never have declared that Adam was going to die. On the other hand, he might well have declared Bob to be the condemned man. In fact, there was a 50-50 chance he would have done so. Therefore, the fact that he didn’t name Bob provides valuable information as to the likelihood that Bob was pardoned while telling us nothing as to whether Adam was.
This is an example of the reality that belief updates must depend not merely on the facts observed but also on the method of establishing those facts.
In case there is still any doubt, imagine that there were 26 prisoners instead of 3. Adam asks the warden not to reveal his own fate but to name in random order 24 of the other prisoners who are to be executed. So what is the chance that Bob will be the lucky one of 26 before the warden reveals any names? It is 1/26, the same chance as each of the other prisoners. Every time, however, that the warden names a dead man walking, say Charlie or Daniel, that reduces their chances to zero and increases the chance of all those left except for Adam, who has expressly asked not be named, regardless of whether he is to be executed. So it means a lot to learn that the warden has eliminated everyone but Bob given that he had every opportunity to name Bob as one of those going to the gallows. It means nothing that he has not named Adam because he was expressly told not to, whatever his fate.
In a 26-man line-up, where the warden in random order names who are condemned, once everyone but Bob has been named for execution by the warden, Adam’s chance of surviving stays at 1/26. Bob’s chance of being pardoned rises to 25/26. This is despite the fact that there are only two remaining prisoners who have not been named for execution by the warden. Would you take 20/1 now that Adam will be spared? You might, if you were Bob, but you are not getting a good price, and you will not have long to spend it!
You are presented with three identical boxes. You are made aware that one of the boxes contains two gold coins, another contains two silver coins, and the third contains one gold coin and one silver coin. You do not know which box contains which.
Now, choose a box at random. Reach without looking under the cloth covering the coins and take out one of the coins. Now you can look. It is gold.
So you can be sure that the box you chose cannot be the box containing the two silver coins. It must be either the box containing two gold coins or the box containing one gold coin and one silver coin.
Withdrawing the gold coin from the box doesn’t provide you with the information to identify which of these two boxes it is. So the other coin must either be a gold coin or a silver coin.
Given what you now know, what is the probability the the other coin in the box is also gold, and what odds would you take to bet on it?
This is essentially the so-called ‘Bertrand’s Box’ paradox, first proposed by Joseph Bertrand in 1889 in his opus, ‘Calcul des probabilités’.
Spoiler alert (Solution)
After withdrawing the gold coin, there are only two boxes left. One is the box containing the two gold coins and the other is the box containing one gold and one silver coin. It seems intuitively clear that each of these boxes is equally likely to be the one you chose at random, and that therefore the chance it is the box with two gold coins is 1/2, and the chance that it is the box containing one gold and one silver coin is also 1/2. Therefore, the probability that the other coin is gold must be 1/2.
This sounds right, but it is in fact the wrong answer.
In fact, there are three equally likely scenarios that might have led to you choosing that shiny gold coin.
Let us separately label all the coins in the boxes to make this clear.
In the box containing two gold coins, there will be Gold Coin 1 and Gold Coin 2. These are both gold coins but they are distinct, different coins.
In the box containing the gold and silver coins, we have Gold Coin 3,which is a different coin to Gold Coin 1 and Gold Coin 2. There is also what we might label Silver Coin 3 in the box with Gold Coin 3. This silver coin is distinct and different to what we might label Silver Coin 1 and Silver Coin 2, which are in the box containing two silver coins, which was not selected.
So here are the equally likely scenarios when you withdrew a gold coin from the box.
a. You chose Gold Coin 1.
b. You chose Gold Coin 2.
c. You chose Gold Coin 3.
You do not know which of these gold coins you withdrew from the box.
If it was Gold Coin 1, the other coin in the box is also gold.
If it was Gold Coin 2, the other coin in the box is also gold.
If it was Gold Coin 3, the other coin in the box is silver.
Each of these possible scenarios is equally likely (i.e. each has a probability of being the true state of the world of 1/3), so the probability that the other coin is gold is 2/3 and the probability that the other coin is silver is 1/3. So, if you are offered even money about the other coin being gold, the edge is very much with you.
Before withdrawing the gold coin, the chance that the box you had selected was that containing two gold coins was 1/3. By revealing the gold coin, however, you not only excluded the box containing two silver coins but also introduced the new information that you could potentially have chosen a silver coin (if the selected box was that containing one gold and one silver coin) but in fact did not. That made it more likely (twice as likely) that the box you withdrew the gold coin from was that containing the two gold coins than the box containing one gold and one silver coin.
And that is the solution to the Bertrand’s Box paradox.
When Theresa May announced on April 18 that she would call a snap general election, most commentators viewed the precise outcome of the vote as little more than a formality. The Conservatives were sailing more than 20% ahead of the Labour party in a number of opinion polls, and most expected them to be swept back into power with a hefty majority.
Even after a campaign blighted by manifesto problems and two terrorist attacks, the Conservatives were by election day still comfortably ahead in most polls and in the betting markets. According to the spread betting markets, they were heading for an overall majority north of 70 seats, while a number of forecasting methodologies projected that Jeremy Corbyn’s Labour could end up with fewer than 210.
In particular, an analysis of the favourite in each of the seats traded on the Betfair market gave the Tories 366 seats and Labour 208. The Predictwise betting aggregation site gave the Conservatives an 81% chance of securing an overall majority of seats, in line with the large sums of money trading on the Betfair exchange.
The PredictIt prediction market, meanwhile, estimated just a 15% chance that the Tories would secure 329 or fewer seats in the House of Commons (with 326 technically required for a majority), while the Oddschecker odds comparison site rated a “hung parliament” result an 11/2 chance (an implied probability of 15.4%). Only the Almanis crowd forecasting platform expressed any real doubt, putting the chance of a Conservative overall majority at a relatively paltry 62%.
In reality, the Conservative party lost more than a dozen seats net, ending up with 318 – eight short of a majority. Labour secured 262 seats, the Scottish National party 35, and the Liberal Democrats 12. Their projected vote shares are 42.4%, 40%, 3% and 7.9% respectively.
So did the opinion polls do any better than the betting markets? With the odd exception, no.
In their final published polls, ICM put the Tories on 46%, up 12% on Labour. ComRes predicted the Tories would score 44% with a 10-point lead. BMG Research was even further out, putting the Conservatives on 46% and a full 13% clear of Labour. YouGov put the Tories seven points clear of Labour (though their constituency-level model did a lot better), as did Opinium; Ipsos MORI and Panelbase had them eight points clear on 44%.
Other polls were at least in the ballpark. Kantar Public put the Tories 5% ahead of Labour, and SurveyMonkey (for the Sun) called the gap at 4%. Survation, the firm closest to the final result in their unpublished 2015 poll, this time put the Conservatives on 42% and Labour on 40%, very close to the actual result. Qriously (for Wired)was the only pollster to put Labour ahead, by three points.
According to the Chris Hanretty 2017 UK Parliamentary Election Forecast polling model, the Conservatives were heading for 366 seats, Labour 207, and the Liberal Democrats seven. Allowing for statistical uncertainty, the projection was of an “almost certain” overall majority for the Conservatives. The probability of a hung parliament was put at just 3%. All very bad misses.
Many others were wrong, too. The 2017 General Election Combined Forecast, which aggregates betting markets and polling models, forecast a Conservative majority of 66 seats. Other “expert” forecasts came from Britain Elects (Tories 356 seats, Labour 219 seats), Ashcroft (363, 217), Electoral Calculus (358, 218), Matt Singh (374, 207), Nigel Marriott (375, 202), Election Data (387, 186), Michael Thrasher (349, 215), Iain Dale (392, 163) and Andreas Murr and his colleagues (361, 236).
So what went wrong?
In the wake of the 2015 election, the Brexit referendum and Donald Trump’s victory, forecasters are getting used to fielding that question. But the answer isn’t that difficult: the problem is in quantifying the key factor in the common forecasting meltdown in advance. That factor is turnout, and notably relative turnout by different demographics.
In the Brexit referendum and 2016 US presidential election, turnout by poorer and less educated voters, especially outside urban areas, hit unprecedentedly high levels, as people who had never voted before (and may never vote again) came out in droves. In both cases, forecasters’ pre-vote turnout models had predicted that these voters wouldn’t show up in nearly the numbers they did.
In the 2017 election, it was turnout among the young in particular that rocketed. This time the factor was widely expected to matter, and indeed get-out-the-vote campaigns aimed at the young were based on it. But most polling models failed to properly account for it, and that meant their predictions were wrong.
Polling is a moving target, and the spoils go to those who are most adept at taking and changing aim. So will the lesson be learned for next time? Possibly. But next time, under-25s might not turn out in anything like the same numbers – or a different demographic altogether might surprise everyone. We might not have long to wait to find out.
References:
Leighton Vaughan Williams. Report card: How well did UK election forecasters perform this time? Article in The Conversation. Link below:
https://theconversation.com/report-card-how-well-did-uk-election-forecasters-perform-this-time-79237
If there is a set of ‘game’ strategies with the property that no ‘player’ can benefit by changing their strategy while the other players keep their strategies unchanged, then that set of strategies and the corresponding payoffs constitute what is known as the ‘Nash equilibrium’.
This leads us to the classic ‘Prisoner’s Dilemma’ problem. In this scenario, two prisoners, linked to the same crime, are offered a discount on their prison terms for confessing if the other prisoner continues to deny it, in which case the other prisoner will receive a much stiffer sentence. However, they will both be better off if both deny the crime than if both confess to it. The problem each faces is that they can’t communicate and strike an enforceable deal. The box diagram below shows an example of the Prisoner’s Dilemma in action.
| Prisoner 2 Confesses | Prisoner 2 Denies | |
| Prisoner 1 Confesses | 2 years each | Freedom for P1; 8 years for P2 |
| Prisoner 1 Denies | 8 years for P1; Freedom for P2 | 1 year each |
The Nash Equilibrium is for both to confess, in which case they will both receive 2 years. But this is not the outcome they would have chosen if they could have agreed in advance to a mutually enforceable deal. In that case they would have chosen a scenario where both denied the crime and received 1 year each.
So a Nash equilibrium is a stable state that involves interacting participants in which none can gain by a change of strategy as long as the other participants remain unchanged. It is not necessarily the best outcome for the parties involved, but it is the outcome we would most likely predict.
The Prisoner’s Dilemma is a one-stage game, however. What happens in games with more than one round, where players can learn from the previous moves of the other players?
Take the case of a 2-round game. The payoff from the game will equal the sum of payoffs from both moves.
The game starts with two players, each of whom is given £100 to place into a pot. They can then secretly choose to honour the deal or to cheat on the deal, by means of giving an envelope to the host containing the card ‘Honour’ or ‘Cheat’. If they both choose to ‘Honour’ the deal, an additional £100 is added to the pot, yielding each an additional £50. So they end up with £150 each. But if one honours the deal and the other cheats on the deal, the ‘Cheat’ wins the original pot (£200) and the ‘Honour’ player loses all the money in that round. A third outcome is that both players choose to ‘Cheat’, in which case each keeps the original £100. So in this round, the dominant strategy for each player (assuming no further rounds) is to ‘Cheat’, as this yields a higher payoff if the opponent ‘Honours’ the deal (£200 instead of £150) and a higher payoff if the opponent ‘Cheats’ (£100 instead of zero). The negotiated, mutually enforceable outcome, on the other hand, would be to agree to both ‘Honour’ the deal and go away with £150.
But how does this change in a 2-round game.
Actually, it makes no difference. In this scenario, the next round is the final round, in which you may as well ‘Cheat’ as there are no future rounds to realise the benefit of any goodwill realised from honouring the deal. Your opponent knows this, so you can assume your opponent who wishes to maximise his total payoff, will be hostile on the second move. He will assume the same about you.
Since you will both ‘Cheat’ on the second and final move, why be friendly on the first move?
So the dominant strategy is to ‘Cheat’ on the first round.
What if there are three rounds? The same applies. You know that your opponent will ‘Cheat’ on the final round and therefore the penultimate round as well. So your dominant strategy is to ‘Cheat’ on the first round, the second round and the final round. The same goes for your opponent. And so on. In any finite, pre-determined number of rounds, the dominant strategy in any round is to ‘Cheat.’
But what if the game involves an indeterminate number of moves? Suppose that after each move, you roll two dice. If you get a double-six, the game ends. Any other combination of numbers, play another round. Keep playing until you get a double-six. Your score for the game is the sum of your payoffs.
This sort of game in fact mirrors many real-world situations. In real life, you often don’t know when the game will end.
What is the best strategy in repeated play? For the game outlined above, we shall denote ‘Honour the deal’ as a ‘Friendly’ move and ‘Cheat’ as a hostile move. But the notion of a Friendly or Hostile approach can adopt other guises in different games.
There are seven proposed strategies here.
- Always Friendly. Be friendly every time
- Always Hostile. Be hostile every time
- Retaliate. Be Friendly as long as your opponent is Friendly but if your opponent is ever Hostile, you be Hostile from that point on.
- Tit for tat. Be Friendly on the first move. Thereafter, do whatever your opponent did on the previous move.
- Random. On each move, toss a coin. If Heads, be Friendly. If tails, be Hostile.
- Alternate. Be Friendly on even-numbered moves, and Hostile on odd-numbered moves, or vice-versa.
- Fraction. Be Friendly on the first move. Thereafter, be Friendly if the fraction of times your opponent has been Friendly until that point is less than a half. Be Hostile if it is less than or equal to a half.
Which of these is the dominant strategy in this game of iterated play? Actually, there is no dominant strategy in an iterated game, but which strategy actually wins if every strategy plays every other strategy.
‘Always Hostile’ does best against ‘Always Friendly’ because every time you are Friendly against an ‘Always Hostile’, you are punished with the ‘sucker’ payoff.
‘Always Friendly’ does best against Retaliation, because the extra payoff you get from a Hostile move is eventually negated by the Retaliation.
Thus even the choice of whether to be Friendly or Hostile on the first move depends on the opponent’s strategy.
For every two distinct strategies, A and B, there is a strategy C against which A does better than B, and a strategy D against which B does better than A.
So which strategy wins when every strategy plays every other strategy in a tournament? This has been computer simulated many times. And the winner is Tit for Tat.
It’s true that Tit for Tat can never get a higher score than a particular opponent, but it wins tournaments where each strategy plays every other strategy. In particular, it does well against Friendly strategies, while it is not exploited by Hostile strategies. So you can trust Tit for Tat. It won’t take advantage of another strategy. Tit for Tat and its opponents both do best when both are Friendly. Look at this way. There are two reasons for a player to be unilaterally hostile, i.e. to take advantage of an opponent or to avoid being taken advantage of by an opponent. Tit for Tat eliminates the reasons for being Hostile.
What accounts for Tit for Tat’s success, therefore, is its combination of being nice, retaliatory, forgiving and clear.
In other words, success in an evolutionary ‘game’ is correlated with the following characteristics:
Be willing to be nice: cooperate, never be the first to defect.
Don’t be played for a sucker: return defection for defection, cooperation for cooperation.
Don’t be envious: focus on how well you are doing, as opposed to ensuring you are doing better than everyone else.
Be forgiving if someone is willing to change their ways and co-operate with you. Don’t bear grudges for old actions.
Don’t be too clever or too tricky. Clarity is essential for others to cooperate with you.
As Robert Axelrod, who pioneered this area of game theory in his book, ‘The Evolution of Cooperation’: Tit for Tat’s “niceness prevents it from getting into unnecessary trouble. Its retaliation discourages the other side from persisting whenever defection is tried. Its forgiveness helps restore mutual cooperation. And its clarity makes it intelligible to the other player, thereby eliciting long-term cooperation.”
How about the bigger picture? Can Tit for Tat perhaps teach us a lesson in how to play the game of life? Yes, in my view it probably can.
Further Reading and Links
Axelrod, Robert (1984), The Evolution of Cooperation, Basic Books
Axelrod, Robert (2006), The Evolution of Cooperation (Revised ed.), Perseus Books Group
Axelrod, R. and Hamilton, W.D. (1981), The Evolution of Cooperation, Science, 211, 1390-96. http://www-personal.umich.edu/~axe/research/Axelrod%20and%20Hamilton%20EC%201981.pdf
The El Clasico game between Real Madrid and Barcelona is in the 23rd minute at the Santiago Bernabeu when Lionel Messi is brought down in the penalty box and rewarded with a spot kick against the custodian of the Los Blancos net, Keylor Navas.
Messi knows from the team statistician that if he aims straight and the goalkeeper stands still, his chance of scoring is just 30%. But if he aims straight and Navas dives to one corner, his chance of converting the penalty rises to 90%.
On the other hand, if Messi aims at a corner and the goalkeeper stands still, his chance of scoring is a solid 80%, while it falls to 50% if the goalkeeper dives to a corner.
We are here simplifying the choices to two distinct options, for the sake of simplicity and clarity.
Navas also knows from his team statistician that if he dives to one corner and Messi aims straight, his chance of saving is just 10%. But if he stands still and Messi aims at one corner, his chance of saving the penalty rises to 50%.
On the other hand, if Navas stands still and Messi aims at a corner, his chance of making the save is just 20%, while it rises to 70% if Messi aims straight.
So this is the payoff matrix, so to speak, facing Messi as he weighs up his decision.
| Goalkeeper – Stands Still | Goalkeeper – dive to one corner | |
| Lionel Messi – Aims straight | 30% | 90% |
| Lionel Messi – Aims at corner | 80% | 50% |
So what should he do? Aim straight or to a corner. And what should Navas do? Stand still or dive?
Here is the payoff matrix facing Navas.
| Messi – Aims straight | Messi – Aims at a corner | |
| Navas – Stands still | 70% | 20% |
| Navas – Dives to one corner | 10% | 50% |
Game theory can help here.
Neither player has what is called a dominant strategy in game-theoretic terms, i.e. a strategy that is better than the other, no matter what the opponent does. The optimal strategy will depend on what the opponent’s strategy is.
In such a situation, game theory indicates that both players should mix their strategies, in Messi’s case aiming for the corner with a two-thirds chance, while the goalkeeper should dive with a 5/9 chance.
These figures are derived by finding the ratio where the chance of scoring (or saving) is the same, whichever of the two tactics the other player uses.
The Proof
Suppose the goalkeeper opts to stand still, then Messi’s chance (if he aims for the corner 2/3 of the time) = 1/3 x 30% + 2/3 x 80% = 10% + 53.3% = 63.3%
If the goalkeeper opts to dive, Messi’s chance = 1/3 x 90% + 2/3 x 50% = 30% + 33.3% = 63.3%
Adopting this mixed strategy (aim for the corner 2/3 of the time and shoot straight 1/3 of the time), the chance of scoring is therefore the same. This is the ideal mixed strategy, according to standard game theory.
From the point of view of Navas, on the other hand, if Messi aims straight, his chance of saving the penalty kick (if he dives 5/9 of the time) = 5/9 x 10% + 4/9 x 70% = 5.6% + 31.1% = 36.7%
If Messi opts to aim for the corner, Navas’ chance = 5/9 x 50% + 4/9 x 20% = 27.8% + 8.9% = 36.7%
Adopting this mixed strategy (dive for the corner 5/9 of the time and stand still 4/9 of the time), the chance of scoring is therefore the same. This is the ideal mixed strategy, according to standard game theory.
The chances of Messi scoring and Navas making the save in each case add up to 100%, which cross-checks the calculations.
Of course, if the striker or the goalkeeper gives away real new information about what he will do, then each of them can adjust tactics and increase their chance of scoring or saving.
To properly operationalise a mixed strategy requires one extra element, and that is the ability to truly randomise the choices, so that Messi actually does have exactly a 2/3 chance of aiming for the corner, and Navas actually does have a 5/9 chance of diving for the corner. There are different ways of achieving this. One method of achieving a 2/3 ratio is to roll a die and go for the corner if it comes up 1, 2, 3 or 4, and aim straight if it comes up 5 or 6. Or perhaps not! But you get the idea.
For the record, Messi aimed at the left corner, Navas guessed correctly and got an outstretched hand to it, pushing it back into play. Leo stepped forward deftly to score the rebound. Cristiano Ronaldo equalised from the spot eight minutes later. And that’s how it ended at the Bernabeu. Real Madrid 1 Barcelona 1. Honours even in El Clasico.
Appendix
Messi’s strategy
x = chance that Messi should aim at corner
y = chance that Messi should aim straight
So,
80x + 30y (if Navas stands still) = 50x + 90y (if Navas dives)
x + y = 1
So,
30x = 60y
30x = 60 (1-x)
90x = 60
x = 2/3
y=1/3
Navas’ strategy
x = chance that Navas should dive to corner
y = chance that Navas should stand still
So,
10x + 70y (if Messi aims straight) = 50x + 20y (if Messi aims at corner)
x+y = 1
So,
10x + 70y = 50x + 20y
40x = 50y
40x = 50(1-x)
90x = 50
x = 5/9
y = 4/9
Professor Leighton Vaughan Williams 