Skip to content

The Tenth Hole – A Short Story

The Tenth Hole
A short story – based on a real event

By Leighton Vaughan Williams

Based closely on a meeting with a stranger on a golf course, it is a meeting
that has inspired me to write this story. It is a story that I didn’t expect to
need to tell, but have needed to tell. If nobody reads it, no matter. But I
hope people do read it. Especially the people who have lessons still to
learn and dreams still to dream. Especially the people who want to play the
tenth hole! Especially those people. But it is a story for everyone.

Synopsis
It all started on the first green when a little white ball mysteriously appeared over our heads, landing next to the pin. Enter nobody! Then a stranger! The journey had not yet begun but would soon do so. As we parted company at the final green with a firm handshake and a salutary farewell, we realised that we had been part of one of those experiences which challenges one’s fundamental perceptions and pre-conceptions, illuminating us without dazzling us, humbling us without diminishing us. Yet we know not the stranger’s name nor philosophy. We did not ask, nor did we desire to know. We had learned enough without needing to seek further. As I placed my trusty white putter into the bag for the final time in the round, I realised I would never reach for it again in quite the same way. I realised also that our journey could only really be experienced, not told. Yet at the same time I knew that our journey offers a tale that needs to be told and re-told, for as long as there remain in golf, and life, lessons to be learned, dreams to be dreamed. Steve knew so too. He knew so because of what had happened the night before.

Chapter 1
The Third Hole
“Sorry about the shot back there. I didn’t know there was anyone on the green. I should have checked.” In truth, the apologised-for strike up the demanding uphill first hole was not a shot many would have expected to reach the skirts of the putting surface, let alone come to rest blowing a six inch kiss to the flag. I brushed it off with a dismissive wave. It is part of the happenstance that populates any game of golf, and especially one in which the opening challenge is played blind over the brink of a hill peering barely 60 yards to the front edge of the polestick’s personal domain. It was not until we were standing astride the third tee that the stranger had made his presence known, in the guise of one of the series of sauntering singleton players who increasingly grace the summer weekday afternoons with their individual displays of personal golfing panache.

A stranger, a singleton, that much we established on sight. A player of the game whose pace, and possibly skill, somewhat outstripped ours, we now established by simple deduction. “Go ahead of us. Play through.” Steve made the offer, as if in response to the words of apology, though it came out more like a barked command. “But I have all the time in the world.” The stranger’s words could have been spoken with weary regret or mock ribaldry or simply matter-of-factly, but there was no flicker of any of these sentiments. None. They were spoken instead with an inflection of what might be described as detached satisfaction lightly brushed with a hint of passion. Such a vocal inflection was strange, but only because it was unexpected. But it somehow sounded neither odd nor unusual. Nor was it discomforting. Yet we were discomforted, for another reason. That offer to go ahead of us, such a superficially selfless act of generosity, was in fact neither selfless nor generous. It was born rather out of a simple desire to see the stranger proceed on his way. And to free us from the burden of a pressing presence to our rear. As well as potential exposure to a dimpled projectile rising bullet-like in undetected flight toward our unprotected frames. And to free us from the burden of unbidden external inspection and inner judgement. We played on, in the quiet hope, even expectation, that a couple of wayward and horizontally challenged strikes off the tee would achieve the result which invitation had regrettably failed to achieve. The concept of limitless time would be put to the test of the real ticking clock.

I struck the ball first, to see it simply disappear from sight. In my experience this always means one thing – that the ball, while technically still in its own existence, would never again be part of mine. It was lost. Steve kept track of his own ball, but only because it had travelled a bare 30 yards towards its destination roughly a quarter of a mile away. We looked to the stranger. “Bad luck” , he intoned. “It happens to us all.” “Play through?”, I enquired. The stranger seemed to consider for an extended moment. “I know where your ball is. I’ll show you.” The ball was soon retrieved, from some of the rough grass designed to challenge players on the adjacent hole. Meanwhile, Steve had played his second shot. Eventually we reached the putting green. As we considered line and length, a ball could be seen descending from its high trajectory. Struck from the third tee, it landed short of our feet but long of reason. An obvious fluke, this was a ball delivered to its flight path as if by mechanical, not human, means. Inspired, we sank our putts. We bagged our respective putters, one made for the left handed and one for the right, and turned, a little lighter of foot, to tee off at the short fourth. Our path meandered edgingly close to the little white missile which awaited the stranger’s next smack. We paused as he came closer. “That was a big hit”, I ventured. “It achieved its purpose,” he smiled. “After all, you sunk your putts.” I considered how he knew our green play from so afar, but set the thought aside. “By the way, thanks for finding my ball. I was sure it was lost.” He looked at me quizzically. “If you know where something is, can it be lost?” He paused for a moment. “I’m not just talking about golf, you know. I’m really not!”

Chapter 2
The Fourth Hole
It is a relatively short hole, so the course designers had felt the need to compensate. They dug a deep bunker to the front side of the green, another to the right side and a steep grassy bank running down its back side. But we were confident. That’s what a hole-dropping long putt does for you. But the tee is set atop a flight of short steps, offering a balcony view to any short strike to the third. So we viewed. The stranger obliged. He knew where the hole was. And he knew how to reach it. Two feet from the hole. Now a simple tap-in, he marked the ball, cleaned it, set it down again. He tapped but it stayed out. He turned and looked up to us atop our temporary perch. He was smiling broadly. He was happy. Now it was my turn. To be happy. To sidestep or overfly sandy trap, to shy short of flirtation with treacherous back side.

I visualised. I swung golf’s version of bat at ball. I succeeded. In my own way. I was satisfied. But I was not happy. Not happy like the stranger. As Steve stepped up, the happy face, now muted, stood with him, still, close yet distant. Club hit ball, ball responded. Then misbehaved. “Difficult hole.” Steve and I were silent, then Steve spoke. “We’re not on song today. You go ahead. Please proceed.” Proceed he did, to place tee in ground, ball on tee, administer smart blow of short club to small ball. Perfect shot, perfect line, perfect distance, if aimed at the deep forward sandy trap. “Beached!”, he noted. It was a simple statement of fact. “I could be quite some time. You two play ahead.” We did.

Steve withdrew a club of middling length and swung freely at the ball. A pleasing crack. It was well timed. We all knew it. It found the green, albeit the right edge, opposite end to the stick. But nice enough. This was a turnaround. Both Steve and I with at least a chance of potting the hole in three, and the stranger’s ball buried deep below a gaping, uninviting, curling upper lip. This was an unequal contest between skill and natural sand, tipped heavily in favour of nature. And tipped in favour of us. I had clear sight of the pin, about 40 feet away, angled just forward of my left shoulder. Maybe 12 feet to the relatively smooth surface which surrounded the flag. I was furthest from the hole, if measured by distance. I locked my wrists and applied blade to ball. Ball bounced away to order. Slow surface. Good for once. Just short and below the hole. Steve was on the green. Still separated, ball to cup, by more than the stranger. But Steve’s ball had the advantage. It could see the objective. So could Steve. He took his time. He took his time on that putt as if he had all the time in the world. He had the line. He sort of had the length. Came up a nose short, queried its options, then decided to drop. Three. Par. He seemed to wave. Almost imperceptibly, but he seemed to wave. He seemed to wave to the stranger.

Now it was my turn. A five footer to share the hole. Don’t leave it short. Don’t leave it short. I didn’t. Hole shared. Move on or wait for splash of grainy sand. We waited and got a shower, and a show, as the flecked sphere soared from its tomb aloft a chariot of spray. It was long of the pin, before the backspin, and less long after, but still the length of a longshot. “You go ahead,” he shouted. “I might be quite some time.”

Chapter 3
The Fifth Hole
We walked on to the difficult fifth. My honour again to tee off first. 480 yards to the middle of the green. No need to guess. Modern technology has seen to that. Smooth swing. This time it flew straight, but not particularly long, and so tracked by eye through its entire flight. “I’m happy enough with that”. Could have been longer but a shot to make you happy. Now Steve inserted his tee into the sloping ground, placed his ball on it and prepared to play. “Same action, using a higher tee, and you would get the same result and a lot more length.” The stranger spoke softly but it interrupted Steve’s pre-shot ritual. Steve stopped and took a few steps backward. Hesitant now, where he had been confident, he sort of tiptoed to tee, imparted a dull thwack to ball and stood back to inspect the damage. He had topped it, had hit just its very top. It had at least gone straight.

It was in the short grass, that much could be said for it. But if only he had cradled the ball upon a higher tee. He remembered the stranger’s words. He looked at the stranger. The stranger said nothing. He had no need to. Steve was still a good 400 yards short of the green, I a little closer, but there was someone with us who placed his ball on a somewhat higher tee than either of us. I saw an opportunity. “Why don’t you show us how you tee up your ball, and show us the result?” He obliged, propelling the ball into an unusually steep trajectory, unusual by our standards.

“How did you do that?”, I asked. “Practice,” he murmured. “And belief.”
“So you can do it every time?” I was curious. “Not at all,” he replied.”You saw. I just missed a simple putt back there. It happens.” He paused. “Acceptance. When we fail we need acceptance. It’s about belief and acceptance. And practice.”
“Easier when the putts are dropping, and you’re hitting the greens. It’s when they’re not that the doubt creeps in.” I looked at Steve, still mulling over his mishit.
The stranger smiled, as if he’d heard it all before. “That, of course, is when your belief in your game is most important of all. Trust me!”

I did trust in something. I trusted in his own belief in his game. I shared his belief in his game. It was my game that I didn’t trust, that I didn’t believe in. I told him just that. “Well, it’s a start,” he said. “It’s a good start.” And he was off. “See you!” We bid the stranger well as he strode off over the artificial horizon created by the dip down the hill. “Try transferring some weight onto your back foot if you want lift.” He was calling back to us. “But don’t expect miracles. Genuine miracles are rare.”

Soon out of sight, he was not out of mind. We played on. To play safe or straddle the lake? Doubt or belief? We both chose the middle path, left of lake and far short of target. And found, in each case, the creek. Doubt played no further part. Lost ball, dropped ball, lose a shot, carry on. Lost ball? “It’s not lost if you know where to find it!” It was the same voice, of the stranger. “But I don’t know where to find it.” “Nor I”, echoed Steve. “Here they are!” Balls both found, thanks to stranger. We could barely see them, but there they were, buried treacherously, all but invisibly deep. How did he spot them? Why was he looking? Questions soon rendered academic. We could not retrieve them. Thank you stranger, we thought, you have shown us what we looked for. We know more but have gained nothing. But we were thinking of golf. We were only thinking of golf. We had learned nothing. But the stranger, he wasn’t just talking about golf, you know. He really wasn’t! We didn’t mind that. Get a new ball out of our bags. Time to play our shots. So we did. We were playing golf. And the game, to win or lose,was on. So to the green.

Preparing to putt for the hole. I’m confident. I bring the putter head straight back. Now don’t decelerate on the shot! A cry from the direction of adjacent tee. ‘Fore!’ “Watch out!” I react, instinctively, and miss. In truth, the rocketing wayward ball missed my person by all of six feet. “Unlucky,” Steve commiserated. But he was happy really. He shared the hole. I was lucky in another sense. Six feet to the left and I might have played my last shot. I hadn’t prepared for that. You never do. You’re concentrating too much on playing the game.

Chapter 4
The Sixth Hole
I like the sixth hole, except for the tree, the tree that always grabs my ball out of its soaring greenward flight. A neighbouring golf club had taken a vote on whether to demolish a tree. Their golfers had the same problem. It stopped the ball going where it was hit. It was a hazard, like a sand trap, only taller and more explicitly menacing. They voted to chop it down. Ours is much the same, but there’s no plan for a vote. One of my balls is still up it, trapped for posterity. I guess it will still be there when I am gone. I think about that every time I pass that tree. It was there as I considered my second shot. We had both hit pretty standard drives up the hill, off the tee. If the ball flies far enough that it disappears over the brow of the hill, I always jump a little. I don’t actually leave the ground, but I jump inside, if you understand. I suppose Steve does too, though not when I do it. Then he has the opposite feeling. There’s not much room for empathy on the golf course. Maybe there should be. It’s only a game, they say. But they’re wrong. Games are important. They teach you things. About yourself. And about others.

Standing atop the crest of the hill, and a few strides beyond, I prepare to avoid the tree. It’s to my left, about 100 yards, maybe a bit more. Should be able to avoid it this time. Hit straight or perhaps a little right. Then it’s a safe, controlled punch to the middle of the green. I can dream. That’s part of the game. But do I believe? I think I do. Then I see the branches. I always see the branches. I try to see the smooth, even grass of the fairway beyond the tree. I believe I will reach it. I’m sure I will reach it. But what about the branches? It’s always about the branches. There are so many branches. I prepare to play. A shout to me, this time heralding advice not danger. “It’s not about the tree!” It was the stranger. “It really isn’t about the tree. It’s about the fairway. Focus on that and you’ll be safe.” Easy for him to say that, I thought. He knows the game. He may as well have invented it. But I didn’t know the game. Well, I knew the rules, at least the ones that let me play the game, but I didn’t really know the game. Not like the stranger. “Trust me!” I looked at him, bag of clubs slung over his right shoulder, left hand propped against tree. “Keep standing there and it’s you who’ll be needing to trust,” I shouted back. Addressing the ball, I took aim. He didn’t take cover. He seemed to know. As did I. I simply knew that I wasn’t going to hit that tree. I trusted him. Despite the branches.

As I walked past the tree toward the pleasant fairway lie, I thanked him. I didn’t even ask him why he had waited, and not moved on to the next hole. I didn’t need to. And I didn’t want to. He returned the thanks. “You won’t always miss that tree,” he said. “But you will always know that you can.” He walked on to the next hole. My next shot found the deep rough, then the sand, then three more shots. Steve raised his hand. But he didn’t jump. Not in the air. Just inside. So did I. For a different reason, so did I.

Chapter 5
The Seventh Hole
Hit it down the fairway. Straight as you can. Easy shot to the green. One putt. In the hole. Shout of ‘Birdie.’ Job done. Before we hit the ball, we can all dream. I had the dream that day. Until I swung the club. The ball bore no blame. It had no will, no self-control. It simply obeyed instructions. But I had will, I had the freedom to instruct it as I liked. And I had the dream. So why was the ball buried deep in the bushes? I had teed it up high as I should, swung club with smoothness of action, completed follow through with self-referenced elegance. So why? I could have asked the stranger, had he been there. Steve was there, but he didn’t know the answer, even if it had been a real question.

Then I saw a dog. Looked like a nice dog, a spaniel, I think. I don’t know a lot about dogs so I can’t be sure. I just know that there are nice dogs and nasty dogs. This one looked nice. The one that bit Steve a few weeks back was the other kind. That dog wasn’t on a lead. It was free to bite and it bit. It bit Steve on the belly. But he didn’t tell anyone, not anyone in authority. That was his choice. He chose to hope it wouldn’t do it again. Steve wasn’t doing anything wrong, not when he was bit. He had no choice about that. The choice came later. I had a choice. I could have chosen to give the ball better instructions. But it was not a real choice. I did try. But now I had a real choice. To forget the fluffed shot, drop another ball, lose a shot, address the ball, play as if this was the ball’s natural resting place. Or to continue asking ‘why’ in plaintive cry. I knew what the stranger would do. So I played the ball, as best I could. Not in anger but in thanks. Thanks to the stranger. And now to the lady with the dog, who shouts “Well played!” The kindness of strangers. Quite pleased with the outcome. Not as pleased as Steve who will soon be at the flag. Or at least near enough to ensure the hole. But I was playing against the course. When Steve plays well, I play against the course. When Steve plays badly, I play against Steve. Confident now, unhurried, undisturbed. I dream of holing it in one from here. Hit the flag. That’ll do. I’ll be happy with that. I look around. Nobody to impress this time.

The lady has moved on, taking her nice dog with her. It had no choice. It was on a lead. The man in the motorised grass-cutter does have a choice. He can choose to buzz around when you are trying to play, or buzz off until you’ve played. He chooses to buzz around. He always does. It stops him getting bored. He’s near enough now that the ball might hit him if you take your shot quickly. The stranger could definitely hit him. He has the skill. But you’d have to be lucky. He’s not even wearing a helmet. His own silly fault if I hit him. I feel lucky. Then I see him. Back of the green. Watching. I see the stranger. Embarrassed, I aim for the green, and almost hit the flag. I’m playing the game now. That’s why games are important. They teach you things. Steve wins the hole, but I’ve won too. The man in the machine has moved on. So has the stranger. And so, I feel, have I.

Chapter 6
The Eighth Hole
The weather forecast had said heavy showers. But I didn’t believe the forecast. They say bad things and if they’re wrong, you will be happy anyway. And forget their mistake. But if they say good weather, and you are drenched, you will not easily forgive them. Clever strategy. But no good if you want to know how the weather will be. So I didn’t believe, and persuaded Steve I was right. He believed me. Now he was wet. I saw the lady with the dog in the distance. She had opened an umbrella. I wondered about the stranger. Now we had a decision to make. We could take whatever partial shelter we could or play on and be fully sheltered sooner. We played on.

The weather did its worst but only served to help us focus. Two shots. Straight off the tee. Both off the putting green. But neither by much. Then a flash. Count to three and the crash of thunder. Not far away now. I know someone who knew someone who was struck by lightning. The girl she knew was on a golf course when it happened. Did everything right. Made all the right choices once she saw the electric fork. But she hadn’t believed the forecast. Clever strategy, she had said, to forecast storms. You’d forgive and forget if the weather was good. This time it wasn’t. The forecasters were right. She was wrong. She paid the price. A high price for getting it wrong, but life is like that sometimes. This was on my mind as I lined up my shot, designed as a little chip to the flag. Maybe my wrist quivered. I suppose the ball obeyed my instructions, it always does, but not my intent. Short, very short of the pin. Steve wasn’t scared like me. He said that it was very long odds to be struck dead. But he didn’t know someone who knew someone who succumbed despite the odds. I think it makes a difference. At least it made a difference that day. He holed it. And would probably live to tell the tale. Very probably, according to the odds. We move on to the closing hole of this nine hole course. You can play eighteen holes if you like, but not eighteen different holes. You just play the nine holes twice. We play them just once. For us, there is no tenth hole.

Chapter 7
The Ninth Hole
The heavens continued to flash, almost in time now with thunderous clash, but Steve was unconcerned. He was chatting with someone in what the British call a golf buggy and the Americans call a cart. Steve climbed in. Then climbed out. “Safer in there”, he said. But he was wrong. First advice is to get clear of open frame vehicles. I told him. “But they’re earthed, grounded. Look at those rubber tyres.” Steve was sure of his physics. I didn’t know the physics but I knew the advice. Get clear, it said. “Anyway, meet Chris, who knows about these things.” My heart sank. The stranger, in whom I’d placed so much trust. Sitting in the nippy four-wheeler. So sure of himself. Placing his trust in a death trap. Now it made sense. I thought as we left the fourth green that I’d seen Steve wave at him. Almost imperceptibly, but definitely an acknowledgment. I had put it out of my mind, put it down to courtesy, but now it all came together. He was not a stranger, not to Steve. I went over to Chris, back turned but obviously impervious to danger. Blissful in ignorance. I tapped on the shoulder of the white collared shirt. Chris turned. She turned and looked at me. “Always best to hire the car when there’s a chance of thunder”. It was one of the local lady golfers, good player, better than us, at golf, but no better at physics. I laughed. Out of relief, really. The stranger would never sit in an open vehicle in a thunderstorm. We were free to do what we wanted, but not the stranger. He would only do what was right. I trusted in him. Then I saw him, at the crest of the hill, over which we hoped to propel our first volleys.

He was calling us on. I felt safe now. And Steve seemed to wave. He seemed to wave to the stranger. We cleared the hill, first bounce I prefer to recall, and took good advantage of the downward sloping smooth grassed fairway. Two further blows apiece, of club on ball, and we were on the green. Then two putts from Steve, one from me. “Well played, Sirs!” He joined us on the green, shook our hands firmly. “The game is important,” he said. It teaches us things, about ourselves, and about others. The question is how we play it.” We wanted to play it like him.

On to the tenth hole?” he asked. “There is no tenth hole,” we said, almost in unison. “There are only the ones you see. You keep going round and round. Until you stop. That’s all there is. Didn’t you know?” The stranger smiled. He knew that we were still talking about golf. “There is a tenth hole,” he said. But you need to look for it. And to believe in it.” He proceeded to hand each of us a little white ball. They were the same two balls, with their distinctive markings, that we had lost in the creek. I was momentarily staggered. “But how? They were lost, unreachable.” “Yes, those balls truly were lost, they were unreachable,”‘said the stranger. “But on the tenth hole, there are no lost balls. Nothing is lost on the tenth. Trust me.”

We did trust him now. We had seen and we believed. “You remember that shot off the tee, the one that soared so high, over the very tall pines. I saw you watching. Well, you can do that too. You can do that, when you reach the tenth hole.”
Then, with a wave, he started to walk away. “It’s about belief and acceptance”, he called back, “and practice. So, practise these things.” The stranger now out of sight, Steve turned to me. “He gave me a lift home last night,” he said, “when I missed the last bus, and was caught three miles from home in that terrible storm. He stopped his car and offered me a lift. Went totally out of his way, took me to my front door, then he was gone. I didn’t even have chance to thank him.”

“So why didn’t you say anything before?” I was perplexed. “Because I didn’t recognise him at first, though I did feel that I knew him. I didn’t recognise him until he called us on in the storm.” “Steve, are you absolutely sure it was him?” “Not for certain,” he said, “But that doesn’t really matter, does it?” “I suppose not,” I agreed. “I suppose not.” The kindness of strangers. What a wonderful thing. We looked to the sky. The clouds were slowly parting. We knew where we were now. We knew now that there was a tenth hole!

Strictly Come Dancing: the luck of the draw really does matter!

A viscountess, a radio DJ, a reality star, a vlogger, a comedian, several sportspeople and an assortment of actors and presenters. These, more or less, are the celebrities lined up to compete in the 2019 season of Strictly Come Dancing.

Outside their day jobs, few people know much about them yet. But over the 13 weeks or so of shows up until Christmas, viewers will at least learn how well the contestants can dance. But how much will their success in the competition have to do with their foxtrot and to what extent will it be, literally, the luck of the draw that sees the victors lift the trophy in December?

seminal study published in 2010 looked at public voting at the end of episodes of the various Idol television pop singing contests and found that singers who were later on in the bill got a disproportionately higher share of the public vote than those who had preceded them.

This was explained as a “recency effect” – meaning that those performing later are more recent in the memory of people who were judging or voting. Interestingly, a different study, of wine tasting, suggested that there is also a significant “primacy effect” which favours the wines that people taste first (as well, to some extent, as last).

A little bias is in order

What would happen if the evaluation of each performance was carried out immediately after each performance instead of at the end – surely this would eliminate the benefit of going last as there would be equal recency in each case? The problem in implementing this is that the public need to see all the performers before they can choose which of them deserves their vote.

Dress rehearsal for Strictly Come Dancing, August 2019. madathanu / Shutterstock.com

You might think the solution is to award a vote to each performer immediately after each performance – by complementing the public vote with the scores of a panel of expert judges. And, of course, Strictly Come Dancing (or Dancing with the Stars if you are in the US) does just this. So there should be no “recency effect” in the expert voting – because the next performer does not take to the stage until the previous performer has been scored.

We might expect in this case that the later performers taking to the dance floor should have no advantage over earlier performing contestants in the expert evaluations – and, in particular, there should be no “last dance” advantage.

We decided to test this out using a large data set of every performance ever danced on the UK and US versions of the show – going right back to the debut show in 2004. Our findings, published in Economics Letters, proved not only surprising, but almost a bit shocking.

Last shall be first

Contrary to expectations, we found the same sequence order bias by the expert panel judges – who voted after each act – as by the general public, voting after all performances had concluded.

We applied a range of statistical tests to allow for the difference in quality of the various performers and as a result we were able to exclude quality as a reason for getting high marks. This worked for all but the opening spot of the night, which we found was generally filled by one of the better performers.

So the findings matched the Idol study in demonstrating that the last dance slot should be most coveted, but that the first to perform also scored better than expected. This resembles a J-curve where there are sequence order effects such that the first and later performing contestants disproportionately gained higher expert panel scores.

Although we believe the production team’s choice of opening performance may play a role in this, our best explanation of the key sequence biases is as a type of “grade inflation” in the expert panel’s scoring. In particular, we interpret the “order” effect as deriving from studio audience pressure – a little like the published evidence of unconscious bias exhibited by referees in response to spectator pressure. The influence on the judges of increasing studio acclaim and euphoria as the contest progresses to a conclusion is likely to be further exacerbated by the proximity of the judges to the audience.

When the votes from the general public augment the expert panel scores – as is the case in Strictly Come Dancing – the biases observed in the expert panel scores are amplified.

All of which means that, based on past series, the best place to perform is last and second is the least successful place to perform.

The implications of this are worrying if they spill over into the real world. Is there an advantage in going last (or first) into the interview room for a job – even if the applicants are evaluated between interviews? The same effects could have implications in so many situations, such as sitting down in a dentist’s chair or doctor’s surgery, appearing in front of a magistrate or having your examination script marked by someone with a huge pile of work to get through.

One study, reported in the New York Times in 2011, found that experienced parole judges granted freedom about 65% of the time to the first prisoner to appear before them on a given day, and the first after lunch – but to almost nobody by the end of a morning session.

So our research confirms what has long been suspected – that the order in which performers (and quite possibly interviewees) appear can make a big difference. So it’s now time to look more carefully at the potential dangers this can pose more generally for people’s daily lives – and what we can do to best address the problem.

The Inspection Paradox – in a nutshell.

The bus arrives every twenty minutes on average, though sometimes the interval between buses is a bit longer and sometimes a bit shorter. Still, it’s 20 minutes taken as an average, or an average of three buses an hour. So you emerge onto the main road from a side lane at some random time, and come straight upon the bus stop. How long can you expect to wait on average for the next bus to arrive?
The intuitive answer is 10 minutes, since this is exactly half way along the average interval between buses, and if your usual wait is rather longer than this, then you have been unlucky.
But is this right? The Inspection Paradox suggests that in most circumstances you will actually be quite lucky only to wait ten minutes for the next bus to arrive.
Let’s examine this more closely. The bus arrives every 20 minutes on average, or three times an hour on average. But that is only an average. If they actually do arrive at exactly 20 minute intervals, then your expected wait is indeed 10 minutes (the mid-point of the interval between the bus arrivals). But if there is any variation around that average, things change, for the worse.

Say for example, that half the time the buses arrive at a ten minute interval and half the time at a 30 minute interval. The overall average is now 20 minutes, but from your point of view it is three times more likely that you’ll turn up during the 30 minute interval than during the ten minute interval. Your appearance at the stop is random, and as such is more likely to take place during a long interval between two buses arriving than during a short interval. It is like randomly throwing a dart at a timeline 30 minutes long. You could well hit the ten minute interval but it is much more likely that you will hit the 30 minute interval.
So let’s see what this means for our expected wait time. If you randomly arrive during the long (30 minute) interval, you can expect to wait 15 minutes. If you randomly arrive during the short (10 minute) interval, you can expect to wait 5 minutes. But there is three times the chance you will arrive during the long interval, and therefore three times the chance of waiting 15 minutes as five minutes. So you expected wait is 3×15 minutes plus 1x 5 minutes, divided by four. This equals 50 divided by 4 or 12.5 minutes.
In conclusion, the buses arrive on average every 20 minutes but your expected wait time is not half of that (10 minutes) but more in every case except when the buses arrive at exact 20 minute intervals. The greater the dispersion around the average, the greater the amount by which your expected wait time exceeds the average wait time. This is the ‘Inspection Paradox’, which states than whenever you ‘inspect’ a process you are likely to find that things take (or last) longer than their ‘uninspected’ average. What seems like the persistence of bad luck is actually the laws of probability and statistics playing out their natural course.
Once made aware of the paradox, it seems to appear everywhere.
For example, take the case where the average class size at an institution is 30 students. If you decide to interview random students from the institution, and ask them how big is their class size, you will usually obtain an average rather higher than 30. Let’s take a stylised example to explain why. Say that the institution has class sizes of either ten or 50, and there are equal numbers of both class sizes. So the overall average class size is 30. But in selecting a random student, it is five times more likely that he or she will come from a class of 50 students than of ten students. So for every one student who replies ‘10’ to your enquiry about their class size, there will be five who answer ’50.’ So the average class size thrown up by your survey is 5×50 + 1 x 10, divided by 6. This equals 260/6 = 43.3. So the act of inspecting the class sizes actually increases the average obtained compared to the uninspected average. The only circumstance in which the inspected and uninspected average coincides is when every class size is equal.
The range of real-life cases where this occurs is almost boundless. For example, you visit the gym at a random time of day and ask a random sample of those who are there how long they normally exercise for. The answer you obtain will likely well exceed the average of all those who attend the gym that day because it is more likely that when you turn up you will come across those who exercise for a long time than a short time.

Once you know about the Inspection Paradox, the world and our perception of our place in it, is never quite the same again.

 

Exercise

You arrive at someone’s home and are ushered into the garden. You know that a train passes the end of the garden every half an hour on average but the trains are actually scheduled so that half pass by with an  interval of a quarter of an hour and half with an interval of 45 minutes. Given that you have no clue when the last train passed by and the scheduled interval between that train and the next, how long can you expect to wait for the next train?

 

Links and References

Amir D. Aczel. Chance: A Guide to Gambling, Love, the Stock market and Just About Everything Else. 18 May, 2016. NY: Thunder’s Mouth Press.

On the Persistence of Bad Luck (and Good). Amir Aczel. Sept. 4, 2013. http://blogs.discovermagazine.com/crux/2013/09/04/on-the-persistence-of-bad-luck-and-good/#.XXJL0ihKh3g

The Waiting Time Paradox, or, Why is My Bus Always Late? https://jakevdp.github.io/blog/2018/09/13/waiting-time-paradox/

Probably Overthinking It. August 18, 2015. The Inspection Paradox is Everywhere. http://allendowney.blogspot.com/2015/08/the-inspection-paradox-is-everywhere.html

 

The Expected Value Paradox – in a nutshell.

To illustrate the Expected Value Paradox, let us propose a coin-tossing game, in which you gain 50% of what you bet if the coin lands Heads and lose 40% if it lands Tails. What is the expected value of a single play of this game?

The Expected Value can be calculated as the sum of the probabilities of each possible outcome in the game times the return if that outcome occurs.

Say, for example, the unit stake for each play of the game is £10. In this case, the gain if the coin lands Heads is 50% x £10 = £5, and the loss if the coin lands Tails is 40% x £10 = £4.

In this case, the expected value (given a fair coin, with 0.5 chance of Heads and 0.5 chance of Tails) = 0.5 x £5 – 0.5 x £4 = £0.5, or 50 pence.

So the Expected Value of the game is 5%. This is the positive net expectation for each play of the game (toss of the coin).

Let’s see how this plays out in an actual experiment in which 100 people play the game. What do we expect would be the average final balance of the players?

The expected gain from the 50 players tossing Heads = 50 x £5 = £250.

The expected loss from the 50 players tossing Tails = 50 x £4 = £200.

So, the net gain over 100 players = £250 – £200 = £50.

The average net gain of the 100 player = £50/100 = £0.5, or 50 pence.

Expected Value = 0.5 x £1.5 + 0.5 x 60p. = £1.05. As above, this is an expected gain of 5%.

From two coin tosses, our best estimate is 25 Heads-Heads, 25 Tails-Tails, 25 Heads-Tails and 25 Tails-Heads.

The Expected Value over the two coin tosses = 0.25 x (1.5)2 + 0.25 x (0.6)2 + 0.25 (1.5 x 0.6) + 0.25 (0.6 x 1.5) = £1.0575.

However many coin tosses the group throws, the Expected Value is positive.

Take now the case of one person playing the game through time. Say there are four coin tosses, for a stake of £10.

From four coin tosses, our best estimate is 2 Heads and 2 Tails.

Expected value for 2 Heads and 2 Tails = £10 x 1.5 x 1.5 x 0.6 x 0.6.

Expected value goes from £10 to £15 to £22.50 to £13.50 to £8.10. This is a net loss.

To clarify, we bet £10. The coin lands Heads. We now have £15. We bet £15 now on the next coin toss. It lands Heads again. We now have £22.50. We bet £22.50 now on the next coin toss. It lands Tails. Now we are back to £13.50. We bet this £13.50 on the next coin toss. It lands Tails again and we are down to £8.10. This is a net loss on the original stake of £10.

If we throw the same number of Heads and Tails after tossing the coin N times, we would expect more generally to earn the following.

1.5N/2 x 0.6N/2 = (1.5 x 0.6)N/2 = 0.9N/2

Eventually, all the stack used for betting is lost.

Herein lies the paradox. When many people play the game a fixed number of times, the average return is positive, but when a fixed number of people play the game many times, they should expect to lose most of their money.

This is a demonstration of the difference between what is termed ‘time averaging’ and ‘ensemble averaging.’

Thinking of the game as a random process, time averaging is taking the average value as the process continues. Ensemble averaging is taking the average value of many processes running for some fixed amount of time.

Processes where there is a difference between time and ensemble averaging are called ‘ergodic processes.’ In the real world, however, many processes, including notably in finance, are non-ergodic.

Say that in an election two parties, A and B, attract some percentage of voters, x% and y% respectively. This is not the same thing as saying that over the course of their voting lives, each individual votes for party A in x% of elections and for party B in y% of elections. These two concepts are distinct.

Again, if we wish to determine the most visited parts of a city, we could take a snapshot in time of how many people are in neighbourhood A, how many in neighbourhood B, etc. Alternatively, we could follow a particular individual or a few individuals, over a period of time and see how often they visit neighbourhood A, neighbourhood B, etc. The first analysis (the ensemble) may not be representative over a period of time, while the second (time) may not be representative of all the people.

An ergodic process is one which in which the two types of statistic give the same results. In an ergodic system, time is irrelevant and has no direction. Say, for example, that 100 people rolled a die once, and the total of the scores is divided by 100. This finite-time average approaches the ensemble average as more and more people are included in the sample. Now, take the case of a single person rolling a die 100 times, and the total scored is divided by 100. This finite-time average would eventually approach the time average.

An implication of ergodicity is that the result ensemble averaging will be the same as time averaging.

And here is the key point: In the case of ensemble averages, it is the size of the sample that eventually removes the randomness from the sample. In the case of time averages, it is the time devoted to the process that removes randomness.

In the dice rolling example, both methods give the same answer, subject to errors. In this sense, rolling dice is an ergodic system.

However, if we now bet on the results of the dice rolling game, wealth does not follow an ergodic system. If a player goes bankrupt, he stays bankrupt, so the time average of wealth can approach zero over time as time passes, even though the ensemble value of wealth may increase.

As a new example take the case of 100 people visiting a casino, with a certain amount of money. Some may win, some may lose, but we can infer the house edge by counting the average percentage loss of the 100 people. This is the ensemble average. This is different to one person going to the casino 100 days in a row, starting with a set amount. The probabilities of success derived from a collection of people does not apply to one person. The first is the ‘ensemble probability’, the second is the ‘time probability’ (the second is concerned with a single person through time).

Here is the key point: No individual person has sure access to the returns of the market without infinite pockets and an absence of so-called ‘uncle points’ (the point at which he needs, or feels the need, to exit the game). To equate the two is to confuse ensemble averaging with time averaging.

If the player/investor has to reduce exposure because of losses, or maybe retirement or other change of circumstances, his returns will be divorced from those of the market or the game. The essential point is that success first requires survival. This applies to an individual in a different sense to the ensemble.

So where does the money lost by the non-survivors go? It gets transferred to the survivors, some of whom tend to scoop up much or most of the pool, i.e. the money is scoped up by the tail probability of those who keep surviving, which may just be by blind good luck, just as the non-survivors may have been forced out of the game/market by blind bad luck. So the lucky survivors (and in particular the tail-end very lucky survivors) more than compensate for the effect of the unlucky entrants.

The so-called Kelly approach to investment strategy, discussed in a separate chapter, is an investment approach which seeks to respond to the survivor issue.

Say, for example, that the probability of Heads from a coin toss is 0.6, and Heads wins a dollar, but Tails (with a probability of 0.4) loses a dollar. Although the Expected Value of this game is positive, if the response of an investor in the game is to stake all their bankroll on each toss of the coin, the expected time until bankroll bankruptcy is just 1/(1-0.6) = 2.5 tosses of the coin.

The Kelly strategy to optimise the growth rate if the bankroll is to invest a fraction of the bankroll equal to the difference in the likelihood you will win or lose.

In the above example, it means we should in each game bet the fraction of x = 0.6 – 0.4 = 0.2 of the bankroll.

The optimal average growth rate becomes: 0.6 log (1.2) + 0.4 log (0.8) = 0.2.

If we bet all our bankroll on each coin toss, we will most likely lose the bankroll. This is balanced out over all players by those who with low probability win a large bankroll. For the real-life player, however, it is most relevant to look at the time-average of what may be expected to be won.

In trying to maximise Expected Value, the probability of bankroll bankruptcy soon gets close to one. It is better to invest, say, 20% of bankroll in each game, and maximise long-term average bankroll growth.

In the coin-toss example, it is like supposing that various “I”s are tossing a coin, and the losses of the many of them are offset by the huge profit of the relatively small number of “I”s who do win. But this ensemble-average does not work for an individual for whom a time-average better reflects the one timeline in which that individual exists.

Put another way, because the individual cannot go back in time and the bankruptcy option is always actual, it is not possible to realise the small chance of making the tail-end upside of the positive expectation value of a game/investment without taking on the significant risk of non-survival/bankruptcy. In other words, the individual lives in one universe, on one time path, and so is faced with the reality of time-averaging as opposed to an ensemble average in which one can call upon the gains of parallel investors/game players on parallel timelines in essentially parallel worlds.

To summarise, the difference between 100 people going to a casino and one person going to the casino 100 times is the difference between understanding probability in conventional terms and through the lens of path dependency.

 

References and Links

Time for a change: Introducing irreversible time in economics. https://www.gresham.ac.uk/lectures-and-events/time-for-a-change-introducing-irreversible-time-in-economics

What is ergodicity? https://larspsyll.wordpress.com/2016/11/23/what-is-ergodicity-2/

Non-ergodic economics, expected utility and the Kelly criterion. https://larspsyll.wordpress.com/2012/04/21/non-ergodic-economics-expected-utility-and-the-kelly-criterion/

Ergodicity. http://squidarth.com/math/2018/11/27/ergodicity.html

Ergodicity. http://nassimtaleb.org/tag/ergodicity/

Solution: Kelly Criterion – in a nutshell.

x = 70% minus 30% = 40%.

The Newton-Pepys Problem – in a nutshell.

One of the most celebrated pieces of correspondence in the history of probability and gambling, and one of which I am particularly fond, involves an exchange of letters between the greatest diarist of all time, Samuel Pepys, and the greatest scientist of all time, Sir Isaac Newton.

The six letters exchanged between Pepys in London and Newton in Cambridge related to a problem posed to Newton by Pepys about gambling odds. The interchange took place between November 22 and December 23, 1693. The ostensible reason for Mr. Pepys’ interest was to encourage the thirst for truth of his young friend, Mr. Smith. Whether Sir Isaac believed that tale or not we shall never know. The real reason, however, was later revealed in a letter written to a confidante by Pepys indicating that he himself was about to stake 10 pounds, a considerable sum in 1693, on such a bet. Now we’re talking!

The first letter to Newton introduced Mr. Smith as a fellow with a “general reputation…in this towne (inferiour to none, but superiour to most) for his maistery [of]…Arithmetick”.

What emerged has come down to us as the aptly named Newton-Pepys problem.

Essentially, the question came down to this:

Which of the following three propositions has the greatest chance of success.

  1. Six fair dice are tossed independently and at least one ‘6’ appears
  2. 12 fair dice are tossed independently and at least two ‘6’s appear.
  3. 18 fair dice are tossed independently and at least three ‘6’s appear.

Pepys was convinced that C. had the highest probability and asked Newton to confirm this.

Newton chose A as the highest probability, then B, then C, and produced his calculations for Pepys, who wouldn’t accept them.

So who was right? Newton or Pepys?

Well, let’s see.

The first problem is the easiest to solve.

What is the probability of A?

Probability that one toss of a coin produces a ‘6’ = 1/6

So probability that one toss of a coin does not produce a ‘6’ = 5/6

So probability that six independent tosses of a coin produces no ‘6’ = (5/6)6

So probability of AT LEAST one ‘6’ in 6 tosses = 1 – (5/6)6 = 0.6651

So far, so good.

The probability of problem B and probability of problem C are more difficult to calculate and involve use of the binomial distribution, though Newton derived the answers from first principles, by his method of ‘Progressions’.

Both methods give the same answer, but using the more modern binomial distribution is easier.

So let’s do it, along the way by introducing the idea of so-called ‘Bernoulli trials’.

The nice thing about a Bernoulli trial is that it has only two possible outcomes.

Each outcome can be framed as a ‘yes’ or ‘no’ question (success or failure).

Let probability of success = p.

Let probability of failure = 1-p.

Each trial is independent of the others and the probability of the two outcomes remains constant for every trial.

An example is tossing a coin. Will it lands heads?

Another example is rolling a die. Will it come up ‘6’?

Yes = success (S); No = failure (F).

Let probability of success, P (S) = p; probability of failure, P (F) = 1-p.

So the question: How many Bernoulli trials are needed to get to the first success?

This is straightforward, as the only way to need exactly five trials, for example, is to begin with four failures, i.e. FFFFS.

Probability of this = (1-p) (1-p) (1-p) (1-p) p = (1-p)4 p

Similarly, the only way to need exactly six trials is to begin with five failures, i.e. FFFFFS.

Probability of this = (1-p) (1-p) (1-p) (1-p) (1-p) p = (1-p)5 p

More generally, the probability that success starts on trial number n =

(1-p)n-1 p

This is a geometric distribution. This distribution deals with the number of trials required for a single success.

But what is the chance that the first success takes AT LEAST some number of trials, say 12 trials?

One method is to add the probability of 12 trials to prob. of 13 trials to prob. of 14 trials to prob. of 15 trials, etc.  …………………………

Easier method: The only time you will need at least 12 trials is when the first 11 trials are all failures, i.e. (1-p)11

In a sequence of Bernoulli trials, the probability that the first success takes at least n trials is (1-p)n-1

Let’s take a couple of examples.

Probability that the first success (heads on coin toss) takes at least three trials (tosses of the coin)= (1-0.5)2 = 0.25

Probability that the first success (heads on coin toss) takes at least four trials (tosses of the coin)= (1-0.5)3 = 0.125

But so far we have only learned how to calculate the probability of one success in so many trials.

What if we want to know the probability of two, or three, or however many successes?

To take an example, what is the probability of exactly two ‘6’s in five throws of the die?

To determine this, we need to calculate the number of ways two ‘6’s can occur in five throws of the die, and multiply that by the probability of each of these ways occurring.

So, probability = number of ways something can occur multiplied by probability of each way occurring.

How many ways can we throw two ‘6’s in five throws of the die?

Where S = Success in throwing a ‘6’, F = Fail in throwing a ‘6’, we have:

SSFFF; SFSFF; SFFSF; SFFFS; FSSFF; FSFSF; FSFFS; FFSSF; FFSFS; FFFSS

So there are 10 ways of throwing two ‘6’s in five throws of the dice.

More formally, we are seeking to calculate how many ways 2 things can be chosen from 5. This is known as ‘5 Choose 2’, written as:

5 C 2= 10

More generally, the number of ways k things can be chosen from n is:

nC k = n! / (n-k)! k!

n! (known as n factorial) = n (n-1) (n-2) … 1

k! (known as k factorial) = k (k-1) (k-2) … 1

Thus, 5C 2 = 5! / 3! 2! = 5x4x3x2x1 / (3x2x1x2x1) = 5×4/(2×1) = 20/2=10

So what is the probability of throwing exactly two ‘6’s in five throws of the die, in each of these ten cases? p is the probability of success. 1-p is the probability of failure.

In each case, the probability = p.p.(1-p).(1-p).(1-p)

= p2 (1-p)3

Since there are 5 C 2 such sequences, the probability of exactly 2 ‘6’s =

10 p2 (1-p)3

Generally, in a fixed sequence of n Bernoulli trials, the probability of exactly r successes is:

nC r x pr (1-p) n-r

This is the binomial distribution. Note that it requires that the probability of success on each trial be constant. It also requires only two possible outcomes.

So, for example, what is the chance of exactly 3 heads when a fair coin is tossed 5 times?

5C 3 x (1/2)3 x (1/2)2 = 10/32 = 5/16

And what is the chance of exactly 2 sixes when a fair die is rolled five times?

5 C 2x (1/6)2 x (5/6)3 = 10 x 1/36 x 125/216 = 1250/7776 = 0.1608

So let’s now use the binomial distribution to solve the Newton-Pepys problem.

  1. What is the probability of obtaining at least one six with 6 dice?
  2. What is the probability of obtaining at least two sixes with 12 dice?
  3. What is the probability of obtaining at least three sizes with 18 dice?

First, what is the probability of no sixes with 6 dice?

P (no sixes with six dice) = n C x . (1/6)x . (5/6)n-x, x = 0,1,2,…,n

Where x is the number of successes.

So, probability of no successes (no sixes) with 6 dice =

n!/(n-k)!k! = 6!/(6-0)!0! x (1/6)0 . (5/6)6-0 = 6!/6! X 1 x 1 x (5/6)6 = (5/6)6

Note that: 0! = 1

Here’s the proof: n! = n. (n-1)!

At n=1, 1! = 1. (1-1)!

So 1 = 0!

So, where x is the number of sixes, probability of at least one six is equal to ‘1’ minus the probability of no sixes, which can be written as:

P (x≥ 1) = 1 – P(x=0) = 1 – (5/6)6 = 0.665 (to three decimal  places).

i.e. probability of at least one six = 1 minus the probability of no sixes.

That is a formal solution to Part 1 of the Newton-Pepys Problem.

Now on to Part 2.

Probability of at least two sixes with 12 dice is equal to ‘1’ minus the probability of no sixes minus the probability of exactly one six.

This can be written as:

P (x≥2) = 1 – P(x=0) – P(x=1)

P(x=0) in 12 throws of the dice = (5/6)12

P (x=1) in 12 throws of the dice = 12 C 1 . (1/6)1 . (5/6)11nC k = n! / (n-k)! k!

So 12 C 1

= 12! / (12-1)! 1! = 12! / 11! 1! = 12

So, P (x≥2) = 1 – (5/6)12 – 12. (1/6) . (5/6)11

= 1 – 0.112156654 – 2 . (0.134587985) = 0.887843346 – 0.26917597 =

= 0.618667376 = 0.619 (to 3 decimal places)

This is a formal solution to Part 2 of the Newton-Pepys Problem.

Now on to Part 3.

Probability of at least three sixes with 18 dice is equal to ‘1’ minus the probability of no sixes minus the probability of exactly one six minus the probability of at exactly two sixes.

This can be written as:

P (x≥3) = 1 – P(x=0) – P(x=1) – P(x=2)

P(x=0) in 18 throws of the dice = (5/6)18

P (x=1) in 18 throws of the dice = 18 C 1 . (1/6)1 . (5/6)17

nC k = n! / (n-k)! k!

So 18 C 1

= 18! / (18-1)! 1! = 18

So P (x=1) = 18.  (1/6)1 . (5/6)17

P (x=2) = 18 C 2 . (1/6)2 .(5/6)16

18 C 2

     = 18! / (18-2)! 2! = 18!/16! 2! = 18. (17/2)

So P (x=2) = 18. (17/2) (1/6)2 (5/6)16

So P(x=3) = 1 – P (x=0) – (P(x=1) – P (x=2)

P (x=0) = (5/6)18

= 0.0375610365

P (x=1) = 18. 1/6. (0.0450732438) = 0.135219731

P (x=2) = 18. (17/2) (1/36) (0.0540878926) = 0.229873544

So P(x=3) = 1 – 0.0375610365 – 0.135219731 – 0.229873544 =

P(x≥3) = 0.597345689 = 0.597 (to 3 decimal places, )

This is a formal solution to Part 3 of the Newton-Pepys Problem.

So, to re-state the Newton-Pepys problem.

Which of the following three propositions has the greatest chance of success?

  1. Six fair dice are tossed independently and at least one ‘6’ appears.
  2. 12 fair dice are tossed independently and at least two ‘6’s appear.
  3. 18 fair dice are tossed independently and at least three ‘6’s appear.

Pepys was convinced that C. had the highest probability and asked Newton to confirm this.

Newton chose A, then B, then C, and produced his calculations for Pepys, who wouldn’t accept them.

So who was right? Newton or Pepys?

According to our calculations, what is the probability of A? 0.665

What is the probability of B? 0.619

What is the probability of C? 0.597

So Sir Isaac’s solution was right. Samuel Pepys was wrong, a wrong compounded by refusing to accept Newton’s solution. How much he lost gambling on his misjudgement is mired in the mists of history. The Newton-Pepys Problem is not, and continues to tease our brains to this very day.

 

References and Links

Newton and Pepys. DataGenetics. http://datagenetics.com/blog/february12014/index.html

Newton-Pepys problem. Wikipedia. https://en.wikipedia.org/wiki/Newton%E2%80%93Pepys_problem

 

Solution: Deadly Doors Problem – in a nutshell.

Solution to Exercise

Question 1. You should switch to either the purple box or the magenta box.

There was a 1 in 4 chance at the outset that your original choice, the red box, contained the prize. This does not change when I open the box which I know to be empty. There was a 3 in 4 chance that it was either the orange box, the purple box or the magenta box before I opened the box and by opening the orange box, which I know to be empty, that can be eliminated. So the chance it is either the purple box or the magenta box is now 3 in 4 in total (or 3/8 each), compared to 1 in 4 for your original choice, the red box.

Question 2. It makes no difference whether you switch or not.

There was a 1 in 4 chance at the outset that your original choice, the black box, contained the prize. There was a 3 in 4 chance that it was either the white box, the grey box or the brown box. By randomly opening a box (I don’t know which box contains the prize), I am giving you no new information. It is the same as asking you to choose a box to open. If you randomly opened the white box, which might have contained the prize, this means there are now two boxes left (grey and brown). Each of these started with a 1 in 4 chance of containing the prize. I have not deliberately eliminated a box potentially containing the prize, so I have given you no new information to indicate which box contains the prize. So the chance of each of the remaining boxes rises to 1/3 in each case. So it makes no difference whether you switch or not.