The Wonderful World of Mr. Bayes
A version of this article appears in TWISTED LOGIC: Puzzles, Paradoxes, and Big Questions, by Leighton Vaughan Williams. Published by Chapman & Hall/CRC press. 2024.
When Should We Update Our Beliefs?
Imagine emerging from a cave for the first time and watching the sun rise. You have never witnessed this before, and in this thought experiment, you are unable to tell whether it’s a regular occurrence, an infrequent event, or a once-in-a-lifetime happening.
As each day passes, however, you observe the dawn again and again: you gradually grow to expect it. With each sunrise, you become more confident that this is a regular event. With this growing confidence, you forecast that the sun will rise again the next day.
This is an illustration of what so-called Bayesian reasoning is about. Bayes’ theorem is a tool that allows us to adjust our understanding of the world based on our observations over time. It represents a process of continuous learning and understanding, pushing us gradually nearer to the truth as we are exposed to more experiences and to more information.
That’s the essence of Bayesian reasoning: adjusting our beliefs based on new information.
THE BIRTH OF BAYESIAN THINKING
The Bayesian perspective on the world can be traced to the Reverend Thomas Bayes, an 18th-century clergyman, statistician, and philosopher. The Bayesian approach advocated predicting future events based on past experiences. His ideas were in a fundamental sense different from the prevailing philosophical ideas of his time, notably those of Enlightenment philosopher David Hume.
Hume argued that we should not justify our expectations about the future based on our experiences of the past, because there is no law stating that the future will always mirror the past. As such, we can never be certain about our knowledge derived from experience. For Hume, therefore, the fact that the sun had risen every day up to now was no guarantee that it would rise again tomorrow. In contrast, Bayes provided a tool for predicting the likelihood of such events based on past experiences and observations. His method can be applied consistently to the sciences, social sciences, and many aspects of our everyday lives.
Unlike the philosopher David Hume, who argued that past experiences don’t guarantee future outcomes, Bayes focused on how we can use past events to predict the likelihood of future ones. Bayes’ approach is not just academic; it’s a practical tool.
BAYES’ THEOREM: AN EVERYDAY TOOL FOR REFINING PREDICTIONS
So how does what is known as Bayes’ theorem help us in our everyday lives and beyond? As it turns out, it’s an important way of helping us to refine our belief of what is true and what is false. Let’s look more closely into this by breaking Bayes’ theorem down into its key components:
Establish a Prior Hypothesis: The starting point in Bayesian reasoning involves the establishment of an initial hypothesis, which may or may not be true. This hypothesis, also known as the ‘prior’ belief or ‘prior probability’ that you assign to this belief being true, is based on the information available to you. For instance, if you’re trying to predict whether it will rain tomorrow, you might estimate the initial likelihood (or ‘prior probability’) based on your personal observation of current weather patterns or conditions.
Observe New Evidence: Once you establish a prior probability, you’ll then need to consider updating this when any new information becomes available. In the weather example, evidence could be anything from new dark clouds gathering or else dispersing to a sudden rise or drop in temperature.
Assess to What Extent This New Evidence Is Consistent with Your Initial Hypothesis: Bayesian reasoning doesn’t stop at just gathering evidence. It also involves considering evidence that is consistent with, or inconsistent with, your initial hypothesis. For example, if there is an increase or decrease in wind speed, this might be considered additional evidence that you should take into account in estimating the probability of rain.
Let’s break down again how Bayes’ theorem helps us refine our beliefs:
Establishing a Starting Point (The Prior Hypothesis): Imagine you’re trying to predict if it will rain tomorrow. Your ‘prior hypothesis’ is your initial estimate, based on what you currently know about the weather conditions.
Incorporating New Information (New Evidence): Now, suppose you observe unexpected dark clouds gathering in the sky. This new information should logically influence your prediction about the weather.
Combining Old and New Insights (Assessing Consistency): Bayesian reasoning involves integrating the new evidence with your initial estimate. You assess whether the appearance of dark clouds increases the likelihood of rain tomorrow.
By applying Bayes’ theorem, you adjust your belief based on the new evidence. If dark clouds often lead to rain, you increase your belief that it will rain. If not, you adjust accordingly.
Visualising Bayes’ Theorem
Think of Bayes’ theorem as a formula that combines your initial estimate with new information to give you a better estimate.
Beyond Weather: The Broad Applications of Bayes’ Theorem
Bayesian reasoning isn’t just about predicting the weather. It’s used in medicine to interpret test results, in finance to assess investment risks, in sports for game strategies, and so on. It’s a tool that refines our understanding, helping us make more informed decisions.
HOW BAYES’ THEOREM ALLOWS US TO UPDATE OUR BELIEFS
In essence, Bayes’ theorem permits us to establish an initial hypothesis, and to enter any supportive and contradicting evidence into a formula which can be used to update our belief in the likelihood that the hypothesis is true.
Consider a scenario where we evaluate our initial hypothesis. For simplicity, we label the probability that this hypothesis is correct as ‘a’. This probability is our starting point, reflecting our initial estimate based on prior knowledge or assumptions before encountering new data.
Next, we introduce ‘b’, which represents the likelihood that some new evidence we come across is consistent with our initial hypothesis being true. This is a critical element of Bayesian updating.
Conversely, ‘c’ is used to denote the probability of observing the same new evidence but under the condition that our initial hypothesis is false. This estimate is equally essential because it helps us understand the significance of the evidence in the context of our hypothesis not being true.
With these definitions in place, Bayes’ Theorem provides a powerful formula: Revised (posterior) probability that our initial hypothesis is correct = ab/[ab + c(1-a)]
This formula is a mathematical tool that updates our initial belief ‘a’ in light of the new evidence.
The result is an updated (or ‘posterior’) probability that reflects a more informed stance on the initial hypothesis.
This process, termed Bayesian updating, is a methodical approach that enables us to refine our beliefs incrementally. As we gather more evidence, we iteratively apply this updating process, allowing our beliefs to evolve closer to reality with each new piece of information. This ongoing refinement is a cornerstone of the Bayesian approach, emphasising the importance of evidence in shaping our understanding and beliefs.
BAYES’ THEOREM: A POWERFUL TOOL
Bayes’ theorem offers us a weapon against biases in our intuition, which can often mislead us. For example, intuition can sometimes lead us to ignore previous evidence or to place too much weight on the most recent piece of information. Bayes’ theorem offers a roadmap that assists us in balancing the weight of previous and new evidence correctly. In this way, it provides a method for us to fine-tune our beliefs, leading us gradually closer to the truth as we gather and consider each new piece of evidence.
CONCLUSION: THE BAYESIAN BEACON
Bayes’ theorem is more than a mathematical concept; it’s a guide through the uncertain journey of life. It teaches us to be open to new information and to continually adjust our beliefs. From daily decisions like weather predictions to complex scientific theories, Bayes’ theorem is a bridge from uncertainty to better understanding, helping us navigate life’s puzzles with more confidence and precision.
It does so in a structured way, dealing with new evidence, guiding us gradually to more informed beliefs. It encourages us always to be open to new evidence and to adjust our beliefs and expectations accordingly. Bayes’ theorem is in this sense a master key to understanding the world around us.
Professor Leighton Vaughan Williams 