Can we believe Nate Silver?

A Balanced View

As a long-time follower of political forecasts, and creator of some, I’ve often found Nate Silver’s insights both intriguing and valuable. Silver, the founder of FiveThirtyEight and now running his own platform, The Silver Bulletin, has built a reputation as one of the most influential data analysts in politics. But with the 2024 U.S. presidential election approaching, I find myself asking: Can we still trust Nate Silver’s predictions?

Understanding Why Polling Aggregators Differ

To answer this question, it’s important to understand why different polling aggregators might come to different conclusions, even when using similar data. Most aggregators, such as FiveThirtyEight, The Economist, and The Silver Bulletin, rely on state-level polls, national polls, approval ratings, and economic fundamentals to make their predictions.

However, the differences arise in two critical areas of judgment:

1. How state-level polls are aggregated to determine each candidate’s chances of winning a state’s electoral votes. In a tight race like 2024, small adjustments in key states can significantly affect the overall probability.
2. How these state-level probabilities are aggregated into an overall probability of winning the election, especially considering correlated polling errors across states. This was crucial in 2016 when correlated polling errors across states led to a surprise outcome.

Silver’s approach has drawn attention this cycle because of how it handles these judgments, and the resulting forecasts sometimes differ significantly from other models.

The “Convention Bounce” Adjustment: Reasonable or Overdone?

One of the most debated aspects of Silver’s current methodology is his use of a “convention bounce” adjustment. This adjustment accounts for the temporary bump candidates often receive in the polls following their party conventions. Silver argues that polls taken just after the Democratic National Convention (DNC) might temporarily overstate Kamala Harris’s support, so his model includes an adjustment to correct for this.

On the surface, this seems like a logical adjustment. However, it has sparked some debate. For example, Silver’s model recently moved more than three percentage points against Harris during a period when no significant new data emerged—no major events, no new battleground state polls, no economic changes. While other models, like The Economist’s or Decision Desk HQ’s, remained stable during the same period, reflecting the lack of new information, Silver’s swung noticeably.

This has led some observers to question whether the “convention bounce” adjustment is appropriately calibrated. Silver defends the adjustment as “highly defensible” and notes that it will phase out gradually, but this episode does raise questions about how such adjustments impact the overall forecast.

Is Silver’s Model Overly Reactive to Limited Information?

Another critique is that Silver’s model appears unusually reactive, responding to minor changes in data with relatively large movements. In a situation where there’s no significant new information, most models would show minimal movement. Yet, Silver’s has shifted markedly at times.

Silver’s supporters might argue that his model is designed to be highly responsive to any new data, reflecting the inherent uncertainty and volatility of the electoral landscape. However, this reactivity has led to speculation that the model might be more prone to showing movement, potentially to maintain engagement, even when the underlying data doesn’t fully justify such shifts.

The underlying question remains: Is this reactivity a strength, showing that Silver’s model adapts quickly to any new information, or a weakness, suggesting it might overemphasise minor fluctuations?

The Electoral College Focus: An Essential Consideration or Overplayed?

Silver also places considerable emphasis on the challenges posed by the Electoral College. He points out that even if Harris wins the popular vote by a few points, she could still lose the presidency due to the distribution of votes across key states.

This focus on the Electoral College is certainly valid; history has shown that a candidate can win the popular vote but lose the presidency. However, some critics question whether Silver’s emphasis on this factor might be overstating its impact compared to other models. Every major aggregator considers the Electoral College in their forecasts, but Silver’s model arguably amplifies its importance relative to the others.

Here, too, there are two sides. Silver’s supporters argue that his focus on the Electoral College is a necessary reminder of how U.S. elections actually work, ensuring we don’t overlook its impact. On the other hand, some might see it as a way to make his model appear more unique or insightful than it truly is.

Conclusion: Should We Trust Nate Silver’s Forecast in 2024?

So, where does that leave us? Can we trust Nate Silver’s model for the 2024 election? On the one hand, Silver’s experience, his focus on “tail risks” (scenarios where less likely but still possible outcomes could occur) and correlated polling errors, make his forecasts a valuable tool in understanding the electoral landscape. On the other hand, some of his adjustments and the model’s reactivity have raised questions about whether his approach this cycle might be about engagement as much as precision.

Ultimately, it may be wise to consider multiple perspectives. Nate Silver’s forecasts should certainly be a part of that consideration, but not the only voice in the conversation. Given the complexities of this election, looking at what other models and aggregators are saying—and understanding their different methodologies and assumptions—might offer a more balanced view.

Looking Ahead

As we get closer to Election Day, it will be interesting to see how Silver’s model and others evolve with new data. Staying informed through multiple sources and understanding their approaches will be key to navigating this race. Ultimately, no single model is likely to have all the answers, or even to be asking all the right questions.