Editor's note: The following series of pieces on polling originally published on Divided We Fall. It features perspectives from Dr. Allan J. Lichtman and Eric Loepp. Learn more about the authors below. The following was published on IVN with permission from Divided We Fall. Photo by Alex Shuper on Unsplash.
Political Polls Cannot Predict Election Outcomes
By Allan J. Lichtman – Distinguished Professor of History, American University
Take horse race polls on the presidential election and, as the philosopher David Hume recommended for works of superstition, consign them to the flames. Such polls are only ephemeral snapshots, but poll-driven pundits abuse them as predictors. Whether the polls are right or wrong is unknown because there is no independent check on their findings. During the election year and even sometimes close to the election, the polls often falsely anticipate the outcome.
A History of Polling Failures
In 1948, President Harry Truman lost every pre-election poll to Tom Dewey, with deficits reaching 15 percent. Yet, Truman won reelection with 49.5 percent of the popular vote to 45.1 percent for Dewey. In March 1980, the Gallup poll had President Jimmy Carter leading Ronald Reagan 58 percent to 33 percent, missing Reagan’s win by thirty-five percentage points. Polls showed Michael Dukakis seventeen points ahead of George H.W. Bush in July 1988, underestimating Bush’s win by nearly twenty-five points. A Gallup poll from late October 1992 showed President Bush within one point of Bill Clinton, missing Clinton’s win by nearly five points. After President Barack Obama’s disastrous first debate with Mitt Romney, he fell behind by three points in the Gallup poll and remained behind by one point in Gallup’s final poll. Obama won reelection with a popular vote margin of four points.
In 2016, the eminent Princeton Election Consortium of poll compilers gave Hillary Clinton a 99 percent chance of winning the White House. “It is totally over. If Trump wins more than 240 electoral votes, I will eat a bug,” said the group’s leader, Dr. Sam Wang. To his credit, Wang did just that.
Errors Are Embedded in the System of Horse Race Polls
Moreover, the pollsters’ professed error margin of about plus or minus three percent is only the statistical error that would, for example, apply to a sample of green and red balls drawn from a limitless supply. People are not green and red balls. Most do not respond to polls. Respondents may lie to pollsters or change their minds, and pollsters must guess at the likely voters in their sample.
The non-statistical errors are substantial and unidirectional, not random; they cannot be cured through poll averages. In 2016, the polls substantially underestimated Republican voting strength. Like the general fighting the last war, the pollsters tried to correct this problem but instead underestimated Democratic voting strength in elections from 2022 to 2024. For example, in the 2024 special election for the New York congressional seat previously held by the disgraced George Santos, a poll on the eve of the election showed the Democrat ahead by one point. He won by eight points.
Moving Past the Pressures of the Polls
The power of America’s political-industrial complex explains our addiction to horse race polls. Like the military-industrial complex President Dwight Eisenhower warned against in his 1961 farewell address, an iron triangle sustains the political version. At one point are the pundits, pollsters, handlers, and consultants who make millions of dollars from the conventional horse race version of presidential elections. The media, which is under pressure to cover elections daily, occupies the second point. At the third point are the politicians who fear going against purveyors of conventional wisdom and their echo chamber in the media.
My prediction system, The Keys to the White House, which gauges the incumbent party’s strength and performance, has been right since 1984. It challenges the horse-race polling that sustains the political-industrial complex.
The Broader Impact of Polls on Politics, Beyond Predictions
By Eric Loepp – Associate Professor of Political Science and Director of the Learning Technology Center, University of Wisconsin-Whitewater
National polls have taken a lot of heat in recent years for their failure to predict election outcomes. However, these criticisms often overlook the necessary nuance to contextualize the value of polls.
For all the criticism, it remains true that candidates leading in national polling before the November elections usually win the most votes. This was even the case in the infamous polling “miss” of 2016, as detractors often overlook that the candidate leading in national polls that year—Hillary Clinton—did win more votes. Polls just cannot account for quirks of the Electoral College.
Election Polls Are Not Precise Predictors
The real issue here is that polls are not supposed to be instruments of prediction. They never have been. Let’s imagine we polled 1,000 Americans and found that 52 percent plan to vote for Vice President Kamala Harris and 48 percent for former President Donald Trump. Those two values—52 and 48 percent—could be off (up or down) by as much as 3 percentage points due to random chance. Polls will likely over- or under-capture candidate support, so no scientific poll in a highly competitive race can claim to provide any more clarity than, “It’s going to be close.”
Do polls’ inevitable errors render them untrustworthy? No. It just means they are not great at providing a statistical foundation to precisely predict future events. But polls can be useful in other ways if we acknowledge their limitations. Think of polls not as crisp forecasts etched in glass, but as blurry snapshots of moments in time.
Parties, politicians, donors, and other political elites trust polls to do the same in elections. Polls provide a feedback loop by which candidates and their advisors can see whether their campaign efforts are resonating with voters and adjust as necessary. For example, if polls show a candidate’s slogan is not memorable, they may try a catchier pitch. Indeed, without polling, how would candidates identify the swing states in which they spend a majority of their time campaigning?
Nonetheless, Election Polls Still Are Valuable
Polls do not need to be perfectly precise to provide valuable feedback. In a representative democracy like ours, is there anything more important than gauging the hopes, fears, wisdom, and priorities of the American people? Isn’t an imperfect gauge better than none at all? Voters should not be limited to collectively expressing their attitudes to representatives on Election Day alone.
In short, polls’ “trust problem” is often a problem of assumptions about what polls can actually deliver. It’s on us to use them well. Pollsters need to conduct polls transparently and scientifically. The media needs to convey results in context. Voters need to familiarize themselves with the basics before drawing conclusions. This means, first and foremost, recognizing that polls cannot predict elections.
But that doesn’t mean they cannot be trusted to provide the people with a means to communicate their views. Indeed, political actors rely heavily on polls to guide strategic decisions. They wouldn’t spend all that money—or risk electoral defeat—if polls could not be counted on to provide useful insight.
Election Polls Are Ineffective at Guiding Candidates’ Strategies
By Allan J. Lichtman – Distinguished Professor of History, American University
There are broad areas of agreement between my critique of horse race polls and Mr. Loepp’s thoughtful response. We agree that polls are snapshots, not predictors. We agree that they cannot accurately predict who will win or lose a presidential election. We agree that for close elections of recent years, the error margin of the polls often exceeds the gap between candidates. However, there remain areas of disagreement.
Mr. Loepp underestimates the polling error margin. He references only the approximately plus or minus 3 percent statistical error margin of polls. Mr. Loepp ignores non-statistical errors, including non-response rates, respondents who lie or may not have focused on the election, and the need to guess at so-called “likely voters.” These errors are unidirectional, not random, favoring one party over another.
Mr. Loepp claims that polls can help guide candidates’ strategic decisions. However, if horse race polls are useless for predicting wins and losses, they cannot reliably inform candidates about winning strategies. Mr. Loepp confuses horse race polls with surveys of issues and candidate ratings, which are not predictive but could provide information to campaigns. Still, he fails to demonstrate that campaign strategies influence outcomes. My Keys to the White House system has accurately forecast election outcomes since 1984 without referencing campaign strategies or events. Similarly, a massive study of 49 field experiments found that campaign effects had zero impact on outcomes.
Horse Race Polls Fail to Provide Useful Information
Mr. Loepp says that horse race polls can at least predict popular vote leaders. There are three problems with this analysis. First, Mr. Loepp only presents evidence that the average of late horse race polls has accurately forecasted the popular vote winner. However, these polls misrepresent the popular vote at any point in the election year. In my initial critique, I cited examples from the 1948, 1980, 1988, 1992, and 2012 elections as well.
Second, the popular vote has become irrelevant in recent elections, given the majorities Democrats garner in New York and California with no offsetting Republican margins. From 1992 to 2020, Republicans won the popular vote once, by a slim margin in 2004. Third, despite Mr. Loepp’s admonishing, pundits continue using horse race polls to call winners and losers in presidential elections, as demonstrated by Nate Silver’s ongoing forecasts from compilations of polls.
Finally, Mr. Loepp claims that horse race polls indispensably identify the swing states that decide elections. However, in recent elections, the swing states have already been established based on the results of past elections, and state polls are often misleading. For example, in 2020, the average of late polls gave Biden a 2.1 percent edge in Florida, which he lost by 3.5 percent. Some time ago, I suggested that my horse-race polling friends should take a vacation on a far-off Pacific island during a presidential election year. Perhaps Mr. Loepp would agree.
Voters Should Be Aware That All Prediction Models Have Limitations
By Eric Loepp – Associate Professor of Political Science and Director of the Learning Technology Center, University of Wisconsin-Whitewater
It is important to affirm that the value of election polls does not lie in their perfect predictive power. Rather, they can capture a general pulse about how things are going that provides actionable feedback to candidates. Maybe the poll was slightly off, or perhaps certain campaign efforts paid off and moved the numbers. But, candidates cannot make strategic decisions based purely on years-old election results. Old election results simply tell you what the people who showed up that day had to say. Voters join and leave the electorate all the time. New circumstances and issues can emerge.
Still, it’s true that some pundits misuse polls, failing to account for the unavoidable uncertainty–both random and otherwise–they contain. Dr. Lichtman and I share a frustration with these unthoughtful commentators. Polls are not predictors, and they shouldn’t be treated as such. Yet consumers enjoy inferring election outcomes from polling results nonetheless. It is thus reasonable to contrast polling with forecasting simply to illustrate an unequivocal truth: no prediction method is perfect. Polls are limited, but so is any model or prediction, including Dr. Lichtman’s 13 Keys approach.
Consulting Multiple Prediction Models May Yield the Best Results
Let me be clear: no one should be criticized for not perfectly anticipating close elections like those in 2000 and 2016. Any effort to gauge how people will behave in the future is subject to bias based on human judgment. Pollsters have to make judgment calls on matters such as how to word their questions or how to aggregate multiple polls. Users of the Keys Approach have to make judgment calls on matters, such as what counts as a significant third-party campaign, a major change in national policy, and whether the candidates are charismatic.
Predictions are hard, but polls that paint a general if imperfect picture of the race create feedback loops that inform campaign activities along the lines I described above. The Keys Approach can help shape this conversation, too. For instance, the Keys emphasize the influence of governing over the impact of campaigning in determining election outcomes. Many political scientists posit that campaigns also play an important role in determining election outcomes. Comparing notes and evidence is how we all learn and improve. Ultimately, considering multiple approaches will often yield the best projection possible.
The point of this conversation, then, should not be to proclaim that polls or Keys can or cannot be trusted indiscriminately. The point is that the best we can do is consider a variety of methods in good faith and refine them transparently over time as necessary. When I teach about polls and forecasting, we analyze the Keys as well as Nate Silver’s (formerly of FiveThirtyEight, now at Silver Bulletin) model, among others. Readers should, too. Most importantly, remember that polls and Keys seek to capture different things, and they both have limits.
About The Authors
Professor Lichtman authored 13 books and hundreds of scholarly and popular articles. He has been an expert witness in 110 civil rights cases and has provided commentary to major media outlets worldwide. His prediction system, The Keys to the White House, has successfully forecasted the outcome of US presidential elections since 1984. His books won the National Jewish Book Award in American Jewish History and were finalists for the National Book Critics Circle Award in nonfiction and the Los Angeles Book Prize in History. He received the Marquis Lifetime Achievement Award and ranked # 85 among the world’s 100 most influential geopolitical experts per rise.global.
Eric Loepp is an associate professor of political science and director of the Learning Technology Center at the University of Wisconsin-Whitewater. He teaches courses in American government, political behavior, and research methods. His research has been published in such journals as Electoral Studies; the Journal of Elections, Public Opinion, and Parties; Research & Politics; Election Law Journal; American Politics Research; PS: Political Science & Politics; and Social Science Quarterly.
