Washington, D.C.- You don’t often see this sort of disclaimer:
“We made 25,975 calls, and 642 people spoke to us.But remember: It’s just one poll, and we talked to only 642 people. Each candidate’s total could easily be four points different if we polled everyone in the state. And having a small sample is only one possible source of error.”
That is right at the top of a single poll result on the New York Times website's live polling section. It posted results in real time with every phone call it made. In the race referred to above, the battle for US Senate in Nevada, the Times got it wrong. It predicted that incumbent Republican Senator Dean Heller would take 47% of the vote, leaving Democratic candidate Jackie Rosen with 45%. But Rosen took home the victory with 50.4 %. The time's methodology used cell phones and landlines to access respondents.
Access to a broad range of voters is more difficult than ever. Landlines produce older voters. Cell phones provide a younger demographic more likely to block the call. Online research is challenging to conduct in rural areas. It means pollsters are pulling at threads and producing errors in many cases.
Triangulation Is A Step Forward
Cliff Young, President of Ipsos Public Affairs, a polling company, says a multi-source, triangulated forecasting approach is the best way to measure voting at this point because it is more robust than relying on a single source such as a poll or a survey. “You don't have to be exact with every data source. What you really want to look for is convergence or divergence,” he says. “So all of my sources whether it be polling, modeling, expert opinion, social media, or finance in campaigns if all the different sort of data sources are converging and telling us the same kind of story then you could feel more comfortable with your predictions."
Compounding the problem of access to voters is that poll questioning can skew the responses. Do people know for sure that they are registered, and registered correctly? Do they lie about their past participation in elections? If the interviewer is female and asking about equal pay, or if they are a Hispanic interviewer asking about immigration – these can skew how the person responds.
Incorrect also on the part of the New York Times was the Florida governorship, a seat which is a significant indicator of the direction it will swing in the 2020 elections. The Times predicted that Tallahassee mayor, Democrat Andrew Gillum would take 48% of the vote, with Republican Ron De Santis taking 43%. It called 37,724 people, and 737 people responded. But Ron De Santis took the governorship.
Young says Ipsos could see through the momentum in the days leading up to the election. “I could see the campaigns swing back and forth concerning enthusiasm. We were tracking enthusiasm and we saw a very serious uptick in the last few days among Republicans,” he said. “That was the effect of Trump and others really hitting up the base, really hammering home those visceral messages. I wasn't surprised at the expression in a place like Florida.”
The Times also predicted that the state’s lone Senate seat would go to the Democrats, but it went GOP. Even Ipsos got it wrong.
“If is divergent then you have to step back and think a little bit. And what I could say overall is that the sources converged, for the most part, they were telling the same story,” says Young. “It made us comfortable in our general calls.”
Conventional polling has used a lot of historical modeling – an if-then approach - ‘If 2014 turnout was this, then 2016 will be this’. And we all know that didn’t work. Young foresees a disaster in two years time especially in Missouri and Indiana, known as “trump readouts”, where the polling even up to the day of the election was off on the Senate side by seven to 10 points; the Republican candidates were underestimated in both places.
“I think the challenge of polling going into 2020 is, are we correctly capturing that white, lower-educated voter that doesn't live in suburbia for urban areas, " he said. "And I think at this point we can say in certain places we are having problems as an industry and we have to figure that out."
Until a new and revolutionary tool is found, at the very least we may hope that pollsters will become more transparent and understandable in their methodology as the presidential race heats up.
