Can we still trust opinion polls after 2015, Brexit and Trump?

The opinion pollsters have had a bruising time after failing to predict the result of the 2015 general election, last November’s US presidential election and the EU referendum.

In all three cases, the outcome was the result of a narrow contest. The polls were, for example, right to suggest that Hillary Clinton would win the popular vote but both the last general election and the unexpected Brexit vote last June led to some head scratching among the pollsters.

After the 2015 debacle, when psephologists widely predicted a hung parliament instead of an outright David Cameron victory, the British Polling Council set up an inquiry. It ruled out possible explanations such as a failure to identify “shy Tories” or the traditional “tardy Russian” explanation of a surge in people changing their minds at the last minute.

Instead, Dr Jonathan Mellon of Nuffield College, Oxford, and Dr Christopher Presser of Manchester University, have written to the Guardian suggesting the 2015 miss was down to the pollsters not contacting enough people from hard-to-reach groups who do not usually vote in elections. Pollsters, who are scrupulous in weighting their samples to look like those who vote, ended up overcounting the voting intentions of those who demographically resembled the missing non-voters.

“These voters were Labour-leaning in 2015 – for example, those under the age of 25 who turned out in low numbers but were likely to support Labour when they did. By including too many of these voters in their samples, pollsters inflated Labour’s apparent support in 2015.”

Peter Kellner, former president of YouGov, says in his BPC guide for journalists that such sampling errors should not lead to the conclusion that the polls cannot really be trusted.

“Polls may not be perfect, but they are the best, or least bad, way of measuring what the public thinks. In most countries where poll results can be compared with actual results (such as elections), well-designed polls are usually accurate to within 3%, even if they occasionally stray outside that margin of error.

“Moreover, much of the time, polls provide a good guide to the state of opinion, even allowing for a larger margin of error. If a well-designed, representative survey finds that the public divides 70%-30% on an issue, then a margin of error of even 10% cannot alter the fact that one view is expressed far more widely than the other.”

He adds that in closely fought elections – such as the US presidential race and the EU referendum – a polling lead of 5% or less cannot be regarded as a certain indicator of victory – as the eventual result in both cases demonstrated.

There is, however, a discrepancy between the current party vote shares being indicated by the opinion polls and the projected national shares based on the 4 May local election results which gave the Conservatives 38%, Labour 27% and Liberal Democrats 16%. This could be the result of further sampling errors or simply down to the fact that some people vote differently in local elections than general elections. Precedent suggests the latter.

It should be borne in mind that when British pollsters do get it wrong, it has tended to be an underestimate of the Tory vote and an overestimate of the Labour vote, so a policy of ignoring the polls is unlikely to provide much comfort for Jeremy Corbyn.

In this general election, it appears that the relative shares of the vote for each party – which should be the main focus of attention rather than the lead – seem so far apart that if the pollsters really have got it wrong this time, they really should shut up shop for good.