There’s natural alarm over “churnalism”: overstretched newsrooms simply shovelling PR handouts through the system, no time for inquiry, no time for thought. But what about the greatest, most humiliating churn of the lot? The simple, jaw-dropping fact that all the newspapers and broadcasters got last May’s general election wrong.
Ah! But that wasn’t us, guv. It was the blinking pollsters. They misled gallant journalists; they gulled bemused politicians; they sent the spin doctors into a tailspin. Maybe their pointers to a hung parliament, with much scary prognostication over an SNP-Lab pact, even decided the whole campaign. Call Mr Benjamin Disraeli to rail about “lies, damned lies and statistics”.
So the Royal Statistical Society inquest on what went wrong (its analysis officially revealed last week, recommendations promised in March) matters. A lot. The Times headline “Election polls asked wrong people” doesn’t quite do the job, somehow. There were 1,942 political polls in the UK between May 2010 and 2015, 91 of them during the election itself. They filled acres of newspaper columns. They were 6.6 points agley at the end. And it’s frankly dismaying to read what Professor Pat Sturgis and his team have to say.
What didn’t go wrong? Most of the usual subjects. No, it wasn’t postal voting, drooping registration levels, differential turnout or the fabled “late swing”. There’s little evidence that respondents fibbed to researchers, or that online and telephone polls produced different answers. Therefore there must have been something wonky about the size and weighting of the samples: too many of the Times’s “wrong people”, too few of the right ones. So any answer, in a few weeks’ time, is bound to be very technical and, as the Sturgis report admits, “there will be no silver bullet: the risk of polling misses in the future can be reduced, but not removed”.
In short, the churn of the pollsters may delude us again. So perhaps will the odd spot of “herding” (which, in some cases last May, meant pollsters adjusting their weightings in midstream so that they all more or less agreed on a dud prediction in the end). There’s no safety in the mantra of “what everybody says” when everybody adjusts to say the same thing at the end. And there are also some very hard questions.
Here’s another legion of polls coming down the tracks. The phone ones show clear “Remain” pluralities in that EU referendum; the online ones are closing to neck-and-neck or “Leave”. If that bit of methodology didn’t matter much in election reports, why is it so glaring now? Remember the way the Scottish referendum gap once seemed to flake away, before returning once the boxes were open.
It’s fair enough to conclude (with a Guardian editorial) that the polls “are flawed, but better than nothing”. It’s not fair, though, to stop at that point. Papers and broadcasters must test the information they display. They have a duty not merely to mention sample size or methods used, but comparative costs of various surveys (more expense should mean more skilled resource) and the record of individual pollsters. They need someone to hand like Professor John Curtice who can crunch his own numbers. They need the utmost caution when they blithely turn data into a shock headline. And if that entails much less zippy certainty at too high a cost so we don’t get another 1,942 polls by 2020? Well, into every media life, a little chastened scepticism must eventually fall.