Political polling is having an existential crisis

In the wake of its most unambiguous failure at a federal election since at least 1980, Australia’s polling industry is licking its wounds.

The Nine/Fairfax papers announced last week that the Ipsos poll series will be put on ice. Those pollsters who do return to the field shortly will face catcalls — whether they persist in recording a Labor lead we now know doesn’t exist, or only now start detecting a Coalition lead that eluded them through the entirety of the past parliamentary term.

Despite it all, though, the pollsters’ performance hasn’t been without its defenders.

David Briggs, who as principal of YouGov Galaxy is responsible for Newspoll, argued in The Australian last week that expectations of a Labor win had arisen less from the polls’ inaccuracy than with the failure of commentators to interpret them properly.

Lest we be too quick to file this under “he would say that, wouldn’t he”, a not dissimilar view was advanced last week by no less an authority than Nate Silver, FiveThirtyEight publisher and long-standing poster boy of American psephology.

Arguing that a judicious interpretation of the polls would have allowed a one-in-three chance of a Coalition win, Silver wrote that “journalists” who spoke of a “massive polling failure” had shown themselves to be “not particularly numerate” — referring, would you believe it, to Antony Green.

With Nate Silver having called it from on high, should we not conclude that our pollsters merely hit a burst of the noise that inevitably distorts their signal from time to time? Should we leave it to them to do better next time with the same methods and, hopefully, a bit more luck?

Well, not so fast.

Silver’s one-in-three calculation derives not from the conventional considerations of sampling error, but from what he imagines to be an established record of mediocre performance by Australian pollsters. This is based entirely on a tweet-length assessment of the polling record going all the way back to 1943, well before any polling organisation here or abroad had any real idea what it was doing.

Local observers knew better about the three-decade reign of Newspoll, which has consistently produced large sample polling of the highest standard on the eve of each election, in a successful strategy by the Murdoch organisation to bolster the prestige of the newspaper that bears it, The Australian.

Until last Saturday, Newspoll had hit close to the bullseye before every federal election since 1993, excepting a partial failure in 2004 when inaccurate preference allocations threw out the two-party total.

Such a record would be the envy of any international pollster, and could probably only be accomplished under Australia’s enforced regime of stable high turnout.

The fly in the ointment may be that Newspoll as we know it actually came to an end when the company of that name was wound up in 2015, after which Newspoll became a brand name for polling conducted by YouGov Galaxy. Where the old Newspoll was a live interview phone poll — which had persisted, increasingly contentiously, in targeting landline phones only — the new Newspoll combined automated “robo-polling” with surveys of an online panel.

In its first major test at the 2016 federal election, this approach produced an eye-wateringly accurate set of numbers that seemed to dispel any doubts as to the new Newspoll’s right to claim the mantle of the old.

This time though, a big sample pre-election poll with a theoretical error margin of 1.8% appeared to confirm the established trend, only to prove nearly 4% amiss on the primary votes of both major parties. And with that, the long apparent danger that the entire polling industry might one day be herded off a cliff was realised.

Whatever some pollsters might offer for public consumption, all will be acutely aware that they now face a serious challenge in restoring their credibility, and that they have no reason to think they can accomplish it by persisting with their existing methods.