The end of the Silver gold standard in polling

Amid the brutal cultural warfare of the presidential election campaign, each side was equally convinced that the other was in a state of denial about the electoral backlash awaiting its candidate. Liberals expected that Hillary Clinton would be swept to victory on a tide of well-founded hostility towards Donald Trump among women and minorities, and found encouragement for the notion in the poll aggregates and forecast models that had given accurate assurance of Democratic victory in 2008 and 2012.

For the Trumptimists, the polls were just one more sign of a system rigged by out-of-touch elites who had closed their ears to the groundswell of anger in America’s industrial cities and small towns against Clinton and all she
represented.

Both were wrong in their way, but it’s the delusions of the liberal viewpoint that stand exposed by the result. For those who hoped for and expected a Clinton win, any poll or forecast that might give cause for doubt tended to be dispatched with help from our constant cognitive companions: confirmation bias and motivated reasoning.

The most intensively analysed poll methodology of the campaign was that constructed by the University of Southern California for its Los Angeles Times poll, which happened to be the strongest series for Trump.

When Nate Cohn of The New York Times determined that the poll had one 19-year-old Trump supporter from
Illinois standing in for a large segment of the black population, the unidentified respondent became an icon of the campaign to rival jumper-wearing town hall debate schlub Ken Bone.

The poll’s bias to Trump was real enough — its final result credited him with a three-point lead on the national vote, whereas The New York Times is projecting Clinton to finish about one point ahead, for all the good it will do her.

However, polls that proved at least as wrong in the other direction tended not to receive such detailed scrutiny. To the extent that any such result might look wonky, it could be rationalised as a point in a distribution that had Trump-biased polls showing a tight race at the other end, with the truth landing somewhere in between.

With that in mind, practised liberal poll watchers could correct for the known wrongness of the USC/LA Times outlier by factoring a bonus for Clinton into the crude average published by RealClearPolitics (which, in a measure
of the polling industry’s overall performance, came in about two points too high for Clinton — not great, but not as bad as some might think).

Assured of Clinton’s solid lead, they might then cast an eye over the election forecasters, soak up the dopamine hit from the Princeton Election Consortium’s confirmation that Clinton was still a mathematical certainty — and wonder how it had all gone so horribly wrong for Nate Silver, whose FiveThirtyEight model scandalously insisted that a Trump win was still a 30% possibility.

These tendencies were typified by two quintessentially modern media outlets, Wired and Huffington Post, which each produced “Dewey beats Truman” moments for the digital age.

On Sunday, an article by Huffington Post‘s Washington bureau chief, Ryan Grim, ran under a headline that accused Silver of “Unskewing Polls — All of Them — in Trump’s Direction”.

The wording invoked a legendary 2012 campaign website that accused pollsters of tilting the ground in favour of Barack Obama with fraudulent turnout models, and “corrected” them to show Mitt Romney with his rightful lead.

Grim went on to accuse Silver of “sparking a collective global freakout” with a projection that, among other outrages, rated Trump as the favourite in Florida, “even as we and others (and the early vote) see it as a comfortable Clinton lead”.

At Wired, Jeff Nesbit was calling it: “Forget Nate Silver. There’s a new king of the presidential election data mountain. His name is Sam Wang, Ph.D.”

Wang’s Princeton Election Consortium site predicted the 2012 result with an accuracy comparable to Silver, but his probability estimates had showed Obama was essentially certain to win, whereas FiveThirtyEight customers had to make do with 90%.

Wang was scarcely less sure of a Democratic triumph this time around, whereas Silver had Clinton’s probability falling into the mid-60s over the weekend, before recovering to 71.4% by election day.

So far as Wired was concerned, that alone seemed sufficient to demonstrate the superiority of Wang’s model, given the manifest unthinkability of a Trump victory.

Wang explained that the secret of his model’s imminent success lay in its being “built upon state polls, which are more accurate than national polls”.

But that proved not to be the case, particularly with respect to Trump’s decisive hat trick of narrow victories in the industrial states of Michigan, Wisconsin and Pennsylvania.

The three states between them produced 62 polls of presidential voting intention from the start of October to polling day, of which a grand total of two had Trump in the lead — both conducted by a Republican-aligned pollster at the close of the campaign.

In spite of everything, Silver’s and Wang’s models were actually in perfect agreement as to the likely shape of the electoral map, in that both rated Clinton as favourite in five states won by Trump (North Carolina, Florida and the aforementioned industrial states).

But it was the wider allowance for uncertainty in Silver’s model that allowed him to emerge with some credibility intact, by way of acknowledging that poll aggregates can only ever be as good as the polls they are aggregating.

In the particularly challenging environment pollsters face today, that can never amount to the level of certainty attributed to them by Wang.