After the November 2014 midterm elections, I wrote that "Despite all of their supposed science, improved methodologies, and sophisticated turnout models, nation’s pollsters have just suffered through their worst midterm elections drubbing in 20 years. The last time they were off this badly was when they woefully underestimated Republican gains in the Newt Gingrich 'Contract with America' midterms of 1994." I also predicted that "If they’re right from now on, it will it only be by accident."
Very few, if any, such "accidents" occurred this year. In key contests, double-digit and worse variances from polled predictions were the norm.
In the Kentucky gubernatorial contest, the pre-election polling average at RealClearPolitics.com had Republican Matt Bevin trailing Democrat Jack Conway by an average of three points. Bevin rolled up a nine-point victory margin accompanied by an unheard of near-sweep of statewide offices by the GOP.
In Ohio, Issue 3, the "ResponsibleOhio" initiative would have legalized recreational marijuana use and put the state's entire pot industry into the hands of a select few who stood to make millions. Pre-election polls indicated either that the election was too close to call or that the ballot measure had the upper hand. The dreams of the one-percenters who would have been granted a windfall-profits business went up in smoke, as the initiative went down in flames by 28 points.
In Houston, the city's horribly misnamed "Houston Equal Rights Ordinance" had a nine-point lead in the polls in mid-October.
Closer to the election, "HERO" was either "too close to call" or losing slightly:
Mark Jones, another Rice political scientist, agreed that early voting returns, along with public opinion polls showing only a slim margin in favor of the ordinance, should be cause for concern for HERO supporters.
“If I had to do an even-money bet, I’d say it may not pass, but I think it really is too close to call,” Jones said.
HERO failed by over 20 points.
As I noted a year ago, these polling disasters shouldn't be surprising (bolds are mine):
There are two ... fundamental problems pollsters face. I don’t see how they can overcome either.
As succinctly stated by Pew Research’s Center for the People & the Press in 2012: “It has become increasingly difficult to contact potential respondents and to persuade them to participate.”
A chart at Pew’s related report based on its own experience lays it out:
... Combining these two problems caused Pew’s survey completion rate to plummet by 27 points, or 75 percent, in 15 years. In 2012, fewer than one in ten contacts ultimately resulted in a completed interview. ... it’s quite easy to imagine that the completion rate is even lower now. There’s also no reason to believe that Pew’s experience is materially different from what other pollsters are facing.
... For all of this not to matter, one would have to believe that there’s no significant difference between the views of those who can and can’t be reached, or between participants and refusers. That premise falls flat on its face once one recognizes that Republicans are twice as likely to distrust the media as Democrats, and that pollsters are usually and properly seen as representatives of the media.
What's utterly disappointing is that the press insists on taking their polls seriously, despite the clearly well-known problems Pew laid out, and fails to tell readers, listeners and viewers about the polls' fundamental vulnerability to serious error.
I should also add, to counter the arguments of pollsters who claim that they end up with roughly equal numbers of Republicans and Democrats in their completed surveys, that people who identify with either party but won't talk to pollsters are far more likely to have a number of genuinely conservative or at least center-right views, and that their completed interviews will typically contain a disproportionate number of far-left Democrats and moderate Republicans.
Pollsters should be willing to routinely disclose their contact rate, cooperation rate and completion rate. The betting here is that they won't, because by doing so, they'll expose their dirty little non-representative secret.
Bottom line: Pollsters surely know that they are failing to reach representative samples of the electorate. Based on how 2015 has turned out to be a miniature version of 2014's failure, they apparently don't care — perhaps because their busts almost invariably understate support for conservative candidates and causes.
Cross-posted at BizzyBlog.com.
