(This article first appeared in The Federalist.)
Polling guru Nate Silver is not happy. The avalanche of criticism for polling failures has reduced him to expletive-laced crankiness. Silver is doubtful the so-called hidden Trump voter exists. He also claims that since the polls called the correct winner for most states, errors are excused, but that’s like saying it’s okay to drive drunk as long as you make it home in one piece. The fact is, the polling was terrible and the pollsters deserve all the criticism they get.
With few votes left to register, we have a good handle on the final results: approximately 50.6 percent for Joe Biden to 47.7 percent for President Trump, at the time of this writing. Investor’s Business Daily got closest with a 4-point Biden advantage, but the overall RealClearPolitics average for October was a 9-point Biden lead. Special mention goes to CNN, which judged Biden to have a 12-point advantage, and Quinnipiac, which predicted an 11-point edge.
In the Senate races, Emerson was off a whopping 15 points in Maine. Quinnipiac had Lindsey Graham in a tie race in South Carolina (he won by 10 points), Theresa Greenfield beating Joni Ernst in Iowa by 2 points (Ernst won by 7 points), and Sara Gideon ahead of Susan Collins by 12 points in September, a mammoth 21-point miss. Even races where pollsters called the winner, they flopped badly on margin. CNN had Gary Peters beating John James in Michigan by 12 points, with the final margin being less than 2 points.
More than a ‘Margin of Error’
Some polling outfits claim the misses are just “margin of error,” but this is a misuse of the term. The problems for the public polls are more systemic than simply picking out the worst misses.
I have written about margin of error before, but let me describe it in the current context, not perfect statistical theory. Margin of error is kind of a measure of bad luck. Think of flipping a coin. If you flip a coin 10 times and get seven heads, that’s a lot — but not too crazy because it’s a small sample. Now, flip that coin 1,000 times, and the split will not be 70-30; it will be much closer to 50-50. If you repeated every day for a month, some days you would have more tails than heads and vice versa. It is very unlikely you will ever get exactly 50-50.
Good polling would have yielded margins of victory that see-sawed around the true result, with some a little above and some a little below. Of the last 20 national polls, however, only one overestimated in Trump’s favor. Looking at the state polls, only in Georgia and Arizona did the polls behave nearly as they should: Of the last 20 polls, 10 overestimated Biden in Georgia and 14 in Arizona, six overestimated Trump in Georgia and five in Arizona, and four and one were on the nose in those states, respectively.
Elsewhere, the results were disastrous. Not a single poll overestimated Trump in Ohio or Iowa. In fact, 13 polls had Biden winning Ohio, although he lost by 8 points. In Wisconsin and Florida, only one poll in each slightly overestimated Trump (Susquehanna by 1 point in both states), and only Trafalgar got the exact margin in each.
These two polling outfits, generally considered Republican-friendly, performed well across all battleground states, along with Rasmussen, which is also largely derided as a GOP partisan. These three did err a bit toward Trump at times but mostly by 1 or 2 points, in contrast to the public polls, which were often far off the mark. ABC/Washington Post had Biden up 17 points in Wisconsin, when the real margin turned out to be 1 point.
Public Polling Flunked
In contrast, top-flight political polling firms look at the voter file, first work to develop their modeled sample (which controls for likely bias in response rates), and then start dialing. Accounting for problems on the front end is more expensive because certain voters are more difficult to reach or refuse at a higher rate. Fixing problems upfront, however, means not having to monkey with the number on the back end.
The back-end fix that public pollsters use makes their misses even worse. They know any poll is susceptible to problems, such as non-responsive Trump voters, yet even after “fixing” the poll, their results are still way off.
FiveThirtyEight doles out letter grades to dozens of pollsters. Among the polls mentioned in this column, the New York Times and ABC/Washington Post have A+ grades, Emerson A-, Quinnipiac B+, CNN’s pollster SRSS and Morning Consult B/C. That’s some serious grade inflation.
As for Nate Silver, clean up your language — and give yourself an F.
