2022 Mid-Term Polling: Non-Response Bias
We keep hearing that several marquee races for Senate and Governor in next week’s midterm elections are very close – within the margin of error. It will be interesting to see how these seven races come out:
- Wisconsin Senate
- Wisconsin Governor
- Arizona Senate
- Arizona Governor
- Pennsylvania Senate
- Georgia Senate
- Nevada Governor.
This week’s polls in all seven races are within two points. The Republican candidate is ahead in 5 of them, and the Democrat is ahead in two – but that’s not really significant because the average margin of error in all seven polls is roughly four percent.
The key question for us to look at today on American Radio Journal is whether these polls are reasonably accurate, or if they are off by more than their margin of error. I’ll make a bold prediction: In at least five of the seven races, the results won’t be close at all: the winners’ margins of victory will be greater than the polls’ margins of error. Furthermore, I’ll predict that the difference between this week’s polling estimate and next week’s actual results will be in favor of Republicans. What leads me to make this prediction? There’s a factor that the pollsters are failing to include in their elaborate statistical models, and that factor is the rapidly declining response rate to pollsters’ calls, whether they use live calls or automated ones. Furthermore, there appears to be a meaningful difference between Republicans’ and Democrats’ response rates.
Nate Cohn is the New York Times’ chief political analyst, and he understands statistical patterns affecting polling better than anyone else I know. He may lean personally in a liberal/progressive/leftward direction in his narrative comments, but he’s enough of a professional to be intellectually honest in his statistical analysis. Here’s what Cohn said just a few days ago about what’s known as “nonresponse bias” in this year’s polling:
“In the aftermath of the 2020 election, most pollsters concluded that the polls probably underestimated Donald J. Trump because of something called nonresponse bias. Mr. Trump’s supporters were less likely to respond to surveys than Joe Biden’s supporters, even among people who had the same demographic characteristics.
While nonresponse bias is challenging to prove, there was one possible marker of it in the New York Times … data in 2020: White registered Democrats were more than 20 percent likelier to respond to our surveys than white registered Republicans.
In our final wave of Senate and House polls in the last few days, … nonresponse bias looks as if it’s back. Overall, white registered Democrats were 28 percent likelier to respond to our Senate polls than Republicans — a disparity exceeding that from our pre-election polling in 2020… the wide disparity in Democratic and Republican response rates was most likely symptomatic of a deeper nonresponse bias: Biden voters, regardless of their party, were probably likelier to respond than Trump voters. This drove up the Democratic response rate, but it did more than that. It meant there were too many Biden Democrats; too many Biden Republicans; too many Biden independents. Weighting by party wasn’t enough. This time around, the response patterns by district and state certainly raise the possibility that there’s a similar challenge.”
One theme that is often heard on inaccuracies in polling is that most polling sample bases are biased towards Democrats — that there are not enough Republicans in the samples. Nate Cohn’s analysis suggest that the problem does not lie in the mathematical distribution of polling respondents, but rather in the psychology of polling respondents. Perhaps because of the relentless demonization and denigration of Republicans in the media, many Republican respondents don’t tell the truth about their preferences, so they behave differently in casting a ballot then they do in responding to pollsters. If Cohn is right, and I believe he is, then the actual voting patterns of Republicans will be stronger than the polling would indicate. Look for strong Republican results across the board next Tuesday. And in my next American Radio Journal commentary, I’ll report on my own predictions in the seven races I highlighted.
