In today’s Science Friday, we explore biases in polling – BUT we explore it from the angle of the interpretation of polls.
In a paper from Madison & Hillygus (both from Duke), they conclude that while most political nerds will evaluate a poll by reviewing the methodology, sample size, and question wording, “we find a significant factor in respondent assessments of polling credibility to be the poll results themselves.” Said a different way they “found that polls were perceived as more credible when they matched a respondent’s prior opinions and less credible when they did not.”
Experiments
The researchers conducted two experiments – one with polling a candidate and one with polling a policy issue. It was a relatively simple experiment design – measure your priors, introduce polling, measure the change in your perceptions.
In both instances, the researchers find motivate reasoning.
“Overall, these results of attitude polarization, together with the findings above showing a poll’s perceived credibility being conditional on congruence with prior beliefs, indicates that evaluations of polling information are biased by motivated reasoning. “
Conclusions
This finding is concerning. We often use polls to pop “bubbles” that politicians and consultants find themselves to be in.
We position survey research and polling as “objective” research and a way to check critical assumptions. This study illustrates that polling results are not being absorbed objectively.
However, if the hyper-partisan political atmosphere is allowing political actors to disregard any research finding, we are in dangerous territory.
citation: Madson, G.J., Hillygus, D.S. All the Best Polls Agree with Me: Bias in Evaluations of Political Polling. Polit Behav 42, 1055–1072 (2020). https://doi.org/10.1007/s11109-019-09532-1
