What causes bumps in polls and polling?
An astute listener to the radio show, the Ward Scott Files, that deals with political strategy and polling asks, “What causes bumps in polling?”
Bumps in Polling
Often after a candidate announces they are running for President, or after a major party’s political convention or some other major event, we will observe a bump in polling numbers. In most cases, if we wait several weeks, the bump will disappear.
What are we actually observing with these “bumps” in polling numbers?
In an attempt to further answer the question, I came across a paper, “The Mythical Swing Voter” by Andrew Gelman, Sharad Goel, Douglas Rivers, and David Rothschild of Columbia University, Stanford University and Microsoft Research.
Political scientists have debated whether swings in the polls are a response to campaign events or are merely reversions to predictable positions as voters become more informed about the candidates (page 2)
The paper follows an interesting methodology in that it conducted 750,148 interviews with 345,858 unique respondents on the X-box gaming platform during the 45 days preceding the 2012 Presidential election – creating a large on-line panel for studying shifts.
What is interesting is that the data showed with demographic adjustments, the data reproduced swings found in media polls during the 2012 campaign. HOWEVER, if we don’t use demographics, but instead use partisanship and ideology “most of the apparent swing in voter intention disappear.”
What the authors found was selection bias playing a role in the “bumps” candidates receive after a large announcement, meaning certain groups of people were much less/more likely to participate in a survey, and if we correctly estimate voters using items other than demographics, most of the apparent swings were sample artifacts, not actual change.
Only a small sample of individuals (3%) switched their support from one candidate to another.
We estimate that in fact only 0.5% of individuals switched from Obama of Romney in the weeks around the first debate, with 0.2% switching from Romney to Obama.
Conclusion(s)
The paper’s authors conclude “that vote swings in 2012 were mostly sample artifacts and that real swings were quite small.”
In today’s highly polarized partisan politics, the percentages of the Mythical Swing Voter is much smaller than polling would indicate.
Meaning, there is little to no actual bump or people switching sides – there is more of a selection bias in the polls. “The polls do indeed swing—but it is hard to find people who have actually switched sides.” (page 2)
The temptation to over-interpret bumps in election polls can be difficult to resist, so our findings provide a cautionary tale. The existence of a pivotal set of voters, attentively listening to the presidential debates and switching sides is a much more satisfying narrative, both to pollsters and survey researchers, than a small, but persistent, set of sample selection biases.