Space & Innovation

Election 2016: The Problem with Polls

Polls can shape coverage of an entire race. So why are they so often wrong come Election Day?

[slideshow 16837]

When New Hampshire voters cast their ballots in today's primary election, the candidates won't be the only ones tested by the results that emerge out of the the Granite State. Political pollsters will also find out how reliable their predictions turn out to be. Polls are supposed to provide an unbiased, authoritative view of the state of an election, but the strength of the reputation of any polling firm is the accuracy of its data. Results of elections in recent years have tested those reputations. In 2012,

Gallup, one of the most trust political polling organizations, wrongly picked

Republican presidential candidate Mitt Romney to emerge victorious over incumbent President Barack Obama, and Gallup wasn't alone. Last week, the

Des Moines Register/Bloomberg Politics poll

released a day before voting incorrectly predicted the outcome of the Republican presidential caucuses. Similar polling mishaps occurred abroad in national elections in the

United Kingdom

and

Israel

in 2015. Polls aren't simply a predictive measure of who will win an election. They can influence media coverage of competing campaigns, especially given that news organizations often focus on the horse race in day-to-day coverage. Candidates who perform well in polls tend to get more attention from voters as the media spotlight focuses on who's ahead. Polls even determined who could and couldn't participate in recent debates when the Republican field was too large to accommodate all of the candidates on the stage. Given how important survey data are to election dynamics, why is it that polls occasionally miss the mark on major elections, even when accounting for their margin of error? What are the underlying issues facing polling firms as an industry?

Why Betting Markets and Polls Don't Agree

Technology poses one of the biggest challenges to the reliability of the data traditional polling firms are capable of gathering. These public opinion analysts rely on phone calls in order to conduct a survey. When over-the-phone dialogue is increasingly displaced by texting, instant messages or other means of communication, the traditional pollster has a more difficult time connecting with audiences.

Federal regulations restricting auto-dialing cell phone lines

, meant to protect mobile users from intrusive solicitation calls, make the work of polling firms more difficult and expensive, as operators have to dial numbers manually and often cannot connect with the intended respondent.

Caller ID

after all makes it easier for potential voters who don't want to be bothered answering a number they're not familiar with to ignore polling firms. Potential survey participants still reachable via landline tend to be older than wireless-only households. Online polls offer one possible alternative. Although they provide the benefit of instant access and feedback and are less costly than traditional polling, Internet-based public opinion survey face their own challenges in terms of response bias and under-representation of certain demographic groups.

Your Politics Stink: People Can Skew Ideology

For public opinion analysts to know what the state of the race, someone has to pick up the phone when pollsters call. A lot of people need to answer, in fact, particularly for a national survey that might need around 1,000 survey.

In piece for the New York Times last summer

, Cliff Zukin, past president of the American Association for Public Opinion Research and a Rutgers University political science professor, cited "a decline in people willing to answer surveys" as one of the chief obstacles to increasing unreliability of public opinion data. Although response rates by themselves do not determine whether a poll will be accurate, they can create the potential for bias in the results by leading to under representation of certain demographic groups. Weighting can correct over representation by certain segments of the electorate, but cannot completely eliminate potential bias from a poll.

How Will We Vote in 2020?

It's no secret that the face of the American public is changing with each generation, but pollsters are still catching up with demographic trends. A poll can only predict the outcome of an election if it can appropriately survey the people who compose the electorate. Doing so has historically proven a challenge for pollsters, who have admitted as much. Following its mistaken 2012 prediction,

Gallup conducted a sweeping review of its polling practices

and promised an overhaul to its operations. Certain minority groups were underrepresented in its sample, a problem faced by other pollsters as well. Polling firms also struggle to reach young and new voters with its data collection efforts. Better-educated are more likely to respond to surveys than the less educated, and likely voters who are more engaged are also more likely to participate in polls.

Surprising Factors That Could Affect Your Vote

While the main focus of election coverage has understandably been the presidential contest, candidates running for office in local, state and congressional races will eventually win the attention of media organizations, voters and pollsters alike. Down-ballot races are more difficult to predict than national elections. Although this partly has to do with the fact that fewer voters have a greater influence on an election outcome, pollsters may also struggle to reach a representative sample of likely voters. Sample sizes are smaller in state and congressional races, particularly for primary elections, increasing the possibility of significant discrepancies between vote tallies and polling data.

According to FiveThirtyEight

, according to their database of House polls that show one candidate leading a contest by between 5 and 10 percentage points, 23 percent picked the wrong winner.

Voter Suppression: Old Strategy, Modern Tactics

In order to get an inside view of the state of the race and also to drive the narrative of horse-race coverage, candidates may choose to commission a polling firm to conduct a survey. Naturally, when the person paying for the poll has an interest in the outcome, results can be skewed. Since 2002, partisan polls have had an average bias of 4.25 points favoring the side that commissioned the survey,

FiveThirtyEight reported in 2014

. Although these polls are often taken with a grain of salt when reported in the media, candidates can make the mistake of relying too heavily on their results. In 2014, internal polling by former House majority leader Eric Cantor showed him with a comfortable 34-point lead over Dave Brat in a primary. Instead, Brat successfully ousted Cantor despite the disadvantage in terms of funding and establishment support, pulling off one of the biggest upsets in congressional electoral history.

Counting Votes Is a Tricky Business

Polls are at snapshot in time. When results are released, the data always indicate a time period over which the survey was conducted. Campaigns aren't static events, of course. News events within and outside of the race dynamics can mold and reshape perceptions of those running for office. Candidates can build momentum that ultimately sways the last on-the-fence voters to fall in with one camp or another come Election Day. Momentum is one of the reasons pollster

J. Ann Selzer cites in her examination

of the tighter-than-expected outcome on the Democratic side during the Iowa caucuses. In a dynamic race, numbers can shift even overnight.

Notorious Political Ads of the Past: Photos

Polls can only do so much to correct for the issues listed in this slideshow, and ultimately the voters can skew results at the last minute. A voter might be undecided until he or she reaches the voting booth. Voters change their minds in light something as consequential as major political or economic or seemingly as minor as a gut-feeling. Even details as simple as how weather affects turnout can influence results. Pollsters can do their best to forecast the outcome of a race, but voters are the ones making the final decision.

Video: Why Polling Accuracy Is Trending Downward