Beyond exit polls: A new way to survey voters

Instead of stationing in-person interviewers outside of polling places, NORC conducted telephone interviews for AP and Fox News with a random sample of registered voters in New Jersey and Virginia in the Nov. 7 general election and in Alabama for the U.S. Senate special election on Dec. 12. In each state, NORC also interviewed a much larger non-probability sample via the internet, and used sophisticated statistical techniques to combine the two surveys. All interviews -- about 4,000 in each state -- were conducted beginning 96 hours before Election Day until the polls closed in each state.  The results were promising. At poll close, the new survey approach estimated that Democrat Ralph Northam would win the Virginia governor’s race by a 52-46 margin. His final margin was 54-45. The New Jersey survey estimated at poll close that Democrat Phil Murphy would win that state’s governor’s race 57-38. His actual margin was 56-42. And in Alabama, the survey estimated at poll close that Democrat Doug Jones would win 50-47, while his actual margin was 50-48. In all three states, the surveys provided detailed crosstabs on the demographic makeup of the electorate and the issues that motivated different voter groups. The tests also provided solid support for using this survey methodology as a part of AP’s race calling operation. While the results supported a call of Murphy winning as soon as polls closed, AP didn’t make that call at poll close only because we decided in advance to treat the survey as a test.

The surveys also provided estimates of the size of the voting group that cast ballots before Election Day, as well as the candidate percentages among that group. In Virginia, for example, the survey reported Northam winning the absentee vote 61-37, contributing to his wider than expected victory margin. Northam won the actual absentee vote 60-40. Advance voting is playing an increasingly important role in American elections. Absentee and early voting now accounts for more than 20 percent of the total vote in about two-thirds of all states. Using random samples of registered voters combined with online surveys is a far more cost-effective way of estimating the size and direction of advance voters than the traditional telephone surveys of all voting age adults. AP and Fox’s election experiments also tested a new way of modeling turnout, the most important factor in determining the outcome of close elections. Normally, pollsters use answers to a series of questions to rate all respondents’ likelihood of voting on a scale from 1 to 10. In Alabama, NORC relied on a similar rating system, but used the scores to identify likely voters at several different levels of turnout, adjusting the amount of evidence needed to identify a respondent as a likely voter. That enabled survey experts to compare the demographics of those interviewed based on each turnout level, to determine which model most closely matched known demographics and race expectations.

NORC and Fox will each present research based on the 2017 experiments conducted with AP at the American Association for Public Opinion Research conference in Denver on May 19.

AP will use this methodology to power its new Election Day survey, AP VoteCast, in November. Read the AP announcement.