By Dr. Ken Broda Bahm:
2016 will be remembered as the year the polls and the pundits were wrong. After enjoying a relatively clear lead in the polls throughout the election, and being predicted as the winner by previously credible poll aggregators like the Huffpolster and Nate Silver’s FiveThirtyEight blog, and after the majority of talking heads talked about Trump having only a “narrow path” to victory and needing to “pull an inside straight” in order to win, the brash businessman apparently did just that. It came as a shock, not only to Clinton supporters but even to Republicans and Trump’s own supporters, as nearly all the big swing states, and even some not expected to swing (like Wisconsin), swung in Trump’s direction.
Now, your own politics will determine whether you greeted Tuesday night as a dream or a nightmare. But one thing on which practitioners and students of persuasion on both sides should agree is that the difference between what we expected and what happened is worth a closer look. The full lessons will play out in the weeks ahead. Academics are likely to study this election for years. And the major political parties, both of them really, are apt to do some soul-searching on what they represent, and the extent to which they speak to and for the American people. But more immediately, I believe that for those interested in public attitudes and persuasion – including litigators — a reality check is in order. In this post, I want to share a few initial reactions that apply to legal persuaders and others paying attention to the pulse of public opinion.
I see four important conclusions.
1. Social Desirability Skews Measurement
I have written before about social desirability bias, or the tendency for people to want to present themselves in a favorable way and to lean toward answers that will meet with approval. In the run up to the election, there was the perception, touted by the campaign itself, that Trump tended to outperform the polls and to do better in automated polls (“Push 1 for Trump”) than in live operator polls, in part due to “shy Trump voters” who were embarrassed to tell another person, even an anonymous data collector on the other end of the line, that they support Trump. It’s certainly possible that the polls undercounted Trump support in part because citizens would say one thing on the phone and then do another in the privacy of the voting booth. This is the same phenomena that has a huge effect on the reliability of the information obtained in voir dire: What the potential juror says in open court during jury selection is often molded by a preference for what seems to be the right answer, and when a judge sitting next to a flag is asking if you can be fair, the potential juror knows that the right answer is “Yes.” Juror questionnaires, in contrast, convey at least a temporary feeling of privacy while filling out the form, and for that reason, they are more reliable than traditional questioning by the judge or by counsel.
2. Opinion Research is Only as Good as Your Sample
Another reason polls were wrong has to do with the polls’ sampling. The national popular vote, which Hillary Clinton seems to have won, was actually pretty close to what the polls were predicting. But when it comes to the electoral college and the state contests, the pollsters have to rely on state-level polling, and that is where the quality drops off fairly substantially. Factor in less-frequent polling and regional variations in non-response bias and likely voter predictions, and sometimes those polls are wrong. A 95 percent level of confidence sounds high, but will still fail in one out of twenty times. One common problem, according to Geoff Garin, a long-time Democratic pollster, is that most of the polls undersampled non-college-educated white males, the group that broke for Trump by a factor of two-to-one. For those of us who use surveys, focus groups, and mock trials as a tool in litigation, it is a reminder to be very careful about our methods: Ask the wrong people, get the wrong answers.
3. Motivation Matters
Viewing the results on election night, it seemed like Trump may have made good on his promise to bring an army of new voters into the system. Further analysis, however, shows that isn’t the explanation: At present, it looks like Trump won fewer votes than either Romney or McCain in the past two cycles. Instead, the explanation seems to be an “enthusiasm gap” that held down turnout for Clinton, leading her to underperform in many of the geographic areas and the demographics that Obama won overwhelmingly in 2008 and 2012. The steady drip of Wikileaks, along with the FBI’s return to the topic of Clinton’s private email server, was a reminder of the “corruption” and “above the law” themes that make independent voters and liberals wary or simply exhausted. The Clinton camp had been saying that these attitudes were already “baked into” the poll results, but the late emphasis might still have been a punch-to-the-gut for questioning Clinton voters, and probably also bouyed the hopes of Trump voters, getting more of them to the polls. It is just like in a jury: There is a big difference between someone who will passively agree with you on the issues and someone who is motivated to work for you when push comes to shove in deliberations.
4. The Rural Urban Split is as Big as Ever
The racial split was big (with Clinton winning 88 percent of African Americans and 65 percent of Latino voters), but Obama had won both groups by even bigger margins, and with greater turnout. The main story for 2016 seems to be the urban-rural divide, with Clinton taking the population centers and Trump taking pretty much everywhere else. The margins in both areas were lopsided 60-30 ratios or more. Watching CNN’s coverage on election night, every time John King went to the large digital map, you would see the same pattern when he opened up the detail on the states: Rural red and urban blue. That same pattern played out in state after state, with the state’s chance of becoming a red or blue state being determined chiefly by whether the urban areas were large enough to outweigh the rural areas. By the end of the night, the county-level map looked like a satellite picture of the country at night, with Clinton votes coming from wherever you could see city lights. For litigators who spend time in those big cities, it is a useful reminder: City jurors and country jurors can be quite different, even when they’re neighbors. Drive a few miles and you can be in a different world.
______
Other Posts on Polling and Public Opinion:
______
Photo credit: Michael Vadon, Flickr Creative Commons