A lot of things went wrong in 2020. And presidential polls were no exception. If Trump does pull it out, it would be the complete and utter destruction of the polling industry, at least in politics. Joe Biden was supposed to win the 2020 presidential election by eight points, according to the polls, which were wrong.
The polls are so tight right now. An eight-point lead for Biden in Wisconsin the president trailing Biden by eight points. Now, Joe Biden leading President Trump by 10 points. He won by five points. He was supposed to win Wisconsin by 10 points. Instead, Biden eked out a victory there with less than one percent of the vote between him and incumbent President Donald Trump.
The polls were very wrong in Wisconsin. The polls also had Biden winning Florida and North Carolina. Yup. The polls were wrong there, too. Can we trust these polls? Here’s why and what’s being done about it.
WHAT WENT WRONG IN 2020
I’m looking at two key areas that I think were really important demographically in understanding the difference between the actual outcomes and what the polls suggested was going to happen. And that is one the big diverse mix of Latino voters in southern Florida and in the Rio Grande Valley, in particular in Texas, and then white voters without a college degree in the Midwest.
Because if we look at the states where we had the biggest problem, that those were the states, we had the biggest problem, Texas, Florida and the Midwest. If you look at other polling, well, it was off a little bit in places like Georgia and Arizona, even Pennsylvania. Those were much closer and much more within the realm of our threshold for uncertainty that we would expect in an election.
So, I think we have to look at why would these certain demographics be off in the poll, in the election, in this particular election. But they weren’t often two years ago in the midterms. That’s where we’re scratching our heads. It was a better performance for pollsters that in 2016, what about twenty eighteen, there wasn’t a presidential election then there were midterm elections instead.
The polls didn’t do so poorly, not nearly as poorly as 2016 and 2020. One big reason, there was one name missing from the ballot, Donald Trump. And when that name gets on the ballot, it has a huge effect on both polling and what happens on Election Day.
This is one thing that we’ve seen in our polling that’s different for President Trump than for any other president before. That is that when we ask voters whether they approve or disapprove of the job the president is doing for past presidents, been a matter of do you agree or disagree with his basically his policies, his overall approach to governance.
With Donald Trump, that question has actually become a reflection of people’s own identities, like if you approve or disapprove of Donald Trump and you say that publicly, that conveys to other people a lot about who you are as a person intrinsically. And that’s much different than anything we’ve ever seen before.
THE TRUMP EFFECT
In 2016, one theory with a catchy name emerged to explain why polls fared so poorly, the so-called shy Trump voter, that’s a voter who, because of Trump’s unending string of controversies, is shy to admit to pollsters that they’re voting for him in 2020. That theory has been refined a bit. It’s no longer about being shy.
It comes down to trust. The pollsters that I talked to this year talked more about kind of this different phenomenon where it’s not really that these voters are too shy to talk about wanting to vote for Trump. It’s just that they don’t want to talk to the pollsters at all. When they look at their numbers, they notice that in some cases they’re accurately measuring where Biden ended up netting out in these early results.
So, like in Wisconsin, for example, the person who leads Marquette Law School’s polls told me that Biden support in Wisconsin was 48 percent of likely voters were going to vote for him within, you know, a plus or minus four-point four percent margin of error. And he ended up with a little over forty-nine percent of the vote based on these early tallies.
So that’s not all that far off. But then of likely Trump voters, that poll found that forty-three percent of them were going to favor Trump. And really, he pulled out with nearly forty-nine percent as of the early tally. So even if they are measuring the Biden supporters and Democrats fairly accurately, they’re clearly missing something with the Trump supporters.
LANDLINES VS. TEXT MESSAGES
The top-rated polls on 538 all have something in common, they’re all called live-caller polls. That’s a mixture between phone calls on mobiles and landlines where pollsters actually talk to potential voters. Getting people to talk to pollsters over the phone is proving harder and harder, even without the trust issues that’s theorized to have affected the 2020 election.
These are the response rates for Pew polls going back to 1997, they show a steady march downward. There is not a great deal of standardization. This variation is something that consumers need to be aware of. So, when I see variation, I’m talking about, of course, the length of the survey, the amount of questions or the amount of time it takes to complete the survey, the mode of the survey.
So, are we doing face-to-face surveys, landlines, cell phones, robocalling? Are we doing online polls variation in the times that the surveys are in the field, which can go from, you know, one to seven days or longer. So, for those of us consuming polls, you know, we have to be aware of this wide variation and really take it upon ourselves to click the link that provides more information about the poll. Take a recent Monmouth poll, for example.
The pollster spoke with two hundred and eighty-nine people on landlines and 521 people on cell phones. That’s expensive. This is all forcing polls to go increasingly online. And there’s a whole host of issues with trying to reach potential voters without using live polling. You know, a lot of young people don’t have landlines.
They’re more likely to respond over the Internet or to a text message. So, people who maybe have traditionally been harder to reach by the polls are maybe going to be incorporated in them more often. And, you know, I think different types of polls are incorporating these methods to a varying extent. So, it’s not necessarily going to be the same across the board.
Those of us who still use live interviewing for most of our polling, the vast majority of the numbers, we’re dialling our cell phone numbers with some landline numbers thrown in there because some people still use their landlines, but not a lot. The question, though, that that’s raised there is not the telephone numbers, which we have for the vast majority of voters. So that’s not a problem. Problem is, some people say, is that you’re not going to get the shy Trump voter when you’re talking to a live interviewer.
But what we’re finding is that folks who use noninteractive polling, such as IVR, which is that where you punch in the number on the telephone or online polling or text message, polling had the same problems. So, it wasn’t that the shy Trump voter wasn’t talking to a live interview.
It was if there is shy Trump voter out there, they simply weren’t responding at all. So that’s one of the things that we have to look at. But I’ve tried some other methods as well. And we actually did some of that using email samples from the voter lists where we not only matched up phone numbers, but we matched up emails to some of those voters and conducted it that way.
We found that, you know, the email samples tend to actually be slightly more democratic than our live telephone sample. So that raises questions about whether that technology is going to get you there. Similarly, text message polling, we actually did a test of a text message polling and found that more Democratic. We didn’t release it, but we just wanted to do the test and we found it was more Democratic than our live interviewing poll.
So, there’s not a magic bullet out there in terms of the type of methodology that you use. You really have to focus on just simply, do I have the right mix of folks who are willing to answer this? This poll?
The American Association of Public Opinion Researchers is an organization representing pollsters and public survey professionals across the United States. It’s in the middle of researching the reasons why polling might have missed the mark in 2020.
The AAPOR did a similar post-mortem after the 2016 election in 2016, it took several months for the main organization that reviews these polls to come out with their post-mortem about why the polls were off that year.
So, it’s going to take a while to understand the full impact of the polls. I think maybe the main lesson for the public and maybe for the pollsters to consider when they’re putting out these results is just, you know, the polls are only as good as the people who respond to them.
They’re only as good as their methods. So, it may be more about having the public reframe how they’re thinking about the polls and, you know, not taking them necessarily as a predictor until maybe we have a better way of sorting that out.
We’re looking at these election polls and what went wrong with the election polls, but I’ve been kind of beating this drum for the past four years and pointing out situations where we ask what seems to be innocuous questions about life in general, such as do you plan to take a vacation this year where we’re suddenly getting these huge partisan splits on a question that never had anything to do with partisanship. And that, I think, is part and parcel of, you know, our deepening partisan tribalism. But that has gone, you know, into hyperdrive. And, you know, so we’re talking about these elections and what went wrong in the elections.
But there’s a lot that I think I as a pollster, have questions about that we’ve been measuring for the past few years that we might have been off by a few points because of this huge partisan divide where people interpret everything that we ask about through this lens of how does it reflect on me? How does it reflect on me as a Trump supporter, as a Trump opponent?
I’m going to answer this question in a way that makes Trump look good or look bad has been something that we’ve been experiencing and quite frankly is a lot more problematic for the polling industry than what happened in this past election.