During an election season, public opinion polls are always big news. It’s important to understand the fundamentals of polls and how the news media covers them.
We received many texts from Wisconsinites with questions about polls for our Informed Voter series — a collaboration between WUWM and the Milwaukee Journal Sentinel to answer questions texted to us from potential Wisconsin voters.
Rich from Burlington wants to know:
What determines the mix of Democrats/Republicans/Independents in the polls?
The answer is random chance.
“We don’t say ahead of time what percentage Democrats, Republicans and independents there should be because we have no way of knowing that. We don’t have registration by party in this state, so you couldn’t go to the voter registration list and find that out. Literally, the only way to know that party balance is by asking people,” Franklin says.
The Marquette Law School poll and most media polls, he explains, start with a random sample that gives all phone numbers (cell phones, landlines) in the state an equal chance to take part. It’s the answer to a question about party preference that determines a poll’s party balance.
Diane of Milwaukee:
How does margin of error affect interpretation of polls such as this one? Does a 5-point lead really mean Evers is ahead, or are the results "within the margin of error," a phrase some media use?
Put simply, the margin of error is a mathematical formula that tells us the amount of random sampling error in a survey’s results. It’s easy to get lost on the specifics of the margin of error, but here’s one basic rule.
“The tighter the race, the more cautious we should be about characterizing who may be ahead or who may be behind. But if the lead is within the margin of error, it doesn’t make it meaningless either,” Gilbert says.
So, how should you interpret the numbers? If someone is ahead by 10 points with a margin of 4 points, odds are they’re in the lead. But if someone is ahead by 1 point with a margin of 4 points, odds aren’t as good that they’re in the lead.
When viewing poll results, Franklin says it's good to keep a few things in mind:
- If the lead is twice the margin of error, then you’re justified in saying that person has the lead.
- If the lead is about equal to the margin of error, then that person might be ahead but not to a statistical level of certainty.
- If the lead is less than the margin of error, then you should be cautious saying anyone is ahead.
So, how does someone — a voter, reader or journalist — decide which polls are credible?
“As a journalist you don’t want to be a gatekeeper … at the same time, you do have to make some judgments because some polls are really bad and some polls are very respectable," Gilbert says. "A lot goes into those judgments. It’s not just margin of error, it’s not just sample size, it’s kind of looking at the totality of the poll. But it’s a real balancing act for journalists."
Franklin says some polls, including the Marquette Law School poll and many media polls, will identify themselves on the phone. But some, like campaign polls or interest group, might not.
You can ask the person on the phone what the poll is for, but there’s no guarantee you’ll get an answer. So, you likely won’t know who’s doing it or how it’ll be used. And you certainly won’t know the poll’s methodology, explains Gilbert.
Lynn from Kenosha:
How can they get an accurate representation of opinions when so many people don't have landlines, have unlisted numbers, screen calls, and block numbers?
Franklin says 60 percent of Marquette Law School poll's interviews are on cell phones, so the decline of landlines is no longer a problem.
“Call screening is probably the biggest issue. With telemarketing explosions, illegal calls to cell numbers by telemarketing scams, response rates are lower,” he explains. “The saving grace from my point of view is that Democrats and Republicans are completely united in hating scam phone calls, so as a result, we don’t get a partisan bias because of people being reluctant to answer.”
Still, Franklin says 80-90 percent of calls made by the Marquette Law School poll go unanswered.
A curious member of the Milwaukee community wants to know:
I am very interested in the Bradley Effect and the impact that has on polling. Also, is there a similar effect based on sexism vs race?
The Bradley Effect tries to explain the differences between voter opinion polls and election outcomes when white candidates run against minority candidates. The concept is named after Tom Bradley, an African-American and former Los Angeles mayor, who ran for governor of California in 1982 but narrowly lost after leading in the polls up to Election Day. Broadly, this can be thought of as a person's reluctance to give answers that go against social norms.
“It also came up in the so-called 'shy Trump voter.' In 2016, allegedly people [were] reluctant to admit they were voting for Trump. So, those are issues in responses and response bias, and they are concerns,” explains Franklin.
But he says things like the Bradley Effect have been difficult to confirm. And experiments with the shy Trump voter found limited evidence that people were more likely to say they were voting for Donald Trump if they were talking to machine versus a real person.
Ahead of Election Day in 2016, most polls predicted Democratic candidate Hillary Clinton would win the presidential race. But Republican candidate Donald Trump was the one celebrating a win — a reminder that polls aren't always right.
“Those of us, including me, who got 2016 wrong — we had Clinton ahead by six in our final poll here in Wisconsin — it certainly is a vivid reminder that polls can be wrong,” Franklin says. “And I think that’s a valuable lesson to the public as well as to the pollsters: that we do our best to get the most accurate responses we can, but we’re fallible too and those things can go wrong.”