These Days, Political Polling Has Its Problems
Credit: www.usatoday.com
August 25, 2015

I read this in the local paper's politics column and thought I'd pass it along, since it made a few points I wasn't aware of. Sounds like campaigns, more than ever, will end up relying on internal polling. Via the Philadelphia Daily News:

Terry Madonna, of Franklin & Marshall College, and Chris Borick, of Muhlenberg College, run respected polls.

After the 2012 elections, both were rated among the nation's most accurate, on a level with the New York Times/CBS News poll.

And guess what? They both say their business is a mess.(Full disclosure: Madonna's the Daily News' pollster.)

"Polling is in a tremendous state of flux," says Madonna, "in an incredibly changing environment that is stressful."

Says Borick, "We're in a very turbulent period in terms of the industry."

Why? Technology, rules and lifestyle.

With the rise of cellphones, more than 40 percent of American households no longer have landlines, up from 25 percent just five years ago.

There's a federal ban against automated calls to cellphones. And it applies not only to robo-calls from computers but also to automated calls that, when answered, connect to an actual person.

So, pollsters start in a significant hole, one that grows each election.

There also are the far-too-busy (or disengaged) lifestyles of much of the electorate, insulated from pollsters with the aforementioned caller-ID.

As a result, response rates of 30 to 40 percent and higher a decade ago are down to 10 percent or lower. So to get a decent Pennsylvania sample of 500 registered voters, you need to make 5,000 calls.

[...] Even if a pollster makes the necessary number of calls to get a large enough sample, it's far less representative than samples in years past.

That's because the total is likely to have too many seniors or too many women and must be massaged or, as pollsters say, "weighted."

That's done by applying data from voting records and census info to adjust for gender, geography, demographics and party registration.

In other words, we know what's supposed to be out there, let's make our sample look like that.

But although weighting seems to mostly work, Borick says, "Every time you weight you're making assumptions that data you're looking at now is going to be same today as when it was collected. . . . I don't think anybody takes for granted that will be the case."

This issue of unreliable samples is most problematic in predicting who votes.

Madonna notes that's why pollsters missed the overly Republican vote in 2014 and the overly Democratic vote in 2012.

And Borick says, "We need to guard against socially desirable outcomes. Many say they're voting because it sounds good. So we ask questions to see if they're following the race or know where their polling place is."

So how to get polls with more real people and less statistical adjustment?

One way is a larger mix of live calls and call-backs to cellphones combined with offers of incentives (gift cards, small amounts of money) to those willing to talk.

But the real future of polling, Madonna and Borick suggest, is in building a sample for Internet polling.

This requires using voting records and demographics to amass a large representative panel of voters, then inviting them to take part in online polling.

"I think that's where we're headed," says Borick.

Meanwhile, both say, be wary of candidate or party polls, or polls with ideological leans (Rasmussen, right; Public Policy Polling, left), look to polls that fully explain methodology and rely on averages of several polls.

Discussion

We welcome relevant, respectful comments. Any comments that are sexist or in any other way deemed hateful by our staff will be deleted and constitute grounds for a ban from posting on the site. Please refer to our Terms of Service for information on our posting policy.
Mastodon