PulsePoll.com : News : The Oregonian : 04.12.00

Web Pollster Hopes to Win Credibility

Wednesday, April 12, 2000

John Marling sits in his Beaverton office and admits his new Internet political polls violate some basic tenets of survey research.

In conventional scientific surveys, pollsters query a random sample of the population. But Marling's company, PulsePoll.com, has put its poll on sites scattered across the Internet. Anybody who sees it can register an opinion with a few clicks of a mouse.

In some ways, the PulsePoll is similar to call-in surveys by many local TV stations. They often get thousands of responses but may draw from only one demographic group. For example, a question about gun control may attract mostly gun owners who are passionate about the issue.

But after his experience in four presidential primary states this year, Marling insists PulsePoll.com is on to something.

"We proved in the presidential primary that the Web-based survey was as accurate, or more accurate, than the telephone survey," he says, declaring that his polls came reasonably close to the results.

Or did he prove anything?

"There are so many problems, you can't put a lot of weight in their results," says Howard Fienberg of the Statistical Assessment Service, which monitors the use of statistics in social research.

"They're not polls," says Arnold Ismach, a survey expert and retired University of Oregon professor. "They just dirty up the name of survey research. . . . They're just for entertainment."

Whatever the case, PulsePoll.com is another example -- along with such innovations as Web-based fund raising -- of how the Internet is changing politics. Once the province of organizations with deep pockets, polls are now as plentiful as mushrooms after a spring rain. The question is how seriously to take many of them.

Marling, 53, is a former newspaper publisher who for 15 years has run a company that specializes in market research for small and midsized newspapers. Last summer, he helped form PulsePoll.com and joined the Internet gold rush.

The company's idea was to spread polling questions across many sites in hopes of generating responses from a broad cross-section of Internet users. Marling says the presidential primaries are a perfect chance to test the model.

Using his newspaper contacts, Marling put his first primary poll on 21 New Hampshire sites, most connected to news organizations.

The company's final poll, finished the day before the state's Feb. 1 election, showed Bill Bradley slightly ahead of Vice President Al Gore, who won the Democratic primary by 4 percentage points. It predicted Arizona Sen. John McCain's win over Texas Gov. George Bush in the Republican primary but understated his margin of victory. However, traditional surveys also missed McCain's surprisingly large 18-point win.

The PulsePoll was closer in Arizona's Feb. 22 primary, where it was 3 percentage points off McCain's winning mark. In Washington on Feb. 29, the poll closely tracked Bush and McCain's support levels, although it had McCain slightly ahead. Bush won by fewer than 2 percentage points. PulsePoll also was relatively close in Colorado's March 10 primary.

Survey experts say that, assuming Marling is accurately reporting his results, he may just have gotten lucky. Generally speaking, they say his methodology has at least two fundamental flaws.

First, despite the Internet's growth, most people still don't regularly use it. A recent survey by the Stanford Institute for the Quantitative Study of Society found that 55 percent of Americans have Internet access but only a third use it more than five hours a week. Minorities and the elderly in particular have lower usage rates.

"You need to get all segments of the population" to have a legitimate survey, says Allan McCutcheon, director of the Gallup Research Center at the University of Nebraska-Lincoln.

Marling counters that Internet users, particularly those who surf news sites, are much more likely to vote in primaries. And he points out that traditional pollsters increasingly face heavy refusal rates, eroding their ability to draw an accurate sample.

But the biggest problem critics see with the PulsePoll is its voluntary nature. You don't select the respondents; they select you. You get people who feel the strongest about a topic or happen to hit the right Web sites.

"There's so many different variables you can't control for," Fienberg says. "You don't really know who the audience is."

Marling says his company takes steps to keep people from voting multiple times -- at least from the same computer. But he concedes his poll "goes against everything that is accepted in survey methodology. . . . We're pioneering here."

McCutcheon says some traditional survey companies use the Internet without sacrificing acceptable methodology. He says some assemble representative groups by giving people free computers and Internet access in exchange for participation. Others form large panels of Internet respondents by providing small payments or the chance to participate in sweepstakes. They also can conduct telephone surveys to pick up underrepresented demographic groups.

At the same time, there are a number of other self-selected polls on the Web. One, sponsored by President Clinton's former pollster, Dick Morris, has been heavily criticized by survey experts.

Marling says that he is undeterred by the criticism and that he wants to make the PulsePoll nationally known. He hopes to eventually make money from his poll by attracting advertising and doing commercial research.

"Very simply, we want to be the dominant entity for Web-based research," says Marling, adding that he wants to do for polling what "Amazon did for books."

Back to PulsePoll.com : News