CGG Weekly, May 28, 2004

"He who builds to every man's advice will have a crooked house."
Danish Proverb

I have a problem with polls, especially those polls that become national news. The problem is that they are fundamentally worthless. Beyond that, the national media uses them, not necessarily to provide a snapshot of Americans' opinions, which was their original purpose, but to influence opinion. Last week, for example, CNN used a Gallup poll on high gasoline prices to back calls for tapping the national strategic petroleum reserve, which President Bush has steadfastly refused to do. Polls can be deftly crafted and cunningly reported to move people in a certain direction.

For instance, polls come into their own during the run-up to the presidential election. Various media outlets and think tanks produce polls on a regular basis, trying to document the candidates' rise and fall over the weeks. Though the polls claim to be scientific, a look at polling methods shows that, in a good many cases, only a relative handful of people are actually asked questions, and from this tiny sample, the pollsters project their "national findings." Do those 1,200 people's opinions really mirror those of the other nearly 300 million Americans?

And they are not often getting the opinion of Joe or Susie American. Most polls select their respondents very carefully. Pollsters say they do this in order to get a truly representative sampling of the population. However, this selection process is ripe for tampering in the same way lawyers use all legal means to select just the right jury members to obtain a verdict in their favor. It would not be all that hard for a poll, purported to be of registered voters likely to vote in an upcoming election, to select a disproportional number of respondents from a particular party, locale, race, economic level, occupation, age, sex, etc. to skew a poll in a certain direction.

Notice this proviso from the Gallup organization on a recent poll regarding the high price of gasoline:

Results are base [sic] on telephone interviews with 1,002 national adults, aged 18 and older, conducted May 21-23, 2004. For results based on the total sample of national adults, one can say with 95% confidence that the margin of sampling error is ±3 percentage points. In addition to sampling error, question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of public opinion polls.

Many national polls have similar error rates. This seems small on the surface until one realizes that each answer to a polling question floats in a six-percentage-point range—and that the pollster is only 95 percent confident that this range is valid. For instance, if respondents to a voters' poll pick President Bush and Senator Kerry at 43 percent each, it is possible that Bush may lead or trail Kerry by as much as six points. How valuable, then, is the poll? It has really told us nothing.

Let us not disregard the second point of the Gallup provis The wording of questions and "practical difficulties" can and do introduce error or bias into polls. Sometimes pollsters try so hard to make their questions "fair" that they become almost unintelligible. Even if a respondent is asked to choose one among several choices, the order of the choices can introduce bias. And what are "practical difficulties"? It sounds suspiciously like a catchall term for whatever excuse might be necessary to fend off criticism or accusations of bias.

These polls, with all their potential deficiencies, are reported as serious news. At Ambassador College, students of freshman mathematics were required to read a small book, How to Lie with Statistics. The author, Darrell Huff, writes, "The secret language of statistics, so appealing in a fact-minded culture, is employed to sensationalize, inflate, confuse, and oversimplify." He cites examples in which even unfavorable results can be spun to seem positive. He could have said the same thing about polls, which are at heart statistics.

This is not to say that every poll is full of error or slanted. Many of them are as good as a human organization can make them. Nevertheless, whether they are true snapshots of the public's opinions or vehicles for media manipulation, they are at best fuzzy generalities and apt to be used for partisan purposes. We would do well to read them with a wary eye, especially the politically charged ones.

In contrast, the Bible does not contain one opinion poll. Perhaps this fact says all we need to know about their real worth.