You should play it by the numbers

We’ve all seen the ads: 4 out of 5 dentists agree that Ta-Ta Tartar gum is the best for tartar control or 9 out of 10 doctors think that ingesting Fart-Away will eliminate gas and stomach bloating. Whenever I see these types of claims, my mind begins racing. So, who exactly are these noble members of the medical profession? How many doctors or dentists were asked? 10? 100? 1,000? What does the other disagreeing doctor or dentist think? Why could he not play along and agree that the product is best? Is the product flawed?

{sidebar id=44}

I know these statements leave obvious question marks in my mind, and I hope that people don’t base all of their purchase choices just on the opinions of five dentists. This might not seem an issue, but the numbers game goes deeper than that.

Think of how many times a day your customers are barraged with meaningless numbers. Are you adding to this numerical clutter, or are you giving them information they can really use?

Call me the unbeliever

By most accounts, I am not a numbers person. I like numbers. I just don’t believe them. Or, more precisely, I don’t believe the people who interpret the data or manipulate the questions. With enough of a spin, you can make numbers say anything you want.

I know I’m a journalist, so the numbers blame game is partly my profession’s fault. But I have tried my darnedest to only provide accurate, reliable numbers that mean something.

I used to work for a magazine that conducted an annual reader survey focusing on a random topic every year. The editor would group the responses by region (a common and logical step to take), then list the answers as the Top 3 or 5 for the region.

The first year I proofed the survey I noticed one location had responses that were completely different from the rest of the regions in the survey.

Ever the questioning soul, I asked the editor what was up with the Southwest. Were people in Arizona and New Mexico trendsetters for the rest of the country? Or, were residents of this region way off base? The editor tried to avoid my queries, and then finally replied that only two people in the region had answered the question. So, here we were. Presenting to the readers that the Top 3 choices for the entire Southwest were X, Y and Z. Nowhere in the article did the editor list how many people were questioned in the survey or how many responded. The editor couldn’t understand why I challenged that. We finally came to an agreement, and he disclosed the number of respondents and toned down the rhetoric.

Do numbers lie?

The Utne Reader explored this in a September 1995 article, “Can numbers lie?”. The article pointed out that poll wording is often overlooked. It reported that when the Yankelovich polling organization asked respondents the question “Should laws be passed to eliminate all possibilities of special interests giving huge sums of money to candidates?” that 80 percent of the sample said yes, and 17 percent said no. When the same organization reposed the question as “Should laws be passed to prohibit interest groups from contributing to campaigns, or do groups have the right to contribute to the candidate they support?,” 40 percent said yes and 55 percent said no.

Hmm... A few words do make a difference.

For more: Utne Reader, www.utne.com/archives/CanNumbersLie.aspx.

{sidebar id=1}

- Jyme Mariani

May 2008