“42.7 percent of all statistics are made up on the spot.” – Steven Wright

6 Jun

Too Many Variables

I see lots of press releases like this one citing third party research to prove the efficacy of marketing and advertising campaigns, and a lot of them are pretty good. This particular example, a study which demonstrates the effectiveness of “Peel n’ Taste,” is not particularly useful for a number of reasons. The campaign tested was for Welch’s Grape Juice, and it measured the effectiveness of “dissolving flavor strips” inserted into PEOPLE magazine on intent to purchase.

The study concluded the following:

Purchase consideration more than doubled when readers tried the Peel ‘n Taste(R) flavor strip — 59% of those who tried the flavor strip were likely to buy Welch’s, compared to only 25% of those who did not sample the Peel ‘n Taste(R) strip

Now, there are certainly occasions where it is not necessary to conduct both pre- and post- exposure research, but there are so many variables here that needed controlling that I cannot believe only post-exposure research was conducted. Here are a few:

  • What percentage of people don’t like grapes or grape juice? If the 71% of the sample that didn’t try the strips are more likely to dislike grapes or grape juice (or even Welch’s) beforehand, than the higher purchase intent amongst strip-tasters becomes axiomatic, not predictive.
  • What percentage of people would never put a strip from a magazine into their mouth? Count me in that bunch.
  • Is it possible that persons who would eat their magazines are just more adventurous, and thus more likely to indicate that they would try a new product?

Really, the first point on this list is a doozy. We don’t have any pre-exposure data to show what percentage of the sample would even consider grape juice (strip or no strip) prior to exposure, so we can’t read any causality into these numbers.

Now, this may or may not be an effective marketing tool, and I am not passing judgement on “flavor strips”—there is some other compelling data in this release that indicates that the strips were the second most-noted of any ad in the magazine, and that there was no negative impact upon the brand amongst strip tasters, so I assume the strips have enough tasty grape goodness to theoretically get the job done. But until you know more about the 71% who didn’t even try the strips, you can’t read anything into the behavior of the 29% who did.

And that’s more on flavor strips than you ever though possible on a Friday morning.