Knowing Your Sources

Market research, like all social science research, falls into one of two categories: quantitative and qualitative. Quant data can be categorised and statistically measured, whereas qual data is non-objective, and cannot be categorised systematically (e.g. numerically).

The difference is pretty stark, but market research is particularly susceptible to pulling quant statistics out of the ass of qual data. This kind of research, unsurprisingly, tells us nothing, and I have found the perfect example to demonstrate why.

Check out this juicy statistic.

Sounds good right? Compelling enough, at least, to pool all marketing dollars together and invest solely in digital platforms. However, a quick look at the following chart shows the methodology behind it.

Ah. So, all this tells us is that websites have, at some point in every person’s life no matter how isolated, influenced a decision to buy. In comparison to what? In what context? We can only assume that they didn't give a shit.

The problem with this statistic (and others like it) is that it is inherently useless, especially when the sources of influence are not even touched upon. A quick look at the difference in influence between online stores such as eBay or Amazon and all banner ads will quickly show a massive discrepancy.

Of course almost every person will be influenced to buy when browsing through online stores, because that’s exactly the reason they’re there. Anyone taking this survey would be incredibly hard-pressed to report that, throughout their entire lives, they have never been nudged toward a purchase by anything they have seen online ever.

The only situation in which that might be applicable is if the respondent did not use the Internet at all, which, judging by the results of this survey, could well be the case with the remaining 3%.

It's not hard to believe that 6 out of the 200 respondents surveyed did not use the internet at all. Smaller samples do produce more polarised results, after all.

Here are some substitute headlines that convey just as much information as the original:

• Going to a Mall Looking to Buy Something Influences 97% of Consumers’ Decisions to Buy Something While Looking in a Mall.
  
  
• Being Exposed to Whatever Product of Whatever Type at Whatever Time Influences 97% of Consumers’ Decisions.
  
  
• 97% of People Have Thought About Buying Something.

Fucking revolutionary, right?

Parading statistics like this around might not do any harm to the average consumer, but for marketers who haven’t been exposed to the methodology behind them, they are easily taken at face value.

And it can quickly lead to a whole lot of wasted dollars and not a lot of revenue.

a.ce