Tom Webster, writing and speaking

A Field Guide To Ignoring Market Research

Added on by Tom Webster.
Charles Babbage's notebook at Wroughton museum

I recently spoke at the Vocus Demand Success conference (thanks for the invite, Geoff Livingston!) on the topic of consumer insights. Before I launched into my talk, however, I asked a very basic, yet provocative question: why are you here? No, I haven't been binging on Camus and Sartre again. Instead, I hoped to get a sense of what people expect to get when they go to a public presentation of market research.

I asked a similar question on Facebook, and got an altogether sensible answer from Chris Brogan, who replied that he wanted to know when he needed to conduct research, and when he could just go out, experiment, and possibly fail, without needing to do any market research.

Good question, Chris. Now here's your long distance dedication.

The answer I gave him on Facebook may have seemed flippant, but it wasn't--I think it was a pretty sound strategy, actually: if the cost of failure is low, easily bearable, or quickly recoverable, then you don't need me or my consumer insights brethren. Oh, sure, you could give us a call now and then. We miss your voice. But if the only thing you have to lose by pursuing an idea is a paycheck or two, or just your fear, then you don't need me.

The opposite is also true, of course: if the cost of failure is high in terms of time, treasure, or head count, it is incumbent upon you to invest some of that anticipated cost into doing a little market research.

Let me give you two quick examples. Yo, an app that essentially allows you to instantly ping a friend with a one "word" message ("Yo," natch) is not what you would call a slam dunk of an idea. In fact, Julien Smith notes that many great ideas are a mix of a good idea and a bad idea. Since most people won't work on the bad idea, there's a pretty wide lane to pursue it without competition. (He's had more great ideas than I have, so I trust his judgment on this matter. Go use Breather already!)

Yo basically replicates a feature Facebook deprecated long ago--the "poke"--and allows users to "YO" each other to their hearts' content, or until they go blind, whichever comes first. If Yo began their product development cycle with a market research study, what they probably would have gotten back is that most people don't have a great need to Yo each other. But a) you don't need "most" people to do anything to be successful, and b) it's hard for us to judge something we've never seen or used. But, more importantly, Yo took 8 hours to build, and was recently funded to the tune of one million dollars. I recently spent 8 hours building an Ikea armoire, and was funded to the tune of zero million dollars.

Yo didn't need any market research--in fact, I wouldn't have recommended it.

But here is another example: Path. If you are not familiar with Path, it started as a bit of a "velvet rope" social network--it was only on mobile, you couldn't post links, and it had a "friend" cap of 150. It was meant for a private social circle, and not to replicate Facebook or Twitter. Well, until this week, anyway. In the recently released Version 4 of Path, the company introduced a new messaging app called "Talk," which apparently we are going to want to use so much that it made sense to lift the 150-person friend cap, though they buried the lede a bit on that one in the release notes.

Now, I have been a Path user since the beginning, and I have a small 30-person circle on Path with whom I can occasionally be intemperate. (Just kidding. I am a well-regulated human.) Now, I do not claim that my collection of Pathogens is representative of anything, but I didn't see too much love for Path's changes in my circle. In fact, the reaction was so negative, that while I won't extrapolate that to the general population of Path users (I'd have to turn in my MRA card if I did that) it does beg--nay demand--asking the question: was this really what Path users wanted?

Now here is an example where the cost of failure is high. Path reportedly has at least 10 million users (though no one has a good sense of active users save Path itself.) Losing 10 million users is, well, a potentially costly risk. The thing is, I don't think Path needed to conduct a costly study to get some sense of that risk--all they needed to do was to release a Version 3.99 prior to version 4.0 that had ONE new feature: a short, one question survey that users had to click through to use the app. You'd only need to ask "How important to you is the fact that Path is capped at 150 users on a scale of 0-10" and count the 8s, 9s, and 10s. 

I don't think they did this with a representative population of Path users, or if they did, I missed it. All I know is that Path today is existentially different to the Path I signed up for, and I'm not sure it was a user-driven change. Time will tell. But in any case, Path represents a great example of an app that started with a low cost of failure and didn't need market research, and became one with a much greater cost of failure, that maybe could use some. It can be hard for a growing company to recognize that inflection point.

Would you recognize that point in your business?