Asking Better Questions

IStock 000017421813SmallDuring my recently completed Blogworld keynote, I sketched out four steps towards turning data into insight, instead of chartjunk. I'll post the slides soon, but to recap, my four steps were as as follows:

  1. Know What You Don't Know.
  2. Ask Better Questions.
  3. Prove Yourself Wrong.
  4. Do Your Own Work.

I was pleased to see these themes also raised throughout Blogworld (particularly in Amber Naslund's fine keynote, and a fantastic talk by kindred soul Jason Falls.) Specifically, the theme of asking better questions really seemed to strike a chord, and it's been gratifying to see so many of the post-conference tweets and articles return to that point, which was the central focus of my talk.

It's not difficult to ask better questions - often, it simply a matter of asking the wrong questions first, but remaining curious. If you continue to seek to disconfirm, and not to confirm, you will eventually be led to the right questions - and that is the key to insight.

I've picked on the recent spate of data-dredging research enough, so I won't continue to flog a very dead horse. What makes these sorts of studies (i.e., the "best day/time to tweet," and their ilk) so pernicious is that they are resolutely incurious. "What is the best time to post a press release" is the wrong question, or at least it's a question that puts the cart well before the horse.

No, the better question is this: does day/time of posting have any effect upon the success of your messaging? Data dredging will never give you this answer. The only way to determine the "if" (before you can tackle the "then") is to conduct a series of controlled experiments, eliminating or mitigating other variables, to determine just how much (if anything) day, time, message length, etc. all contribute to the overall variance of the success of your communication. If it's significant, then and only then can you go about the business of determining which day or time to post is right for you. You know, do your own work.

Saying, by the way, that this sort of data dredgery is a "starting point" is a bit of a cop-out. That implies that while the "optimal" tweet is, say, 130 characters, yours might be different, so you need to sift through your data as well. Sounds reasonable, doesn't it? But, again, that seemingly benign point makes the assumption that the hypothesis has been "proven." It hasn't. The right questions haven't even been asked. Using it as a starting point does not mean to determine which day, time or length of tweet is right for you. No, it means that your next step should be to try and prove it wrong.

It's important to realize this, as well: data generated for the purposes of content creation is inherently incurious.

You know what it's like - you have a blog, and some sort of editorial calendar (or at least an internal voice that tells you "I need a post by 5:00!") Data for the purposes of content creation can't afford to keep asking better questions. It's not evil, per se, but it stops when it gets to the first answer, because the "answer" is the generation of an article, a chart, or an infographic. The goal is to create content, not to keep uncovering rocks on what might end up as a wild-data-goose chase.

Data for the purposes of content generation is incurious because it seeks to prove or show something, and not to learn something. If you seek to confirm something, you probably will - it's an ancient and sinister bias amongst humans. Finding the real truth is a painstaking process of disconfirmation. You have a hypothesis, and you seek to prove it wrong. Eventually, you will reach a point where you either do, or you cannot. In either case, you have reached a truth. This is what I mean when I cite my constant refrain to do the work.

Finally, friends who know me know that I don't use the word "incurious" lightly, or academically. To me, it's a vulgarity - the research equivalent of the F-bomb. Please don't be incurious. Curiosity is not cynicism, it's skepticism. Skeptics ask better questions. Do so with regularity, and better answers are assured.