The myth of the “crystal ball” is not a new topic for this blog, but the election brought it back into stark focus. Intuition and instinct are crucial components to business success—indeed, there is no such thing as “perfect” information, and no amount of data, research or corroborating evidence will ever give you 100% decision support. If it does, you waited too long to make the decision.
Still, I often see intuition and data pitted against each other, like Sharks and Jets. Business fables celebrate the folks with vision who had the courage to buck the trends and succeed despite the odds—to skate where the puck is going, to quote The Great One. You don’t see so many Malcolm Gladwell books about the quiet analysts who prove their dashing, risk-taking CEO’s wrong, because those don’t make good stories, I guess (I’d read the hell out of them, though).
This year’s U.S. Election, was, however, as I wrote in part one of this series, a bit of a triumph for the number crunchers. In nearly every election I can recall, some political consultant or strategist emerges as the “hero” of a campaign. You know the names: Carville. Begala. Noonan. Trippi. Rove. Each justifiably praised for making the right moves, and having the more accurate crystal ball.
This year was different. For the first time that I can recall, the emergent election hero was not an “operative,” strategist or consultant; rather, it was Nate Silver, who served as the popular face for a number of quantitative analysts who successfully predicted the outcome of the election based upon polls. What made Silver’s “predictions” accurate? A better reading of the present. In short, not a crystal ball—but a clear-eyed reading of existing data.
One of the central “data vs. instinct” conflicts of this election was the battle between pollsters who based their models on Party ID data (which indicated that there was a larger base of Democrats than Republicans) and those who simply did not believe that data, and believed instead that the two parties were roughly equal.
As it turned out, the pollsters who modeled their findings on Party ID did a lot better than those pollsters and pundits who tried to unsuccessfully “unskew” the polls based upon their intuition—and NOT data. In hindsight, it was a brutally simple distinction—one side read the cards they were dealt, while another tried to predict where the cards should be.
There’s a real discipline involved in looking at data and apprising it fully for what it is, and not jumping directly to what it could or should be. It’s that skill—the reliable discernment of the present—that often trumps the prediction of the future, though the futurists get all the plumb speaking gigs.
With this election, though, we may be seeing a little balance return to The Force. On election night, after one of the news networks called Ohio for Obama, a pundit on that same network gazed into his own crystal ball and disputed the decision. To their endless credit, the network in question literally brought their cameras into the room where their quantitative team was busily crunching the numbers they were getting from Edison and from the vote count, and put the head of their Decision Team on camera, where he calmly explained exactly how that call was made, and exactly why they could be secure in making it.
For we few, we band of brothers—we pollsters, it was our equivalent of St. Crispin's day (and kudos to the network in question for making that process transparent!) But the lesson here is this—when you lack data, then intuition and instinct are essential. And when you have conflicting data, it’s often those same instincts that separate the visionary from the failure. But when you have good data, and it largely agrees, recognizing when your crystal ball is wrong and acting accordingly is a far more practical and valuable business skill to hone.
Business fables are built upon celebrating the outliers and exceptions—and there is a pretty clear survivor bias around that data. The mark of leadership is sometimes not the ability to ignore the data and damn-the-torpedoes towards a vision of the future. Sometimes, it’s the ability to recognize the data for what it is, turn on a dime, and pivot accordingly. Often, in hindsight, you’ll end up looking psychic anyway.