Over the past few weeks, two events took place that had a greater than 99% chance of not taking place. In the first instance, Bernie Sanders won the Michigan Democratic Primary, despite an array of pre-election polls showing Secretary Clinton leading by as much as 37 points. Noted polling aggregator Nate Silver extrapolated those polls into a “probability” of a Clinton victory: greater than 99%.
The second instance involved the recent NCAA Men’s Basketball Tournament game between Northern Iowa and Texas A&M. Northern Iowa lost this game despite being up by 12 points with just 35 seconds remaining. Once again, the “probability calculators” were brought into play, and with the game almost in hand, Northern Iowa was given a 99.99% chance of winningInstead, Texas A&M staged (wait for it…) an “improbable” comeback.
I’m a little disturbed by the sudden creep of the “probability calculator” into everyday parlance. Probabilities are a part of my job, so I think I have a pretty good understanding of how they work. A coin has two sides. The chances that a given coin flip will turn up tails is 1 in 2, or 50%. Add a third side to that coin, and the probability becomes 1 in 3, or 33%, and you have a most curious coin indeed.
Of course, you could flip 15 heads or 7 tails in a row. It’s unlikely, but possible. A probability distribution, however, would hold that over time, you’ll flip as many heads as you do tails as long as the coin is fair. A coin is a coin. But here is where you can get into trouble with lazy thinking: say I flip the coin 9 times — all heads. What’s the chance that the next flip will be tails? The wrong answer is “It’s gotta be tails — tails is due!” (that won’t get you asked back for that second interview at Edison.) The sorta right answer is “it doesn’t matter — probability is independent of previous results.” In other words, no matter how many times you flipped heads, the chance of flipping tails on the next turn is still 50%. (The clever answer, by the way, is to predict another “heads.” Not strictly correct, but that coin sounds like it’s loaded.)
Let’s return to our earlier examples for a moment. Both events — wins by Clinton and Northern Iowa — were assigned a probability of greater than 99%. But what are those “probabilities” based on? Past events, not the hard, fixed sides of a coin. In the case of the Michigan primary, we had a series of polls showing that Clinton was leading at the time of each poll. Polls are great for telling you what is. At some point, however, we became consumed with taking all of that information about what is, and turning it into a projection of what will be. This isn’t the same thing as a coin flip, because people aren’t coins.
We’ve fallen in love with these “prediction markets,” but consider this: FiveThirtyEight.com aggregated all of those polls into an estimated 20-point win for Clinton, and assigned that win a >99% probability. Clinton did not win. So what happens in 2020 when the Michigan primary again shows a 20 point lead for a candidate going into the primary? Will that also be assigned a >99% probability? Will the next team down 12 with 35 seconds left be assigned a 99.99% lock to win?
What I am getting at here is that when events like the Sanders win in Michigan, or Texas A&M’s comeback occur, the “predictors” talk about how improbable these events were. But isn’t it equally valid to question the “probability calculator” itself? It’s easier for me to believe that Clinton didn’t actually have a 99% chance of winning, than to believe Sanders pulled off a 1-in-10,000 upset.
I’m not a forecaster, and that’s a badge I do not choose to wear. As a pollster, I think our brethren and sistren do a pretty good job of describing the current landscape in any given situation — and, Michigan aside, the pre-election polls have been pretty good this year. But taking all of these snapshot polls and turning them into “prediction percentages” is a cottage industry whose utility I fail to see. I feel the same way about this practice as I do about timing the stock market vs. playing the index over the long haul. Like every investment prospectus says, past results are not indicative of future performance.
Anyway, put the calculator down and watch the damn game. It’s more fun that way.