Prognosticating big games

Brian Burke’s explanation of his skepticism about his own Super Bowl predictions in a NY Times blog is the best thing I’ve come across over in this fortnight between the NFL conference championships and the Big Game. This is a period of time when the ratio of chatter-to-football approaches infinity. What’s most interesting is not the prediction (Colts in a close one), but his self-reflectiveness of the scientific limits and psychological pitfalls of single-event forecasting like this.

It is never clear to me what most pundits think they are doing when they make confident predictions of upcoming games. Of course, part of what they are doing is providing entertainment, often in simulated-debate format, where someone else is playing the role of confidently predicting the other side. Fair enough. But in fact, many pundits are confident. They give reasons for their predictions, and they seem to believe that of the thousands (millions?) of factors that can effect the outcome of the game — many of which are entirely unpredictable or random — the three or four factors they’ve cited provide sufficient confidence in a prediction. Even if one of those factors is something like “Smith is in a zone these days” or “Jones wants it more.”

There are many ways to explain what is going wrong in this kind of reasoning. There are very sophisticated models in the philosophy of social science that clarify the limits on our explaining events involving humans. But in this short blog post, Burke suggests another way we could usefully sharpen our own predictions; or at least be appropriately skeptical of seemingly sure bets. He alludes to a booming area of research in cognitive psychology over the past decade that has tried to catalogue “cognitive biases” to which almost all of us are susceptible. In the case of sports prognosticators,

many people overestimate how easy it is to pick straight-up winners. Much of our overconfidence is probably due to what psychologists call hindsight bias, the inclination to think that past events were more predictable than they really were. People also tend to whitewash their own predictions, recalling them as more accurate than they truly were.

And of course,

These lessons go far beyond sports. Experts are often no better than the rest of us in predictions. Sometimes they’re just skilled at explaining away or sweeping aside their past errors. People tend to seek out confident predictions to allay anxieties about the future. The overconfidence of experts, whether they’re in Congress, the Pentagon or on Wall Street, can be very costly for the rest of us.

“Hindsight bias,” however, is but one of a long list of cognitive biases that systematically distort the “order” and the “narrative” we try to impose on the unknowable and chancy elements in sports, politics, business, etc.

Now if this blog were a lot further off the ground that it is now — that is, it it had actual followers — there would be a prize for whoever could spot the most cognitive biases in play at ESPN and the NFL Network between now and the Super Bowl. But if you’re reading this in the first week of February 2010, you can certainly play along at home. Keep this list handy when you’re watching Around the Horn or PTI.