Looking forward and making predictions and judgments on what will happen in the future is incredibly difficult, and very few people can reliably make good predictions about what will happen. But when we look to the past, almost all of us can describe what did happen. It is easy to look back and see how a series of events unfolded, to make connections between certain conditions and eventual outcomes, and to be confident that we understood why things unfolded as they did. But this confidence is misleading and is something we can reliably expect from people.
The hindsight fallacy is the term which describes our overconfidence in describing what happened in the past and determining which causal factors influenced the outcomes we observed. When the college football playoff is over this year, sports commentators will have a compelling narrative as to why the winning team was able to pull through. When the stock market makes a jump or dip in the next year, analysts will be able to look backward to connect the dots that caused the rise or fall of the market. Their explanations will be confident and narratively coherent, making the analysts and commentators sound like well reasoned individuals.
However, “every point in history is a crossroads,” writes Yuval Noah Harari. Strange and unpredictable things could happen at any time in history, and the causal factors at work are hard to determine. It is worth remembering that the best social science studies return an R value of about .4 at most (the R value is a statistical value reflecting how well the model fits reality). This means that the best social science studies we can conduct barely reflect the reality of the world. It is unlikely that any commentator, even a seasoned football announcer or stock market analyst, really understands causality well enough to be confident in what caused what, even in hindsight. Major shifts could happen because someone was in a bad mood. Unexpected windfalls could create new and somewhat random outcomes. Humans can think causally, and this helps us better understand the world, but we can also be overconfident in our causal reasoning.
Harari continues, “the better you know a particular historical period, the harder it becomes to explain why things happened one way and not another. Those who have only a superficial knowledge of a certain period tend to focus only on the possibility that was eventually realized.” What Harari is saying in this quote is that we can get very good at describing how things happened in the past, but not exactly very good at describing why. We can look at each step and each development that unfolded ahead of a terrorist attack, a win by an army, or as Harari uses for demonstration in his book, the adoption of Christianity in the Roman Empire. But we can’t always explain the exact causal pathway of each step. If we could, then we could identify the specific historical crossroads where history took one path and not another and make reasonable predictions about how the world would have looked had the alternative option been the one that history followed. But we really can’t do this. We can look back and identify factors that seemed important in the historical development, but we can’t always explain exactly why those factors were important in one situation relative to another. There is too much randomness, too much chance, and too much complexity for us to be confident in the causal pathways we see. We won’t stop thinking in a causal way, of course, but we should at least be more open to a wild range of possibilities, and less confident in our assessments of history.
One of my favorite examples of the hindsight bias in action is in Good to Great by Jim Collins. In the book published in 2001, Collins identifies 11 companies that had jumped form being good companies to great companies. One of the companies identified, Circuit City, was out of business before Collins published his subsequent book. Another, Wells Fargo, is now one of the most hated companies in the United States. A third, Fannie Mae, was at the center of the 2008 financial crisis, and a fourth, Gillette, was purchased by P&G and is no longer an independent entity. A quick search suggests that the companies in the Good to Great portfolio have underperformed the market since the books publication. It is likely that the success of the 11 companies included a substantial amount of randomness, which Collins and his team failed to incorporate in their analysis. Hindsight bias was at play in the selection of the 11 companies and the explanation for why they had such substantial growth in the period that Collins explored.
Novel Learning 