In the past, I have encouraged attaching probabilities and statistical chances to the things we believe or to events we think may (or may not) occur. For example, say Steph Curry’s three point shooting percentage is about 43%, and I am two Steph Currys confident that my running regiment will help me qualify for the Boston Marathon. One might also be two Steph Currys confident that leaving now will guarantee they are at the theater in time for the movie, or that most COVID-19 restrictions will be rescinded by August 2021 allowing people to go to movies again. However, the specific percentages that I am attaching in these examples may be meaningless, and may not really convey an important message for most people (Myself included!). It turns out, that modern day statistics and the messaging attached to it is not well understood.
In his book Risk Savvy, Gerd Gigerenzer discusses the disconnect between stats and messaging, and the mistake most people make. The main problem with using statistics is that people don’t really know what the statistics mean in terms of actual outcomes. This was seen in the 2016 US presidential election when sources like FiveThirtyEight gave trump a 28.6% chance of winning and again in 2020 when the election was closer than many predicted, but was still well within the forecasted range. In both instances, a Trump win was considered such a low probability event that people dismissed it as a real possibility, only to be shocked when Trump did win in 2016 and performed better than many expected in 2020. People failed to fully appreciate that FiveThirtyEight’s prediction meant that in 28.6% of election simulations, Trump was predicted to win in 2016, and in 2020 many of their models predicted races both closer than and wider than the result we actually observed.
Regarding weather forecasting and statistical confusion, Gigerenzer writes, “New forecasting technology has enabled meteorologists to replace mere verbal statements of certainty (it will rain tomorrow) or chance (it is likely) with numerical precision. But greater precision has not led to greater understanding of what the message really is.” Gigerenzer explains that in the context of weather forecasts, people often misunderstand that a 30% chance of rain means that on 30% of days when when the observed weather factors (temperature, humidity, wind speeds, etc…) match the predicted weather for that day, rain occurs. Or that models taking weather factors into account simulated 100 days of weather with those conditions and included rain for 30 of those days. What is missing, Gigerenzer explains, is the reference class. Telling people there is a 30% chance of rain could lead them to think that it will rain for 30% of the day, that 30% of the city they live in will be rained on, or perhaps they will misunderstand the forecast in a completely unpredictable way.
Probabilities are hard for people to understand, especially when they are busy, have other things on their mind, and don’t know the reference class. Providing probabilities that don’t actually connect to a real reference class can be misleading and unhelpful. This is why my suggestion of tying beliefs and possible outcomes to a statistic might not actually be meaningful. If we don’t have a reasonable reference class and a way to understand it, then it doesn’t matter how many Steph Currys likely I think something is. I think we should take statistics into consideration with important decision-making, and I think Gigerenzer would agree, but if we are going to communicate our decisions in terms of statistics, we need to ensure we do so while clearly stating and explaining the reference classes and with the appropriate tools to help people understand the stats and messaging.
Novel Learning 