Understanding False Positives with Natural Frequencies

Understanding False Positives with Natural Frequencies

In a graduate course on healthcare economics a professor of mine had us think about drug testing student athletes. We ran through a few scenarios where we calculated how many true positive test results and how many false positive test results we should expect if we oversaw a university program to drug tests student athletes on a regular basis. The results were surprising, and a little confusing and hard to understand.

 

As it turns out, if you have a large student athlete population and very few of those students actually use any illicit drugs, then your testing program is likely to reveal more false positive tests than true positive tests. The big determining factors are the sensitivity of the test (how often it is actually correct) and the percentage of students using illicit drugs. A false positive occurs when the drug test indicates that a student who is not using illicit drugs is using them. A true positive occurs when the test correctly identifies a student who does indeed use drugs. The dilemma we discussed occurs if you have a test with some percentage of error and a large student athlete population with a minimal percentage of drug users. In this instance you cannot be confident that a positive test result is accurate. You will receive a number of positive tests, but most of the positive tests that you receive are actually false positives.

 

In class, our teacher walked us through this example verbally before creating some tables that we could use to multiply the percentages ourselves to see that the number of false positives will indeed exceed the number of true positives when you are dealing with a large population and a rare event that you are testing for. Our teacher continued to explain that this happens every day in the medical world with drug tests, cancer screenings, and other tests (including COVID-19 tests as we are learning today).  The challenge, as our professor explained, is that the math is complicated and it is hard to explain to person who just received a positive cancer test that they likely don’t have cancer, even though they just received a positive test. The statistics are hard to understand on their own.

 

However, Gerd Gigerenzer doesn’t think this is really a limiting problem for us to the extent that my professor had us work through. In Risk Savvy Gigerenzer writes that understanding false positives with natural frequencies is simple and accessible. What took nearly a full graduate course to go through and discuss, Gigerenzer suggests can be digested in simple charts using natural frequencies. Natural frequencies are numbers we can actually understand and multiply as opposed to fractions and percentages which are easy to mix up and hard to multiply and compare.

 

Rather than telling someone that the actual incidence of cancer in the population is only 1%, and that the chance of a false positive test is 9%, and trying to convince them that they still likely don’t have cancer is confusing. However, if you explain to an individual that for every 1,000 people who take a particular cancer test that only 10 actually have cancer and that 990 don’t, the path to comprehension begins to clear up. With the group of 10 true positives and true negatives 990, you can explain that of those 10 who do have cancer, the test correctly identifies 9 out of 10 of them, and provides 9 true positive results for every 1,000 test (or adjust according to the population and test sensitivity). The false positive number can then be explained by saying that for the 990 people who really don’t have cancer, the test will error and tell 89 of them (9% in this case) that they do have cancer. So, we see that 89 individuals will receive false positives while 9 people will receive true positives. 89 > 9, so the chance of actually having cancer with a positive test still isn’t a guarantee.

 

Gigernezer uses very helpful charts in his book to show us that the false positive problem can be understood more easily than we might think. Humans are not great at thinking statistically, but understanding false positives with natural frequencies is a way to get to better comprehension. With this background he writes, “For many years psychologists have argued that because of their limited cognitive capacities people are doomed to misunderstand problems like the probability of a disease given a positive test. This failure is taken as justification for paternalistic policymaking.” Gigerenzer shows that we don’t need to rely on the paternalistic nudges that Cass Sunstein and Richard Thaler encourage in their book Nudge. He suggest that in many instances where people have to make complex decisions what is really needed is better tools and aids to help with comprehension. Rather than developing paternalistic policies to nudge people toward certain behaviors that they don’t fully understand, Gigerenzer suggests that more work to help people understand problems will solve the dilemma of poor decision-making. The problem isn’t always that humans are incapable of understanding complexity and choosing the right option, the problem is often that we don’t present information in a clear and understandable way to begin with.
Can We Employ Simple Health Nudges?

Can We Employ Simple Health Nudges?

In their book Nudge, Cass Sunstein and Richard Thaler write, “Libertarian paternalists see countless opportunities for improving people’s health. Social influences could obviously be enlisted: if most people think that most people are starting to avoid unhealthy foods, or to exercise, more people will avoid unhealthy foods and will exercise.” The book was published in 2008, and while the authors imagined many ways in which nudges could make a big impact for the health of individuals and populations, few nudges seem to be making an impact in the US today. The lack of successful nudges, and the health challenges of the last few years raise the question, can we employ simple health nudges to solve our problems?

 

The COVID-19 pandemic has shown us how hard it is to adopt simple healthcare practices in the United States. Nudges, like signs, reminders, and commercials about preventing airborne transmission of the virus through the use of masks doesn’t seem to be as effective as we would like. It has often taken mask mandates and fines for business to compel people to actually wear masks. Nudges, in the case of encouraging mask wearing in the face of a deadly pandemic and highly transmissible disease seemed to be ineffective.

 

Before the COVID-19 pandemic, two public health ideas that were being tested were limiting the size of sodas that people could purchase at restaurants and convenience stores and taxing sugary drinks. I’m not sure if Sunstein and Thaler would consider bans on overly large soda cups or taxes on sugary beverages as nudges, but I think they count. No one was limiting the number of sodas an individual could buy, and the taxes on sugary drinks were very low. The idea behind each measure was to marginally reduce some sugary beverage consumption, hopefully helping people reduce their caloric intake and improve their dental health. But even these small measures were met with fierce backlash. Very few people would really be impacted by the limited sizes of large soda cups, and few people would meaningfully feel the price of the soda taxes, but both measures were attacked and only a few places were actually able to pass such measures. If such limited actions are met with such strong resistance, then it doesn’t seem like we can rely on nudges that will meaningfully move people toward more healthy lives.

 

Sunstein and Thaler also write about social influencers as being important in nudging people toward diets and exercise, but in the years since 2008, social influencers have been less successful at encouraging diets than they have been at getting people to take cool pictures wearing athleisure wear. Body positivity movements have possibly encouraged people to be more accepting of non-model/Avenger body shapes, rather than encouraging them to spend more time at the gym and eat more salads. I think it is a healthy movement, but the nudge of body positivity movements are not tied to the same health goals that are written about in the book. From my perspective, it seems that there are larger structural issues that shape and limit our exercising and influence our diets beyond what nudges can hope to influence.

 

While I wish we could employ simple health nudges to improve individual and population health, I don’t think it is possible. We have trouble communicating the effectiveness of masks and encouraging people to wear masks during a global pandemic, and people will fight against marginal measures to limit soda consumption. Encouraging more exercise and getting people to eat healthier requires action beyond what a nudge can do, and require real structural changes to the systems and incentives that create our current health problems. Beyond nudges, we need larger creative solutions that will truly change people’s behavior.
Constructive Thoughts on Wellness

Constructive Thoughts on Wellness

There is an argument in the world of public health that the American medical system is too focused on solving problems rather than preventing problems. This argument that is presented in Sam Quinones’ book Dreamland, expressed by Dr. Alex Cahana, “The U.S. medical system is good at fighting disease, … and awful at leading people to wellness.”

 

The difference between fighting disease and leading people to wellness has to do with where you step in to help with people’s health. Our country generally focuses on providing medical care and attention after someone has gotten sick. We ask doctors, nurses, and medical professionals to correct a huge range of problems, many of which stem from bad habits, unhealthy environmental factors, and conditions that are generally beyond the control of an individual, and not open to medical interventions. Attacking the problem once it has already developed, once a set of factors have set in that promote the health problem, makes any real changes expensive and difficult.

 

Wellness requires that we think about medical care, costs, and health further upstream, before anyone ever gets sick. Consider the idea of wellness in the context of car maintenance (I know, I know, I just wrote about the problems with comparing ourselves to cars, but this will be helpful).

 

If you regularly change your oil, rotate your tires, and drive as if your grandma was in the car with you, then your vehicle is going to operate more smoothly with fewer major costs (in general) throughout its entire life. You are making small interventions along the way to make sure your car is operating optimally. The costs of changing your oil and putting in the necessary effort to keep it working well are not trivial, but we know that those costs are less than what we might face otherwise.

 

Failing to maintain our vehicle could lead to a catastrophic engine failure. Driving our car like a teenager that just downed two Redbulls is going to put a lot of strain on the vehicle, wearing out our tires and breaks much faster. When things wear our quicker, when unexpected failures occur, we suddenly have to pay a lot more money to keep the car going.

 

Our bodies are similar, and whether it is our national Medicaid or Medicare systems, or our private health insurance systems, the cost we pay for healthcare is interconnected with where we step in to try to make people healthy. Paying for interventions downstream, once we already have health problems is expensive. It is equivalent waiting until our human check engine lights turn on before we consider doing anything to help our health. The solution that many medical professionals and many public health researchers encourage is moving upstream from the actual health problems that develop to focus on interventions before anyone develops terrible disease. The idea is to focus on wellness first, and hope we don’t have to pay for as much medical care for the prohibitively expensive diseases down the road. Rather than focusing all our effort on solving disease, we can redirect some of the money and effort into improving our environments, finding new ways to help people adopt healthy lifestyles, and finding more ways to connect and help us share in wellness as a community.