Dose-Response Curves

Dose-Response Curves

One limitation of linear regression models, explains Judea Pearl in his book The Book of Why is that they are unable to accurately model interactions or relationships that don’t follow linear relationships. This lesson was hammered into my head by a statistics professor at the University of Nevada, Reno when discussing binomial variables. For variables where there are only two possible options, such as yes or no, a linear regression model doesn’t work. When the Challenger Shuttle’s O-ring failed, it was because the team had run a linear regression model to determine a binomial variable, the O-ring fails or it’s integrity holds. However, there are other situations where a linear regression becomes problematic.
 
 
In the book, Pearl writes, “linear models cannot represent dose-response curves that are not straight lines. They cannot represent threshold effects, such as a drug that has increasing effects up to a certain dosage and then no further effect.”
 
 
Linear relationship models become problematic when the effect of a variable is not constant over dosage. In the field of study that I was trained in, political science, this isn’t a big deal. In my field, simply demonstrating that there is a mostly consistent connection between ratings of trust in public institutions and receipt of GI benefits, for example, is usually sufficient. However, in fields like medicine or nuclear physics, it is important to recognize that a linear regression model might be ill suited to the actual reality of the variable.
 
 
A drug that is ineffective at small doses, becomes effective at moderate doses, but quickly becomes deadly at high doses shouldn’t be modeled with a linear regression model. This type of drug is one that the general public needs to be especially careful with, since so many individuals approach medicine with a “if some is good then more is better” mindset. Within physics, as was seen in the Challenger example, the outcomes can also be a matter of life. If a particular rubber for tires holds its strength but fails at a given threshold, if a rubber seal fails at a low temperature, or if a nuclear cooling pool will flash boil at a certain heat, then linear regression models will be inadequate for making predictions about the true nature of variables.
 
 
This is an important thing for us to think about when we consider the way that science is used in general discussion. We should recognize that people assume a linear relationship based on an experimental study, and we should look for binomial variables or potential non-linear relationships when thinking about a study and its conclusions. Improving our thinking about linear regression and dose-response curves can help us be smarter when it comes to things that matter like global pandemics and even more general discussions about what we think the government should or should not do.

A Bias Toward Complexity

A Bias Toward Complexity

When making predictions or decisions in the real world where there are many variables, high levels of uncertainty, and numerous alternative options to chose from, using a simple rule of thumb can be better than developing complex models for predictions. The intuitive sense is that the more complex our model the more accurately it will reflect the real complexity of the world, and the better job it will do with making a prediction. If we can see that there are multiple variables, then shouldn’t our model capture the different alternatives for each of those variables? Wouldn’t a simple rule of thumb necessarily flatten many of the alternatives for those variables, failing to take into consideration the different possibilities that exist? Shouldn’t a more complex model be better than a simple heuristic?

 

The answer to these questions is no. We are biased toward complexity for numerous reasons. It feels important to build a model that tries to account for every possible alternative for each variable, we believe that always having more information is always good, and we want to impress people by showing how thoughtful and considerate we are. Creating a model that accounts for all the different possibilities out there fits those preexisting biases. The problem, however, is that as we make our model more complex it becomes more unstable.

 

In Risk Savvy, Gerd Gigerenzer explains what happens with variance and our models by writing, “Unlike 1/N, complex methods use past observations to predict the future. These predictions will depend on the specific sample of observations it uses and may therefore be unstable. This instability (the variability of these predictions around their mean) is called variance. Thus, the more complex the method, the more factors need to be estimated, and the higher the amount of error due to variance.”  (Emphasis added by me – 1/N is an example of a simple heuristic that Gigerenzer explains in the book.)

 

Our bias toward complexity can make our models and predictions worse when high levels of uncertainty with many alternatives and relatively limited amounts of data exist. If we find ourselves in the opposite situation, where there is low uncertainty, few alternatives, and a plethora of data, then we can use very complex models to make accurate predictions. But when we are in the real world, like making stock market or March Madness predictions, then we should rely on a simple rule of thumb. The more complex our model, the more opportunities for us to misestimate a given variable. Rather than having one error be offset by numerous other point estimates within our model to reduce the cost of a miscalculation, our model ends up creating more variance and a greater likelihood that our prediction will be further from reality than if we had flattened the variables with a simple heuristic.
Value in Healthcare

Value in Healthcare

A common complaint about healthcare in the United States is that it has traditionally operated on a fee for service (FFS) based model. It is a natural and easy to understand system, and generally the type of system that both patients and providers prefer. The idea is that you pay for the services you receive from a healthcare provider. So if you need a tooth extracted, you go and have the tooth extracted and pay for the extraction. If you need a skin check, you go and get a skin check and pay for it. However, this FFS model can encourage a lot of waste through unnecessary medical procedures, and the value in healthcare is sometimes lost when we wait until someone has a problem before we help them with their health.

 

A lot of government programs, employers, and insurance companies are making efforts to push against FFS in an effort to provide greater value in the healthcare services we pay for, but it is worth asking, what is value and how can healthcare systems provide it? Is value just better health? Is it services that a patient said they were happy about? Is it care that saves a life or can it just be care that makes a life somewhat more comfortable? Dave Chase helps explain one aspect of value in healthcare in his book The Opioid Crisis Wake-Up Call, “Value is defined as the ratio of quality to cost. Value increases as the quality of the care increases or the cost of care decreases.”

 

FFS encourages short appointments where doctors cram as much as they can bill for into the shortest possible time before moving on to the next patient to do the same. Value based models, on the other hand, seek to improve the quality of the care provided without adding more costs to the patient and their insurer. As opposed to simply cramming in more tests, treatments, and procedures to get more money, value based systems that increase quality focus on improving health outcomes while keeping costs stable.

 

Alternatively, value based models might seek to keep quality the same, but reduce overall costs. This can wade into territory we don’t necessarily want to support, such as cutting nurse management staff to keep overhead low, but it could also look like more comprehensive care to reduce costly re-admissions after a procedure. When we think about value and try to build systems around value, we ultimately have to think about quality and cost, and how those are related. We can cut pieces out of the system that are just meant for signaling and cut pieces out that might be unnecessary without diminishing quality. But at the same time, we really need to examine whether the pieces we want to cut really do help with the quality of the care, especially over the long run.

 

Thinking about value in healthcare isn’t entirely new, but it is receiving increased focus, which is important if we want to have a healthcare system that people actually trust and are willing to engage with when necessary.