Month: May 2021

Ignorability

Ignorability

Posted on by jabittan
The idea of ignorability helps us in science by playing a role in randomized trials. In the real world, there are too many potential variables to be able to comprehensively predict exactly how a given intervention will play out in every case. We almost always have outliers that have wildly different outcomes compared to what we would have predicted. Quite often some strange factor that could not be controlled or predicted caused the individual case to differ dramatically from the norm.
Thanks to concepts of ignorability, we don’t have to spend too much time worrying about the causal structures that created a single outlier. In The Book of Why Judea Pearl tries his best to provide a definition of ingorability for those who need to assess whether ignorability holds in a given outlier decision. He writes, “the assignment of patients to either treatment or control is ignorable if patients who would have one potential outcome are just as likely to be in the treatment or control group as the patients who would have a different potential outcome.”
What Pearl means is that ignorability applies when there is not a determining factor that makes people with any given outcome more likely to be in a control or treatment group. When people are randomized into control versus treatment, then there is not likely to be a commonality among people in either group that makes them more or less likely to have a given reaction. So a random outlier in one group can be expected to be offset by a random outlier in the other group (not literally a direct opposite, but we shouldn’t see a trend of specific outliers all in either treatment or control).
Ignroability does not apply in situations where there is a self-selection effect for control or treatment. In the world of the COVID-19 Pandemic, this applies in situations like human challenge trials. It is unlikely that people who know they are at risk of bad reactions to a vaccine would self-select into a human challenge trial. This same sort of thing happens with corporate health benefits initiatives, smart phone beta-testers, and general inadvertent errors in scientific studies. Outliers may not be outliers we can ignore if there is a self-selection effect, and the outcomes that we observe may reflect something other than what we are studying, meaning that we can’t apply ignorability in a way that allows us to draw a conclusion specifically on our intervention.
Co-opting Mental Machinery

Co-opting Mental Machinery

Posted on by jabittan
The human mind is great at pattern recognition, but it is not the only brain that can recognize a pattern. Pigeons can recognize patterns for food distribution with button presses, mice can remember mazes and navigate through complex patterns to a reward, and other animals can recognize patterns in hunting, mating, and other activities. What humans do differently is use pattern recognition to determine causal structures by imagining and testing alternative hypotheses. This is a crucial step beyond the pattern recognition of other animals.
In The Book of Why Judea Pearl writes, “It is not too much of a stretch to think that 40,000 years ago, humans co-opted the machinery in their brain that already existed for pattern recognition and started to use it for causal reasoning.” This idea is interesting because it explains our pattern recognition linkage with other animals and helps us think about how brain structures and ways of thinking may have evolved.
In isolation, a brain process is interesting, but not as interesting as when considered alongside similar brain processes. When we look at pattern recognition and its similarities to causal reasoning, we see a jumping off point. We can see how brain processes that helped us in one area opened up new possibilities through development. This helps us think more deeply about the mental abilities that we have.
The ways we think and how our brains work is not static. Different cultural factors, environmental factors, and existing brain processes can all shape how our brains work and evolve individually and as a species.  As Pearl notes, it is likely that many of our brain processes co-opted other mental machinery for new purposes. Very few of what see in human psychology can be well understood in isolation. Asking why and how evolution could have played a role is crucial to understanding who we are now and how we got to this point. Causality is not something that just existed naturally in the brain. It was built by taking other processes and co-opting them for new purposes, and those new purposes have allowed us to do magnificent things like build rockets, play football, and develop clean water systems.
The Representation Problem

The Representation Problem

Posted on by jabittan
In The Book of Why Judea Pearl lays out what computer scientists call the representation problem by writing, “How do humans represent possible worlds in their minds and compute the closest one, when the number of possibilities is far beyond the capacity of the human brain?”
 
 
In the Marvel Movie Infinity War, Dr. Strange looks forward in time to see all the possible outcomes of a coming conflict. He looks at 14,000,605 possible futures. But did Dr. Strange really look at all the possible futures out there? 14 million is a convenient big number to include in a movie, but how many possible outcomes are there for your commute home? How many people could change your commute in just the tiniest way? Is it really a different outcome if you hit a bug while driving, if you were stopped at 3 red lights and not 4, or if you had to stop at a crosswalk for a pedestrian? The details and differences in the possible worlds of our commute home can range from the miniscule to the enormous (the difference between you rolling your window down versus a meteor landing in the road in front of you). Certainly with all things considered there are more than 14 million possible futures for your drive home.
 
 
Somehow, we are able to live our lives and make decent predictions of the future despite the enormity of possible worlds that exist ahead of us. Somehow we can represent possible worlds in our minds and determine what future world is the closest one to the reality we will experience. This ability allows us to plan for retirement, have kids, go to the movies, and cook dinner. If we could not do this, we could not drive down the street, could not walk to a neighbors house, and couldn’t navigate a complex social world. But none of us are sitting in a green glow with our head spinning in circles like Dr. Strange as we try to view all the possible worlds in front of us. What is happening in our mind to do this complex math?
 
 
Pearl argues that we solve this representation problem not through magical foresight, but through an intuitive understanding of causal structures. We can’t predict exactly what the stock market is going to do, whether a natural disaster is in our future, or precisely how another person will react to something we say, but we can get a pretty good handle on each of these areas thanks to causal reasoning.
 
 
We can throw out possible futures that have no causal structures related to the reality we inhabit.  You don’t have to think of a world where Snorlax is blocking your way home, because your brain recognizes there is no causal plausibility of a Pokémon character sleeping in the road. Our brain easily discards the absurd possible futures and simultaneous recognizes the causal pathways that could have major impacts on how we will live. This approach gradually narrows down the possibilities to a level where we can make decisions and work with a level of information that our brain (or computers) can reasonably decipher. We also know, without having to do the math, that rolling our window down or hitting a bug is not likely to start a causal pathway that materially changes the outcome of our commute home. The same goes for being stopped at a few more red lights or even stopping to pick up a burrito. Those possibilities exist, but they don’t materially change our lives and so our brain can discard them from the calculation. This is the kind of work our brains our doing, Pearl would argue, to solve the representation problem.

Objective Reality, Rationality, & Shared Worlds - Joe Abittan

Objective Reality, Rationality, & Shared Worlds

Posted on by jabittan
The idea of an objective reality has been under attack for a while, and I have even been part of the team attacking that objective reality. We know that we have a limited ability to sense and experience the world around us. We know that bats, sharks, and bees experience phenomena that we are blind to. We can’t know that the color red that I experience is exactly like the color red that you experience. Given our lack of sense, the fact that physical stimuli are translated into electrical brain impulses, and that there appears to be plenty of subjectivity in how we experience the same thing, an objective reality doesn’t really seem possible. We seemingly all live within a world created by many subjective measures within our own brains.
But is this idea really accurate? I recently completed Steven Pinker’s book Enlightenment Now in which he argues that reason depends on objectivity and that our efforts toward rationality and reason demonstrate that there is some form of objectivity toward which we are continually working. The very act of attempting to think rationally about our world and how we understand the universe demonstrates that we are striving to understand some sort of objective commonality. A quote from The Book of Why by Judea Pearl seems to support Pinker’s assertion. Pearl writes:
“We experience the same world and share the same mental model of its causal structure. … Our shared mental models bind us together into communities. We can therefore judge closeness not by some metaphysical notion of similarity but by how much we must take apart and perturb our shared model before it satisfies a given hypothetical condition that is contrary to fact.”
Pearl wrote this paragraph while discussing the human ability to imagine alternative possibilities (specifically writing about the sentence Joe’s headache would have gone away if he had taken aspirin). The sentence acknowledges a reality (Joe has a headache) and proposes a different reality that doesn’t actually exist (Joe no longer has a headache because he took aspirin). It is this ability to envision different worlds which forms the basis of our causal interpretations of the world, but it also reveals a shared world in which we live and from which we can imagine different possible worlds. It hints at an objective reality shared among individuals and distinct from unreal and imagined, plausible worlds.
Reason and rationality demonstrate that there seems to be an objective reality in which we are situated and in which we experience the world. There are undoubtedly subjective aspects of that world, but we nevertheless are able to share a world in which we can imagine other possible worlds and consider those alternative worlds as closer or further from the world in which we live. Doing this over and over again, among billions of people, helps us define the actual objective reality which constitutes the world we share and from which we have subjective experiences. It is from this world that we can discuss what is subjective, what causes one phenomenon or another, and from which we can imagine alternative realities based on certain interventions. If there was no objective reality for us to all share, then we would never be able to distinguish alternative worlds and compare them as more or less close to the world we share and exist within.
Alternative, Nonexistent Worlds - Judea Pearl - The Book of Why - Joe Abittan

Alternative, Nonexistent Worlds

Posted on by jabittan
Judea Pearl’s The Book of Why hinges on a unique ability that human animals have. Our ability to imagine alternative, nonexistent worlds is what has set us on new pathways and allowed us to dominate the planet. We can think of what would happen if we acted in a certain manner, used a tool in a new way, or if two objects collided together. We can visualize future outcomes of our actions and of the actions of other bodies and predict what can be done to create desired future outcomes.
In the book he writes, “our ability to conceive of alternative, nonexistent worlds separated us from our protohuman ancestors and indeed from any other creature on the planet. Every other creature can see what is. Our gift, which may sometimes be a curse, is that we can see what might have been.”
Pearl argues that our ability to see different possibilities, to imagine new worlds, and to be able to predict actions and behaviors that would realize that imagined world is not something we should ignore. He argues that this ability allows us to move beyond correlations, beyond statistical regressions, and into a world where our causal thinking helps drive our advancement toward the worlds we want.
It is important to note that he is not advocating for holding a belief and setting out to prove it with data and science, but rather than we use data and science combined with our ability to think causally to better understand the world. We do not have to be stuck in a state where we understand statistical techniques but deny plausible causal pathways. We can identify and define causal pathways, even if we cannot fully define causal mechanisms. Our ability to reason through alternative, nonexistent worlds is what allows us to think causally and apply this causal reasoning to statistical relationships. Doing so, Pearl argues, will save lives, help propel technological innovation, and will push science to new frontiers to improve life on our planet.
Causal Illusions

Causal Illusions

Posted on by jabittan
In The Book of Why Judea Pearl writes, “our brains are not wired to do probability problems, but they are wired to do causal problems. And this causal wiring produces systematic probabilistic mistakes, like optical illusions.” This can create problems for us when no causal link exists and when data correlate without any causal connections between outcomes. According to Pearl, our causal thinking, “neglects to account for the process by which observations are selected.” We don’t always realize that we are taking a sample, that our sample could be biased, and that structural factors independent of the phenomenon we are trying to observe could greatly impact the observations we actually make.
Pearl continues, “We live our lives as if the common cause principle were true. Whenever we see patterns, we look for a causal explanation. In fact, we hunger for an explanation, in terms of stable mechanisms that lie outside the data.” When we see a correlation our brains instantly start looking for a causal mechanism that can explain the correlation and the data we see. We don’t often look at the data itself to ask if there was some type of process in the data collection that lead to the outcomes we observed. Instead, we assume the data is correct and  that the data reflects an outside, real-world phenomenon. This is the cause of many causal illusions that Pearl describes in the book. Our minds are wired for causal thinking, and we will invent causality when we see patterns, even if there truly isn’t a causal structure linking the patterns we see.
It is in this spirit that we attribute negative personality traits to people who cut us off on the freeway. We assume they don’t like us, that they are terrible people, or that they are rushing to the hospital with a sick child so that our being cut off has a satisfying causal explanation. When a particular type of car stands out and we start seeing that car everywhere, we misattribute our increased attention to the type of car and assume that there really are more of those cars on the road now. We assume that people find them more reliable or more appealing and that people purposely bought those cars as a causal mechanism to explain why we now see them everywhere. In both of these cases we are creating causal pathways in our mind that in reality are little more than causal illusions, but we want to find a cause to everything and we don’t always realize that we are doing so.
Regression Coefficients

Regression Coefficients

Posted on by jabittan
Statistical regression is a great thing. We can generate a scatter plot, generate a line of best fit, and measure how well that line describes the relationship between the individual points within the data. The better the line fits (the more that individual points stick close to the line) the better the line describes the relationships and trends in our data. However, this doesn’t mean that the regression coefficients tell us anything about causality. It is tempting to say that a causal relationship exists when we see a trend line with lots of tight fitting dots around and two different variables on an X and Y axis, but this can be misleading.
In The Book of Why Judea Pearl writes, “Regression coefficients, whether adjusted or not, are only statistical trends, conveying no causal information in themselves.” It is easy to forget this, even if you have had a statistics class and know that correlation does not imply causation. Humans are pattern recognition machines, but we go a step beyond simply recognizing a pattern, we instantly set about trying to understand what is causing the pattern. However, our regression coefficients and scatter plots don’t always hold clear causal information. Quite often there is a third hidden variable that cannot be measured directly that is influencing the relationship we discover in our regression coefficients.
Pearl continues, “sometimes a regression coefficient represents a causal effect, and sometimes it does not – and you can’t rely on the data alone to tell you the difference.” Imagine a graph with a regression line running through a plot of force applied by a hydraulic press and fracture rates for ceramic mugs. One axis may be pressure, and the other axis may be thickness of the ceramic mug. The individual points represent the point at which individual mugs fractured We would be able to generate a regression line by testing the fracture strength of mugs of different thickness, and from this line we would be able to develop pretty solid causal inferences about thickness and fracture rates. A clear causal link could be identified by the regression coefficients in this scenario.
However, we could also imagine a graph that plotted murder rates in European cities and the spread of Christianity. With one axis being the number of years a city has had a Catholic bishop and the other axis being the number of murders, we may find that murders decrease the longer a city has had a bishop.  From this, we might be tempted to say that Christianity (particularly the location of a Bishop in a town) reduces murder. But what would we point to as the causal mechanism? Would it be religious beliefs adopted by people interacting with the church? Would it be that marriage rules that limited polygamy ensured more men found wives and became less murderous as a result? Would it be that some divinity smiled upon the praying people and made them to be less murderous? A regression like the one I described above wouldn’t tell us anything about the causal mechanism in effect in this instance. Our causal-thinking minds, however, would still generate causal hypothesis, some of which would be reasonable but others less so (this example comes from the wonderful The WEIRDest People in the World by Joseph Henrich).
Regression coefficients can be helpful, but they are less helpful when we cannot understand the causal mechanisms at play. Understanding the causal mechanisms can help us better understand the relationship represented by the regression coefficients, but the coefficient itself only represents a relationship, not a causal structure. Approaching data and looking for trends doesn’t help us generate useful information. We must first have a sense of a potential causal mechanism, then examine the data to see if our proposed causal mechanism has support or not. This is how we can use data and find support for causal hypothesis within regression coefficients.
Laboratory Proof

Laboratory Proof

Posted on by jabittan
“If the standard of laboratory proof had been applied to scurvy,” writes Judea Pearl in The Book of Why, “then sailors would have continued dying right up until the 1930’s, because until the discovery of vitamin C, there was no laboratory proof that citrus fruits prevented scurvy.” Pearl’s quote shows that high scientific standards for definitive and exact causality are not always for the greater good. Sometimes modern science will spurn clear statistical relationships and evidence because statistical relationships alone cannot be counted on as concrete causal evidence. A clear answer will not be given because some marginal unknowns may still exist, and this can have its own costs.
Sailors did not know why or how citrus fruits prevented scurvy, but observations demonstrated that citrus fruits managed to prevent scurvy. There was no clear understanding of what scurvy was or why citrus fruits were helpful, but it was commonly understood that a causal relationship existed. People acted on these observations and lives were saved.
On two episodes, the Don’t Panic Geocast has talked about journal articles in the British Medical Journal that make the same point as Pearl. As a critique of the need for randomized controlled trials, the two journal articles highlight the troubling reality that there have not been any randomized controlled trials on the effectiveness of parachute usage when jumping from airplanes. The articles are hilarious and clearly satirical, but ultimately come to the same point that Pearl does with the quote above – laboratory proof is not always necessary, practical, or reasonable when lives are on the line.
Pearl argues that we can rely on our abilities to identify causality even without laboratory proof when we have sufficient statistical analysis and understanding of relationships. Statisticians always tell us that correlation is not causation and that observational studies are not sufficient to determine causality, yet the citrus fruit and parachute examples highlight that this mindset is not always appropriate. Sometimes more realistic and common sense understanding of causation – even if supported with just correlational relationships and statistics – are more important than laboratory proof.
Nature Answers the Questions We Pose

Nature Answers the Questions We Pose

Posted on by jabittan
I have not read A Hitchhiker’s Guide to the Galaxy, but I know there is a point where a character asks what’s the meaning of life, the universe, and everything, and receives a response of 42. The answer was certainly not what anyone was expecting, but it was an answer. Much of science is like the answer 42. We ask grand questions of nature and receive answers we didn’t quite expect and can’t always make sense of.
In The Book of Why Judea Pearl writes, “Nature is like a genie that answers exactly the question we pose, not necessarily the one we intend to ask.” We learn by making observations about the world. We can make predictions about what we think will happen given certain conditions and we can develop and test hypotheses, but the answers we get may not be answers to the questions we intended to ask. I frequently listen to the Don’t Panic Geocast and the hosts often talk about scientific studies that go awry because of some unexpected interaction between lights, between an experimental set-up and the sun, or because an animal happened to have messed with equipment in the field. Real results are generated, but they don’t always mean what we think they do on first look. The hosts have a frequent line that, “any instrument can be a thermometer,” to note how subtle changes in temperature can cause misleading noise in the data.
Pearl’s quote is meant to demonstrate how challenging science can be and why so much of science has taken such a long time to develop. Humans have often thought they were receiving answers to the questions they were asking, only to find out that nature was answering a different question, not the one the scientists thought they had asked. Pearl states that randomness has been one of the ways that we have gotten past nature, but writes about how counter-intuitive randomized controlled trials were when first developed. No one realized that the right question had to be asked through experimental set-ups that involved randomness. On the benefits of randomness he writes, “first, it eliminates confounder bias (it asks Nature the right question). Second, it enables the researcher to quantify his uncertainty.”
In the book, Pearl takes observations and statistical methods combined with causal insights to a level that is honestly beyond my comprehension. What is important to note, however, is that nature is not obligated to answer the questions we intend to ask. It answers questions exactly as we pose them, influenced by seemingly irrelevant factors in our experimental design. The first answer we get may not be very reliable, but randomness and statistical methods, combined as Pearl would advocate, with a solid understanding of causality, can helps us better pose our questions to nature, to be more confident that the responses we get answer the questions we meant to ask.
The Cigarette Wars - Judea Pearl The Book of Why - Joe Abittan

The Cigarette Wars

Posted on by jabittan
Recently I have been writing about Judea Pearl’s The Book of Why, in which Pearl asks if our reliance on statistics and our adherence to the idea correlation is not causation has gone too far in science. For most people, especially students getting into science and those who have studied politics, reiterating the idea that correlation does not imply causation is important. There are plenty of ways to misinterpret data and there is no shortage of individuals and interest groups who would love to have an official scientist improperly assign causation to a correlation for their own political and economic gain. However, Pearl uses the cigarette wars to show us that failing to acknowledge that correlations can imply causation can also be dangerous.
“The cigarette wars were science’s first confrontation with organized denialism, and no one was prepared,” writes Pearl. For decades there was ample evidence from different fields and different approaches linking cigarette smoking to cancer. However, it isn’t the case that every single person who smokes a cigarette gets cancer. We all know people who smoked for 30 years, and seem to have great lungs. Sadly, we also all know people who developed lung cancer but never smoked. The causation between cigarettes is not a perfect 1:1 correlation with lung cancer, and tobacco companies jumped on this fact.
For years, it was abundantly clear that smoking greatly increased the risk of lung cancer, but no one was willing to say that smoking caused lung cancer, because powerful interest groups aligned against the idea and conditioned policy-makers and the public to believe that in the case of smoking and lung cancer, correlation was not causation. The evidence was obvious, but built on statistical information and the organized denial was stronger. Who was to say if people more susceptible to lung cancer were also more susceptible to start smoking in the first place? Arguments such as these hindered people’s willingness to adopt the clear causal picture that cigarettes caused cancer. People hid behind a possibility that the overwhelming evidence was wrong.
Today we are in a similar situation with climate change and other issues. It is clear that statistics cannot give us a 100% certain answer to a causal question, and it is true that correlation is not necessarily a sign of causation, but at a certain point we have to accept when the evidence is overwhelming. We have to accept when causal models that are not 100% proven have overwhelming support. We have to be able to make decisions without being derailed by organized denialism that seizes on the fact that correlation does not imply causation, just to create doubt and confusion. Pearl’s warning is that failing to be better with how we think about and understand causality can have real consequences (lung cancer in the cigarette wars, and devastating climate impacts today), and that we should take those consequences seriously when we look at the statistics and data that helps us understand our world.