Month: May 2021

Scrutinizing Causal Assumptions

Scrutinizing Causal Assumptions

Posted on by jabittan
Recently I have been writing about my biggest take-away from The Book of Why by Judea Pearl. The book is more technical than I can fully understand and grasp since it is written for a primarily academic audience with some knowledge of the fields that Pearl dives into, but I felt that I still was able to gain some insights from the book. Particularly, Pearl’s idea that humans are better causal thinkers than we typically give ourselves credit for was a big lesson for me. In thinking back on the book, I have been trying to recognize our powerful causal intuitions and to understand the ways in which our causal thinking can be trusted. Still, for me it feels that it can be dangerous to indulge our natural causal thinking tendencies.
However, Pearl offers guidance on how and when we can trust our causal instincts. he writes, “causal assumptions cannot be invented at our whim; they are subject to the scrutiny of data and can be falsified.”
Our ability to imagine different future states and to understand causality at an instinctual level has allowed our human species to move form hunter-gatherer groups to massive cities connected by electricity and Wi-Fi. However, our collective minds have also drawn causal connections between unfortunate events and imagined demons. Dictators have used implausible causal connections to justify eugenics and genocide and still to this day society is hampered by conspiracy theories that posit improbable causal links between disparate events.
The important thing to note, as Pearl demonstrates, is that causal assumptions can be falsified and must be supported with data. Supernatural demons cannot be falsified and wild conspiracy theories often lack any supporting data or evidence. We can intuit causal relations, but we must be able to test them in situations that would falsify our assumptions if we are to truly believe them. Pearl doesn’t simply argue that we are good causal thinkers and that we should blindly trust the causal assumptions that come naturally to our mind. Instead, he suggests that we lean into our causal faculties and test causal relationships and assumptions that are falsifiable and can be either supported or disproven by data. Statistics still has a role in this world, but importantly we are not looking at the data without making causal assumptions. We are making predictions and determining whether the data falsifies those predictions.
The Screening-Off Effect

The Screening-Off Effect

Posted on by jabittan
Sometimes to our great benefit, and sometimes to our detriment, humans like to put things into categories – at least Western, Educated, Industrialized, Rich, Democratic (WEIRD) people do. We break things into component parts and categorize each part as belonging to a category of thing. We do this with things like planets, animals, and players within sports. We like established categories and dislike when our categorization changes. This ability has greatly helped us in science and strategic planning, allowing our species to do incredible things and learn crucial lessons about the world. What is remarkable about this ability is how natural and easy it is for us, but how hard it is to explain or program into a machine.
One component of this remarkable ability is referred to as the screening-off effect by Judea Pearl in The Book of Why. Pearl writes, “how do we decide which information to disregard, when every new piece of information changes the boundary between the relevant and the irrelevant? For humans, this understanding comes naturally. Even three-year-old toddlers understand the screening-off effect, though they don’t have a name for it. … But machines do not have this instinct, which is one reason that we equip them with causal diagrams.”
From a young age we know what information is the most important and what information we can ignore. We intuitively have a good sense for when we should seek out more information and when we have enough to make a decision (although sometimes we don’t follow this intuitive sense). We know there is always more information out there, but don’t have time to seek out every piece of information possible. Luckily, the screening-off effect helps us know when to stop and makes decision-making possible for us.
Beyond knowing when to stop, the screening-off effect helps us know when to ignore irrelevant information. The price of tea in China isn’t a relevant factor for us when deciding what time to wake up the next morning. We recognize that there are no meaningful causal pathways between the price of tea and the best time for us to wake up. This causal insight, however, doesn’t exist for machines that are only programmed with the specific statistics we build into them. We specifically have to code a causal pathway that doesn’t include the price of tea in China for a machine to know that it can ignore that information. The screening-off effect, Pearl explains, is part of what allows humans to think causally. In cutting edge science there are many factors we wouldn’t think to screen out that may impact the results of scientific experiments, but for the most part, we know what can be ignored and can look at the world around us through a causal lens because we know what is and is not important.
Causal Hypotheses

Causal Hypotheses

Posted on by jabittan
In The Book of Why Judea Pearl argues that humans have a unique superpower among animals and living creatures on earth. We are great at developing causal hypotheses. Animals are able to make observations about the world and some are even able to use tools to open fruit, find insects, and perform other tasks. However, humans alone seem to be able to take a tool, develop a hypothesis for why a tool works, and imagine what could be done to improve its functioning. This step requires that we develop causal hypotheses about the nature and reality of tools and how they interact with the objects we wish to manipulate. This is a hugely consequential mental ability, and one that humans have developed the ability to improve overtime, especially through cultural learning.
Our minds are imaginative and can think about potential future states. We can understand how our tools work and imagine ways in which our tools might be better in order for us to better achieve our goals. This is how we build causal hypotheses about the world, and how we go about exploring the world in search of evidence that confirms or overturns our imagined causal structures.
In the book, Pearl writes, “although we don’t need to know every causal relation between the variables of interest and might be able to draw some conclusions with only partial information, Wright makes one point with absolute clarity: you cannot draw causal conclusions without some causal hypothesis.”  (Sewall Wright is who Pearl references)
To answer causal questions we need to develop a causal hypothesis. We don’t need to have every bit of data possible, and we don’t need to perfectly intuit or know every causal structure, but we can still understand causality by investigating imagined causal pathways. Our brains are powerful enough to draw conclusions based on observed data and imagined causal pathways. While we might be wrong and have historically made huge errors in our causal attributions about the world, in many instances, we are great causal thinkers, to the point where causal structures that we identify are common sense. We might not know exactly what is happening at the molecular level, but we can understand the causal pathway between sharpening a piece of obsidian to form a point that could penetrate the flesh of an animal we are hunting. While some causal pathways are nearly invisible to us, a great deal are ready for us to view, and we should not forget that. We can get bogged down in statistics and become overly reliant on correlations and statistical relationships if we ignore the fact that our minds are adept at identifying and imagining causal structures.
The Quest of Science & Life

The Quest of Science & Life

Posted on by jabittan
“It is an irony of history that Galton started out in search of causation and ended up discovering correlation, a relationship that is oblivious of causation,” writes Judea Pearl in his book The Book of Why. Pearl examines the history of the study of causation in his book suggesting that Galton abandoned his original quest to define causation. Galton, along with Karl Pearson is a titanic figure in the study of statistics. The pair are in many ways responsible for the path of modern statistics, but as Pearl describes it, that was not the original intent, at least for Galton.
Pearl describes Galton as trying to work toward universal theories and approaches to causation. Correlation, the end product of Galton’s research is helpful and a vital part of how we understand the world today, but it is not causation. Correlation does not tell us if one thing causes another, only that a relationship exists. It doesn’t tell us which way the arrow of causation moves and whether other factors are important in causation. It tells us that as one thing changes, another changes with it, or that as other variables adjust, outcomes in the specific thing we want to see also adjust. But from correlation and statistical studies, we don’t truly know why the world works the way it does. I think that Pearl would argue that in its best form, statistics helps us narrow down causal possibilities and pathways, but it never tells us with any certainty that a relationship exists because of specific causal factors.
The direction of Galton’s research is emblematic of science and of our lives in general. Galton set out in search of one thing, and gave rise to an entirely different field of study. For his work he clearly became successful, influential, and well regarded, but today (as Pearl argues) we are living with the consequences of his work. We haven’t been able to move forward from the paradigm he created. A paradigm he didn’t really set out to establish.
Quite often in our lives we follow paths that we don’t fully understand, ending up in places we didn’t quite expect. We can make the most out of where our journeys take us and live full lives, even if we didn’t expect to be where we are living. We can’t fully control where the path takes us, and if we chose to stop, there is no reason the path has to stop as well. What we set out to do can become more than us, and can carry far beyond our imaginations, and the world will have to live with those consequences, even if we walk away or pass away.
They key point in this post is to remember that the world is complex. Remember that what you see is only a partial slice, that your causal explanations of the world may be inaccurate, and that the correlations you see are not complete explanations of reality. The path you walk shapes the future of the world, for you and for others, so you have a responsibility to make the best decisions you can, and to live well with the destination you reach, even if it isn’t the destination you thought you were walking toward. Your journey will end at some point, but the path you start could keep going far beyond your end-point, so consider whether you are leaving a path that others can continue to follow, or if you are forging a trail that will cause problems down the road. The lesson is to be considerate and make the most out of the winding and unpredictable path ahead of you as you set out on your quest.
Slope is Agnostic to Cause and Effect

Slope is Agnostic to Cause and Effect

Posted on by jabittan
I like statistics. I like to think statistically, to recognize that there is a percent chance of one outcome that can be influenced by other factors. I enjoy looking at best fit lines, seeing that there are correlations between different variables, and seeing how trend-lines change if you control for different variables. However, statistics and trend lines don’t actually tell us anything about causality.
In The Book of Why Judea Pearl writes, “the slope (after scaling) is the same no matter whether you plot X against Y or Y against X. In other words, the slope is completely agnostic as to cause and effect. One variable could cause the other, or they could both be effects of a third cause; for the purpose of prediction, it does not matter.”
In statistics we all know that correlation is not causation, but this quote helps us remember important information when we see a statistical analysis and a plot with linear regression line running through it. The regression line is like the owl that Pearl had described earlier in the book. The owl is able to predict where a mouse is likely to be and able to predict which direction it will run, but the owl does not seem to know why a mouse is likely to be in a given location or why it is likely to run in one direction over another. It simply knows from experience and observation what a mouse is likely to do.
The regression line is a best fit for numerous observations, but it doesn’t tell us whether one variable causes another or whether both are influenced in a similar manner by another variable. The regression line knows where the mouse might be and where it might run, but it doesn’t know why.
In statistics courses we end at this point of correlation. We might look for other variables that are correlated or try to control for third variables to see if the relationship remains, but we never answer the question of causality, we never get to the why. Pearl thinks this is a limitation we do not need to put on ourselves. Humans, unlike owls, can understand causality, we can recognize the various reasons why a mouse might be hiding under a bush, and why it may chose to run in one direction rather than another. Correlations can help us start to see where relationships exist, but it is the ability of our mind to understand causal pathways that helps us determine causation.
Pearl argues that statisticians avoid these causal arguments out of caution, but that it only ends up creating more problems down the line. Important statistical research in areas of high interest or concern to law-makers, business people, or the general public are carried beyond the cautious bounds that causality-averse statisticians place on their work. Showing correlations without making an effort to understand the causality behind it makes scientific work vulnerable to the epistemically malevolent who would like to use correlations to their own ends. While statisticians rigorously train themselves to understand that correlation is not causation, the general public and those struck with motivated reasoning don’t hold themselves to the same standard. Leaving statistical analysis at the level of correlation means that others can attribute the cause and effect of their choice to the data, and the proposed causal pathways can be wildly inaccurate and even dangerous. Pearl suggests that statisticians and researchers are thus obligated to do more with causal structures, to round off  their work and better develop ideas of causation that can be defended once their work is beyond the world of academic journals.
The Fundamental Nature of Cause and Effect

The Fundamental Nature of Cause and Effect

Posted on by jabittan
In my undergraduate and graduate studies I had a few statistics classes and I remember the challenge of learning probability. Probability, odds, and statistics are not always easy to understand and interpret. There are some concepts that are pretty straightforward, and others that seem to contradict what we would expect if we had not gone through the math and if we had not studied the concepts in depth. To contrast the difficult and sometimes counter-intuitive nature of statistics, we can think about causality, which is a challenging concept, but unlike statistics, is something we are able to intuit from very young age.
In The Book of Why Judea Pearl writes, “In both a cognitive and a philosophical sense, the idea of cause and effect is much more fundamental than probability. We begin learning causes and effects before we understand language and before we understand mathematics.”
As Pearl explains, we see causality naturally and experience causality as we move through our lives. From a young child who learns that if they cry they receive attention to a nuclear physicist who learns what happens when two atoms collide at high energy levels, our minds are constantly looking at the world and looking for causes. It begins by making observations of phenomena around us and continues as we predict what outcomes would happen based on certain system inputs. Eventually, our minds reach a point where we can understand why our predictions are accurate or inaccurate, and we can imagine new ways to bring about certain outcomes. Even if we cannot explain all of this, we can still understand causation at a fundamental and intuitive level.
However, many of us deny that we can see and understand the world in a causal way. I am personally guilty of thinking in a purely statistical way and ignoring the causal. The classes I took in college helped me understand statistics and probability, but also told me not to trust my intuitive causal thinking. Books like Kahneman’s Thinking Fast and Slow cemented this mindset for me. Rationality, we believe, requires that we think statistically and discount our intuitions for fear of bias. Modern science says we can only trust evidence when it is backed by randomized controlled trials and directs us to think of the world through correlations and statistical relationships, not through a lens of causality.
Pearl pushes back against this notion. By arguing that causality is fundamental to the human mind, he implies that our causal reasoning can and should be trusted. Throughout the book he demonstrates that a purely statistical way of thinking leaves us falling short of the knowledge we really need to improve the world. He demonstrates that complex tactics to remove variables from equations in statistical methods are often unnecessary, and that we can accept the results of experiments and interventions even when they are not fully randomized controlled trials.  For much of human history our causal thinking nature has lead us astray, but I think that Pearl argues that we have overcorrected in modern statistics and science, and that we need to return to our causal roots to move forward and solve problems that statistics tells us are impossible to solve.
Predictions & Explanations

Predictions & Explanations

Posted on by jabittan
The human mind has incredible predictive abilities, but our explanatory abilities do not always turn out to be as equally incredible. Prediction is relatively easy when compared to explanation. Animals can predict where a food source will be without being able to explain how it got there. For most of human history our ancestors were able to predict that the sun would rise the next day without having any way of explaining why it would rise. Computer programs today can predict our next move in chess but few can explain their prediction or why we would make the choice that was predicted.
As Judea Pearl writes in The Book of Why, “Good predictions need not have good explanations. The owl can be a good hunter without understanding why the rat always goes from point A to point B.” Prediction is possible with statistics and good observations. With a large enough database, we can make a prediction about what percentage of individuals will have negative reactions to medications, we can predict when a traffic jam will occur, and we can predict how an animal will behave. What is harder, according to Pearl, is moving to the stage where we describe why we observe the relationships that statistics reveal.
Statistics alone cannot tell us why particular patterns emerge. Statistics cannot identify causal structures. As a result, we continually tell ourselves that correlation is not causation and that we can only determine what relationships are truly causal through randomized controlled trials. Pearl would argue that this is incorrect, and he would argue that this idea results from the fact that statistics is trying to answer a completely different question than causation. Approaching statistical questions from a causal lens may lead to inaccurate interpretations of data or “p-hacking” an academic term used to describe efforts to get the statistical results you wanted to see. The key is not hunting for causation within statistics, but understanding causation and supporting it through evidence uncovered via statistics.
Seeing the difference between causation and statistics is helpful when thinking about the world. Being stuck without a way to see and determine causation leads to situations like tobacco companies claiming that cigarettes don’t cause cancer or oil and gas companies claiming that humans don’t contribute to global warming. Causal thinking, however, utilizes our ability to develop explanations and applies those explanations to the world. Our ability to predict different outcomes based on different interventions helps us interpret and understand the data that the world produces. We may not see the exact picture in the data, but we can understand it and use it to help us make better decisions that will lead to more accurate causal understandings over time.
Tool Use and Causation - Judea Pearl - The Book of Why - Joe Abittan

Tool Use and Causation

Posted on by jabittan
Judea Pearl’s book The Book of Why is all about causation. The reason human beings are able to produce vaccines, to send rockets into space, and maintain green gardens is because we understand causation. We have an ability to observe events in the world, to intervene, and to predict how our interventions produce specific outcomes. This allows us to develop tools to specifically achieve desired ends, and it is not a small feat.
In the book Pearl describes three levels of causation based on Alan Turing’s proposed system to classify cognitive systems in terms of the queries systems can answer. The three levels of causation are association, intervention, and counterfactuals. Pearl explains that many animals observe the world and detect patterns, but that fewer animals use tools to intervene in the world. Fewer still, Pearl explains, possess the ability to actually develop and improve new tools. As he writes, “tool users do not necessarily possess a theory of their tool that tells them why it works and what to do when it doesn’t. For that, you need to have achieved a level of understanding that permits imagining. It was primarily this third level that prepared us for further revolutions in agriculture and science and led to a sudden and drastic change in our species’ impact on the planet.”
The theory of tool use that Pearl mentions in the quote is our ability to see and understand causation. We can observe that rocks can be used to cut plant fibers, and then we can identify the qualities in some rocks that make them better at cutting fibers than others. But to get to the point where we are sharpening an edge of a rock to make it even better at cutting fibers, we have to have a causal understanding of what allows the rock to cut and we need sufficient imagination to predict what would happen if the rock had a sharper edge. We have to imagine an outcome in a future world where something was different, and that something different caused a new outcome.
This point is small, but is actually quite profound. Our minds are able to conceptualize causality and build hypothesis about the world that we can test. This can improve our tool usage, improve the ways we act and behave, and can allow us to achieve desired ends through study, prediction, imagination, and experimentation. The key, however, is that we have a theory of the tools and how they work, that we have an ability to intuit causation.
We hear all the time that correlation is not causation and in our modern technological age we are looking to statistics to help us solve massive problems. However, as Pearl’s quote shows, data, statistics, and information is useless unless we have a theory of the tools we can use based on the knowledge we gain from the data, statistics, and information. We have to embrace causation and our ability to imagine and predict causal structures if we want to do anything with the data.
This all reminds me of the saying, when the only tool you have is a hammer, everything begins to look like a nail. This represents an inability to understand causality, a lack of imagination and predictive prowess. Statistics without a theory of causality, without an ability to use our power to identify and predict causation, is like the hammer and nail saying. It is useless and throws the same toolkit and approach at every problem. Statistics alone doesn’t build knowledge – you also need a theory of causation.
Pearl’s message throughout the book is that statistics (tool use) and causation is linked, that we need a theory and understanding of causation if we are going to do anything with data, statistics, and information. For years we have relied on statistical relationships to help us understand the world, but we have failed to apply the same rigorous study to causation, and that will make it difficult for us to use our new statistical power to achieve the ends that big data and statistical processing promise.
Hope in Big Data

Hope in Big Data

Posted on by jabittan
Most of us probably don’t work with huge data sets, but all of us contribute to huge data sets. We know the world of big data is out there, and we know people are working with big data, but there are not many of us who truly know what it means and how we should think about any of it. In The Book of Why, Judea Pearl argues that even many of those doing research and running companies based on big data don’t fully understand what it all means.
Pearl is critical of researchers and entrepreneurs who lack causal understandings but pursue new knowledge and information by pulling correlations and statistics out of large data sets. There are some companies that are taking advantage of the fact that huge amounts of computing power can give us insights into data sets that we never before could have generated, however, these insights are not always as meaningful as we are lead to believe.
Pearl writes, “The hope – and at present, it is usually a silent one – is that the data themselves will guide us to the right answers whenever causal questions come up.”
My last post was about the overuse of the phrase: correlation is not causation. Finding correlations and relationships in data is meaningless if we don’t also have causal understandings in mind. This is the critique that Pearl makes with the quote above. If we don’t have a way of understanding basic causal structures, then the phrase is right, correlations don’t mean anything. Many companies and researchers are in a stage where they are finding correlations and unexpected statistical results in big data, but they lack causal understandings to do anything meaningful with the data. In the world of public policy this feels like the saying, a solution in search of a problem or in the world of healthcare like a pay and chase scenario.
Pearl argues throughout the book that we are better at identifying causal structures than we are lead to believe in our statistics courses. He also argues that understanding causality is key to unlocking the potential of big data and actually getting something useful out of massive datasets. Without a grounding in causality, we are wasting our time with the statistical research we do. We are running around with solutions in the forms of big data correlations that don’t have a causal underpinning. It is as if we are paying fraudulent claims, then chasing down some of the money we spent and congratulating ourselves on preventing fraud. The end result is a poor use of data that we prop up as a magnanimous solution.
Correlation and Causation - Judea Pearl - The Book of Why - Joe Abittan

Correlation and Causation

Posted on by jabittan
I have an XKCD comic taped to the door of my office. The comic is about the mantra of statistics, that correlation is not causation. I taped the comic to my office door because I loved learning statistics in graduate school and thinking deeply about associations and how mere correlations cannot be used to demonstrate that one thing causes another. Two events can correlate, but have nothing to do with each other, and a third thing may influence both, causing them to correlate without any causal link between the two things.
But Judea Pearl thinks that science and researchers have fallen into a trap laid out by statisticians and the infinitely repeated correlation does not imply causation mantra. Regarding this perspective of statistics he writes, “it tells us that correlation is not causation, but it does not tell us what causation is.”
Pearl seems to suggest in The Book of Why that there was a time where there was too much data, too much humans didn’t know, and too many people ready to offer incomplete assessments based on anecdote and incomplete information. From this time sprouted the idea that correlation does not imply causation. We started to see that statistics could describe relationships and that statistics could be used to pull apart entangled causal webs, identifying each individual component and assessing its contribution to a given outcome. However, as his quote shows, this approach never actually answered what causation is. It never actually told us when we can know and ascertain that a causal structure and causal mechanism is in place.
“Over and over again,” writes Pearl, “in science and in business, we see situations where mere data aren’t enough.”
To demonstrate the shortcomings of our high regard for statistics and our mantra that correlation is not causation, Pearl walks us through the congressional testimonies and trials of big tobacco companies in the United States. The data told us there was a correlation between smoking and lung cancer. There was overwhelming statistical evidence that smoking was related or associated with lung cancer, but we couldn’t attain 100% certainty just through statistics that smoking caused lung cancer. The companies themselves muddied the water with misleading studies and cherry picked results. They hid behind a veil that said that correlation was not causation, and hid behind the confusion around causation that statistics could never fully clarify.
Failing to develop a real sense of causation, failing to move beyond big data, and failing to get beyond statistical correlations can have real harms. We need to be able to recognize causation, even without relying on randomized controlled trials, and we need to be able to make decisions to save lives. The lesson of the comic taped to my door is helpful when we are trying to be scientific and accurate in our thinking, but it can also lead us astray when we fail to trust a causal structure that we can see, but can’t definitively prove via statistics.