Laboratory Proof

Laboratory Proof

Posted on by jabittan
“If the standard of laboratory proof had been applied to scurvy,” writes Judea Pearl in The Book of Why, “then sailors would have continued dying right up until the 1930’s, because until the discovery of vitamin C, there was no laboratory proof that citrus fruits prevented scurvy.” Pearl’s quote shows that high scientific standards for definitive and exact causality are not always for the greater good. Sometimes modern science will spurn clear statistical relationships and evidence because statistical relationships alone cannot be counted on as concrete causal evidence. A clear answer will not be given because some marginal unknowns may still exist, and this can have its own costs.
Sailors did not know why or how citrus fruits prevented scurvy, but observations demonstrated that citrus fruits managed to prevent scurvy. There was no clear understanding of what scurvy was or why citrus fruits were helpful, but it was commonly understood that a causal relationship existed. People acted on these observations and lives were saved.
On two episodes, the Don’t Panic Geocast has talked about journal articles in the British Medical Journal that make the same point as Pearl. As a critique of the need for randomized controlled trials, the two journal articles highlight the troubling reality that there have not been any randomized controlled trials on the effectiveness of parachute usage when jumping from airplanes. The articles are hilarious and clearly satirical, but ultimately come to the same point that Pearl does with the quote above – laboratory proof is not always necessary, practical, or reasonable when lives are on the line.
Pearl argues that we can rely on our abilities to identify causality even without laboratory proof when we have sufficient statistical analysis and understanding of relationships. Statisticians always tell us that correlation is not causation and that observational studies are not sufficient to determine causality, yet the citrus fruit and parachute examples highlight that this mindset is not always appropriate. Sometimes more realistic and common sense understanding of causation – even if supported with just correlational relationships and statistics – are more important than laboratory proof.
Nature Answers the Questions We Pose

Nature Answers the Questions We Pose

Posted on by jabittan
I have not read A Hitchhiker’s Guide to the Galaxy, but I know there is a point where a character asks what’s the meaning of life, the universe, and everything, and receives a response of 42. The answer was certainly not what anyone was expecting, but it was an answer. Much of science is like the answer 42. We ask grand questions of nature and receive answers we didn’t quite expect and can’t always make sense of.
In The Book of Why Judea Pearl writes, “Nature is like a genie that answers exactly the question we pose, not necessarily the one we intend to ask.” We learn by making observations about the world. We can make predictions about what we think will happen given certain conditions and we can develop and test hypotheses, but the answers we get may not be answers to the questions we intended to ask. I frequently listen to the Don’t Panic Geocast and the hosts often talk about scientific studies that go awry because of some unexpected interaction between lights, between an experimental set-up and the sun, or because an animal happened to have messed with equipment in the field. Real results are generated, but they don’t always mean what we think they do on first look. The hosts have a frequent line that, “any instrument can be a thermometer,” to note how subtle changes in temperature can cause misleading noise in the data.
Pearl’s quote is meant to demonstrate how challenging science can be and why so much of science has taken such a long time to develop. Humans have often thought they were receiving answers to the questions they were asking, only to find out that nature was answering a different question, not the one the scientists thought they had asked. Pearl states that randomness has been one of the ways that we have gotten past nature, but writes about how counter-intuitive randomized controlled trials were when first developed. No one realized that the right question had to be asked through experimental set-ups that involved randomness. On the benefits of randomness he writes, “first, it eliminates confounder bias (it asks Nature the right question). Second, it enables the researcher to quantify his uncertainty.”
In the book, Pearl takes observations and statistical methods combined with causal insights to a level that is honestly beyond my comprehension. What is important to note, however, is that nature is not obligated to answer the questions we intend to ask. It answers questions exactly as we pose them, influenced by seemingly irrelevant factors in our experimental design. The first answer we get may not be very reliable, but randomness and statistical methods, combined as Pearl would advocate, with a solid understanding of causality, can helps us better pose our questions to nature, to be more confident that the responses we get answer the questions we meant to ask.
The Cigarette Wars - Judea Pearl The Book of Why - Joe Abittan

The Cigarette Wars

Posted on by jabittan
Recently I have been writing about Judea Pearl’s The Book of Why, in which Pearl asks if our reliance on statistics and our adherence to the idea correlation is not causation has gone too far in science. For most people, especially students getting into science and those who have studied politics, reiterating the idea that correlation does not imply causation is important. There are plenty of ways to misinterpret data and there is no shortage of individuals and interest groups who would love to have an official scientist improperly assign causation to a correlation for their own political and economic gain. However, Pearl uses the cigarette wars to show us that failing to acknowledge that correlations can imply causation can also be dangerous.
“The cigarette wars were science’s first confrontation with organized denialism, and no one was prepared,” writes Pearl. For decades there was ample evidence from different fields and different approaches linking cigarette smoking to cancer. However, it isn’t the case that every single person who smokes a cigarette gets cancer. We all know people who smoked for 30 years, and seem to have great lungs. Sadly, we also all know people who developed lung cancer but never smoked. The causation between cigarettes is not a perfect 1:1 correlation with lung cancer, and tobacco companies jumped on this fact.
For years, it was abundantly clear that smoking greatly increased the risk of lung cancer, but no one was willing to say that smoking caused lung cancer, because powerful interest groups aligned against the idea and conditioned policy-makers and the public to believe that in the case of smoking and lung cancer, correlation was not causation. The evidence was obvious, but built on statistical information and the organized denial was stronger. Who was to say if people more susceptible to lung cancer were also more susceptible to start smoking in the first place? Arguments such as these hindered people’s willingness to adopt the clear causal picture that cigarettes caused cancer. People hid behind a possibility that the overwhelming evidence was wrong.
Today we are in a similar situation with climate change and other issues. It is clear that statistics cannot give us a 100% certain answer to a causal question, and it is true that correlation is not necessarily a sign of causation, but at a certain point we have to accept when the evidence is overwhelming. We have to accept when causal models that are not 100% proven have overwhelming support. We have to be able to make decisions without being derailed by organized denialism that seizes on the fact that correlation does not imply causation, just to create doubt and confusion. Pearl’s warning is that failing to be better with how we think about and understand causality can have real consequences (lung cancer in the cigarette wars, and devastating climate impacts today), and that we should take those consequences seriously when we look at the statistics and data that helps us understand our world.
Scrutinizing Causal Assumptions

Scrutinizing Causal Assumptions

Posted on by jabittan
Recently I have been writing about my biggest take-away from The Book of Why by Judea Pearl. The book is more technical than I can fully understand and grasp since it is written for a primarily academic audience with some knowledge of the fields that Pearl dives into, but I felt that I still was able to gain some insights from the book. Particularly, Pearl’s idea that humans are better causal thinkers than we typically give ourselves credit for was a big lesson for me. In thinking back on the book, I have been trying to recognize our powerful causal intuitions and to understand the ways in which our causal thinking can be trusted. Still, for me it feels that it can be dangerous to indulge our natural causal thinking tendencies.
However, Pearl offers guidance on how and when we can trust our causal instincts. he writes, “causal assumptions cannot be invented at our whim; they are subject to the scrutiny of data and can be falsified.”
Our ability to imagine different future states and to understand causality at an instinctual level has allowed our human species to move form hunter-gatherer groups to massive cities connected by electricity and Wi-Fi. However, our collective minds have also drawn causal connections between unfortunate events and imagined demons. Dictators have used implausible causal connections to justify eugenics and genocide and still to this day society is hampered by conspiracy theories that posit improbable causal links between disparate events.
The important thing to note, as Pearl demonstrates, is that causal assumptions can be falsified and must be supported with data. Supernatural demons cannot be falsified and wild conspiracy theories often lack any supporting data or evidence. We can intuit causal relations, but we must be able to test them in situations that would falsify our assumptions if we are to truly believe them. Pearl doesn’t simply argue that we are good causal thinkers and that we should blindly trust the causal assumptions that come naturally to our mind. Instead, he suggests that we lean into our causal faculties and test causal relationships and assumptions that are falsifiable and can be either supported or disproven by data. Statistics still has a role in this world, but importantly we are not looking at the data without making causal assumptions. We are making predictions and determining whether the data falsifies those predictions.
The Screening-Off Effect

The Screening-Off Effect

Posted on by jabittan
Sometimes to our great benefit, and sometimes to our detriment, humans like to put things into categories – at least Western, Educated, Industrialized, Rich, Democratic (WEIRD) people do. We break things into component parts and categorize each part as belonging to a category of thing. We do this with things like planets, animals, and players within sports. We like established categories and dislike when our categorization changes. This ability has greatly helped us in science and strategic planning, allowing our species to do incredible things and learn crucial lessons about the world. What is remarkable about this ability is how natural and easy it is for us, but how hard it is to explain or program into a machine.
One component of this remarkable ability is referred to as the screening-off effect by Judea Pearl in The Book of Why. Pearl writes, “how do we decide which information to disregard, when every new piece of information changes the boundary between the relevant and the irrelevant? For humans, this understanding comes naturally. Even three-year-old toddlers understand the screening-off effect, though they don’t have a name for it. … But machines do not have this instinct, which is one reason that we equip them with causal diagrams.”
From a young age we know what information is the most important and what information we can ignore. We intuitively have a good sense for when we should seek out more information and when we have enough to make a decision (although sometimes we don’t follow this intuitive sense). We know there is always more information out there, but don’t have time to seek out every piece of information possible. Luckily, the screening-off effect helps us know when to stop and makes decision-making possible for us.
Beyond knowing when to stop, the screening-off effect helps us know when to ignore irrelevant information. The price of tea in China isn’t a relevant factor for us when deciding what time to wake up the next morning. We recognize that there are no meaningful causal pathways between the price of tea and the best time for us to wake up. This causal insight, however, doesn’t exist for machines that are only programmed with the specific statistics we build into them. We specifically have to code a causal pathway that doesn’t include the price of tea in China for a machine to know that it can ignore that information. The screening-off effect, Pearl explains, is part of what allows humans to think causally. In cutting edge science there are many factors we wouldn’t think to screen out that may impact the results of scientific experiments, but for the most part, we know what can be ignored and can look at the world around us through a causal lens because we know what is and is not important.
Causal Hypotheses

Causal Hypotheses

Posted on by jabittan
In The Book of Why Judea Pearl argues that humans have a unique superpower among animals and living creatures on earth. We are great at developing causal hypotheses. Animals are able to make observations about the world and some are even able to use tools to open fruit, find insects, and perform other tasks. However, humans alone seem to be able to take a tool, develop a hypothesis for why a tool works, and imagine what could be done to improve its functioning. This step requires that we develop causal hypotheses about the nature and reality of tools and how they interact with the objects we wish to manipulate. This is a hugely consequential mental ability, and one that humans have developed the ability to improve overtime, especially through cultural learning.
Our minds are imaginative and can think about potential future states. We can understand how our tools work and imagine ways in which our tools might be better in order for us to better achieve our goals. This is how we build causal hypotheses about the world, and how we go about exploring the world in search of evidence that confirms or overturns our imagined causal structures.
In the book, Pearl writes, “although we don’t need to know every causal relation between the variables of interest and might be able to draw some conclusions with only partial information, Wright makes one point with absolute clarity: you cannot draw causal conclusions without some causal hypothesis.”  (Sewall Wright is who Pearl references)
To answer causal questions we need to develop a causal hypothesis. We don’t need to have every bit of data possible, and we don’t need to perfectly intuit or know every causal structure, but we can still understand causality by investigating imagined causal pathways. Our brains are powerful enough to draw conclusions based on observed data and imagined causal pathways. While we might be wrong and have historically made huge errors in our causal attributions about the world, in many instances, we are great causal thinkers, to the point where causal structures that we identify are common sense. We might not know exactly what is happening at the molecular level, but we can understand the causal pathway between sharpening a piece of obsidian to form a point that could penetrate the flesh of an animal we are hunting. While some causal pathways are nearly invisible to us, a great deal are ready for us to view, and we should not forget that. We can get bogged down in statistics and become overly reliant on correlations and statistical relationships if we ignore the fact that our minds are adept at identifying and imagining causal structures.
The Quest of Science & Life

The Quest of Science & Life

Posted on by jabittan
“It is an irony of history that Galton started out in search of causation and ended up discovering correlation, a relationship that is oblivious of causation,” writes Judea Pearl in his book The Book of Why. Pearl examines the history of the study of causation in his book suggesting that Galton abandoned his original quest to define causation. Galton, along with Karl Pearson is a titanic figure in the study of statistics. The pair are in many ways responsible for the path of modern statistics, but as Pearl describes it, that was not the original intent, at least for Galton.
Pearl describes Galton as trying to work toward universal theories and approaches to causation. Correlation, the end product of Galton’s research is helpful and a vital part of how we understand the world today, but it is not causation. Correlation does not tell us if one thing causes another, only that a relationship exists. It doesn’t tell us which way the arrow of causation moves and whether other factors are important in causation. It tells us that as one thing changes, another changes with it, or that as other variables adjust, outcomes in the specific thing we want to see also adjust. But from correlation and statistical studies, we don’t truly know why the world works the way it does. I think that Pearl would argue that in its best form, statistics helps us narrow down causal possibilities and pathways, but it never tells us with any certainty that a relationship exists because of specific causal factors.
The direction of Galton’s research is emblematic of science and of our lives in general. Galton set out in search of one thing, and gave rise to an entirely different field of study. For his work he clearly became successful, influential, and well regarded, but today (as Pearl argues) we are living with the consequences of his work. We haven’t been able to move forward from the paradigm he created. A paradigm he didn’t really set out to establish.
Quite often in our lives we follow paths that we don’t fully understand, ending up in places we didn’t quite expect. We can make the most out of where our journeys take us and live full lives, even if we didn’t expect to be where we are living. We can’t fully control where the path takes us, and if we chose to stop, there is no reason the path has to stop as well. What we set out to do can become more than us, and can carry far beyond our imaginations, and the world will have to live with those consequences, even if we walk away or pass away.
They key point in this post is to remember that the world is complex. Remember that what you see is only a partial slice, that your causal explanations of the world may be inaccurate, and that the correlations you see are not complete explanations of reality. The path you walk shapes the future of the world, for you and for others, so you have a responsibility to make the best decisions you can, and to live well with the destination you reach, even if it isn’t the destination you thought you were walking toward. Your journey will end at some point, but the path you start could keep going far beyond your end-point, so consider whether you are leaving a path that others can continue to follow, or if you are forging a trail that will cause problems down the road. The lesson is to be considerate and make the most out of the winding and unpredictable path ahead of you as you set out on your quest.
Slope is Agnostic to Cause and Effect

Slope is Agnostic to Cause and Effect

Posted on by jabittan
I like statistics. I like to think statistically, to recognize that there is a percent chance of one outcome that can be influenced by other factors. I enjoy looking at best fit lines, seeing that there are correlations between different variables, and seeing how trend-lines change if you control for different variables. However, statistics and trend lines don’t actually tell us anything about causality.
In The Book of Why Judea Pearl writes, “the slope (after scaling) is the same no matter whether you plot X against Y or Y against X. In other words, the slope is completely agnostic as to cause and effect. One variable could cause the other, or they could both be effects of a third cause; for the purpose of prediction, it does not matter.”
In statistics we all know that correlation is not causation, but this quote helps us remember important information when we see a statistical analysis and a plot with linear regression line running through it. The regression line is like the owl that Pearl had described earlier in the book. The owl is able to predict where a mouse is likely to be and able to predict which direction it will run, but the owl does not seem to know why a mouse is likely to be in a given location or why it is likely to run in one direction over another. It simply knows from experience and observation what a mouse is likely to do.
The regression line is a best fit for numerous observations, but it doesn’t tell us whether one variable causes another or whether both are influenced in a similar manner by another variable. The regression line knows where the mouse might be and where it might run, but it doesn’t know why.
In statistics courses we end at this point of correlation. We might look for other variables that are correlated or try to control for third variables to see if the relationship remains, but we never answer the question of causality, we never get to the why. Pearl thinks this is a limitation we do not need to put on ourselves. Humans, unlike owls, can understand causality, we can recognize the various reasons why a mouse might be hiding under a bush, and why it may chose to run in one direction rather than another. Correlations can help us start to see where relationships exist, but it is the ability of our mind to understand causal pathways that helps us determine causation.
Pearl argues that statisticians avoid these causal arguments out of caution, but that it only ends up creating more problems down the line. Important statistical research in areas of high interest or concern to law-makers, business people, or the general public are carried beyond the cautious bounds that causality-averse statisticians place on their work. Showing correlations without making an effort to understand the causality behind it makes scientific work vulnerable to the epistemically malevolent who would like to use correlations to their own ends. While statisticians rigorously train themselves to understand that correlation is not causation, the general public and those struck with motivated reasoning don’t hold themselves to the same standard. Leaving statistical analysis at the level of correlation means that others can attribute the cause and effect of their choice to the data, and the proposed causal pathways can be wildly inaccurate and even dangerous. Pearl suggests that statisticians and researchers are thus obligated to do more with causal structures, to round off  their work and better develop ideas of causation that can be defended once their work is beyond the world of academic journals.
The Fundamental Nature of Cause and Effect

The Fundamental Nature of Cause and Effect

Posted on by jabittan
In my undergraduate and graduate studies I had a few statistics classes and I remember the challenge of learning probability. Probability, odds, and statistics are not always easy to understand and interpret. There are some concepts that are pretty straightforward, and others that seem to contradict what we would expect if we had not gone through the math and if we had not studied the concepts in depth. To contrast the difficult and sometimes counter-intuitive nature of statistics, we can think about causality, which is a challenging concept, but unlike statistics, is something we are able to intuit from very young age.
In The Book of Why Judea Pearl writes, “In both a cognitive and a philosophical sense, the idea of cause and effect is much more fundamental than probability. We begin learning causes and effects before we understand language and before we understand mathematics.”
As Pearl explains, we see causality naturally and experience causality as we move through our lives. From a young child who learns that if they cry they receive attention to a nuclear physicist who learns what happens when two atoms collide at high energy levels, our minds are constantly looking at the world and looking for causes. It begins by making observations of phenomena around us and continues as we predict what outcomes would happen based on certain system inputs. Eventually, our minds reach a point where we can understand why our predictions are accurate or inaccurate, and we can imagine new ways to bring about certain outcomes. Even if we cannot explain all of this, we can still understand causation at a fundamental and intuitive level.
However, many of us deny that we can see and understand the world in a causal way. I am personally guilty of thinking in a purely statistical way and ignoring the causal. The classes I took in college helped me understand statistics and probability, but also told me not to trust my intuitive causal thinking. Books like Kahneman’s Thinking Fast and Slow cemented this mindset for me. Rationality, we believe, requires that we think statistically and discount our intuitions for fear of bias. Modern science says we can only trust evidence when it is backed by randomized controlled trials and directs us to think of the world through correlations and statistical relationships, not through a lens of causality.
Pearl pushes back against this notion. By arguing that causality is fundamental to the human mind, he implies that our causal reasoning can and should be trusted. Throughout the book he demonstrates that a purely statistical way of thinking leaves us falling short of the knowledge we really need to improve the world. He demonstrates that complex tactics to remove variables from equations in statistical methods are often unnecessary, and that we can accept the results of experiments and interventions even when they are not fully randomized controlled trials.  For much of human history our causal thinking nature has lead us astray, but I think that Pearl argues that we have overcorrected in modern statistics and science, and that we need to return to our causal roots to move forward and solve problems that statistics tells us are impossible to solve.
Predictions & Explanations

Predictions & Explanations

Posted on by jabittan
The human mind has incredible predictive abilities, but our explanatory abilities do not always turn out to be as equally incredible. Prediction is relatively easy when compared to explanation. Animals can predict where a food source will be without being able to explain how it got there. For most of human history our ancestors were able to predict that the sun would rise the next day without having any way of explaining why it would rise. Computer programs today can predict our next move in chess but few can explain their prediction or why we would make the choice that was predicted.
As Judea Pearl writes in The Book of Why, “Good predictions need not have good explanations. The owl can be a good hunter without understanding why the rat always goes from point A to point B.” Prediction is possible with statistics and good observations. With a large enough database, we can make a prediction about what percentage of individuals will have negative reactions to medications, we can predict when a traffic jam will occur, and we can predict how an animal will behave. What is harder, according to Pearl, is moving to the stage where we describe why we observe the relationships that statistics reveal.
Statistics alone cannot tell us why particular patterns emerge. Statistics cannot identify causal structures. As a result, we continually tell ourselves that correlation is not causation and that we can only determine what relationships are truly causal through randomized controlled trials. Pearl would argue that this is incorrect, and he would argue that this idea results from the fact that statistics is trying to answer a completely different question than causation. Approaching statistical questions from a causal lens may lead to inaccurate interpretations of data or “p-hacking” an academic term used to describe efforts to get the statistical results you wanted to see. The key is not hunting for causation within statistics, but understanding causation and supporting it through evidence uncovered via statistics.
Seeing the difference between causation and statistics is helpful when thinking about the world. Being stuck without a way to see and determine causation leads to situations like tobacco companies claiming that cigarettes don’t cause cancer or oil and gas companies claiming that humans don’t contribute to global warming. Causal thinking, however, utilizes our ability to develop explanations and applies those explanations to the world. Our ability to predict different outcomes based on different interventions helps us interpret and understand the data that the world produces. We may not see the exact picture in the data, but we can understand it and use it to help us make better decisions that will lead to more accurate causal understandings over time.