I have not read A Hitchhiker’s Guide to the Galaxy, but I know there is a point where a character asks what’s the meaning of life, the universe, and everything, and receives a response of 42. The answer was certainly not what anyone was expecting, but it was an answer. Much of science is like the answer 42. We ask grand questions of nature and receive answers we didn’t quite expect and can’t always make sense of.
In The Book of Why Judea Pearl writes, “Nature is like a genie that answers exactly the question we pose, not necessarily the one we intend to ask.” We learn by making observations about the world. We can make predictions about what we think will happen given certain conditions and we can develop and test hypotheses, but the answers we get may not be answers to the questions we intended to ask. I frequently listen to the Don’t Panic Geocast and the hosts often talk about scientific studies that go awry because of some unexpected interaction between lights, between an experimental set-up and the sun, or because an animal happened to have messed with equipment in the field. Real results are generated, but they don’t always mean what we think they do on first look. The hosts have a frequent line that, “any instrument can be a thermometer,” to note how subtle changes in temperature can cause misleading noise in the data.
Pearl’s quote is meant to demonstrate how challenging science can be and why so much of science has taken such a long time to develop. Humans have often thought they were receiving answers to the questions they were asking, only to find out that nature was answering a different question, not the one the scientists thought they had asked. Pearl states that randomness has been one of the ways that we have gotten past nature, but writes about how counter-intuitive randomized controlled trials were when first developed. No one realized that the right question had to be asked through experimental set-ups that involved randomness. On the benefits of randomness he writes, “first, it eliminates confounder bias (it asks Nature the right question). Second, it enables the researcher to quantify his uncertainty.”
In the book, Pearl takes observations and statistical methods combined with causal insights to a level that is honestly beyond my comprehension. What is important to note, however, is that nature is not obligated to answer the questions we intend to ask. It answers questions exactly as we pose them, influenced by seemingly irrelevant factors in our experimental design. The first answer we get may not be very reliable, but randomness and statistical methods, combined as Pearl would advocate, with a solid understanding of causality, can helps us better pose our questions to nature, to be more confident that the responses we get answer the questions we meant to ask.