Challenges with the Scientific Process: Setting Priorities & Managing Conclusions

Challenges with the Scientific Process: Setting Priorities & Managing Conclusions

Science provides objective answers to questions about the world, but that doesn’t mean that science is an entirely objective enterprise. Science exists within a world dominated by human needs, biases, and prejudices which means that science can be impacted by the same political, discriminatory, and mistaken judgements and decisions that any other human activity can be overwhelmed by. In his book Sapiens, Yuval Noah Harari shows how this happens when it comes to selecting scientific research topics, setting the priorities of science, and when objective conclusions flow into the world where they can be used by less than respectable actors.
 
 
Harari writes, “science is unable to set its own priorities. It is also incapable of determining what to do with its discoveries.” Part of the reason why science cannot set its own priorities because science is expensive. Especially as we continue to make new discoveries, the subsequent steps require more time, energy, and resources. To discover the next quantum particle will require an even more impressive supercollider. To discover the next secret of the Amazon river will require taking new technology further up river. The cost grows, and individuals conducting research need to be able to convince those with resources to commit those resources to their particular interests. This means that science doesn’t unfold uniformly or in equal ways. As Harari puts it, “to channel limited resources we must answer questions such as what is more important and what is good? And these are not scientific questions.”
 
 
But even when good science is done, and even when accurate and objective measurements are obtained with reasonable conclusions drawn from those measurements, the impact of science can be unpredictable. Many scientific studies and results are obscure, with very few people outside a select expert community ever hearing about the results. But other conclusions can be taken out of their original context and can become part of the cultural zeitgeist. How studies and their conclusions are understood can get away from the researchers, and can be used to further specific political or economic goals, even if those goals really don’t have a real relationship to the original conclusion that was drawn. Harari demonstrates how this happened with scientific conclusions being merged with racist ideas about the inferiority of non-white people. He writes, “racist theories enjoyed prominence and respectability for many generations, justifying the Western conquest of the world.” Whether researchers were explicitly racist or not, their research was adopted by people who were, and used to justify unsavory political ends. The science became wrapped up in a political culture that wanted to justify discriminatory and prejudiced behaviors and attitudes.
 
 
This doesn’t only happen with racist ideas, though those ideas can be the most prominent and dangerous. Small scientific findings can be taken up by militaries, by corporations, and by media organizations which may use the research and findings in ways the authors could not have predicted. Research on technology that helps improve light detection could find its way into a guided missile, into mass surveillance systems, or onto the grocery store shelves to be used by advertisers. The science itself cannot control the way that results end up being used in the real world, and that can be problematic.
Science, Money, & Human Activities

Science, Money, & Human Activities

The world of science prides itself on objectivity. Our scientific measurements should be objective, free from bias, and repeatable by any person in any place. The conclusions of science should likewise be objective, clear, and understandable from the outside. We want science to be open, discussed, and the implications of results rigorously debated so that we can make new discoveries and develop new knowledge to help propel humanity forward.
 
 
“But science is not an enterprise that takes place on some superior moral or spiritual plane above the rest of human activity,” writes Yuval Noah Harari in his book Sapiens. Science may strive for objectivity and independence, but it still takes place in the human world and is conducted by humans. Additionally, “science is a very expensive affair … most scientific studies are funded because somebody believes they can help attain some political, economic, or religious goal,” continues Harari.
 
 
No matter how much objectivity and independence we try to imbue into science, human activities influence what, how, and when science is done. The first obstacle, as Harari notes, is money. Deciding to fund something always contains some sort of political decision. Whether we as individuals are looking to fund something, or whether a collective is looking to fund something, there is always a choice between how the final dollars could be used. Funding could be provided for science that helps develop a vaccine that predominantly impacts poor people in a country far away. Funding could be provided for a scientific instrument that could help address climate change. Or funding could be used to make a really cool laser that doesn’t have any immediate and obvious uses, but which would be really cool. Depending on political. goals, individual donor desires, and a host of other factors, different science could be funded and conducted. The cost of science means that it will always in some ways be tied to human desires, which means biases will always creep into the equation.
 
 
It is important to note that science is built with certain elements to buffer the research, results, findings, and conclusions from bias. Peer review for example limits the publication of studies that are not done in good faith or that make invalid conclusions. But still, science takes place in society and culture and is conducted by humans. What those individual humans chose to study and how they understand the world will influence the ways in which they choose and design studies. This means that bias will still creep into science, in terms of determining what to study and how it will be studied. Early material scientists working with plastics were enthusiastic about studies that developed new plastics with new uses, where today materials scientists may be more likely to study the harms of plastics and plastic waste. Both fields of research can produce new knowledge, but with very different consequences for the world stemming from different cultural biases from the human researchers.
 
 
This is not to say that science cannot be trusted and should not be supported by individuals and collectives. Science has improved living standards for humans across the globe and solved many human problems. We need to continue pushing forward with new science to continue to improve living standards, and possibly just to maintain existing living standards and expectations. Nevertheless, we do have to be honest and acknowledge that science does not exist in a magical space free from bias and other human fallacies.
Truth is a Poor Test for Knowledge

Truth is a Poor Test for Knowledge

We live in what is being called a post-truth world, where facts don’t seem to stand up on their own and motivated reasoning drives what people believe. Politicians, activists, and people of note say wild things without regard to accuracy. Against this backdrop, many people have begun to argue that we need more truth in our news, statements, and beliefs.
 
 
This quest for truth is noble, but also has its downsides. The COVID-19 pandemic is an example of how standards around truth can become self-defeating and can contribute to people’s motivated reasoning and cynicism around information. Science has moved very quick with regard to COVID-19, but that has often meant changing recommendations for how to stay healthy. We have changed what we know about infection rates, hospitalization rates, treatment, prevention, and death. This means that what people know and believe about the disease may change on a weekly or monthly basis, and consequently public policy and recommendations change. Unfortunately, that change can be a difficult process. Former Press Secretary Sean Spicer unfairly used the quick changes in science around COVID for political purposes in a tweet. On the other end of the spectrum, people are not happy with how slow some regulations update in the face of changing science, as George Mason Economist Bryan Caplan unfairly mocked in another tweet.
 
 
Yuval Noah Harari would argue that truth shouldn’t be the goal. In his book Sapiens, Harari writes, “truth is a poor test of knowledge. The real test is utility. A theory that enables us to do new things constitutes knowledge.” We treat scientific knowledge and information about the world as clear and deterministic. The reality is that our scientific knowledge and understanding of the world is incomplete, especially on a personal level. We all live with models for reality, and we should not make complete truth and accuracy our goal. We should strive to be as accurate and truthful as possible, but we should recognize that knowledge comes from how well our models work in the real world. Improved information along with more accurate and true knowledge should help us perform better, do new things, make new advances, and improve the world. We don’t have to mock science, policy, or the statements of others. We need to look for ways to update our models and theories so that we can do the most with what we know. We should be willing to update when we learn that our information is not true or accurate. Holding ourselves to impossible truth standards doesn’t help us build knowledge, and can actually be an obstacle to developing knowledge. 
Scientific Observations & Math

Scientific Observations & Math

My last post was about science and newness. Modern science values new information more than existing information and rewards research that pushes forward into new territories. What unites new science in any field with the historical information that the new science rests on, is mathematics. As Yuval Noah Harari writes in his book Sapiens, “mere observations, however, are not knowledge. In order to understand the universe, we need to connect observations into comprehensive theories. Earlier traditions usually formulated their observations into stories. Modern science uses mathematics.”
 
 
Mathematics are used to communicate observations because mathematics can be objective, precise, and evaluated for accuracy.  My experiences of reality and how I may interpret and communicate that reality is not likely to be the same as the way someone in New York City, Tokyo, or Kabul experiences, interprets, and communicates their immediate reality. However, if we chose to measure our worlds through data and agree on the scales to use, we can begin to bring our subjective experiences of reality into a unified and consistent framework. A lot of how we understand the world is subjective. For example, I run a lot and a lot of my friends run, so a three mile run sounds short to me. However, for someone who doesn’t run often and doesn’t have friends who run often, a three mile jog may as well be a 26 mile marathon. Mathematics escapes the subjective, goes beyond stories and narratives that we may develop from our subjective experiences. It ties our collective experiences together into something more objective. Mathematics allows us to go from stories to real theories.
 
 
That still doesn’t mean we all understand and interpret the numbers the same. In his recent book How to Make the World Add Up, Tim Harford shares an example of national statistics in the UK showing that the average rail car has only 100 passengers. However, in Harford’s experience, traveling at rush hour, the average rail car is completely packed with far more than 100 people. The statistics can be viewed through a different reference point, through the average passenger traveling at rush hour, or through the rail car traveling throughout the day. Without mathematics we could never describe this reality in a consistent and unified way. Our descriptions of the world would be based on narrative and story. Mathematics gives us a grounding through which we can understand the universe in a more comprehensive and generalizable manner.

Newness in Science

“Modern science has no dogma,” writes Yuval Noah Harari in his book Sapiens. “Yet it has a common core of research methods, which are all based on collecting empirical observations – those we can observe with at least one of our senses – and putting them together with the help of mathematical tools.” Harari continues to explain that science may not be dogmatic, but that there are two key defining aspects of modern science that set it apart from the ways that humans have traditionally understood the world. Those two aspects are the reliance on mathematics in understanding observations and the desire to seek out new knowledge and observations.
 
 
Mathematics gives us a common language to discuss observations and allows us to compare observations for veracity. Newness pushes our observations and knowledge in a continuously expanding manner. Relying on tradition, historical knowledge, and existing information has not been enough for science to advance. Newness has been a central idea in the basic structure of modern science. Harari writes, “as modern people came to admit that they did not know the answers to some very important questions, they found it necessary to look for completely new knowledge.”
 
 
New information is rewarded in academic institutions and drives the way that modern universities work. You can work at a college as a lecturer without doing research, but the prized positions are primarily research positions. As a researcher at a university you are rewarded for the number of papers you publish, and journals want to publish novel scientific studies. The goal of science today is to take what we already know and push beyond. Science doesn’t just help us better understand what has come before us, but helps us push into new worlds. We use math and the scientific method to make and communicate our discoveries.
 
 
This is a new approach from most of human history. We don’t simply assume we already have the answers or that our current knowledge will be sufficient into the future. We look backward less than we look forward. Science is centered around what we can do with our knowledge, expanding that knowledge, and doing new things with it.
 
 
Admitting Collective Ignorance

Admitting Collective Ignorance

I generally agree that we are too confident in our opinions and judgments about the world. We live with a lot of complexity and very few of us are superforcasters, carefully considering information and updating our knowledge as new information comes along. We rely on personal experiences and allow ourselves to believe things that we want to be true. However, there are some modern institutions which help push back against this knowledge overconfidence.
 
 
In his book Sapiens, Yuval Noah Harari writes, “Modern-day science is a unique tradition of knowledge, inasmuch as it openly admits collective ignorance regarding the most important questions “ (emphasis in original). As Harari notes, physicists openly admit that we cannot study what happened in the first moments of the Big Bang and scientists cannot explain what consciousness is or how it arises. On the largest problems science is great at admitting collective ignorance, something that is very unique among humans. Throughout our history many humans have attempted to answer the most important questions through narratives and stories, resulting in religions and dogmas. For modern science to eschew this trend is rather remarkable.
 
 
I would say that modern science actually goes a step beyond admitting collective ignorance in the largest questions we can ask. Something I often noted during my masters program, where we read a great deal of academic public policy and political science papers, was how often we could argue that authors were confident in their findings in the body of the paper (sometimes overstating the impact of their finding) only to admit in the conclusion that their study was limited in scope and could not be generalized to broader contexts. Within social sciences at least, papers encourage researchers to place their work within an appropriate context, and from my experience, the best papers do a good job of being honest and realistic about their conclusions. They admitted ignorance even when identifying effects that appeared to be real.
 
 
Humans cannot admit ignorance in business, politics, and religion. A CEO who admitted that the company didn’t really know what was happening and that they were operating from a place of ignorance probably won’t be CEO for long – especially not if they face a stretch of bad luck. Very few voters would elect a candidate who admitted to being ignorant on much of the world. Religions (in my view – which could be wrong) seem to provide more answers than admissions of ignorance (although Christians at least seems to admit that humans cannot understand their deity’s decision-making process).
 
 
Science is a unique place where we can admit that we don’t know much, even when announcing findings and things we have learned from careful study. This is one of the strengths of science and something we should do a better job communicating.  An admission of ignorance within science is a sign that scientific institutions are functioning well. That seems to have been forgotten at times during the pandemic, and often has been mocked by people who are unhappy with regulations and decisions by public policy officials.
Discovering That Humans Don't Know Everything

Discovering That Humans Don’t Know Everything

Humans have a great ability to explain the world through narratives that seem to make sense. One problem, however, is that our narratives may feel coherent, but fail to accurately reflect the true nature of reality. We are great at explaining why things happen a certain way, at identifying causal relationships between phenomenon, and creating reasons for why things are the way they are. We are not great at recognizing when we don’t know something and when our explanations couldn’t possibly be correct.
 
 
Yuval Noah Harari sees humans overcoming our tendency to create incorrect explanations of the world as a major development toward the Scientific Revolution. In his book Sapiens, he writes, “The great discovery that launched the scientific revolution was the discovery that humans do not know the answers to their most important questions.” From our modern vantage point, this is obvious. We just launched the JWST into space to help us understand questions to which we do not have answers. And even with the incredible power that the telescope has, it won’t tell us why there is a universe at all rather than nothing. There are questions we realize that we do not have actual scientific answers for.
 
 
In the past, we used narratives to explain the unexplainable. Religious explanations are not scientifically based, but create compelling and relatable narratives to why the world is the way it is. Magic and other supernatural phenomena explained everything from earthquakes to human economic behavior. They felt correct on an intuitive level, but couldn’t possibly explain reality in an accurate way.
 
 
The Scientific Revolution required that humans acknowledged gaps in knowledge. It required acknowledging that narratives and myth were insufficient to explain the true nature of the world. From this starting point humans could begin to make objective measures of the world around them, could test their causal explanations, and could begin to understand the world in a way that assumed there were lessons to learn and that myth and stories didn’t contain all the answers we need.
Imagined Orders Versus Natural Orders

Imagined Orders Versus Natural Orders

Imagined orders are myths that we agree upon and uphold through our actions and beliefs. There is no clear or objective basis to an imagined order to which everyone can agree at all times. Often, imagined orders exist on a continuum with numerous caveats and carve-outs as needed to maintain order and stability. They help shape our institutions and societies by creating a sense of common understanding and accepted beliefs and behaviors.
 
 
Natural orders, on the other hand, are the basis of the scientific theories and observations that humans can make. No matter where we are on the planet we can make the same observations of the speed of light, of protons and electrons, or of gravity. An important distinction is that natural orders exist whether we believe in them or want them to exist. Imagined orders only exist when we believe in them and want them to exist. Yuval Noah Harari describes it this way in his book Sapiens,
 
 
“A natural order is a stable order. There is no chance that gravity will cease to function tomorrow, even if people stop believing in it. In contrast, an imagined order is always in danger of collapse, because it depends upon myths, and myths vanish once people stop believing in them.”
 
 
We can ignore natural order, pretend it isn’t there, and abandon trust and belief in the scientific institutions that deliver knowledge regarding natural orders, but that doesn’t make the natural order itself go away. However, this is something that has occurred throughout human history with our imagined orders. The divine right of kings to rule is an institution that has been discredited and largely abandoned across the globe, but at one time was a powerful institution. Similarly, Roman and Greek religions were abandoned and were left for me to study in English class in high school as mythical stories. The myths which held the Soviet Union together also failed and were abandoned. Once a myth is no longer accepted, it is easily rejected as little more than fiction.
 
 
Harari argues that this fragility of myths is what drives us toward constant vigilance and ritual surrounding myths. Our judges wear long robes to appear more wise to help give credibility to their decisions. We hold large official and serious investigations around events such as the January 6th riot at the US Capitol to help preserve our electoral system. We play the national anthem ahead of sporting events to remind everyone of the fiction of our Nation. The reality, however, is that judges only have authority if we all recognize and agree that their words and declarations are important. Determining what was a violent riot and what was an impassioned plea for freedom can depend on perspective (though when it comes to January 6th and how objectively awful Trump was this one doesn’t seem defensible). And the United States isn’t a real thing. There is no clear reason why our country exists in the exact place that it does – indeed at one point the same territory existed but it was not the United States.
 
 
This doesn’t mean that these myths are bad or are not useful. They help us live our lives, cooperate, and coexist. They are useful fictions, even if they are fragile, built on little more than vague concepts and ideas, and require silly rituals like singing a special song before playing sports. 
The Most Durable Human Species Ever

The Most Durable Human Species Ever

In recent years our DNA sequencing techniques and abilities have become dramatically better. We are able to get DNA from ancient sources in a way that we previously had not been able to, and we are then able to sequence that DNA more accurately are carefully than ever before. What this has started to reveal is a greater diversity of ancient human species, a greater spread of various human species, and more diversity and intermixing of human species than we had previously thought. A lot of this research is cutting edge and evolving daily, but for the last decade this research has been shifting how we view ancient humans, which in turn shifts the way we view ourselves.
In Sapiens, published in 2011, Yuval Noah Harari writes, “the more eastern regions of Asia were populated by Homo erectus, ‘Upright Man’, who survived there for close to 2 million years, making it the most durable human species ever.” It is somewhat strange to think that one species of human existed for 2 million years, or close to that figure, and eventually was outcompeted by a different species of human. Homo sapiens, our modern human species, eventually outcompeted all the other species of humans, including those which existed for hundreds of thousands to millions of years before our species evolved and began to spread.
Since Harari’s book was published we have learned more about species he briefly mentions such as Homo denisova and how widely those species managed to spread across the Earth. Additionally, research during the COVID-19 Pandemic suggested that some individuals may have genetic mutations stemming from the genome of Homo neanderthalensis, which changed their immune response to the disease. To me, research on ancient humans through DNA is a powerful and humbling reminder that my life and experiences are not unique just to me and this moment. It reminds me that human evolution has been a long and complicated process, with many Homo Sapiens and other human species that could think, talk, and experience the world in similar ways coming before me. Harari also stresses that Homo Sapiens may not be the final version of humans to evolve and dominate the planet, or last the longest on the planet. He continues, “this record [the estimated 2 million years that Homo erectus survived] is unlikely to be broken even by our own species. It is doubtful whether Homo sapiens will still be around a thousand years from now, so 2 million years is really out of our league.”
Unorthodox Thinking & Large Budgets

Unorthodox Thinking & Large Budgets

“Surprising, occasionally game-changing things happen when flights of unorthodox thinking collide with large, abiding research budgets,” writes Mary Roach in her book Grunt: The Curious Science of Humans at War. Militaries face far more than just opposing combatants in war. Their needs go far beyond bombs, tanks, and fighter planes. Armies need lots of things for basic survival, and sometimes this means that large research budgets are devoted to small topics where institutionalized dogma has not set in. The results can be weird, sometimes less than what the army hoped for, and – as Roach notes in the quote above – sometimes game changing.
In Grunt, Roach highlights some of the examples of incredible, yet unexpected scientific breakthroughs that have come from military research. Medical trauma research, clothing research, and other less thought of research has been crucial for saving lives during war. These are not the first things we usually think about with armed conflict, but without winning in these small areas, armies may not be able to win on larger geopolitical stages.
Entire industries and societies may deal with problems for years without the huge funding and sometimes unorthodox thinking that an army can bring to a problem. Something small, like sweaty and sticky shirts, may plague people for years and be a minor annoyance, but for an army, where keeping moral and camaraderie up is a key for success, a sweaty and sticky shirt could end up being a life or death matter. Bringing in a scientific research team, that doesn’t have the same constraints as public researchers at a university or researchers for a for-profit corporation can open up new avenues of discovery. A researcher at a public university may be shunned away from research on sweaty, sticky shirts because they don’t want their colleagues to think they are working on a goofy topic. Private companies may not see enough of a profit motive in researching sweaty, sticky clothes and may not hire anyone in their R&D section to focus on the issue. But the army can provide some level of intellectual freedom in asking researchers to tackle strange areas and can bring the necessary funding to find a breakthrough. This idea is at the heart of the research that Roach presents in the book.
The sometimes game-changing breakthroughs are not the result purely of lots of money or purely unorthodox thinking. The breakthroughs are the result of a web of factors that include the money, the intellectual space for unorthodox thinking, and the willingness to allow people to focus on sometimes narrow or obscure topics. It also requires obsessives who are not afraid to spend years researching something strange or off-putting. the kinds of breakthroughs people make often don’t get much attention, even if they are very important and make it into daily life beyond the initial military use, but over time they pile up to become part of our way of life. Perhaps we would have gotten there independently of the military, but sometimes that extra funding and unorthodox thinking is needed to help push new innovations and discoveries.