One of the successes of System 1, the name Daniel Kahneman uses to describe our quick, intuitive part of the brain in his book Thinking Fast and Slow, is recognizing emotions in people’s faces. We don’t need much time to study someone’s face to recognize that they are happy, scared, or angry. We don’t even need to see someone’s face for a full second to get an accurate sense of their emotional state, and to adjust our behavior to interact accordingly with them.
The human mind is great at intuiting emotions from people’s faces. I can’t remember where, but I came across something that suggested the reason why we have white eyes is to help us better see where each other’s eyes are looking, and to help us better read each other’s emotions. Our ability to quickly and intuitively read each others’ faces helps us build social cohesion and connections. However, it can still go wrong, even though we are so adept.
Kahneman explains that biases and baseless assumptions can be built into System 1’s assessment of faces. We are quick to notice faces that share similar features as our own. We are also quick to judge people as nice, competent, or strong based on features in their faces. This is demonstrated in Thinking Fast and Slow with experiments conducted by Alex Todorov. He had showed potential voters the faces of candidates, for sometimes only fractions of seconds and noted that faces influenced votes. Kahneman writes, “As expected, the effect of facial competence on voting is about three times larger for information-poor and TV-prone voters than for others who are better informed and watch less television.”
I’m not here to hate on information-poor and TV-prone voters, but instead to help us see that we can easily be influenced by people’s faces and traits that we have associated with facial characteristics, even if we don’t consciously know those associations exist. For all of us, there will be situations where we are information-poor and ignorant of issues or important factors for our decision (the equivalent of being TV-prone in electoral voting). We might trust what a mechanic or investment banker says if they have a square jaw and high cheekbones. We might trust the advice of a nurse simply because she has facial features that make her seem caring and sympathetic. Perhaps in both situations the person is qualified and competent to be giving us advice, but even if they were not, we might trust them based on little more than appearance. System 1, which is so good at telling us about peoples’ emotions, can jump ahead and make judgement about many characteristics of people simply based on faces, and it may be correct sometimes, but it can also be wrong. System 2 will probably construct a coherent narrative to justify the quick decision made by System 1, but it likely won’t really have to do with the experience and qualifications of the person. We may find that we end up in situations where deep down, we are making judgments of someone based on little more than what they look like, and what System 1 thought of their face.