I don’t like doing online personality quizzes. Part of the reason why I dislike them is because I believe that three of the cognitive errors and biases identified by Daniel Kahneman in his book Thinking Fast and Slow are at play when we take online quizzes.
First, we are influenced by the availability heuristic. Our perception of how common or how accurate something is can be greatly influenced by whether we have an easy or hard time remembering the thing. This can influence how we answer questions about things we normally prefer or normally like to do. We might be answering based on how quickly we remember something, not on how we actually feel about something.
Second, we might substitute the questions being asked with easier to answer questions. In reality, this is what is happening with the availability heuristic. A difficult self-reflection question might not be answered directly. We might switch the question out and instead answer a simpler question. In the case of the availability heuristic, we are answering how easily something came to mind rather than the original question, but this can happen outside of the availability heuristic as well. The result is that we are not really measuring what the question purports to measure.
Third, Kahneman argues that we can think of ourselves as having two operating systems for how we act and feel in the present moment versus how we reflect back and remember previous experiences. The remembering self has different perceptions than the experiencing self, as Kahneman terms the two systems. The remembering self doesn’t have an accurate memory for how much we liked or disliked certain experiences. Think about a vacation. You may be feeling burnt out with work and life, and all you want to do, what you would enjoy the most in the world, is to sit on a familiar beach doing absolutely nothing. But your remembering self won’t take any exciting and novel memories from a week sitting on a beach doing nothing. Your remembering self would much rather have you go on an exciting yet stressful vacation to a new foreign country. This tension between your experiencing and remembering selves makes the reliability of online personality quizzes questionable. Your remembering self answers the questions, not your experiencing self, and they don’t always have the same opinions.
What this means, is that the kind of reflection that goes into online personality quizzes, or really any reflective activity, can potentially be self-deceptive. Quassim Cassam writes about these dangers in his book Vices of the Mind. He writes, “there is always the danger that what critical reflection produces is not self-knowledge, but self-deceptive rationalization.” Our biases and cognitive errors can lead us to incorrect answers about ourselves during self-reflection. This process can feel honest and insightful, but it can often be nothing more than a rationalization for behaviors and actions that we want to believe are true about ourselves. The only way through, Cassam continues to explain, is to cultivate real epistemic virtues, to see the world more clearly, and to recognize our epistemic vices to become better thinkers.
Novel Learning 
Two comments:
First,
I wonder how often the experiencing and remembering selves agree/disagree. Even if they are mostly independent systems, there are surely situations where they tend to agree. My memory of the days just before defending my thesis is that of constant anxiety. If we could go back to that time and sample my experiencing self, how likely is past-Jordan to report feeling anxiety? Of course, the most interesting thing about K’s concept is when there is disagreement between the experiencing and remembering self, but if we’re to extrapolate from the research it would be helpful to know common disagreement occurs in everyday life.
Second,
Perhaps there is a spectrum to consider for the probability that a survey item will encourage self-deception. For example, extroversion scales often ask people how much they agree with statements like “I enjoy having lots of people around to talk with”, but it isn’t obvious that there is a single answer to this question that most people find socially desirable. In this case, introverts often find extroverts exhausting and extroverts often find introverts to be boring.
Conversely, items for something like conscientiousness (e.g., “I often check my work over repeatedly to find any mistakes”) seem much more likely to produce self-deception because there is a strong consensus that low conscientiousness is undesirable.
Extroversion and conscientiousness might not be perfect examples, but the basic point is that there are probably some personality scales that are less vulnerable to self-deception than others.
LikeLike
Oops, first paragraph should end: “[…] would be helpful to know HOW common disagreement occurs in everyday life.”
LikeLike