“If you are aren’t confused by quantum theory, then you haven’t really understood it” – Paraphrased quote by John Wheeler/Neils Bohr?
“Ignorance more frequently begets confidence than does knowledge” – Charles Darwin
Those two quotes, I feel are antecedents to this topic, in that they were the conceptual nuggets that come long before a theory. Although the first quote was really intended to show the counter-intuitive nature of quantum mechanics, I think it applies when discussing this effect.
I found out about this study just the other day, and I felt it gave some structure to various half-thoughts I have had about this topic. Like me, I’m sure you have experienced this.
So what is this effect? In short, it shows the slightly inverse correlation between ‘how good you are’ and ‘how good you think you are’. What the the paper highlights is that, usually people unskilled in an area, as they learn the rudiments, overestimate their abilities (beginners luck?) and those that are skilled underestimate their abilities. An effect attributed to lack of metacognition in the paper.
I think this study is already quite familiar, in that most people have, at one point or another, experienced this. Usually when we start to learn something (a skill, sport, subject) we tend to be a little too confident in our abilities and as we progress a little we begin to recognise our shortcomings.
I’ll go through the paper and studies Dunning and Kruger carried out 16 years ago and then start a discussion as to perhaps why this may occur. I would love to hear your thoughts on this as well!
The graph in the featured image is a little exaggerated for effect, (see the graph below from the study) but it highlights the effect well. It is quite paradoxical, the incompetent believe themselves to be competent, yet the solution to remove this discrepancy and make them realise they are incompetent, is to make them competent.
The paper discusses the 4 predictions the authors made (I’ve paraphrased slightly):
- Incompetent individuals, compared with their more competent peers, will dramatically overestimate their ability and performance relative to objective criteria.
- Incompetent individuals will be less likely to accurately gauge competence when they see it – be it in themselves or others. In short, they won’t know what they don’t know.
- Incompetent individuals will be less able to gain insight into their true level, by social comparison. Since according to prediction 2 cannot guage the competence in other people, they fail to make the necessary comparison between competent people and themselves and thus make an accurate evaluation of their abilities. (Takes one to know one)
- The incompetent recognise their incompetence, (paradoxically) by gaining skills and becoming more competent.
Incompetent is bit of a strong word, what they mean is not people who are incompetent in general (life) but in a particular set of skills which they tested. I just wanted to clarify as I will be using that word with that particular meaning.
They conducted 4 studies to test these predictions, each study consisted of tests that assessed the ability of the candidates in particular areas. Study 1 tested humour, studies 2 and 4 tested logical reasoning whilst study 3 tested english grammar.
The problem with prediction 1 is that, the lower ones score the higher chance of someone overestimating it. So they looked at how much someone’s estimate of their skill was miscalibrated with respect to their test scores.
65 Cornell university students were given jokes that were given a comedic value by several comedians (represents the expert opinion). The students also had to judge the comedic value of the joke and the more their score for the jokes coincided with the expert opinion the better their test score.
This was not asking they found the joke funny as that is quite subjective but whether the joke could be considered funny by most people. Although the test is not purely objective it does give a good indication, since the criteria by which we consider a joke to be funny is if everyone laughs, i.e comedy comes from the harmony of your humour with everyone else.
This is a graphical representation of the data they collected, and as you can see those who performed the poorest (bottom quartile) grossly overestimated their performance or ability to recognise humour. Although they certainly did not think to be as good as the top quartile they believed themselves to be slightly better than average.
They further investigated the empirical basis, by carrying out test 2 which I think was more rigorous and objective than the previous test.
This time 45 undergraduate students were given a 20 minute logical reasoning test, and were later asked to make three estimates.
How they performed compared to their peers (provide a percentile ranking), their logical reasoning ability compared to their peers, and how many test scores they answered correctly.
The results of test 2 reconfirmed the results of test 1, those in the bottom quartile had gross miscalibration between their perceived ability and actual ability and their perceived test scores and actual test scores.
Studies 3 and 4 were conducted in two phases. The first reconfirmed the results of the first two tests. The second phases were to test predictions 3 and 4.
They both delivered positive results, and confirmed their predictions. I am not going through those tests here, mostly because their method is largely repetitive and similar to 1 and 2 and also because the paper does a good job already of showing the results. (I’ll post the link to the paper below if you wish to read the last two studies)
Limitations of their analysis
Of course, for the incompetent to overestimate their abilities, they must have some rudimentary understanding of the area first.
No one would challenge Ronda Rousey, if they did not have any martial prowess. For the incompetent individuals to overestimate their ability, they would first need to have a certain threshold of knowledge or past experiences that give them the boosted confidence in their abilities, otherwise they would not even have intuition on how to respond.
In other areas, as they mentioned, competence does not depend on cognitive abilities but on physical skills. “One need not to look far to find individuals of impressive understanding of the strategies and the techniques of basketballs, for instance, but yet could not ‘dunk’ to save their lives (these people are called coaches)”
Their view (as highlighted in the prediction) is that those who are incompetent lack the metacognitive skills to accurately asses their true ability. Metacognition is I think what just mean self-awareness. Study 4 does reflect that, however, there are several other reasons why such a result could be seen, other than a deficit in metacognitive ability.
The effect has three main criticisms to the suggestion of a metacognitive deficit in low scorers, which are: Regression to the mean, Regression to the mean plus better-than-average, and the role of task difficulty.
Regression to the mean is a statistical artefact, whereby performers that score low, will on subsequent tests, score higher i.e closer to the mean, just by pure chance. The fact that the low scorers overestimated their ability is because perceived performance is dictated by more than actual performance. However, there is a lack of symmetry at the higher end, the high scorers should underestimate by just as much, but their underestimation is nowhere close to the amount of overestimation at the lower end.
Regression to the mean plus better-than-average. Krueger and Muller in a study in 2002, explained that people tended to have a excessively positive view of themselves, so the large overestimation at the lower end was a combination of regression to the mean and the self enhanced view. And at the top end, the underestimation due to regression to the mean was offset by self enhancement.
Task difficulty. Kruger and Dunning discuss this in the paper too, the tasks they carried out were relatively easy and hence confirms the effect, however this consistency disputes for harder tasks. There have been studies done on this by Burson and colleagues in 2006, but in the paper they discuss this too. “In some domains there are clear and unavoidable reality constraints.” No one denies their inability to translate obscure and ancient Sanskrit passages, or being able to build a rocket. In fact in difficult tasks like these, the bias is to rate yourself as bad as the next person.
Why is there such a large disparity between actually ability vs perceived ability at both extremes?
Perhaps that is just a function of how we learn. We learn through failure, negative feedback and what have you. But there are many causes of failure and too many variables (that each can cause failure in equal measure), whilst for success almost everything has to function correctly (ability, work, luck etc.) Failure can occur if even one of those aspects underperform. Thus pinpointing what caused the failure becomes clouded and thus it is hard to gauge which factor is bad and which is good and thus what the meaning of competence is in a particular area.
The paper discussed several other reasons why this might occur, one of which is motivational bias. This makes sense to me. If you were to embark upon learning a new skill, say kickboxing. It would be to the detriment of your learning process to accurately estimate your incompetence; you would be discouraged to even begin learning. But instead you might think, “I think I could learn this I am already quite agile and can throw a decent enough punch.” I recently started getting into Muay Thai, but before I started it is safe to say I definitely thought too highly of my conditioning (in painful retrospect).
This may just be a convenient lie we tell ourselves, before facing a challenge whose difficulty challenges our abilities. It may also shed some light on why we require teachers and mentors in our life; why can’t just do it (whatever it may be) on our own?
We need an experienced person to show us our incompetence and remove the cognitive deficit that causes us to blindside true competence. Furthermore, is this reason because we are social creatures or is this one of the reasons that causes us to be social creatures?
This also confirms why when we are young we tend to be brash and more sure of ourselves.
Of course what I have said here is just speculation based on the results that I read. The paper simply explored why and if people do hold “overly optimistic and miscalibrated views of themselves”. Those with limited knowledge suffering from a dual burden; not knowing enough, not knowing enough to know they do not know enough. There have been lots of other papers, providing reasons why this might occur. Perhaps this is just a statistical effect, and not a psychological one.
Whatever the reason for this may be, what should be taken away from this effect is to recognise constantly you are not as good as you think you are, remove the ego that says so and then expand your knowledge. Reinforce the idea that you know nothing.
Is Ygritte secretly Socrates?
For this work the pair won the Ig Nobel prize 2000, Socrates would have esteemed greatly.