The Dunning-Kruger effect: Why does it happen? Why is it so tricky?

The assessment of one's own knowledge and skills is an important skill. Unfortunately, it is not easy to accurately assess yourself. This is reflected, among other things, in the Dunning-Kruger effect (DKE). In this article I discuss a study by Rachel Jansen and her colleagues into two possible explanations for the DKE. I also explain why the DKE is so tricky and why we intuitively tend to think that especially others are vulnerable to it.

The Dunning-Kruger effect

A well-known problem in self-assessment is the Dunning Kruger effect (DKE). Researchers Kruger & Dunning (1999) showed in a series of studies that people who score low on a task also lack the metacognitive ability to assess their own performance. These low performers overestimate their own performance the most on average. The higher people score, the more accurate their own assessment of their performance becomes. It should be noted that the highest performers, on average, slightly underestimate their own performance. The picture below illustrates the DKE.

Suggested explanations for the Dunning-Kruger effect

The dominant explanation for the DKE is that low achievers have a lower sensitivity for recognizing errors. They not only lack the knowledge to perform well on the task, but also the knowledge to recognize errors in their own performance. Researchers Krueger & Mueller (2002) thought that the DKE may not really exist and suggested a statistic explanation (rather than a psychological one) for the DKE: regression toward the mean. But additional analyzes (here and here) found no evidence for this hypothesis.

Does Bayesian reasoning explain the effect?

Researchers Jansen et al. (2021) investigated another possible psychological explanation that has been suggested for the DKE, namely that of a rational Bayesian inference. Bayesian probability theory boils down to learning through experience. You start with so-called prior beliefs (earlier, plausible assumptions) which you gradually adjust bit by bit on the basis of new available information.

If we view individuals as naive statisticians when analyzing their own behavior, the Bayesian approach is to combine the evidence from experience with one's prior beliefs. If those prior beliefs about yourself are that you generally perform well, this should result in you only slightly downgrading your overall positive self-assessment after a poor performance on one task. This could explain that low performers generally overestimate themselves after one or a few cases of low performance.

The research by Jansen et al.

Jansen et al. compared the following two explanations for the DKE:

1. the DKE reflects a truly reduced sensitivity to error among underperforming people
2. the DKE is a result of a rational Bayesian estimate

The researchers formalized these two competing explanations as mathematical models and performed large-scale replications to assess which model provides a better explanation for the DKE. Firstly, these analyzes revealed new evidence for the occurrence of the DKE. Second, it emerged that model 1 fits the data better than model 2. This study therefore supports the explanation for the DKE that low-performing people are less able to judge their own performance because they are less able to recognize mistakes.

The Dunning-Kruger effect is tricky (1)

The DKE effect is tricky because it is a double burden. The first burden is that of low performance. The second burden is that of a lack of insight into one's own low competence. This also makes it challenging to help people gain insight. We are not aware of our own low performance in the domain, but also of the fact that we are overestimating ourselves. If someone confronts us with our low performance, we can quickly react defensively. Even discussing the DKE can provoke a defensive response.

The Dunning-Kruger effect is tricky (2)

A second tricky aspect is that we may tend to underestimate the occurrence of the DKE in ourselves. We may intuitively think that it is primarily something that others may fall prey to. That is not surprising, because in others we can repeatedly and easily recognize the DKE. If we are quite good at something, we can recognize their low performance as such. And we may find that they overestimate their own performance. When we ourselves overestimate our own performance, we don't notice that we are doing it. Not only do we have a blind spot, but we also have a blind spot to the fact that it is a blind spot.

Intellectual humility

It is not so much the case that everyone is equally sensitive or vulnerable to the DKE. For example, there is some research that suggests that a fixed mindset increases the likelihood of the DKE occurring. But it would probably be wrong to think that certain individuals are not vulnerable to it, if only because everyone may sometimes end up in a fixed mindset from time to time.

I think the wisest way is to try to develop a kind of intellectual humility. It is appropriate to recognize your own competence in those areas in which you have worked (and are still working) for a long time and systematically to build up that competence (including by following the work of experts in those areas). But besides that, we must remain cautious about our own intuitions about our knowledge and skills in other areas. Even if we feel that our assessment is in those areas is accurate this may be very misleading.