The Dunning-Kruger effect is one of the most famous and misunderstood cognitive biases. I want to compare the way researchers think of the Dunning-Kruger effect to the way many members of the general public think of it.

Here are two statements of the Dunning-Kruger effect, characteristic of these two groups:

General public: The Dunning-Kruger effect states that people who are bad at some skill think that they’re better at that skill than people who good at the skill think that they are at the skill.
Researchers: The Dunning-Kruger effect states that people who are bad at some skill have more of a tendency to overestimate their skill than people who are good at the skill.

These two descriptions sound almost the same, but there’s an important difference. Statement 1 is a claim of nonmonotonicity: It claims that as people get better at something, they learn humility and their self-perception diminishes. It claims that thinking you’re really good at something is negatively correlated with ability. Let’s call this “nonmonotonic DK”.

Here’s a diagram making the nonmonotonic DK claim:

The nonmonotonic DK claim is the standard understanding of the Dunning-Kruger effect among members of the general public. If I do a Google Image search of “dunning kruger effect”, 19 of the first 20 images are depictions of the nonmonotonic DK claim.

But if we look inside a research paper on the Dunning-Kruger effect, we see something very different, something much more akin to Statement 2. Rather than a nonmonotonic relationship between true ability and perceived ability, when a cognitive bias researcher talks about the Dunning-Kruger effect, they are making a claim of flattening: As people get better at something, their self-perception increases slowly. More competent people only think they’re a little better at the skill than less competent people. It claims that thinking you’re really good at something is weakly positively correlated with ability. Let’s call this “flattening DK”.

Here’s a diagram making the flattening DK claim:

Notice how as true competence increases, perceived competence increases more slowly. The data for this diagram comes from Dunning’s survey of the topic.

The flattening DK claim is the one that appears in almost every research paper on the Dunning-Kruger effect. In particular, in every single figure depicting researcher results in Dunning’s survey linked above, flattening DK is demonstrated, not nonmonotonic DK. When researchers say that “The Dunning-Kruger effect is extensively verified”, flattening DK is what they’re talking about. Nonmonotonic DK, in contrast, is barely even discussed, and certainly not widely believed.

The source of the confusion

When tracing the source of this confusion, we must go all the way back to the beginning. Researchers including Dunning and Kruger commonly conflate nonmonotonic DK and flattening DK. Nonmonotonic DK draws people in and gets people excited, while flattening DK is all that is actually demonstrated. For instance, in one of the foundational papers on the DK effect, Why the unskilled are unaware: Further explorations of (absent) self-insight among the incompetent, the authors, including Dunning and Kruger, start their paper with the following quote:

One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision. Bertrand Russell (1951)

This is fundamentally a statement of nonmonotonic DK – “those who feel certainty are stupid.” It is a claim of negative correlation between perception and reality.

However, immediately afterwards, the authors go on to state:

As Bertrand Russell noted, those most confident in their level of expertise and skill are not necessarily those who should be. Surveys of the psychological literature suggest that perception of skill is often only modestly correlated with actual level of performance, a pattern found not only in the laboratory but also in the class- room, health clinic, and the workplace.

This is a claim of flattening DK – “only modestly correlated”! The conflation between the two claims has been present from the start. The paper then goes on to demonstrate flattening DK, not nonmonotonic DK. A reader who is not reading very closely could easily come away thinking that nonmonotonic DK had been demonstrated, given the conflation of the claims. This conflation of claims, in this paper and others, is likely the original source of the confusion discussed above.

Further subdivisions within Flattening DK

I don’t want to leave you with an inaccurate understanding of what researchers (and Dunning and Kruger in particular) mean by the Dunning-Kruger effect, so let me get more specific within the flattening DK concept.

When researchers talk about the DK effect, they’re referring to a tendency of less competent people to think more highly of themselves, relative to their actual ability, than more competent people think of themselves, relative to their actual ability. To be mathematical about it, the difference “true ability minus perceived ability” is negative for people with less true ability, and becomes positive for people with more true ability.

This tendency is typically measured to be fairly weak, leading to a flattening DK effect, not a nonmonotonic DK effect.

Even within this concept, the mechanism of this tendency is also important. There are many possible mechanisms that could lead to this tendency, and when a researcher talks about “The Dunning-Kruger effect”, they are referring to a specific flattening mechanism. Here are two potential flattening mechanisms:

People who have less true ability tend to overestimate their ability, and people with more true ability tend to underestimate their ability. These over- and under-estimations are relatively weak, leading to a flattening DK effect.
What is actually measured, and labeled “true competence”, is someone’s performance on some task, like a score on a test. Suppose that this score is only weakly correlated with one’s true ability, perhaps because some questions were vague and people were just guessing. Suppose also that everyone has a perfect knowledge of their true ability, but not of their performance. When asked about their performance on the test, suppose that people report something based on their true ability. This report is labeled “perceived competence”. “True competence” and “perceived competence” are only be weakly correlated, due to the randomness inherent in the measurement. As a result, the observed flattening occurs.

Statement 1 is “the Dunning-Kruger effect”, and is a meta-cognition explantation, while statement 2 is a “regression to the mean” explanation, in which the Dunning-Kruger effect is a statistical artifact, rather than an interesting psychological phenomenon. Here “meta-cognition” refers to one’s ability to think about the quality of one’s own thinking.

While flattening DK is reliably demonstrated, differentiating Statement 1 from Statement 2 is more difficult, and requires more advanced statistical analysis. Nonetheless, when such analysis is done, the meta-cognition explanation comes out on top, both in one of the original studies, and in more modern replication.

This meta-cognitive effect, finally, is what researchers mean when they say “the Dunning-Kruger effect”.

Conclusion

The Dunning-Kruger effect as understood by researchers, is the flattening DK effect with a meta-cognitive mechanism. This is real, high-quality science, which illustrates a real effect in people’s minds.

The Dunning-Kruger effect as understood by the general public, is the nonmonotonic DK effect. This is not good science, is not widely demonstrated, and should not be accepted as a real effect in people’s minds. Rather, it is a way for people to feel smug and superior to others, especially people they think are less smart than themselves. This can be seen by the moralistic or insulting terms that are often thrown around in this context, such as “Mount Stupid”.

Let’s collectively shift towards talking about the real Dunning-Kruger effect, not the mythical, moralized version floating around the popular science information ecosystem.