American Association for Clinical Chemistry
Better health through laboratory medicine
Patient Safety Focus: The Inability of People to Recognize Their Own Incompetence

Patient Safety Logo


The Inability of People to Recognize Their Own Incompetence
An Interview with David Davis, MD

David Davis, MD, is senior director for continuing education and improvement for the Association of American Medical Colleges and an adjunct professor of medicine at the University of Toronto. In 2006, he published an influential systematic review of physicians’ lack of accuracy in assessing their own competence (1). Referred to as the confidence-competence conundrum, the inability of human beings to recognize their own shortcomings isn’t unique to physicians. In this interview, Dr. Davis explores various aspects of this phenomenon, especially its implications for patient safety and continuing education in the lab. 

Michael Astion, MD, PhD, conducted this interview.

Q: Can you briefly summarize the general findings of studies of self-assessment among healthcare workers?
A:
There are three fundamental findings:

  1. Over a broad range of intellectual tasks, healthcare workers overrate their ability.
  2. The relationship between the actual competence of healthcare workers and their self-assessment of competence is unreliable. Often, there is no relationship, and occasionally there is even an inverse relationship.
  3. The most incompetent workers show the greatest gap between self-assessment and actual competence.

Q: Is this problem unique to healthcare workers?
A:
 No, across a wide range of disciplines, human beings are unable to be totally objective in their perceptions of their own performance. This is a human trait that is not unique to healthcare workers, and it appears to be independent of age or gender. The inaccuracy of self-assessment has been demonstrated for a variety of specialties and at all levels of training. 

Q: What are the implications of this finding for patient care?
A:
This finding has important implications for patient safety, especially in situations where healthcare workers believe they are competent, but in fact, have deficiencies when they are objectively assessed. The findings also affect our recommendations regarding the best approach for the continuing education of healthcare workers.

Q: What if I have a lab staff member who tells me she is very highly skilled at urine microscopy? Should I believe her?
A:
The staff member might be very skilled, modestly skilled, or incompetent. The point is that self-assessment is too unreliable to use to determine whether this person should be performing urine microscopy.

Q: Lab supervisors and directors usually manage sizable groups of people. What is your advice to them?
A:
My advice is:

  1. Do not assume that employees can accurately self-assess.
  2. Expect your least competent employees to have the most inaccurate self-assessment, and recognize that this is a danger to patients.
  3. Rely on objective competency assessment to determine competence.
  4. Perform objective competency assessment as frequently as is realistic.
  5. Focus competency assessment and continuing education on tasks that affect patient safety the most.
  6. For each staff member, provide feedback on the results of objective competency assessment.
  7. Make frequent, objective competency assessment followed by feedback a permanent part of work life—no matter how experienced the employee.

Incompetent lab workers frequently overrate their abilities.

Q: What are some examples of objective assessment?
A:
Examples include written exams, audits of a staff member’s work by an expert, lab practical exams, and performance on simulations. These are all useful for evaluating specific healthcare tasks.

Q: What are some examples of audits?
A:
Expert physicians can perform chart reviews of random cases and problematic cases and then provide feedback to the physician being assessed. This general method of case review with feedback is applicable to many other healthcare workers, including lab workers.

Q: Many tasks that are important to patient safety, such as the interpretation of a microscope-based laboratory test, can be objectively assessed using an exam. However, another important aspect of patient safety is teamwork. How do you objectively assess teamwork and other relational aspects of work?
A:
I agree that the ability of healthcare workers to function as a team is an important aspect of providing safe care. Team-work encompasses a variety of characteristics that can be evaluated, including availability for consultation and troubleshooting, flexibility to meet changing clinical demands, accurate and timely communication, and ability to get along with others. For a reasonably objective evaluation of the various aspects of teamwork, I favor the use of 360-degree reviews. In these reviews, the person being assessed is evaluated confidentially by every person on the team, including team members above, below, and at the same level in the work hierarchy. A survey instrument, which often uses a Likert scale (strongly disagree to strongly agree) to judge various components of teamwork, can be used as the assessment tool. The overall results of the assessment are gathered by the supervisor of the person being evaluated, and then the supervisor provides the feedback to help the employee improve performance (See Box below).

Teamwork Can Be Assessed Using a 360-degree Evaluation

Employee: Joe Laboratorian
Area of Assessment: Teamwork

Each characteristic was scored on a scale of 1(poor) to 5 (excellent).

 

Self-Assessment

Average rating by group
(360 degree evaluation,
N= 20 participants)

Availability for consultation

5

3.5

Flexibility to meet changing clinical demands

5

3.1

Accuracy of communication

5

3.3

Timeliness of communication

5

3.8

Ability to get along with team members

5

3.9

This example illustrates the common problem of inflated self-assessment.

Q: Some employees question why supervisors spend so much time on objective assessment. How can a supervisor convince an employee that objective competency assessment is important?
A:
You can do so by educating them about the inaccuracy of self-assessment and the implications for patient safety.

Q: Why is a 360-degree review better than simply having an experienced and trustworthy supervisor provide the feedback?
A: 
On individual tasks, the incompetent person, who also suffers from inflated self-assessment, may not believe the results of objective competency assessment provided by exams. For the more relational aspects of work, one person’s opinion may not be accurate. Even if it is, it might not be sufficient to convince an incompetent person to deal with the problem. The overall opinion of the team often carries a greater weight and is more likely to motivate change.

Q: Can healthcare workers improve their ability to self-assess?
A: 
 Yes, they can improve self-assessment by improving competency and by accepting accurate objective assessments as feedback. As people become more knowledgeable about their performance, and/or competent regarding a particular task or characteristic, their self-assessment tends to become more accurate (Figure 1). In addition, their morale usually improves. 

Figure 1
The Confidence-Competence Conundrum
 

Those scoring in the bottom quartile by objective measures show the largest gap between objective and self-assessment. (See references 1, 3)

Q: Are there methods for improving a person’s ability to globally self-assess over a wide range of areas including teamwork and specific tasks? 
A: 
Unfortunately, nobody has discovered a method for this. You can only make progress by establishing competency for each task or knowledge domain, but this makes the journey to improved self-assessment a slow one.

Q: Do you believe in building teamwork by having elaborate retreats that include participation in ropes courses and group cooking classes?
A:
 
Although those activities sound fun, I think it is better to put the time into working on your work and improving your work competency. 

Q: What are the implications of inflated self-assessment for continuing education?
A:
 The main problem caused by inflated self-assessment is that healthcare workers tend to choose continuing education topics they like but not what they need. Continuing education requirements should actually be based on the results of objective competency assessment. In addition, it is helpful to incorporate the opinions of experts familiar with an individual’s work.

Q: Are you saying that the choice of continuing education topics should not be based on a person’s self-assessment?
A:
 Physicians and most other healthcare professionals are currently asked to pick their own continuing education activities. Essentially, we are asking healthcare workers to determine their own learning needs. Unfortunately, the general tendency of people to inaccurately self-assess means that they often do not pick the topics on which they need the most education. 

Q: Is there a system for assessing the educational needs of healthcare workers that is superior to the self-direction we currently rely on in the U.S.?
A:
I think the recertification system used for primary care physicians in the U.K. is better than that used in the U.S. In the U.K. system, peer mentors spend time with the physician on a regular basis. The mentor reviews charts and then provides some external feedback regarding learning needs. In this way, learning is not guided solely by physician self-assessment.

Q: Thanks for an informative interview on a topic that is often neglected by patient safety experts.
A:
 My pleasure. I hope your audience can use the information to guide their approach to competency assessment and continuing education.

REFERENCES

  1. Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence. JAMA 2006;296:1094–1102.
  2. Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology 1999;77:1121–1134.
  3. Haun DE, Zeringue A, Leach A, Foley A: Assessing the competence of specimen-processing personnel. Laboratory Medicine 2000;31:633–637. Freely available on the Metapress Website, accessed July 28, 2008.

Found in the Scientific Literature
The Inaccuracy of Self-assessment

Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence leads to inflated self-assessments. Journal of Personality and Social Psychology 1999; 77:1121–34.

In this classic paper, Kruger and Dunning compared self-assessment to objective assessment in three areas: grammar, logical reasoning, and humor. The majority of participants over-rated their abilities on all three, with those in the bottom quartile of performance having the largest gap between self-assessment and objective assessment. On average, participants in the bottom quartile scored in the 12th percentile of ability, but they placed themselves in the 62nd percentile. The authors suggest that those who are incompetent face a two-fold burden: they are incompetent, and their incompetence decreases their ability to recognize this.

Haun DE, Zeringue A, Leach A, Foley A. Assessing the competence of specimen-processing personnel. Laboratory Medicine 2000;31: 633–637. Freely available on the Metapress Website, accessed July 28, 2008.

These authors applied the work of Kruger and Dunning to specimen processing. Written exams were used to objectively assess the knowledge and judgment of specimen processing personnel. Exam questions covered three areas:

  • Medical knowledge, such as the urgency of different types of test orders.
  • Medical definitions used in test ordering and specimen processing, such as venous, amniotic, and HIV.
  • Problem solving, such as knowing what to do when a specimen is mislabeled or it was collected in the wrong tube.

The results of the exams were compared to self-assessment by the same specimen-processing personnel. In all three areas, the least competent employees judged themselves to be about as competent as the most competent employees. As in the previous study, the biggest gap between actual competence and self-assessment of competence occurred in the group of least competent employees.


Patient Safety Focus Editorial Board

Chair
Michael Astion, MD, PhD
Department of Laboratory Medicine
University of Washington, Seattle

Members
Peggy A. Ahlin, BS, MT(ASCP)
ARUP Laboratories
Salt Lake City, Utah 
James S. Hernandez, MD, MS
Mayo Clinic College of Medicine
Rochester, Minn.
Devery Howerton, PhD
Centers for Disease Control and Prevention
Atlanta, Ga.

Sponsored by ARUP Laboratories, Inc.
ARUP Logo