I missed a PT challenge. What now?
When proficiency testing (PT) challenges are missed, the laboratory is presumed to have a problem with the test method, and this is an opportunity to identify and fix problems in the total testing process. If the challenge involves one of the approximately 100 analytes explicitly required by CLIA regulations, then a miss causing an unsatisfactory PT event (<80% correct) is likely to have regulatory implications. CLIA requires that the root causes for any PT miss be investigated and fixed, and the outcomes documented.
The consequences of a PT miss depend upon how many challenges were missed and the frequency of errors. PT events usually occur three times per year and most events involve five challenges. For most required analytes, missing only one of five challenges (80% score) is passing and the event is “satisfactory.” An “unsatisfactory” score will involve scrutiny by CLIA surveyors or accreditation organizations. Fortunately, by carefully reviewing PT results from every event, a laboratory can frequently detect and deal with analytical problems before they result in inaccurate patient testing.
What are the odds that a given PT result may be outside the CLIA acceptance limits (ALs)?
Typically, PT results are judged against results from other laboratories that use the same test method; the target value is the mean of equivalent test methods in the peer group. Some ALs are based on percentages and/or concentrations around the peer group mean. In other cases, ALs are three standard deviations, in which case the miss rate per challenge will typically be about 1%—or greater due to non-Gaussian distribution—regardless of the peer group’s overall precision. The risk of a specific PT result failing a challenge depends upon not only the laboratory’s accuracy relative to its peer group, but also the stringency of the analyte’s AL. Analytical performance goals should be tight enough to minimize the risk of missing a PT challenge, but perhaps even tighter to meet clinical needs based on the laboratory director’s judgment.
What should one do after a PT miss?
The response should begin with reviewing all the recorded data surrounding the PT event. Look for obvious transcription errors, including transposed results, miscalculations, and so on. Question the technologist who performed the analysis to assure the PT samples were handled correctly. Barring obvious pre- or post-analytic explanations, focus should include analytic causes, which would include systematic errors (bias) and random errors (imprecision).
To look for bias relative to the peer group, examine results for all challenges for the past few events. Are results consistently running below or above the peer group mean? Review the calibration and calibration verification records looking for shifts in quality control (QC), internal standards, and, if possible, the mean or median of patient results. Review maintenance records and compare reagent lot records against calibration records.
While sources of systematic error are more likely to cause misses for several PT challenges, sources of random error are likely to present as a relatively rare aberration. Reviewing QC results and previous PT results also helps to detect sources of random error. It may be helpful to review training records and competency assessment records. Of course, sometimes both systemic and random errors may simultaneously underlie PT misses.
In short, PT misses and unsatisfactory events help identify and correct underlying problems, as PT is designed to do. It is helpful to know the PT ALs and actively review all PT results, especially when they are approaching ALs. Even when there aren’t any misses, staff can keep ahead of sources of error. CLSI GP27-A2, “Using Proficiency Testing to Improve the Clinical Laboratory,” is especially helpful for interpreting patterns in PT results. PT programs and accreditation organizations can provide helpful advice, too.
Rex Astles, PhD, DABCC, FACB, is a health scientist at the Centers for Disease Control and Prevention (CDC) Division of Laboratory Programs, Standards, and Services. He leads a CDC team that is working with the Centers for Medicare and Medicaid Services to update the CLIA regulations for PT. Email: [email protected]