SLIDES
Download Slides (pdf)
TRANSCRIPT
Download Transcript (pdf)
Slide 1:
Hello. My name is Lakshmi Kuchipudi. I am a senior scientist at Bio-Rad Laboratories. The “QC Design: Things You Need to Know” series is intended to educate the audience on statistical quality control design and testing. In the 1st pearl (Why Do We Need Quality Control?), which is the introduction to this series, we discussed different types of out-of-control conditions, why we need statistical quality control, and briefly about the quality of a patient result. We also vaguely defined an unreliable patient result. In this 2nd Pearl, we will discuss the quality requirements (i.e. the allowable total error, also known as TEa) used to define a patient result as unreliable.
Slide 2:
First of all, let’s discuss why we need these quality specifications. The quality specifications dictate the performance characteristics of our test system so we know when the test system fails. In the absence of the quality specifications, there is no way for us to determine if the control procedures being utilized are appropriate or if the quality of the patient result is acceptable.
Slide 3:
Quality specifications are generally defined in terms of allowable total error limits, TEa. The allowable total error limits specify the measurement error requirements that must be met for patient results to be fit for use. In the 1st pearl, we discussed how every patient result has some kind of measurement error even when the test system is in control. But if there’s an out-of-control condition, additional error is added due to the out-of-control condition. If the difference between the value reported and the actual value that should have been reported is greater than the specified allowable total error limit, then we consider the patient result unreliable, which means that the patient result fails to satisfy its intended purpose.
Slide 4
I am going to demonstrate this concept using animations. The left panel shows patient testing in time. We are using Glucose as an example here. The black dots are Glucose test results. The right panel shows the patient distribution for Glucose. Even when the process is in-control, every patient result has some measurement error. For example, if you test Glucose on a single patient multiple times, you wouldn’t get the exact same value each time. There will be random measurement error added to the actual values that reflects the inherent analytical imprecision of the test method. The black dots here represent the measured results. The black vertical bars represent the degree of imprecision in the patient test results. Now let’s plot the measurement errors shown in the lower left panel. In the lab, we never know the magnitude of the measurement error in each patient result, but here we do because we are simulating the patient results. The dotted horizontal lines represent ±10% TEa limits. The lower right panel is plotting the measurement error distribution. You can see the measurement error is centered at 0 and the distribution of measurement errors appears symmetric with a bell-shaped curve appearance even though the distribution of patient glucose results is skewed. As we test more patients, the distribution grows and you can see that all the measurement errors are within the allowable total error and hence no Glucose test result is unreliable.
Slide 5:
Now let’s say there is an out-of-control condition that caused a 5% upward shift in the process. SE here represents the Systematic Error condition. The lab is unaware that the out-of-control condition exists. Again, every test result has some measurement error; now, with an out-of-control condition, additional error is added. When you look at the Glucose test results (in the top left panel), you can’t easily identify which Glucose test results are unreliable (the red dot is an unreliable Glucose test result). If we plot the measurement errors (as shown in the bottom left panel) that include the out-of-control condition, you can see the measurement errors shift up and that the glucose test results in red are outside the allowable total error limits making those 2 results unreliable. As more glucose measurements are made, you can see only a few red dots which are the unreliable glucose test results. If you look at the measurement error distribution, you can see it’s no longer centered at 0 but centered at 5%. You can clearly see the shift in the measurement errors and its distribution. The measurement error distribution in the lower right panel shows the fraction of patient results (in red) that are unreliable results.
Remember, all we know is the patient test results and the patient distribution we see in the upper panel. We don’t know the measurement errors shown in the lower left panel. However, we can model the distribution of measurement errors shown in the lower right panel and predict the fraction of patient results that are unreliable.
Slide 6:
How to obtain this allowable total error limit? Actually, there’s no set defined formula or a single procedure to follow in the literature. There was a conference held at Stockholm, Sweden in April 1999 by the IFCC, WHO, and the International Union of Pure and Applied Chemistry where a hierarchy for defining quality specifications was developed. There were participants from 23 different countries with published papers on various quality specification models who attended the conference.
Slide 7:
This is the quality specification hierarchy and it defines quality specifications in 5 different areas.
- Quality specifications in specific clinical situations
- Quality specifications based on general clinical use of test results
- Quality specifications from professional recommendations
- Quality specifications based on regulation and external quality assessment
- Quality specifications based on State of the Art.
Let’s briefly discuss each of these.
Slide 8:
Quality specifications in specific clinical situations. The error specifications are designed based on the assessment of analytical performance in specific clinical situations. The problem is very little analysis has been done to specify the error criteria for universal use.
Slide 9:
Quality specifications based on general clinical use of test results fall into two categories:Specifications based on biological variation and specifications based on analysis of clinicians’ opinions. Specifications based on biological variation are firmly based on medical requirements. In Biological Variation: From Principle to Practice (Washington: the Association for Diagnostics & Laboratory Medicine (formerly AACC) Press; 2001), Dr. Callum Fraser explains why it’s important to set your quality specifications based on biological variation, describing the effects of with-in individual variation and between individual variation and why it’s important to include these factors when designing your TEa specifications. I will briefly discuss these.
Slide 10:
The quality specifications based on biological variation are derived by evaluating the inherent biological variation of an analyte and by determining how large imprecision and bias can be before they mask significant biological changes in the analyte. You can use this equation to compute the TEa values based on biological variation. The allowable imprecision is based on within individual variation and allowable bias is based on within individual and between individual biological variation. If you want to compute the TEa using this formula, the book by Dr. Callum Fraser (Biological Variation: From Principle to Practice. Washington: the Association for Diagnostics & Laboratory Medicine (ADLM) Press; 2001) has more details. If not, there are published values for biological variation and you can use them. This is the link to the published values: http://www.qcnet.com/Portals/0/PDFs/BVValues1Final.pdf.
Slide 11:
Three sets of performance specifications have been derived for biological variation values. Minimum, Desirable, and Optimum performance values. Now which specification should be used? The biological variation databases are published using desirable specifications. The quality specifications generated using BV Desirable is generally applicable. They are the most widely and frequently used. The BV optimum quality specifications are more stringent and should be used when the Desirable performance is easily achieved. The BV minimum quality specifications are less stringent and should be used when the Desirable performance specifications are not attainable. Minimum, Desirable, and Optimum factors can be used to tune the biological variation specifications for values achievable and suitable for your laboratory.
Slide 12:
Quality specifications based on biological variation have the following benefits:
- They are firmly based on medical requirements.
- They are usable in all laboratories.
- They are generated using simple models and are widely accepted.
Slide 13:
The 3rd Quality specification in the hierarchy is from professional recommendations. Various groups of experts have published quality specifications for specific sets of analytes. Two of those groups are listed here. The National Cholesterol Education Panel in the US has published recommendations for the precision, accuracy, and total allowable error for lipids. The American Diabetes Association has documented quality specifications for self-monitoring blood glucose.
Slide 14:
The next in hierarchy is the quality specifications based on regulatory agencies. CLIA’88 specifications have a set of mandated performance goals. E.g. CLIA specifies the Total Allowable Error for glucose as target value ± 6 mg/dL or target value ±10%, whichever is greater. For our animation example on slide 4 and 5, we used the CLIA ± 10% as the allowable total error. CLIA specifies the allowable total error for chloride as target value ± 5%. CLIA TEa limits tend to be wider than the other TEa limits and are currently undergoing review. So they should always be the minimum. Outside the US, there are proficiency programs which have a variety of mechanisms for grading performance. They can all be translated into analytical goal specifications.
Slide 15:
The last in the hierarchy is the quality specifications based on “State of the Art,” which refers to deriving quality specifications by what is currently possible. An example of a “State of the Art” precision specification might be to take the median CV from a group of laboratories. “State of the Art” specifications can be derived from proficiency testing programs or inter-laboratory consensus programs.
Slide 16:
In conclusion, consideration should be given to what quality specifications are used. Biological variation- based quality specifications are very defensible and should be the logical selection. CLIA’88 Proficiency specifications should always be the minimum. Later on in this series, I will discuss the concept of Sigma metric and how Sigma metric values are related to the TEa criteria. So stay tuned.
Slide 17: References
Slide 18: References
Slide 19: References
Slide 20: Disclosures
Thank you for joining me on this Pearl of Laboratory Medicine on “Allowable Total Error” from the “QC Design: Things You Need to Know” series. I am Lakshmi Kuchipudi. These Pearls of Laboratory Medicine are part of the Clinical Chemistry Trainee Council.