Listen to the Clinical Chemistry Podcast
Article
W. Korzun, G. Nilsson, L. Bachmann, G. Myers, I. Sakurabayashi, K. Nakajima, M. Nakamura, R. Shamburek, A. Remaley, and W.G.Miller. Difference in Bias Approach for Commutability Assessment: Application to Frozen Pools of Human Serum Measured by 8 Direct Methods for HDL and LDL Cholesterol. Clin Chem 2015;61:1107-1113.
Guest
Dr. Greg Miller is Professor in the Department of Pathology and Director of Clinical Chemistry at Virginia Commonwealth University.
Transcript
[Download pdf]
Bob Barrett:
This is a podcast from Clinical Chemistry, sponsored by the Department of Laboratory Medicine at Boston Children’s Hospital. I am Bob Barrett.
Commutability is an important concept in determining traceability and relationship among different methods used to measure the same analyte, but how to delineate commutable materials is not straightforward and a number of techniques have been proposed. In the August 2015 issue of Clinical Chemistry, a paper examines a difference in bias approach for assessing commutability of materials to be used in HDL and LDL cholesterol assays. The senior author of that article is Dr. Greg Miller. He is a professor in the Department of Pathology and Director of Clinical Chemistry at Virginia Commonwealth University, and he is our guest in today's podcast.
So Dr. Miller, commutability is a somewhat specialized topic, usually reserved for metrologists. Can you start by explaining what commutability is and why it's important to all laboratorians?
Greg Miller:
Well, commutability is a property of a reference material demonstrated by the closeness of agreement in the relationship between two measurement procedures; for a reference material and for a panel of individual clinical samples. So when there are several different measurement procedures, the relationship extends to all combinations of results among the different procedures.
Interestingly, commutability is not a new concept. It was first created as a word in 1973 by clinical chemists—Charles Fasce, Bob Rej, Bill Copeland and Ray Vanderlinde—to describe the ability of a reference or control material to have inter-assay properties comparable to the properties demonstrated by authentic clinical samples when measured by more than one analytical method. The importance of commutability has now been recognized not only by clinical chemists but by analytical chemists and metrologists, who work in the area of standardizing or harmonizing laboratory test results.
Commutability is extremely important in the context of calibration traceability to a reference material or to a reference measurement procedure. A commutable calibrator will ensure that the calibration relationship will be correct for clinical samples. A non-commutable calibrator will introduce an artifactual bias that will make the results or patient samples not traceable to the reference system and consequently not in agreement with results from other measurement procedures that are calibrated with the same non-commutable calibrator.
Commutability is also quite important in the context of external quality assessment also called proficiency testing. When a commutable reference material is used to assess the performance of different measurement procedures such as the frozen pooled sera in this report were intended, the relationship between methods will reflect the relationship expected for clinical samples.
Consequently, the EQA or proficiency testing data can be used to evaluate how well the results agree for patient samples among the measurement procedures. If a non-commutable material is used for external quality assessment then the relationships observed between different measurement procedures will have artifactual biases introduced by the non-commutability, and the results will not correctly identify the agreement expected for clinical samples.
Bob Barrett:
Is a difference in bias approach for assessing commutability, is that something new?
Greg Miller:
Yes, it is. Let me explain a little bit about how commutability has been traditionally evaluated. The usual approach uses regression analysis and a statistic called the prediction interval. The prediction interval reflects the variability among the results from the measurement procedures being evaluated.
Consequently, a criteria to accept a reference material as commutable vary among different combinations of measurement procedures for the same analyte. More imprecise methods will have large scatter among results, and consequently larger acceptance criteria that may label a reference material as commutable when it is close to a limit, when it is actually not suitable for use.
On the other hand, very precise methods may have very small scatter among results that would have very small criteria that would label a reference material as non- commutable when it could actually be quite suitable for use as a calibrator or as an external quality assessment sample to assess performance that properly reflects the performance expected for clinical samples. So the difference in bias approach allows the bias and uncertainty to be estimated and the same fixed criteria for commutability to be applied to all measurement procedures in a commutability assessment.
Bob Barrett:
Can you explain what a difference in bias approach adds to commutability assessment?
Greg Miller:
Yes, the principle thing the difference in bias does, it quantitates this difference between the reference material response and the clinical sample’s response, which allows the closeness of agreement in the commutability definition to be assessed directly rather than simply determining if the reference material and the clinical sample results are equivalent. The closeness of agreement approach allows the uncertainty of the difference in bias between reference material and clinical samples to be estimated and used in the assessment.
Consequently, the criteria used to make a determination of commutability can be based on medical requirements, which is not possible using the traditional approaches that are based on the reference material result being within the variability observed for the relationship of the clinical sample results between two measurement procedures. In addition, the difference in bias can be quantitated which opens a possibility to apply method-specific correction factors when using non-commutable reference materials in certain situations.
Bob Barrett:
Doctor, why did you decide to use this approach in the application to the off-the-clot frozen pools used in the CDC Lipid Standardization Program?
Greg Miller:
These pools that we evaluated were made using the same protocol that’s still in use by the CDC, as well as in use by other external quality assessment programs around the world Since external quality assessment is intended to determine if a laboratory or a measurement procedure meets the criteria for bias as well as criteria for imprecision set in this case, set in this case by the Laboratory Working Group of the National Cholesterol Education Program, the samples used must have commutability that’s fit for that purpose.
Development of an approach that could apply the medically relevant criteria of the NCEP Lab Working Group was important to answer the commutability question properly, using the same patient data we reported in 2010 that there was a substantial amount of scatter in patient sample results when measured by different direct methods for HDL or LDL cholesterol. Consequently, the traditional approach that used acceptance criteria based only on imprecision would have had rather large acceptance criteria, which we showed in the paper.
So we investigated the difference in bias approach, which was derived from the approach we used in that same 2010 paper to assess performance of the measurement procedures for patient samples when compared to the CDC beta-quantification reference measurement procedure. In that approach we estimate all the different types of imprecision and bias error components contained in the data. We can then use that data to assess bias and uncertainty in bias in the case of this commutability assessment.
Bob Barrett:
Doctor, in your paper quite a few results for reference materials were given as inconclusive, now how should laboratorians interpret those?
Greg Miller:
Well, inconclusive in this case means that statistically a conclusion cannot be made with 95% confidence. Stated a little bit differently, the reference material may be commutable or non-commutable, but the data is not sufficient for a conclusion, hence we use the term “inconclusive.” An inconclusive determination generally occurs because the uncertainty from the different sources of random error, including specimen-specific effects, is relatively large.
We showed in the previous paper, and actually repeated in the table in the current paper, that specimen-specific effects is the dominant source of random error in results for these direct HDL and LDL cholesterol methods. So the inconclusive conclusions reflect limitations in the selectivity of the HDL and LDL methods to measure the same biological molecules that are measured by the CDC beta-quantification reference measurement procedure.
Bob Barrett:
What are the implications for the finding that not all frozen pools were commutable for HDL cholesterol and that none were commutable for LDL cholesterol when criteria based on medical requirements were used to interpret the results?
Greg Miller:
Well, for HDL cholesterol, where one of four pools was not commutable based on the medical criteria for bias, the grading criteria used to evaluate a set of EQA or PT results needs to allow the possibility that an occasional apparent failure on one sample could be a commutability limitation of a particular frozen pool and should not be a reason to fail a participant for the entire event.
In the case of LDL cholesterol none of the frozen pools were commutable, which means that the EQA/PT process using frozen serum pools is really not suitable to assess laboratory performance. As a result, comparisons using freshly collected non-frozen clinical samples are needed to evaluate the performance of LDL cholesterol methods.
Bob Barrett:
Well, finally doctor, do you think the difference in bias approach will become more commonly used for commutability assessment?
Greg Miller:
Yes, I do. The key thing about the difference in bias approach is that you can quantitate that difference and you can set criteria based on medically relevant criteria that can be applied uniformly to all measurement procedures in a commutability assessment.
This approach is less arbitrary than the current approaches that are based only on the total variability between two measurement procedures. This approach also has the possibility to determine correction functions that could enable non-commutable reference materials to be used in carefully defined situations for calibration traceability. This approach of correction functions may allow us to overcome some limitations in achieving harmonized results that our profession has identified because not all reference materials will end up being commutable for all the different measurement procedures they might be useful for.
In addition, IFCC Working Group on Commutability is examining the difference in bias approach and is likely to recommend it because of these positive characteristics, in particular it assesses closeness of agreement, which is included in the definition of the term commutability. It also measures the uncertainty and has the flexibility to use whatever the most appropriate criteria are for evaluation of commutability.
Bob Barrett:
Dr. Greg Miller is Professor in the Department of Pathology and Director of Clinical Chemistry at Virginia Commonwealth University. He has been our guest in this podcast from Clinical Chemistry on assessing commutability for materials to be used in HDL and LDL cholesterol assays. I am Bob Barrett. Thanks for listening!