Listen to the Clinical Chemistry Podcast
Article
Liesbet Deprez, et al. Commutability Assessment of Candidate Reference Materials for Pancreatic α-Amylase. Clin Chem 2018;64:1193-1202.
Guest
Dr. Liesbet Deprez is from the European Commission’s Joint Research Center in Belguim.
Transcript
[Download pdf]
Bob Barrett:
This is a podcast from Clinical Chemistry, sponsored by the Department of Laboratory Medicine at Boston Children’s Hospital. I am Bob Barrett.
Pancreatic amylase concentrations are used to detect and diagnose acute pancreatitis and pancreatic injury. When used to aid clinical decisions, amylase results should be accurate and equivalent over time, across laboratories and measurement procedures. However, external quality assessments schemes and international studies have shown a wide interlaboratory variation for pancreatic amylase measurements in serum samples.
Standardization efforts for amylase have been limited by a lack of commutability in the current certified reference material. New candidate reference materials have been developed to potentially overcome this limitation, facilitating their use for calibration and trueness control of routine measurement systems. An original article appearing in the August of 2018 issue of Clinical Chemistry describes the commutability assessments of these new candidate reference materials for pancreatic alpha amylase. The authors investigated if a reference material with an artificial matrix could be commutable for routine methods and compared to different statistical approaches to assess the commutability of the candidate reference materials.
For this podcast, we’re joined by the article’s first author, Dr. Liesbet Deprez of the European Commission Joint Research Center in Belgium. Dr. Deprez specializes in the development of certified reference materials to support standardization of clinical assays. So Dr. Deprez, tell us why are you and your colleagues working on pancreatic alpha amylase?
Liesbet Deprez:
I'm working for the Joint Research Center of European Commission and our role is to support the EU policies. And one of our tasks is to support these policies by supporting measurements on these issues, standardizations in various areas, and so we provide reference materials for this. And one of these areas is the in vitro diagnostics or IVD.
Now for this specific case, we are talking about catalytic activity measurements in science in serum and this is quite particular, because the measurement result completely depends on the conditions of the measurement. Therefore, there are primary reference measurement procedures to measure enzymes in serum and they have been developed by the International Federation of Clinical Chemistry, also called IFCC.
So, these primary reference measurement procedures are the anchor points for the standardization, and we have developed in the past together with IFCC reference materials for these methods. One of them is the reference material for pancreatic amylase, but this reference material will be sold out soon, so we will replace it with a new certified reference material. And that’s why we conducted this commutability study because it's very important that the reference material is commutable. This reference material will have different background of a real clinic examples, it will not be a serum but it will be an artificial background that’s why the commutability for sure needs to be tested. And in this study, we tested four different candidate reference materials all with a different artificial background, and we added one serum sample as a control.
Bob Barrett:
Well let's quickly talk about that term “commutability,” can you shortly explain the concept and why it is so important for reference materials?
Liesbet Deprez:
So commutability of a reference material refers to the fact that the reference material should behave in the same way as a real clinical sample if you measured with different IVD methods. So, when you measure a reference material with the primary reference measurement procedure and with IVD methods, you will have the correlation between the two measurement results. And this correlation should be the same as if you would measure a real clinical sample, because if you will use the reference material to calibrate your IVD methods to work the reference measurement procedure, and your reference material is not commutable, and afterwards, after the calibration you will not have the same measurement result for the real clinical sample if you measured them with the different IVD methods.
So, to assess the commutability of a reference material, you need to do a separate commutability study, in which you analyze a large number of clinical samples and the reference material with the primary reference measurement procedure and with different IVD methods. Afterwards, you apply statistical analysis to see if the reference material is commutable for each of the IVD methods in comparison to the primary reference measurement procedure.
Bob Barrett:
In this study, you applied two different statistical approaches to assess the commutability of the candidate reference materials. What are the main differences between these two approaches and their outcomes?
Liesbet Deprez:
So yeah, on the one hand we applied the recently published guidelines of the IFCC working group on commutability, and on the other hand, we applied a kind of old school standard approach, which was prescribed in the guideline C30 from the Clinical and Laboratory Standards Institute. This guideline was published in 2002 and these methods have been the gold standard for quite some time.
The first main difference between the old approach and the new approach is that with the old approach, you only get a yes or no answer. You get yes, if it’s commutable or it’s non-commutable, but the old method doesn’t really quantify how close the reference material agrees with the clinical samples.
Now with the newer approach, you can quantify how close your agreement is, and you could also calculate the associated uncertainty. So, for the new approach as described at the IFCC working group, you can have three outcomes from the study: the reference material can be commutable, it can be non-commutable, or you can have an inconclusive result in case the uncertainty of the study is too large.
The second main difference is that how you set the criteria to define commutable or non-commutable. For the old approach, this criteria depended on how well the IVG methods correlated with the primarily reference measurement procedures. So the criteria could be different for the different IVD methods that you were testing, in case of the IFCC approach, the new approach, you will set the criterion based on the clinical performance criteria of the IVD methods, so you can apply the same commutability criteria for all methods.
So in the case, what’s now the outcome in our studies for the two different approaches, if we have a conclusive result for the IFCC approach, for the new approach, then this conclusive result is exactly the same as the result we get for the old approach. So, a reference material which is commutable according to the IFCC approach for a certain IVD is also commutable according to the old approach. But in several cases, we had that with the IFCC approach actually the outcome of the study was inconclusive, so actually the uncertainty of our study was too large.
Now the old approach does not quantify the uncertainty so in this case, you might make a wrong conclusion just by the fact that you don’t have the inconclusive outcome as a possibility.
Bob Barrett:
So based on this experience, would you recommend the use of the IFCC approach for further commutability study, and if so, why?
Liesbet Deprez:
Yes, I would for sure recommend this newer approach, for the reasons I said before, but also for the reasons that you can set your criterion also depending on the intended use of the reference material. So, if your intended used of the certified reference material is to use it as a calibration, you would set your criterion much stronger or much stricter than if you would intend to use the reference material just as a quality control.
Now there's one element of using this newer approach, and that is the fact that you need a large number of clinical samples, meaning at least 30, and that you also have to do a large number of replicate measurements to get a small uncertainty and therefore a conclusive outcome.
Now one way to solve this is to -- for a two-stage commutability study, let's say. First you could screen a large group of candidate reference materials in a smaller commutability study where you allow a quite large associated uncertainty, from this study you could select the few reference materials which are good and for which you want to continue. And then you can do a much larger study, in which you really prove the commutability of certain reference materials, and then afterwards, you can select the candidate reference materials which have the best commutability profile, and you can perform with them a more extended commutability study to really prove their commutability.
Bob Barrett:
Well finally doctor, did this study provide you the answers that you need to proceed with the production of a new certified reference material, and what are the next steps?
Liesbet Deprez:
Yes, for sure the study showed us that there was one candidate reference material which has the artificial background, which was as commutable as the reference material with the serum background, so we could go from there, form that point to really produce a new certified reference material. It also showed that the other reference materials were not commutable so we have to pay very much attention to how we construct the artificial matrix.
The study also showed that, for instance, lyophilization of the reference material does not affect its commutability which is good news for us because it makes production of a reference material which is stable over the long term much easier. So yes, the study did provide us with very useful information and it helps us with the production of the new reference material.
Bob Barrett:
Dr. Liesbet Deprez is from the European Commission’s Joint Research Center in Belguim. She is been our guest in this podcast from Clinical Chemistry. I’m Bob Barrett. Thanks for listening.