Listen to the Clinical Chemistry Podcast
Article
E. Lenters-Westra and R.J. Slingerland. Three of 7 Hemoglobin A1c Point-of-Care Instruments Do Not Meet Generally Accepted Analytical Performance Criteria. Clin Chem 2014;60:1062-1072.
Guest
Dr. Erna Lenters-Westra is a researcher in glycated hemoglobin at the Clinical Chemistry Department of Isala, in Zwolle, The Netherlands.
Transcript
[Download pdf]
Bob Barrett:
This is a podcast from Clinical Chemistry, sponsored by the Department of Laboratory Medicine at Boston Children’s Hospital. I am Bob Barrett.
In a 2009 study in Clinical Chemistry, Dr. Erna Lenters and Dr. Robbert Slingerland evaluated eight different point of care instruments measuring hemoglobin A1c and came to the conclusion that six of the eight did not meet generally accepted analytical performance criteria. Since then some manufacturers have either improved their methods or withdrawn devices from the market. New instruments have also since come to market and these authors conducted a second round of evaluations published in a paper in the August 2014 issue of Clinical Chemistry. Erna Lenters-Westra is a researcher in glycated hemoglobin at the Clinical Chemistry Department of Isala, in Zwolle, The Netherlands, and has been involved in hemoglobin A1c standardization since 1993. She is our guest in this podcast.
Doctor, why did you repeat the experiments from 2009?
Erna Lenters:
Well, last year I gave a presentation at the EuroMedLab in Milano about the Analytical Performance of Different HbA1c methods. I used the graph of the performance of one of the point of care instruments I evaluated in 2009, and a few weeks after this presentation I got an email from the CEO of the manufacturer of that particular method and he was not happy that I am still using this graph to show the audience that his method is not performing well.
His company has put a lot of effort and money in improving the methods, and the graph didn’t represent the performance any longer. And he asked me if I was willing to reevaluate their improved methods, and I said that I am more than happy to reevaluate his methods. But also new instruments have since come to market and this initiated the second round of evaluation of different point of care instruments.
We approached all manufactures that joined the first evaluation study and asked them if they were willing to join the second round. Some were still improving their methods and not ready for the second test, and some did not want to join the second round for unknown reasons.
Bob Barrett:
Well, did you do exactly the same experiments as in 2009?
Erna Lenters:
No, in the first study I used the EP-10 protocol for the familiarization study. I didn’t do that this time because I got unexpected result of some methods in the EP-10 in 2009. The reason was probably that you have to make a mixture of low and high sample and that samples gave strange results; for example, the Afinion, which cannot work with hemolyzed material.
So this time I decided to use the IFCC monitoring samples for the methods which can work with frozen material and fresh whole blood samples for the methods which can only work with fresh whole blood. The results of the IFCC monitoring samples tells you exactly what a bias is compared to the IFCC primary reference method, which is, as you probably know, the only anchor for the standardization of HbA1c.
Bob Barrett:
Is there anything else you did differently compared to the first study?
Erna Lenters:
Yes, regarding to EP-5, I also calculated the CPs from the duplicates in the EP-9, and I did a medical decision point analysis from the same results to try to find an answer on the question if the point of care devices can be used to diagnose diabetes.
Besides that, we also decided to investigate the possibly interference of the common Hb variants, like the AS, AC, AD, AE, F and increased A2 on the instruments which can work with frozen material.
Bob Barrett:
Doctor, why is this evaluation study different from other evaluation studies?
Erna Lenters:
Well, our laboratory is together with another laboratory in The Netherlands, the European Reference Laboratory for Glycohemoglobin. We make the IFCC secondary reference material in our lab. This material is the heart of the HbA1c standardization worldwide.
The assigned IFCC values to this secondary reference material is obtained from the mean of approximately 15 approved IFCC primary reference laboratories, running the IFCC primary reference methods. The three methods used in the study were all offline calibrated with the secondary reference material. By using this secondary reference material the results are one step higher in the traceability chain to the IFCC reference method for HbA1c than results produced with a routine method, which is calibrated with the calibrator supplied by the manufacturer.
The step of value assignment to the calibrator at the manufacturer’s side, with a certain error is skipped. Just take a look at the capture of a result of last year and you can see immediately what the impact is on bias when a manufacturer experienced some difficulties with the value assignment of their calibrators.
Bob Barrett:
And what were the results this time?
Erna Lenters:
The most important message is that the results of this study showed that the analytical performance of point of care instruments have improved considerably as compared with the results of the first evaluation study in 2009. I am happy and thankful that most of the manufacturers took the message of the first study seriously.
About the familiarization study, the results were very promising for all manufacturers. There was hardly any bias compared to the IFCC primary reference method, and good to excellence CVs, so all manufacturers gave approval to continue the investigations.
Bob Barrett:
What about the results of the main study?
Erna Lenters:
Well, there were some, let’s say, remarkable results. The results of the EP-9 were for three instruments: the Quo- Test, Quo-Laboratory and InnovaStar very disappointing. When I analyzed the first of the fresh whole samples, I saw immediately the negative bias. First I thought, well, this is an individual patient matrix effect, no problem, it can happen, but when I saw the results of the second sample I became worried. I checked everything to make sure that I didn’t do something wrong, but I couldn’t find anything.
I finished the EP-9 protocol and decided that I should investigate if there is a problem regarding frozen versus fresh whole blood, and indeed these three methods gave different results if you analyze the same sample fresh or frozen. Something is happening while the samples are stored frozen at -80 degrees, but to be honest I have no idea what.
Regarding the Cobas B101, there is some room for improvement for the calibration. It passed the NGSP certification with comparison to the Tosoh G8 and failed with Roche Tina-quant Gen.2 Hb A1c, which is strange. A method should pass or fail NGSP certification compared to all SRMPs, but not pass one and fail another. The Cobas B101 showed also an interference of the HbAE variant. Results are falsely high when the sample contains HbAE.
Regarding the DCA Vantage, in the first study in 2009 the CV of the DCA Vantage with the sample with a high HbA1c value was too high. Unfortunately, the CV is still too high, even if we took a sample with a much lower HbA1c value than as we did in 2009.
Regarding the B-analyst, the analytical performance of the B-analyst is excellent, but we found out that the B-analyst uses a different master equation than the international accepted and published one.
Bob Barrett:
How did you find out?
Erna Lenters:
Well, you can choose, if you want to print DCCT and/or IFCC values. We chose for both units. We checked the relation between the two numbers and we saw that the graphs in the DCCT units were different compared to the graph in SI units and then we discovered that they were using a different master equation.
Bob Barrett:
What are the implications of the results in the main study?
Erna Lenters:
With the Quo-Test, Quo-Laboratory and InnovaStar, patients’ results were falsely lower. The consequences that the patient, of which had their HbA1c measured with these instruments, may not have had a stringent therapy as they should have had. As a result they may develop complications in the future. These three methods need to be calibrated with fresh patient samples instead of frozen material. Also, NGSP certification, IFCC monitoring program, and external quality assessment should be performed with fresh patient samples instead of frozen material.
The Cobas B101 showed interference of HbAE. This is a problem if this method is used in part of the world where the prevalence of HbAE is high and patients are not routinely screened for this variant. The patients will get an HbA1c failure, which is falsely high, with possibly more stringent therapy than is necessary.
And finally, users of the DCA Vantage should be informed not to change therapy based on small differences between two consecutive HbA1c values when the HbA1c value is higher than 64 mmol/mol or 8% in DCCT units.
Bob Barrett:
Were there any limitations to your work?
Erna Lenters:
There are always limitations to the work I have done. However, I try to minimize it as much as possible. One of the limitations to this study is that we haven’t been able to collect fresh patient samples with different Hb variants and varying HbA1c values over the clinically relevant range to investigate whether the B-analyst and the Afinion had interference from these Hb variants.
Also this part of the study needs to be repeated once the Quo-Test, Quo-Laboratory and the InnovaStar have been recalibrated with fresh whole blood. Another limitation is that the B-analyst hasn’t been evaluated at the same time as the other instruments.
Bob Barrett:
So doctor, what are your recommendations?
Erna Lenters:
I have several recommendations. First, the calibration process and certification process of HbA1c point of care instruments need to be changed. Instead of frozen material, fresh whole blood needs to be used for instruments which have a problem with frozen material, or in other words, the frozen material is not commutable with their methods.
In the packet insert of the IFCC calibration material it is stated that a manufacturer should investigate themselves if frozen material is commutable with their methods and also a procedure is given how to test this.
Secondly, external quality assessments should be mandated for uses of HbA1c point for care instruments. Unfortunately, this is still not the case, because if this was the case the manufacturer of the Quo-Test, Quo-Laboratory and InnovaStar probably would have had a signal from the users that the results of the external quality assessments were too low and then the manufacturer would have had a chance to fix it.
Third, it will probably never happen, but I wish and recommend that all patients should be screened at the time of diagnosis for hemoglobinopathies and thalassemia. If the hemoglobin variant is detected, one should consult the NGSP website to choose the appropriate method to determine correctly the value of HbA1c.
Bob Barrett:
Well, finally doctor, is there anything else you would like to share about your work?
Erna Lenters:
Yes. I want to thank the manufacturers who took the message from the first study seriously, because let's be honest, it is not about me, I am not important at all. And it’s also not about the manufacturer, they are also not important. It’s all about the patient, because one day my husband, daughter, or your partner, mother or father might become a patient with diabetes and then we all want the best therapy based on the true value. That is what this is all about.
Bob Barrett:
Dr. Erna Lenters-Westra is a researcher in glycated hemoglobin at the Clinical Chemistry Department of Isala, in Zwolle, The Netherlands, and she has been our guest in this podcast from Clinical Chemistry on Hemoglobin A1c Standardization. I am Bob Barrett. Thanks for listening!