Listen to the Clinical Chemistry Podcast
Article
C. Weykamp et al, on behalf of the IFCC Task Force on the Implementation of HbA1c Standardization. Investigation of 2 Models to Set and Evaluate Quality Targets for Hb A1c: Biological Variation and Sigma-Metrics. Clin Chem 2015;61:752-759.
Guest
Cas Weykamp is the Director of the MCA Laboratory of the Queen Beatrix Hospital in The Netherlands.
Transcript
[Download pdf]
Bob Barrett:
This is a podcast from Clinical Chemistry, sponsored by the Department of Laboratory Medicine at Boston Children’s Hospital. I am Bob Barrett.
A major objective of the International Federation of Clinical Chemistry and Laboratory Medicine, or IFCC, Task Force on implementation of Hemoglobin A1c Standardization is to develop quality targets with the measurement of Hemoglobin A1c. That information is of substantial use for monitoring, diagnosis and screening of diabetes and glucose intolerance. A paper in the May 2015 issue of Clinical Chemistry examined this topic and the development of a model to set and evaluate quality targets for Hemoglobin A1c measurements. Dr. Cas Weykamp is the lead author of that report and he joins us in this podcast. He is Director of the MCA Laboratory at the Queen Beatrix Hospital in The Netherlands.
Dr. Weykamp, why are quality targets required for Hemoglobin A1c?
Cas Weykamp:
Well I think diabetes is by far the most prevalent disease; many people suffer from it, and it’s one of the exceptions where therapy and diagnosis nearly fully depends on results from the laboratory.
So most prevalent and depending on the laboratory and then it’s of upmost importance that all laboratories have a high quality HbA1c, and for that reason the IFCC Task Force on HbA1c works on the model for setting quality targets for this HbA1c.
Bob Barrett:
How did you go about developing quality targets for Hemoglobin A1c?
Cas Weykamp:
Well, we had several considerations beforehand. Once you realize that the IFCC is an international organization that has advantages and disadvantages, the disadvantages that we can’t make the law. We can’t tell for example the US, you have to do this, or you have to do that.
So, what we can offer and what we have to develop, is a flexible model in which criteria can be filled by local, national, or international authorities.
The second consideration is that the model must be easy to understand and apply, otherwise no one will use it; and also the data needed to fill the model are easily accessible, and we also want a wide applicability of the model, not only in an individual laboratory but also to judge the general performance of a whole country, and these requirements are met with the concept of total error and we feel that we have attractively elaborated it through visual graph, with data of proficiency test programs. In fact, readers, users need no statistics at all to interpret this model. And then there are two options, one based on biological variation, and one on the Sigma matrix model, and our Task Force has chosen the Sigma matrix model as the best one.
All you have to do is to set a clinically relevant target and the risk you accept to not achieve that target, and it may seem quite difficult. But to give an example, default setting is a maximum allowable error, in US units 0.46% HbA1c, you wonder, the resale of a laboratory has no bias exceeding at 0.46%, and the risk of not achieving that goal, may not be higher than 1 out of 20. So that is the starting point and that is the way we went to develop quality targets.
Bob Barrett:
Doctor, you say that the model is applicable at the level of an individual laboratory and at the level of a whole country, but let’s say, I am President Obama, I got a promotion, and I want to know a general picture of the quality of Hemoglobin A1c in US labs, can I derive that from your model?
Cas Weykamp:
Well, to paraphrase Barack Obama, I would say, yes, we can, and we can do it literally from figure 2 of the paper. As many people will know there is a countrywide proficiency program for HbA1c, the CAP survey program, and you can simply take the data from that survey program, fill it in the model. You can do it in a couple of minutes and then see the performance, and then you can see the performance of the whole country.
In the model we have done that for a specific survey GH2a of 2014, but you can do it with any survey of every half year, and the star in the figure represents the Mean performance of more than 3,000 laboratories in the United States, and there you can see that the United States in general the laboratories do not meet the default quality target of the model.
Bob Barrett:
Well, Mr. President, if you are listening I apologize for that. Can you also, from the model, find out why labs in the US perform so poorly?
Cas Weykamp:
Yes, it can be taken from the same figure. There are many tests on the market for HbA1c and some test is better than the other. Performance is very highly variable with tests, and from the data, in the figure, you can see that there are eight excellent tests from commercial methods, from tests where laboratories that use such tests have less than 1% chance to fail the quality targets, that’s on the upper end.
At a lower end there are also eight poor tests, laboratories using these tests have more than 20% chance to fail the quality target. So when quality in the US would have to be improved there are two options, A) that poor tests improve, which is a challenge for many of the manufactures or B) laboratories that perform poorly and see that is due to the test they are using they could switch to one of the excellent tests.
If that would be done I think quality of US in general would dramatically improve and clearly fall within the limits of the targets we have set. There can be several reasons why tests performed poorly; for example, poor calibration. When calibration is not okay laboratories will see a systematic bias every time they do assays and we have seen a typical sample, it can be seen in the figure. Well, this is an interview, so you don't have the figure at hand so I can’t refer to it.
Another reason, it can be poor precision. When you repeat the tests in the same laboratory you get quite a different result, then the quality of the system is not under control-- intrinsic poor precision; and that is another reason for poor precision, that is batch to batch variability and reagents or calibrators from one batch to another are different, you will also see variation in the results.
Well there are other reasons of course, but the major is either its calibration or its reproducibility, and laboratories can work on it. Both can also derive from poor instructions to the users, poor maintenance of the instruments and things like that, and that makes it up to the manufacturers and the users to work together on improvement.
Bob Barrett:
Finally doctor, what can an individual lab learn from this model?
Cas Weykamp:
The primary aim for an individual lab is to learn, how do I perform? Do I meet the quality standards or don’t I? If you do, as an individual laboratory you can say, okay, do nothing. If it's not okay you have to do root analysis and then the first question is, that poor performance is it due to what I do in my laboratory or is it at an external reason, for example, the test I am using is not of best quality.
To that last point, the general performance of a test can be judged from the model, from the figure in the paper, and if a laboratory using such a poor test they should seriously consider if they want to change to one of the better tests or put pressure on the manufacturer to improve or switch a test.
If a laboratory has a poor performance, but a general picture of the method they are using is quite good, it's clear that the reason for poor performance is within the laboratory, and then the laboratory should try to find why operating the tests in the laboratory is so poor. It could be instructions for the operators that are not quite good, maintenance of the instrument, a system where clerical errors are easily made, and things like that. Often talking about it, showing poor performance to the technicians, does already raise quality improvement, and that’s the way an individual laboratory can learn from the model.
Bob Barrett:
Cas Weykamp is the Director of the MCA Laboratory of the Queen Beatrix Hospital in The Netherlands. He has been our guest in this podcast from Clinical Chemistry on Hemoglobin A1c Standardization. I am Bob Barrett. Thanks for listening!