Listen to the Clinical Chemistry Podcast
Article
Moderator: Tony Badrick; Experts: W. Greg Miller, Mauro Panteghini, Vincent Delatour, Heidi Berghall, Finlay MacKenzie, and Graham Jones. Interpreting EQA—Understanding Why Commutability of Materials Matters. ClinChem 2022;68:494–500.
Guest
Dr. Tony Badrick is chair of the Education on Laboratory Management Committee of the Asian-Pacific Federation of Clinical Biochemistry, a member of the Joint Committee on Traceability and Laboratory Medicine, and is chief executive of the Royal College of Pathologists of Australasia Quality Assurance Programs in Sydney, Australia.
Transcript
[Download pdf]
Bob Barrett:
This is a podcast from Clinical Chemistry sponsored by the Department of Laboratory Medicine at Boston Children’s Hospital. I’m Bob Barrett. Although there has been progress in harmonizing test results from different test procedures and different laboratories, unfortunately some differences among them still remain. Such differences may arise due to the lack of traceability of the calibrators assay imprecision, differences in analytical specificity, and individual laboratory practices. There is potential risk to patients of misdiagnosis or incorrect treatment due to a bias between results reported from different laboratories for the same measure end in the same patient. These risks can be identified and hopefully reduced by using EQA, external quality assurance, or proficiency testing programs. Those programs provide a form of surveillance of the efforts of traceability.
However, the materials used in those programs must be fit for purpose to accurately identify problems in a cost-effective way. Ideally, the material should have target value assignment from referenced methods using certified reference materials and be commutable. That last attribute, commutability, is the closeness of agreement between results for a referenced material and clinical samples when measured by two or more measurement procedures. In a Q&A feature appearing in the April 2022 issue of Clinical Chemistry, global experts discussed why commutability of materials matters in interpreting external quality assurance results. That feature under the auspices of the Working Group on Traceability Education and Promotion of the Joint Committee on Traceability and Laboratory Medicine was moderated by Dr. Tony Badrick. He is our guest in this podcast.
Dr. Badrick is chair of the Education on Laboratory Management Committee of the Asian-Pacific Federation of Clinical Biochemistry. A member of the Joint Committee on Traceability and Laboratory Medicine and is chief executive of the Royal College of Pathologists of Australasia Quality Assurance Programs in Sydney, Australia. First of all, Dr. Badrick, tell us about the role of EQA and who is the intended audience for the results from these programs?
Tony Badrick:
Thank you. I think they’re both interrrelated, the audience and the value of EQA, so I’ll cover both. They go to the value of EQA or proficiency testing as it’s known in some parts of the world. Basically, why do it? And why we do it, depends on the audience as I say. When you think about what we do in laboratories, we test patient samples for particular analytes, but all those results from those tests are compared against something else. They’re compared against a reference interval to see if the patient is within that reference interval. They’re compared against all the results from the same patient just to see if there’s been a change over time or sometimes as in with therapeutic drug monitoring or some like HbA1c.
They’re compared against a clinical decision level, so is the patient’s result more than 5.5 or something like that. So, all our results really are compared against something else. Therefore, it’s imperative that when we do our testing that we get the same results in our lab over a period of time, that we get the same results as other laboratories in our network where patient might go because they need to have comparable results. We need the same results as everybody else using our method and we also probably need the same results ultimately as every other lab that might test that patient. So, all those things go towards the issue of traceability, but interestingly enough, it’s not a direct responsibility of manufacturers who provide the equipment and the test to get the same results as another manufacturer. That really falls on the shoulders of laboratory people.
So, when I go to the audiences, the various audiences, for laboratories, what they really want from EQA, when they compare their results against other people using their method, their peer group, they want to make sure that they’re getting the same results as their peer group. So, basically, they want to ensure that they’re using the manufacturer’s method or analyzer as per specification, so they can see they get the same results. So, do other laboratories, the next question they might ask is: do other laboratories in my network if I’m part of a network of six or seven laboratories all using the same equipment, do we get the same answer? But more importantly, if we use slightly different pieces of equipment, because the patient might wander around to different sites, do we get the same answer, so that’s important.
The other thing as a laboratory we want is that if we’re not getting the same answer, we want to be able to trouble shoot the issue and apply some quality improvement, so we want it as a part of an educational exercise. What we want over time using EQA or PT samples is to see if there’s lot-to-lot variation. In fact, if we run the same sample, EQA sample, are we finding that we’re going to get a slight shift in our results over long period of time that can indicate that perhaps the manufacturer’s process is not quite as good as they should be.
And certainly, laboratories in many jurisdictions require to be in an EQA just to get accreditation. The next group of people who are interested in EQA results are the manufacturers. It maybe that they need to be part of an EQA scheme themselves because it’s a regulatory requirement, but it can also be a marketing advantage for manufacturer to have good results in EQA. They can demonstrate therefore, independently that in fact the results, the calibration, their lot-to-lot variation, is sort of controlled and that laboratories will get the same results over a long period of time. And remember, this information is available to all users, people who enroll in EQA PT get these results from all the peer groups, from all the manufacturers out there, so they can see the stability of results over time. That information might well influence purchasing decisions for customers.
The next group that are interested in EQA, and these are sort of moving more and more away from laboratories, but the next group are professional organizations or perhaps health organizations. And really, what they’re interested in is, can we apply common reference intervals in a state or a province or a country? Can we just use the same reference interval for a particular analyte as everybody else in the country, but be able to do that, everybody is going to get the same results. Can we combine data from different laboratories in an electronic medical record? And again, you can only do that if in fact everybody gets the same results on the same sample, so EQA provides information about that. And certainly, people who are writing clinical guidelines to say that in this disease, if the patient has result at this level, they need to have extra treatment or diagnosis.
They are interested to see that laboratories are all getting the same answer on the same sample. Similarly, our clinical researchers who kind of put out studies that they want to ensure that everybody’s getting comparable results and certainly the CDC Lipid Standardization Program is an example of that and there are also accreditation bodies who want to see that when they come round to a laboratory and they review that laboratory’s performance, they want to see that when they’re performing well, or if they’re not performing well that at least they go into some sort of process to improve what they doing, so they are the main agents. The value of EQA, I could summarize this, you can use it to identify QC problems. Are you in fact controlling that your method is well? EQA can also provide you with a range of samples that QC doesn’t, so you can see if in fact there’s problems at high or low ends, can identify problems with interference if the EQA provide you samples that have interference in them. And look at the whole testing cycle using EQA, there’s an educative process and you can identify problems with methods and problems in specific laboratories with the same effort.
Bob Barrett:
Well, the title of this session you moderated is why commutability of materials matters in interpreting EQA results? Are commutable EQA materials always required and if not, when are they absolutely essential?
Tony Badrick:
So, I might start by saying what does commutable mean? So, commutable really means that the results that you get for the EQA PT material, and in fact the QC as well, is exactly the same as you get for a patient sample. Is the inflammation that I’m getting from these EQA samples the same situation that’s happening with patient samples, so that’s the first thing. So, when are commutable materials useful? They’re really useful to compare different methods to identify that there's a true bias. You might have an issue with a certain EQA material because of the way the material is being prepared, it reacts differently to a patient sample. So, when you see differences across different instruments or different methods, it might not be because the different methods are returning a different result, it's because the material is behaving differently in different systems and the material may behave differently because of the way we have to make EQA material.
The very concept of EQA is you send out the same sample to a lot of laboratories and you compare their results. But that means that you got to have a lot of sample, so to get a lot of sample, you can’t just use one patient sample. You have to get a lot of samples together and bulk up the volume and then, you have to prepare that material so that it’s stable. You might have to add preservatives, you might have to freeze dry it. You want a range of concentration as well in that material, so you might have to spike the material with additional levels of measurands you’re interested in. You might have to dialyze to remove some measurands and also need, because it's convenient, you needed one bottle of EQA. You don’t want just to have a high sodium or high potassium, you want to have a high level of a whole lot of measurands and in another bottle you want to have a low level of whole lot of measurands.
To get those requirements, we have to produce material that might not behave the same way that patients do, so that’s the issue. But to ensure that we’re truly getting a difference between different instruments, that the difference is not due to those factors, those preparation issues, its due purely to the methods, we need to use a commutable material.
Bob Barrett:
Is there any value to using non-commutable material?
Tony Badrick:
There is, because as I say, one of the primary things that laboratories want to understand is that, am I using the method, the instrument, in the way that the manufacturer specified it should be used? So, the best way I can determine that is to run a sample, the same sample in different sites, but on the same manufacturer’s equipment and reagents. As long as the material is sort of humanlike, it doesn’t have to be commutable, but as long as it’s sort of humanlike. By running that in a whole series of different laboratories, but using the same equipment, I can see if I’m getting the same results. So, what we can see is if I’m getting different result on the same material to a whole lot of other laboratories using the same equipment, I’ve got a problem. So, that means, I’m not running the material as I should.
It also can provide information about quality improvement exercises as I’ve said. If there’s a problem with the material, I’ll have to go through a quality improvement process to identify, well, is it the material or is it the method itself? So, certainly non-commutable material can be useful. Also, I should add that with a lot of material, we just don't know whether it’s commutable or not, so that’s part of the issue. A lot of material that we use now, all we can say is that it’s not verified as being commutable, but we don’t know whether it’s commutable or not.
Bob Barrett:
Do you have some idea of the percentage of EQA materials that are used globally that have been demonstrated to be commutable for most or at least some analytes?
Tony Badrick:
There’s no data that I’m aware of. And most, even recent surveys, what people will say is exactly what I’ve just said-- it’s not verified as commutable. I’m sorry, people who provide EQA material, they might have an idea, but most haven’t been able to prove that it’s commutable.
Bob Barrett:
Well, finally doctor, the concept of commutability was first put forward about 50 years ago. After half a century, one would think that commutable materials would be universally adopted. What are some of the problems in providing commutable materials?
Tony Badrick:
And this goes back to the last point that I made. So, how do you prove that something is commutable? So, to prove that something is commutable, what you need to show is that the material behaves exactly the same way as patient samples do on different methods’ instruments. So, what that means you need to do is to take your material and run it at the same time as you're running a series of patient samples on every method that you’re going to provide that EQA material to. The way you actually do it is using a form of linear regression and in that linear regression you have pairs of methods, so if in your survey you’ve got four analyzers that you’re going to send out your material to, you will need to compare a range of patient samples for every analyte, sodium, potassium, urea, creatinine, every analyte that you include in your EQA.
You need to have a range of patient samples that cover the clinical range or possibilities and you need to run those samples on every analyzer in pairs. You know analyzer A against analyzer B, analyzer A against analyzer C, analyzer B against analyzer C. It’s a lot of samples and you have to show that for all of these analyzers, for all of those measurands, analytes, you need to have your EQA material behaving exactly the same way as patient samples do, which basically means that it lines up on a regression line.
So, when you say you have to have a whole range of patient samples covering every analyte, every concentration, and then subject your material, and all those patient samples, to every pair of analyzers that are in your program, so that’s a lot of cost, it’s a lot of complexity, it’s very difficult to get that range of patient samples. And the volume that you need, so it’s basically a very complex process, and a costly process, that’s why.
Bob Barrett:
Sounds like job security to me.
Tony Badrick:
It’s very difficult and the other thing I should add is that with some patient samples, they themselves may have matrix effects. So, some patient samples might have antibodies or autoantibodies, rheumatoid factor or something, or drugs, and so, they will in fact not behave the same way that other patients will on a particular assay. So, it’s a really quite complex process, hence, why 50 years down the track we’re still trying to come to grips with the problem, but we’re trying to address it. You know, I think up till now it has been seen to be too difficult. But now, at last, we are trying to address it.
Bob Barrett:
That was Dr. Tony Badrick, chief executive of the Royal College of Pathologists of Australasia Quality Assurance Programs in Sydney Australia. He was moderator of the Q&A feature on why commutability of materials matters in interpreting external quality assurance results appearing in the April 2022 issue of Clinical Chemistry and he’s been our guest in this podcast on that topic. I’m Bob Barrett.