Listen to the Clinical Chemistry Podcast
Article
L. Al-Ansary, A. Farmer, J. Hirst, N. Roberts, P. Glasziou, R. Perera, C.P. Price. Point-of-Care Testing for Hb A1c in the Management of Diabetes: A Systematic Review and Metaanalysis Clin Chem 2011; 57: 568–576.
Guest
Dr. Chris Price is a Visiting Professor in Clinical Biochemistry at the University of Oxford, as well as a member of the Diagnostic Horizon Scanning Group in the Department of Primary Health Care in Oxford.
Transcript
[Download pdf]
Bob Barrett:
This is a podcast from Clinical Chemistry. I am Bob Barrett.
Circulating concentrations of glycohemoglobin or hemoglobin A1c provide physicians and patients valuable information on the average blood glucose concentration over the previous two to three months. Hemoglobin A1c has been employed as a surrogate outcome measure in a number of landmark studies of the management of glycemic control and is now an established and well-documented part of the management of patients with diabetes. Hemoglobin A1c measurement along with other tools such as traditional blood glucose monitoring is used to optimize glycemic control in newly diagnosed patients, and then once adequate levels of control has been established, to monitor compliance with therapy. New technology is available allowing the measurement of hemoglobin A1c by nursing staff in a clinic setting at the time of a consultation. Such devices can provide results with an acceptable level of performance, although not all such systems are capable of delivering the required performance.
In a study published in the April issue of Clinical Chemistry a team at the University of Oxford carried out a systematic review of randomized clinical trials to determine if the use of point-of-care testing for hemoglobin A1c enabling immediate feedback of results leads to improved glycemic control compared with the use of a laboratory-based testing service in the management of patients with diabetes. Dr. Christopher Price, a member of that team, is our guest in this podcast.
First of all, Dr. Price, just what is a systematic review, aren’t all reviews systematic?
Christopher Price:
A systematic review formerly is a summary of the literature that uses defined methods, specific methods, for the performance of this comprehensive literature set and then critical appraisals of the studied that are identified from the research. Now, it's often actually performed together with the use of metaanalysis, which again is a sort of fairly formal set of statistical techniques for combining similar information from different studies to derive an overall estimate of the treatment effect or the characteristic of the test.
Now, let’s take the second part of your question, which amused me to a certain extent. In fact, no, all reviews are not systematic, and the ones that are not systematic are sometimes called narrative reviews. And in a way that gives a clue to the difference. They don't have the same degree of rigor in the choices of papers that are discussed and referenced in the review, and often they will span a whole topic.
So for example, in the context of today's discussion, you might find a narrative review on hemoglobin A1c that discusses right through from the biochemistry of the glycation of hemoglobin, how it's measured, the link between hemoglobin A1c and morbidity and mortality, and also the evidence that it's useful as a marker in diabetes care.
So a lot of this is good background information, but it doesn't really help you if you are trying to determine whether and perhaps how well the measurement of hemoglobin A1c might help the physician in his or her practice.
So in some respect, a systematic review is the first step when you are looking at introducing a new test, or for example, the disinvestment in an old test. Is this test A better than test B or what we did before? Or it may be responding to a clinical need by a clinician, or perhaps quite commonly today in the first stages of writing a practice guideline.
So in that respect a systematic review, not only is it a fairly formal process, but it's more focused on summarizing knowledge that will help make decisions about clinical practice and patient outcomes. And then the metaanalysis that goes -- I guess you might ask me what a metaanalysis is really. This comprises statistical techniques that enable shall we say the aggregation of data form a number of studies that have been included in the review to achieve the desired statistical power.
So perhaps if you are not so familiar with statistics, a lot of the studies will be fairly small and there are lots of reasons for that; the cost, the number of patients that might be accessible or agree. So it's important to pull these studies together in a way that they can provide a set of summary statistics, but also have the power that minimizes the variability.
The other thing it will also do, as will the systematic review itself, it would give you summary statistics and show what the variation might be between studies. And so you might have a series of studies that all give you some sort of outcome measure, but they may differ from study to study and that might reflect something unique to the test in relation to the population.
Bob Barrett:
And this review was looking at the impact of point-of-care measurements for hemoglobin A1c on outcomes. Could you elaborate on the specific outcome studied?
Christopher Price:
First of all, the outcome itself, what we are talking about in this specific review we undertook was the outcome of using the test in a particular care pathway. But outcome in many respects could be anything from looking at the accuracy or the precision, right through to what we are interested in, and that is the impact of the test used at the point- of-care on health outcome.
So it can describe a number of different outcomes, and typically when one is looking at the use of a test in a particular care pathway, ultimately we have what are called hard outcomes and those would be morbidity and mortality and a number of then softer outcomes related to patient satisfaction, maybe even care of satisfaction.
Some of these are quite difficult to measure, and if you are looking at morbidity and mortality, as you can imagine, that can take a long time in order to be able to gain sufficient numbers. And so we start to look at surrogates, and this is a bit of a conundrum in the whole piece. Hemoglobin A1c, the value of the hemoglobin A1c is actually a surrogate marker of both morbidity and mortality, and by that it means, as your hemoglobin A1c increases, problems related to morbidity and mortality increase. So it's a surrogate marker.
Then there are two different ways in which one can assess that or that we used in this particular study, and that is to take the average value in the experimental cohort, in other words, the mean hemoglobin A1c and the confidence intervals, or the alternative is to look at the proportion of patients with a hemoglobin A1c of less than 7%. Because 7% is generally the target value the diabetologists and primary care physicians try to keep their patients below. So that's the target they are aiming for.
And in that respect therefore, less than 7% -- proportion of patients with the value of less than 7% is an indication of how successful the treatment has been. So those are what I would call the sort of primary outcomes when one is looking at the use of a test in this context.
There are then a number of secondary outcomes, some of which are one might say slightly softer and a bit more difficult to measure. One of those is patient satisfaction, and certainly that was looked at.
Another important one, very important, and I am sure this is going to figure in our discussion today is, actually is there some way of telling that the clinician actually used those results and made some treatment change? Because as a treatment change in terms of a patient or whatever is a way of showing that the test has been used. And then obviously there is what one might call the health economic side. Depending on the health system, that will be of interest to a different person within an organization or healthcare economy.
But one likes to know as to whether using a particular test has a positive or negative impact on the health economics. And we looked at all of those, although, as I said earlier we did focus on the primary outcomes of the changes in the hemoglobin A1c.
Bob Barrett:
In your review you have included only randomized controlled trials. Was there a reason for this and did you find any disadvantages to using a randomized controlled trial approach?
Christopher Price:
Interesting! The whole issue of trials particularly on diagnostic tests is a fairly controversial area. Some of the controversial areas include cost and the time they take to generate the desired information.
However, the important point of doing such trials is to demonstrate that the test, the treatment, whatever intervention you talk about will benefit patients and also whether it will benefit all patients or particular types of patients.
So when you are looking at the utility of a test or a treatment, you want to design a study that looks at the impact of the intervention in a way that minimizes the risks that the results will be biased, and that’s one of the primary reason for doing a randomized controlled trial.
When I say biased, it may be biased in a number of ways. So interestingly if you are looking at a drug with a randomized controlled trial, then a patient is randomized to have the drug or a placebo. In essence, neither the patient nor the clinician or any other carer within the system can tell the difference, and in a way that minimizes the bias.
That’s much more difficult when you are looking at a diagnostic test. But in a way not only does it mean that patients in both arms of the trial are, shall we say, treated in the same way, but they are also selected in the same way, so there’s no opportunity for bias.
I mean, I implied earlier that trials of drugs are easy to perform. When you are looking at a test, there isn’t a placebo. The reason for that is when you are looking at a test or a way of delivering a test, which is really what we are talking about with point-of-care testing, the clinician has to make a decision on the receipt of the result in a different way. And therein lies in a way the disadvantage of using a randomized control trial, because it automatically introduces a degree of bias because the clinician is aware that he or she is doing something differently.
But I would like to give you a sort of example from previous experiences. A group looked at point-of- care testing in the emergency room using blood gases and electrodes and they showed no benefit from using the point-of-care test.
And interestingly, the investigators advanced the opinion that this might have been due to the fact that the ED physicians didn’t in fact change the way they did things; although they should have done because the results were available after five minutes rather than 40 minutes.
So there is an advantage in reducing bias, but there is a disadvantage in some respect, in that it's very difficult to avoid that. And at this stage I am not sure whether the laboratory community really has got the answer to this.
Bob Barrett:
Well, speaking of bias, are there other issues of bias that need to be considered?
Christopher Price:
Yes, all those biases have to be considered. And interestingly, there are some quite subtle ones in the studies that we looked at.
One of the greatest potential areas of bias is in patient type. Sometimes it is called spectrum bias. To give you a slightly bizarre example, typically the first study with a diagnostic test will look at a group of patients who have a disease and compare that with a group of patients who are considered to be normal; so almost the two extremes.
Whereas typically the result from a diagnostic point of view will be used in patients who are perhaps, shall we say, at the early stages of developing a disease. So that’s an example of bias.
One of the areas of bias in the studies that we were looking at was that the patients sometimes were selected across a broad spectrum. I can recall in a couple of studies a lot of the patients actually had good glycemic control or another way of putting that is the hemoglobin A1c of less than 7%.
And the value of the test in that situation is not as important, because -- well, the benefit to the patient would be that they would be told, you are doing well, carry on as you are, but the real value is in the patients who have significantly or bigger excursions from that optimal range and those are the patients who you really want to use the test for.
So by virtue of the fact that there is a high proportion of patients in good control, the results might appear to have been biased because the results in the patients who needed the test were being masked by those where the tests was of less importance, and that was one of the reasons we were unable to perform some of our metaanalysis.
Bob Barrett:
Well, let's get down to point-of-care testing for hemoglobin A1c. Why would a healthcare provider want to consider point-of-care testing for hemoglobin A1c?
Christopher Price:
Can I sort of broaden that question just a little, but by sort of posing another question, which I will answer rather than expect you to. So who is the customer? The customer could be the healthcare purchaser or a provider, depending on the healthcare system.
I guess in the U.S. it could be an HMO or it could be a hospital provider organization. They would be interested in point-of-care testing, but also the patient would be interested and the carer. So there are variety of reasons which apply to varying degrees depending on who the customer is.
I mean, the first point I think is, patients who are aware of their hemoglobin A1c tend to have better outcomes. And so in the shorter term, patients would be interested in being aware of their hemoglobin A1c, as would day carer, as would providers and purchasers because their raison d'être is getting better outcomes of patients. So knowing their hemoglobin A1c and being able to have that result as part of their consultation could lead to an improvement in their outcomes.
There might as a consequence of that then be a reduction in the rate of onset of complications. That’s another good reason why all parties would be interested in delivery of the test and the service which would improve those outcomes, and it might reduce the rate of hospitalization.
It also might enable care to be delivered in a primary care situation. That’s something that’s becoming of increasing importance in Europe. And so point-of- care testing would enable that, because it reduces the requirement to send the sample off to the laboratory, and overall it might then lead to a reduction in the cost of care.
And I think these, one might call aspirations, that are shared between both purchaser, providers, carers, and patients, and there all good reasons why one might consider the measurement of point-of-care.
Bob Barrett:
You have found seven trials in your review. Is that typical for a systematic review of medical test?
Christopher Price:
Yes, I think it is fairly true. I think it's fair to say within the area of trials looking at outcomes, you will find far fewer in the literature than you would in the case of some almost therapy intervention.
I think part of this is due to the fact that regulation and reimbursement arrangements in countries do not require formal evidence of impact on outcomes. I think that’s going to change, but typically that is not required at the moment.
Secondly, tests are not reimbursed on the basis of their impact on outcomes and it has become very commoditized, and so there aren’t resources available for large studies of the type that are required and we have been talking about. So you will see, we began with 17 reports of trials. I mean, that’s I would say fairly average, but some of those we had to exclude for a variety of reasons to do with bias, as we were talking about earlier.
For example, there was no control group and randomization was not rigorous and it was a retrospective study. So we ended up with certain basic studies. And that should have been enough to generate the results we required, yes.
Bob Barrett:
Well, it appears you measured a number of outcomes but few of the trials reported data on all outcomes. How did you deal with that?
Christopher Price:
Yeah. With lots of tearing of hair I can say. We looked at the impact of point-of-care testing on the hemoglobin as a surrogate outcome.
As I made the point earlier, we couldn’t use morbidity and mortality because it would have just taken so many years. So we used primarily the two measures; one was the mean hemoglobin A1c, but there were very few trials that actually -- so that was the mean change over the period of the study. Actually, very few of the trials used that parameter, although one might think it to be the most robust.
Just as an aside, we did actually write to all of the investigators and asked if we could have the raw data so that we could calculate it ourselves. We were lucky that we got positive feedback from the Australian group who had just published a big study on point-of-care testing for hemoglobin A1c. But in the other studies the information was either locked away or certainly wasn’t available to us.
More of the studies used the proportion of patients with a hemoglobin A1c of less than 7%, or another way of looking at that, the proportion of patients who were successful in achieving target value by the end of the study.
But the problem with that was that a high proportion of patients had hemoglobin A1cs either below the target or very close to it. The concern there was, just by virtue of typical biological variation or natural variation around any one point, you would have people moving across that sort of cutoff value, which would mask the effects that we were looking for in the patients with those bigger hemoglobin A1c excursions.
So although we were able to extract data from all of the studies, in one case, one of the parameters that wasn’t sufficient to really give us a strong outcome in terms of metaanalysis, and in the other the hemoglobin A1c of less than 7%, it meant that we couldn’t undertake any metaanalysis because of this potential contamination of results from the patients who were very close to the target value.
I think the other issue at this point is, we do need to gain some agreement, and that doesn’t just apply to this study, but any study really, we do need to have good agreement at the outset when looking at any intervention as to what the outcome measures are that should be collected, so that we can be confident that all studies when they are undertaken will be to a standard and reporting the right outcomes so that they can be integrated into systemically a metaanalysis.
Bob Barrett:
Your main conclusion is that there is an absence of evidence for the use of point-of-care testing for hemoglobin A1c. Could you amplify that statement a bit?
Christopher Price:
Yeah, happy to. It’s quite a strange wording, but it’s something that I think has been stressed time and time again but actually requires a bit of thinking about.
There was a paper published in the BMJ a few years ago where a caution was noted and the author said, it’s important to understand that an absence of evidence, that is of effect, does not mean evidence of absence of effect. That is often confused. So I added the words of effect because that makes it clearer, at least it does to me. But absence of evidence doesn’t mean evidence of absence. That’s why we really put -- you have made the statement, the absence of evidence, there is an absence of evidence for testing.
So what we are saying that the evidence is being produced on the impact in the management of diabetes, but for several reasons, some of which we have already talked about, it’s not strong enough to make a recommendation from the randomized controlled trials.
We did actually, as an aside, just report on a number of observational studies, maybe we ought just to -- sort of just think about observational studies compared with a randomized controlled trial. In an observational study the point-of-care test has typically been introduced and the experience is compared with earlier practice using a laboratory service.
So to give you an example, a laboratory service is used to provide the hemoglobin A1c to the diabetes clinic and then they go over to using point-of-care testing in the clinic itself, and then they just measure over a period of time the changes in the hemoglobin A1c.
What I have said already gives a clue as to one of the benefits, because it is actually a bit like an audit or practice. So one is able to look at the changes in the hemoglobin A1c in individual patients as well as populations over longer period of time.
Invariably there are studies on larger numbers of people, partly because it’s almost what you do as part of total quality of management of a clinical process. One of the questions that’s always asked about randomized control is whether the short duration of the study or the change in practice can be sustained. That’s actually something that’s covered in these observational studies. So we have got observational studies. The only problem with those -- well, I think there are twofold.
One, you can’t be sure that they can be generalizable to other populations and also, and this is where one of the problems in any trial of a diagnostic test is concerned, you have to ask yourself the question, is that experience actually down to the fact that they are using point-of-care testing or is it down to some other part of the care process? But when all is said and done, that process which includes point-of-care testing has actually shown a better outcome.
But, and I would agree with this, in the first instance you do need robust evidence using a randomized controlled trial, because that helps to show in the least biased way and pinpoints particularly that test and treat sequence of activities around point-of-care testing.
Bob Barrett:
Okay. I know that you have recommended that more attention should be made to study design. Would you like to explain that in a little more detail?
Christopher Price:
Yeah. I think already from our conversation you would appreciate that we have highlighted a number of issues which I think our group believe have compromised the data generated.
I think there are four key issues here. The small number of patients in each of the studies and the fact that as a consequence the studies were underpowered. The fact that knowledge of hemoglobin A1c is most important in those in whom the physician is going to take some action with respect to change in treatment. And so it's important perhaps to stratify the patients when reporting the outcomes in terms of proportion of patients with hemoglobin A1c within the target range or less than 7%, so that the small changes, as I was saying earlier, don’t compromise or contaminate the results from those where the larger excursions are needed.
I think the third point is the need to document clearly the approach to clinical care and treatment in both the experimental and control arms, such the changes as indicative of results being acted upon; for example, the treatment type intensification can be measured.
We were only in a small proportion of the studies able to glean any indication of changes in clinical management. And I think in one of the studies it is wroth mentioning that there was actually a randomized controlled trial which was undertaken in primary care, but in the discussion the investigators made the note that they had made no attempt or they were unable to change the way the physicians arranged their practice and able to use the results at the time of the consultation, which is really the whole point of point-of-care testing.
In the other studies we were left actually thinking, well, when did they discuss the results with the patients in the control arm, because the results might come back a few days later, and in the point- of-care arm, did they actually discuss the results?
So it's really about documenting the process, because this is very much a process and also related to interaction between patient and physician. And finally, because some of our studies had to be excluded on the basis that there wasn’t clarity of how the randomization was undertaken, we think that’s an important thing to do.
As I said, it's difficult to randomize effectively and I think that’s an ongoing issue within the laboratory community on how one can randomize and in so doing reduce the bias amongst the people using the point-of-care testing.
Bob Barrett:
Well, finally Dr. Price, what do you think are the key learning points from this review?
Christopher Price:
I have been thinking about that quite a lot, because interestingly, I have done one or two systematic reviews before, but I think this is the one that has really brought home to me a lot of issues around what one broadly calls evidence-based laboratory medicine. And that is around understanding where the test is used and how it's used in the care pathway.
I don’t actually think this is a particular issue here, but the fact that we have been enabled to define steps in the pathway which are actually important to being confident that results have been used at the time when the results were generated, which is the raison d'être of the whole thing, how to address that has brought home to me that we need to consider that in great detail.
It's sort of rather obvious that the immediate availability of the hemoglobin A1c provides an opportunity for face-to-face discussion, and just in general terms, in laboratory medicine, we don’t always know whether that’s the case.
In several of the studies there were things that were missing and so bizarrely the strongest learning point for me or being rather an obvious one is, you have to look at the test, not just the way it's performed, but actually the way it's used in order to evaluate the impact on outcomes.
So I think I have learned more about how to perform a study and the potential pitfalls in undertaking an outcome study rather than whether point-of-care for hemoglobin A1c improved outcomes. On the other hand, I did learn a lot about how to avoid those pitfalls as well.
Bob Barrett:
Dr. Chris Price is a Visiting Professor in Clinical Biochemistry at the University of Oxford, as well as a member of the Diagnostic Horizon Scanning Group in the Department of Primary Health Care in Oxford. He is also the Clinical Lead for the Cumbria and Lancashire Pathology Commissioning Network, and he has been our guest in this podcast from Clinical Chemistry. I am Bob Barrett. Thanks for listening.