Listen to the Clinical Chemistry Podcast
Article
Aasne K. Aarsand, et al. The Biological Variation Data Critical Appraisal Checklist: A Standard for Evaluating Studies on Biological Variation. Clin Chem 2018;64:501-14.
Guest
Dr. Aasne Aarsand is Chair of the European Federation of Clinical Chemistry and Laboratory Medicine Working Group on Biological Variation and a consultant at both Haukeland University Hospital in Norway and the Norwegian Quality Improvement of Laboratory Examinations.
Transcript
[Download pdf]
Bob Barrett:
This is a podcast from Clinical Chemistry, sponsored by the Department of Laboratory Medicine at Boston Children’s Hospital. I am Bob Barrett.
Data on biological variation can be used in many ways, most notably for the diagnosis and monitoring of disease and also for setting analytical performance specifications. In 2014, after the first strategic conference of the European Federation of Clinical Chemistry and Laboratory Medicine, a task force was established to develop criteria for evaluation of the literature on biological variation.
A report from this group appears in the March 2018 issue of Clinical Chemistry. Dr. Aasne Aarsand was the lead author of that report, and Chair of the European Working Group on Biological variation. She is a consultant at the Norwegian Porfiria Center and Laboratory of Clinical Biochemistry at Haukeland University Hospital and at the Norwegian Quality Improvement of Laboratory Examinations. We’re happy to have Dr. Aarsand as our guest in this podcast. So doctor, your report found that the quality of biological variation data is quite variable. Why should patients and laboratory professionals be concerned about that?
Aasne Aarsand:
Well, biological variation data are essential for a number of the work and also some decisions we make every day based in the laboratory in the clinical practice. And whenever you dismiss important application is to set analytical performance specification which we used to say, what quality are different, analysis or -- instruments should have.
And when we assess such analytical performance specification based on biological variation data, the aim is to keep the analytical variability small compared to the biological variation. In other words, to reduce the analytical noise, so that we pick up the biological signal we are interested in. And obviously, if the estimates we use to set these performance specifications are not appropriate or not correct, then this will mean that our analytical performance specifications and the quality of our instruments are not as we would like them to be either.
So, the more biological variation data used to determine the so-called "reference change values", which we use to assess the significance of change when performing serial measurements within the same person. Say if we have inappropriate estimates of biological variation and based on our reference change values on this, they may cause us inadequately interpret changes in test results when we monitor patients over time.
So, these are just a few examples of how the quality of biological variation estimates may have consequences for our laboratory work and in the follow-up of patients. In reality, biological variation data, reference data, but they are often applied without this understanding. Meaning, that you take estimates from a study and they may often be applied to all the populations done or populations with characteristics that are different to the rest of the population from which the biological variation data has been derived. And this is really the background for the work we report on in this article that we want to provide a standard for critical appraisal of biological variation data in order to assure that the estimates we use in the laboratory are fit for purpose.
Bob Barrett:
Your group’s report indicates that the majority of biological variation studies lack important detail necessary for producing high quality estimates of that variation. How surprised were you by these findings and how can we overcome that?
Aasne Aarsand:
Well, I was quite surprised that so many publications did not deliver on a number of the quality items we defined, and this is specifically referring to some of the most statistical items such as analysis for outliers and assessments of variance homogeneity testing. But I think it’s important to keep in mind that a lot of these studies were published several decades ago and many of these historical papers are performed according to the standards existing at the time of publication.
Generally, the area of biological variation has been suffering from a lack of standards for the generation and reporting of data. It has for example been unusual to include measures of uncertainties such confidence intervals, which is typically required to be reported, and it means all the types of studies. They are often lacking in or very rarely included in studies on biological variation estimates, and the confidence intervals are necessary to evaluate the reliability of these estimates and to compare with the other studies.
So, when you use the literature, you find that estimates for the same measurand obtained from independent studies may deliver estimates that’s very substantially and the lack of harmonization in study design and the analytical methods used, and the data handling between different study may be an important factor to explain this variation we observed.
Bob Barrett:
Doctor, I think we’d like to know more about the Biological Variation Data Critical Appraisal Checklist that you mentioned in the paper. How could it be helpful in providing better biological variation estimates?
Aasne Aarsand:
Well yes. Obviously, we hope that this checklist which can be useful in a number of ways. And firstly, its main purpose is study can be used as an instrument to review studies on biological variation to assure that this deliver on that we can pick out the studies that deliver estimates a better fit for purpose.
And we can also use this quality assessment when we perform meta-analysis to produce global estimates of biological variation. This is one of the things we report on in this publication in Clinical Chemistry.
Secondly, we hope that it can assist researchers in defining and reporting their future studies appropriately, and then that it may help identify measurands for which studies need to be performed, because there’s a lack of high quality studies for some measurands.
Finally, we hope that it may serve as an aid for editors and reviewers when they assess biological variation manuscript script submitted for publications. And clearly, the Biological Variation Data Critical Appraisal Checklist has been inspired and built on the concept of STARD and similar initiatives. It has been reported that the STARD initiative has improved, though slowly, the quality of the reporting of diagnostic accuracy studies and we hope that the BIVAC can have a similar effect on the quality of biological variation studies in the future.
Bob Barrett:
For years now, the main source of biological variation data for the laboratory community has been an online list compiled by the Analytical Quality Commission of the Spanish Society of Laboratory Medicine. You mentioned in your article that the results of your work will be included in a new database, published at the European Federation of Clinical Chemistry and Laboratory Medicine website. In which ways will this new database differ from the one presently available online?
Aasne Aarsand:
Well, in the presently available online database, this was last updated in 2014. In this database, publications on
biological variations were reviewed with regard to sets of criteria, such as the applied statistical model, on the ratio between the so-called within subject biological variation and the analytical variation. However, they did not perform a rigorous appraisal study. This time, reviewing the applied analytical methods and the statistical analysis, which is what Biological Variation Data Critical Appraisal Checklist has been designed to deliver.
Furthermore, and that’s very important, the online database contains results from a number of historical studies performed with analytical methods that are not relevant for the methods currently in use today. And in fact, it may deliver estimates that are not relevant for the methods we're using today.
So, in the EFLM database, all the studies performed in similar population and with a similar study design for example, relevant sampling intervals, performed with updated analytical methods will be included. Furthermore, we’ll make the details of the quality score as well as detailed data on the study of population and design available to view so that they can access the evidence behind the estimates we present.
Finally, in the presently available version online, the median of included biological variation studies is presented as the common estimate. In the new database, we will apply a meta-analysis approach which takes into encompass the reliability of the reported estimates, as well as the quality scoring of the Biological Variation Data Critical Appraisal Checklist delivers, to provide a global estimates with measures of uncertainty.
Bob Barrett:
That’s very, very interesting. When do you expect this new database to be available to users? And finally, where can they find out more about it?
Aasne Aarsand:
Oh, well that’s a good question. We have obviously been -- this work has been ongoing for quite a few years. It’s the result of Task and Finish Groups established by the EFLM. Presently, we are working on finalizing the database and the required functionality. The measurands we have reviewed so far are number of commonly used measurands such as lipids, electrolytes, hemoglobin A1c, hematology markers, et cetera.
The plan is first to make the results of these measurands available in the database. And then to start reviewing include results for all the measurands. We are hopeful that we will be able to officially launch this new EFLM database by the end of the summer. And we are greatly looking forward to being able to deliver updated global biological variation estimates to laboratory professionals worldwide.
It will be available via the EFLM website and we will be using all different channels to advertise for it once it’s updated. Since it’s not available now, I can’t give you an address but we will certainly use all available channels to advertise once it is ready.
Bob Barrett:
Dr. Aasne Aarsand is Chair of the European Federation of Clinical Chemistry and Laboratory Medicine Working Group on Biological Variation and a consultant at both Haukeland University Hospital in Norway and the Norwegian Quality Improvement of Laboratory Examinations. She’s been our guest in this podcast on biological variation. I’m Bob Barrett. Thanks for listening.