LAB SAFETY CONCEPTS
Reference Intervals and Patient Safety
The Ins and Outs of Establishing Reference Intervals
By Frederick Strathmann, PhD
Department of Laboratory Medicine, University of Washington, Seattle
Regardless of whether an assay is qualitative or quantitative, measured using a dipstick, an immunoassay, or a tandem mass spectrometer, the final interpretation by the physician remains relative. In other words, the physician must compare the laboratory value to some reference value or range of values before deciding to take action or not.
Not surprisingly, the utility of a given laboratory value is directly dependent upon the laboratory’s ability to provide analytical and physiological context to a value. Reference intervals—also referred to as reference limits or the reference or normal range—provide both of these. The reference interval tells the physician if the patient’s value is expected in a healthy or diseased individual or if further testing is warranted.
Sometimes labs must alter reference intervals, for example, when bringing on a new method for measuring an analyte. Doing so ensures that physicians can continue to interpret the laboratory value appropriately from the new method. However, defining a new reference interval is not straightforward. While extensive guidelines exist to help laboratories provide a minimum standard of patient care, no consensus exists on how to establish or alter reference intervals.
The First Step: A Look at the Requirements
Clinical laboratory directors often turn to guidance in the CLIA ’88 federal standards, as well as guidelines from agencies given deemed status by the Centers for Medicare and Medicaid Services (CMS), to better understand the regulations surrounding laboratory operations. Verification of reference intervals, outlined by CMS in the Interpretive Guidelines for Laboratories, Appendix C §493.1253, are explained as follows:
Reference Range (Normal Values)
The laboratory may use the manufacturer’s reference range, provided it is appropriate for the laboratory’s patient population (i.e., a normal range that reflects the type of specimen and demographic variables such as age and sex, as applicable). If the manufacturer has not provided reference ranges appropriate for the laboratory’s patient population, the laboratory may use published reference range(s). The laboratory must evaluate an appropriate number of specimens to verify the manufacturer’s claims for normal values or, as applicable, the published reference ranges.
This guidance raises many questions for laboratory directors. What’s the best method for determining comparability of my laboratory’s patient population to the manufacturer’s test population? Which published reference intervals are the most appropriate for my laboratory? How many specimens do we need to validate or establish a laboratory reference interval? Is it acceptable to use the laboratory’s previously established reference intervals?
Let’s look at how to approach these questions and why they are important for patient safety.
Population Comparability and Selection
Determining the characteristics of the patient population used by the manufacturer can be difficult sometimes. A few important characteristics include age, sex, race, disease/clinical status, and environment. Likewise, selecting a target population for one’s own reference range study can be challenging. The laboratory director must consider whether or not to use direct or indirect sampling, a priori or a posteriori methods of direct sampling, or random or nonrandom sampling. Furthermore, defining appropriate inclusion criteria can lead to complicated discussions about how to differentiate absence of disease from disease. Lastly, population selection varies significantly depending on whether you are establishing health-associated reference intervals or decision-based reference intervals.
Published Reference Intervals
Both manufacturers and clinical laboratories commonly use published reference intervals to establish their own intervals. In an informal survey I conducted of product inserts for 38 different tests performed on an automated clinical chemistry analyzer, 24% suggested using reference intervals based upon published literature alone, including one based on a citation more than 20 years old, 3% relied upon published literature, and 13% referred to internal manufacturers’ data. Only two inserts describing internal studies contained descriptions of the population used in the study, and one study was based upon outdated guidelines. At my own institution, 55% of these same tests, performed on the same analyzer, use a different reference interval than the manufacturer’s recommended interval based on internal validation studies and/or literature review. One of the most striking differences is illustrated by alkaline phosphatase (Table 1). We found the reference interval to be age and sex dependent, while the manufacturer only gave a single range.
Reference Interval Comparison for Alkaline Phosphatase (U/L)
Institutional Reference Intervals
In 2001, the College of American Pathologists published the results of a survey of 500 laboratories in which 78% of reporting laboratories indicated that they adopted manufacturers’ recommended reference intervals. Unfortunately, the report did not include data on the percentage of laboratories that performed adequate reference interval validation.
As a rule of thumb, establishing a reference interval requires selecting 120 separate samples for each partition, such as age and sex, with more samples needed for highly skewed values. Because the distribution of laboratory values does not always adhere to a normal or Gaussian distribution, nonparametric methods may be needed to determine appropriate reference intervals. Most often, this includes sorting the values in ascending order, assigning a rank, and selecting the values that bracket the middle 95% of the data as the reference interval.
When validating a manufacturer’s recommended range, guidelines often recommend including as few as 20 samples. Notably, this approach makes the assumption that the reference interval in question was established by adhering to stringent techniques. Tracing the origins of a manufacturer’s recommended range can be problematic if the data resides in internal documents or is based upon publications using outdated or different methodologies.
Transferability of Previously Established Intervals
For patient safety, it is vitally important for laboratories to validate reference intervals when introducing a new method for an established analyte. This is especially important when the new method has greater specificity, such as a mass spectrometry method. Frequently, method validation studies of this type include a comparison to the current method in use. The primary objective of this comparison is to facilitate the continuity of physicians’ interpretations of patient results before and after the method switch. Common data manipulations may include a regression analysis and correlation paired with outlier identification. If predetermined criteria are met, such as a slope of approximately 1 and no constant bias, it is usually acceptable to transfer the currently used reference interval. Conversely, if the results of the comparison do not meet the specified criteria, the lab may need to establish a new reference interval.
Sometimes laboratories transfer reference intervals through several method changes. This practice is not good, as traceability can be lost, potentially leaving a laboratory director unable to definitively identify the origin of the reference interval. Furthermore, because CLIA rules do not explicitly require a method comparison, laboratories deemed compliant may in fact be using reference intervals that have drifted to the point where they are inaccurate, out-dated, and potentially harmful to patients.
Transferring reference intervals multiple times also can lead to significant inter-institutional variation for the same analyte. For example, in a 2007 study, Friedberg et al. reported finding that some laboratories had no overlap in the reference interval for the same analyte, with the upper limit from one laboratory being below the lower limit of another. Furthermore, a number of institutions reported that they did not know the year that their original reference interval had been established or the date of their most recent reference interval revalidation.
The Link to Patient Safety
As electronic medical records become more commonplace, the laboratory is becoming more responsible for interpreting test results. For example, it is up to the laboratory to define and flag abnormal values in a manner that ensures review by physicians. Of course, the physician is responsible for interpreting the patient’s value in the context of his or her clinical findings and determining if any action is needed. Clearly, the accuracy of the initial lab interpretation is dependent upon the accuracy of the established reference interval. Inaccurate reference intervals can lead physicians astray and are a likely contributor to suboptimal diagnosis, suboptimal treatment, and unnecessary follow-up testing.
Despite the disparities on when to establish, validate, or accept a manufacturer’s reference interval, every laboratory is capable of verifying reference intervals for use in their own patient population. Answering the following questions can help the laboratory director decide when to establish or verify a reference interval.
Is the assay being performed in a similar manner and on samples consistent with published references? Did the results of a method comparison study demonstrate comparability? Did the manufacturer provide documentation of a more stringent or extensive study than is possible by a typical laboratory?
Armed with this information, lab directors can make sound choices that help ensure patient safety.
Friedberg R, Souers R, Wagar E, Stankovic A, Valenstein P. The origin of reference intervals. Arch Pathol Lab Med. 2007;131:348–357.
Katayev A, Balciza C, Seccombe D. Establishing reference intervals for clinical laboratory test results: is there a better way? Am J Clin Pathol. 2010;133:180–186.
Solberg HE. Approved recommendation (1987) on the theory of reference values. Part 5. Statistical treatment of collected reference values: determination of reference limits. J Clin Chem Clin Biochem 1987;25:645–656.
Valenstein P, ed. Quality Management in Clinical Laboratories: Promoting Patient Safety Through Risk Reduction and Continuous Improvement. Chicago, Ill: College of American Pathologists; 2005:99–104.
Westgard, James O. Basic Method Evaluation, Third Edition. Westgard Quality Corporation, Madison, Wisc. 2008.
Patient Safety Focus Editorial Board
Michael Astion, MD, PhD
Department of Laboratory Medicine
University of Washington, Seattle
Peggy A. Ahlin, BS, MT(ASCP)
Salf Lake City
Corrine Fantz, PhD
James S. Hernandez, MD, MS
Mayo Clinic Arizona
Scottsdale and Phoenix
Brian R. Jackson, MD, MS
Salt Lake City, Utah