American Association for Clinical Chemistry
Better health through laboratory medicine
January 2011 Clinical Laboratory News: Reference Intervals and Patient Safety


PSF header

LAB SAFETY CONCEPTS

Reference Intervals and Patient Safety
The Ins and Outs of Establishing Reference Intervals

By Frederick Strathmann, PhD
Senior Fellow
Department of Laboratory Medicine, University of Washington, Seattle

Regardless of whether an assay is qualitative or quantitative, measured using a dipstick, an immunoassay, or a tandem mass spectrometer, the final interpretation by the physician remains relative. In other words, the physician must compare the laboratory value to some reference value or range of values before deciding to take action or not.

Not surprisingly, the utility of a given laboratory value is directly dependent upon the laboratory’s ability to provide analytical and physiological context to a value. Reference intervals—also referred to as reference limits or the reference or normal range—provide both of these. The reference interval tells the physician if the patient’s value is expected in a healthy or diseased individual or if further testing is warranted.

Sometimes labs must alter reference intervals, for example, when bringing on a new method for measuring an analyte. Doing so ensures that physicians can continue to interpret the laboratory value appropriately from the new method. However, defining a new reference interval is not straightforward. While extensive guidelines exist to help laboratories provide a minimum standard of patient care, no consensus exists on how to establish or alter reference intervals.

The First Step: A Look at the Requirements

Clinical laboratory directors often turn to guidance in the CLIA ’88 federal standards, as well as guidelines from agencies given deemed status by the Centers for Medicare and Medicaid Services (CMS), to better understand the regulations surrounding laboratory operations. Verification of reference intervals, outlined by CMS in the Interpretive Guidelines for Laboratories, Appendix C §493.1253, are explained as follows:

Reference Range (Normal Values)

The laboratory may use the manufacturer’s reference range, provided it is appropriate for the laboratory’s patient population (i.e., a normal range that reflects the type of specimen and demographic variables such as age and sex, as applicable). If the manufacturer has not provided reference ranges appropriate for the laboratory’s patient population, the laboratory may use published reference range(s). The laboratory must evaluate an appropriate number of specimens to verify the manufacturer’s claims for normal values or, as applicable, the published reference ranges.

This guidance raises many questions for laboratory directors. What’s the best method for determining comparability of my laboratory’s patient population to the manufacturer’s test population? Which published reference intervals are the most appropriate for my laboratory? How many specimens do we need to validate or establish a laboratory reference interval? Is it acceptable to use the laboratory’s previously established reference intervals?

Let’s look at how to approach these questions and why they are important for patient safety.

Population Comparability and Selection

Determining the characteristics of the patient population used by the manufacturer can be difficult sometimes. A few important characteristics include age, sex, race, disease/clinical status, and environment. Likewise, selecting a target population for one’s own reference range study can be challenging. The laboratory director must consider whether or not to use direct or indirect sampling, a priori or a posteriori methods of direct sampling, or random or nonrandom sampling. Furthermore, defining appropriate inclusion criteria can lead to complicated discussions about how to differentiate absence of disease from disease. Lastly, population selection varies significantly depending on whether you are establishing health-associated reference intervals or decision-based reference intervals.

Published Reference Intervals

Both manufacturers and clinical laboratories commonly use published reference intervals to establish their own intervals. In an informal survey I conducted of product inserts for 38 different tests performed on an automated clinical chemistry analyzer, 24% suggested using reference intervals based upon published literature alone, including one based on a citation more than 20 years old, 3% relied upon published literature, and 13% referred to internal manufacturers’ data. Only two inserts describing internal studies contained descriptions of the population used in the study, and one study was based upon outdated guidelines. At my own institution, 55% of these same tests, performed on the same analyzer, use a different reference interval than the manufacturer’s recommended interval based on internal validation studies and/or literature review. One of the most striking differences is illustrated by alkaline phosphatase (Table 1). We found the reference interval to be age and sex dependent, while the manufacturer only gave a single range.

Table 1
Reference Interval Comparison for Alkaline Phosphatase (U/L)
Medical Center Manufacturer
Age Male Female Literature Derived
0–9 115–324 111–281 38–126 32–91
10–11 115–324 132–366    
12–13 119–426 89–285    
14–17 72–400 43–226    
18–24 42–136 26–98    
25–34 35–109 25–100    
35–44 36–122 25–112    
45–54 39–139 34–121    
55–64 37–159 31–132    
65–74 36–161 38–172    
75+ 52–227 49–199    

Institutional Reference Intervals

In 2001, the College of American Pathologists published the results of a survey of 500 laboratories in which 78% of reporting laboratories indicated that they adopted manufacturers’ recommended reference intervals. Unfortunately, the report did not include data on the percentage of laboratories that performed adequate reference interval validation.

As a rule of thumb, establishing a reference interval requires selecting 120 separate samples for each partition, such as age and sex, with more samples needed for highly skewed values. Because the distribution of laboratory values does not always adhere to a normal or Gaussian distribution, nonparametric methods may be needed to determine appropriate reference intervals. Most often, this includes sorting the values in ascending order, assigning a rank, and selecting the values that bracket the middle 95% of the data as the reference interval.

When validating a manufacturer’s recommended range, guidelines often recommend including as few as 20 samples. Notably, this approach makes the assumption that the reference interval in question was established by adhering to stringent techniques. Tracing the origins of a manufacturer’s recommended range can be problematic if the data resides in internal documents or is based upon publications using outdated or different methodologies.

Transferability of Previously Established Intervals

For patient safety, it is vitally important for laboratories to validate reference intervals when introducing a new method for an established analyte. This is especially important when the new method has greater specificity, such as a mass spectrometry method. Frequently, method validation studies of this type include a comparison to the current method in use. The primary objective of this comparison is to facilitate the continuity of physicians’ interpretations of patient results before and after the method switch. Common data manipulations may include a regression analysis and correlation paired with outlier identification. If predetermined criteria are met, such as a slope of approximately 1 and no constant bias, it is usually acceptable to transfer the currently used reference interval. Conversely, if the results of the comparison do not meet the specified criteria, the lab may need to establish a new reference interval.

Sometimes laboratories transfer reference intervals through several method changes. This practice is not good, as traceability can be lost, potentially leaving a laboratory director unable to definitively identify the origin of the reference interval. Furthermore, because CLIA rules do not explicitly require a method comparison, laboratories deemed compliant may in fact be using reference intervals that have drifted to the point where they are inaccurate, out-dated, and potentially harmful to patients.

Transferring reference intervals multiple times also can lead to significant inter-institutional variation for the same analyte. For example, in a 2007 study, Friedberg et al. reported finding that some laboratories had no overlap in the reference interval for the same analyte, with the upper limit from one laboratory being below the lower limit of another. Furthermore, a number of institutions reported that they did not know the year that their original reference interval had been established or the date of their most recent reference interval revalidation.

The Link to Patient Safety

As electronic medical records become more commonplace, the laboratory is becoming more responsible for interpreting test results. For example, it is up to the laboratory to define and flag abnormal values in a manner that ensures review by physicians. Of course, the physician is responsible for interpreting the patient’s value in the context of his or her clinical findings and determining if any action is needed. Clearly, the accuracy of the initial lab interpretation is dependent upon the accuracy of the established reference interval. Inaccurate reference intervals can lead physicians astray and are a likely contributor to suboptimal diagnosis, suboptimal treatment, and unnecessary follow-up testing.

Despite the disparities on when to establish, validate, or accept a manufacturer’s reference interval, every laboratory is capable of verifying reference intervals for use in their own patient population. Answering the following questions can help the laboratory director decide when to establish or verify a reference interval.

Is the assay being performed in a similar manner and on samples consistent with published references? Did the results of a method comparison study demonstrate comparability? Did the manufacturer provide documentation of a more stringent or extensive study than is possible by a typical laboratory?

Armed with this information, lab directors can make sound choices that help ensure patient safety.

RECOMMENDED READING

Friedberg R, Souers R, Wagar E, Stankovic A, Valenstein P. The origin of reference intervals. Arch Pathol Lab Med. 2007;131:348–357.

Katayev A, Balciza C, Seccombe D. Establishing reference intervals for clinical laboratory test results: is there a better way? Am J Clin Pathol. 2010;133:180–186.

Solberg HE. Approved recommendation (1987) on the theory of reference values. Part 5. Statistical treatment of collected reference values: determination of reference limits. J Clin Chem Clin Biochem 1987;25:645–656.

Valenstein P, ed. Quality Management in Clinical Laboratories: Promoting Patient Safety Through Risk Reduction and Continuous Improvement. Chicago, Ill: College of American Pathologists; 2005:99–104.

Westgard, James O. Basic Method Evaluation, Third Edition. Westgard Quality Corporation, Madison, Wisc. 2008.

Patient Safety Focus Editorial Board

Chair

Michael Astion, MD, PhD
Department of Laboratory Medicine 
University of Washington, Seattle

Members

Peggy A. Ahlin, BS, MT(ASCP)
Consultant
Salf Lake City

Corrine Fantz, PhD
Emory University
Atlanta, Ga.

James S. Hernandez, MD, MS 
 Mayo Clinic Arizona 
Scottsdale and Phoenix

Brian R. Jackson, MD, MS
ARUP Laboratories 
Salt Lake City, Utah 



Patient Safety Focus Editorial Board

Chair
Michael Astion, MD, PhD
Department of Laboratory Medicine 
University of Washington, Seattle

Members
Peggy A. Ahlin, BS, MT(ASCP) 
Consultant
Salt Lake City, Utah 

Corinne Fantz, PhD
Emory University
Atlanta, Georgia

James S. Hernandez, MD, MS
 
  Mayo Clinic Arizona 
Scottsdale and Phoenix


Brian R. Jackson
ARUP Laboratories
Salt Lake City, Utah

Sponsored by ARUP Laboratories, Inc.
ARUP Logo