June 2008: Volume 34, Number 6
The Disconnect Between Standardization and Interpretive Criteria
By Bernard Cook, PhD, DABCC, FACB
The American Cancer Society (ACS) estimates that this year 28,660 men will die from prostate cancer in the U.S., making it the second leading cause of cancer deaths in men. Today, prostate specific antigen (PSA) is the preferred serum biomarker for detecting and managing prostate cancer. To detect disease early, the ACS recommends that physicians offer both the PSA test and a digital rectal exam (DRE) annually, beginning at age 50, to men who have at least a 10-year life expectancy. Men at higher risk, including African Americans and those with one or more first-degree relatives diagnosed at an early age, should begin testing at age 45.
Forty years ago, labs measured prostatic acid phophatase in men with suspected prostate cancer. In the late 1970s, researchers recognized PSA as a potentially better marker for the disease, and in 1986, the Hybritech* Tandem-R PSA immunoassay became the first FDA-approved PSA assay to aid in the management of prostate cancer. Results of a large study published in 1991 conclusively demonstrated that prostate cancer screening with both the Hybritech PSA assay and DRE was superior to DRE alone (1). Based upon the results of this and other studies, the FDA approved the PSA test in 1994 as an aid in the early detection of prostate cancer and established 4.0 ng/mL PSA as the threshold concentration for recommending a prostate biopsy.
As demand for PSA testing grew, manufacturers who entered the PSA testing market calibrated their assays to align with the Hybritech Tandem-R test, essentially establishing it as the industry standard. But in reality, differences in antibodies and assay design created the potential for variation in a man’s PSA results if his serum was tested with different manufacturer’s methods. As these wide variations in PSA results among the early assays became apparent, researchers believed a global standardization effort would lead to uniformity of patient results. Unfortunately, the move by some manufacturers to standardize their PSA test caused a lowering of test values, creating confusion for physicians who may be unaware of the need for different interpretive criteria for results from different laboratories and how this affects health management decisions.
This article describes the history of PSA standardization efforts and how using two different materials for calibrating PSA assays has led to differences in PSA values reported to physicians. To help physicians understand and accurately interpret each patient’s PSA results, laboratorians need to become aware of when a PSA assay’s calibration is not matched to an appropriate clinical cutoff. Through diligent laboratory practice, physicians and patients can continue to receive the screening performance that they expect from the PSA assay.
Standardization of PSA Assays
Calibration of the original Tandem-R PSA assay was based on an internal reference preparation of human PSA purified by Hybritech scientists. This calibration material was also used for the clinical performance data that established PSA as a powerful prostate cancer marker.
As the problems regarding standardization and lack of agreement among PSA assays became apparent, Thomas Stamey, MD, a leading prostate cancer researcher at Stanford University, organized two conferences on the topic. A major outcome of the second conference held in 1994 was a proposal to create a standard containing a 90:10 mixture of complexed PSA and free PSA, respectively, which became the basis for the World Health Organization’s International Reference Preparation, designated WHO IRP 96/670. The 1997 CLSI document I/LA-19A, “Primary Reference Preparations Used to Standardize Calibration of Immunochemical Assays for Serum Prostate Specific Antigen (PSA),” details the purification of free and complexed PSA and the preparation of this standard.
While this standard material mitigated disparities in some of the early PSA assays, it didn’t totally resolve them. As it turned out, the mass of total PSA in the WHO IRP 96/670 preparation was approximately 20% higher than that of the original Hybritech calibration material. Therefore, PSA values measured with assays aligned to the WHO IRP 96/970 material produce results that are approximately 20% lower than PSA assays aligned to the Hybritech calibration material (Figure 1). The basis for the difference in the two calibration materials is simple to explain. PSA in the original Hybritech calibration material was quantitated using the widely accepted standard Lowry total protein method with bovine serum albumin as the standard. A decade later, Stamey and his collaborators assigned concentration values for purified PSA using more sophisticated methods, amino acid analysis and ion-spray mass spectrometry. They determined the molecular weight of PSA to be 28,430 Daltons, resulting in a molar absorption coefficient of 1.84 ± 0.04 (mL x mg-l x cm-l) at 280 nm. This value is approximately 22% higher than the previously reported coefficient of 1.42 calculated by Hybritech scientists (2). Based on this 22% difference, if a Hybritech-calibrated PSA assay was aligned with the WHO 96/670 standard, the traditional cutoff of 4.0 ng/mL would need to be adjusted to approximately 3 ng/mL.
Comparison of Traditional and WHO Calibration Curves for PSA
Manufacturers use either a PSA calibrating material similar to that developed by Hybritech (Traditional Calibration) or the WHO IRP 96/670 standard (WHO Calibration) to establish the relationship between samples of known concentration to signal produced in their PSA assays, measured in relative light units (RLU). This calibration curve is used to determine the concentration of PSA in samples of unknown concentration. As shown, for a patient sample with the same RLUs, the PSA value determined from the assay (3 ng/mL and 4 ng/mL points) depends on which calibrator is used to generate the standard curve. In this example, the value produced from the assay calibrated with the traditional material is 4.0 ng/mL, and the value produced from the assay calibrated with the WHO material is 3.2 ng/mL, a difference of 20%.
Impact on Patient Results
As manufacturers switched to the WHO IRP 96/670 standard, they did not aggressively inform physicians how this ~20% reduction would impact the interpretation of PSA results for their patients. The manufacturers also did not provide clinical labs with clear recommendations on how to determine whether new interpretive criteria were necessary with their WHO-calibrated assays. As a result, the medical community was largely unaware that patients’ PSA results from a WHO-standardized method would be substantially lower than those from an assay calibrated to agree with the original Hybritech method.
Surprisingly, only a few reports have appeared in the literature on this topic. Klee, et al. examined approximately 20,000 consecutive PSA results and expressed the analysis in terms of rates per 1,000 patients (3). Of the 176 per 1000 patients with PSA results >4 ng/mL, only 118 of these crossed the 4 ng/mL threshold if the investigators applied a 20% negative analytical bias. The remaining 58 patients (1,160 out of the 20,000 or 5.8%) would be reported as <4 ng/mL when applying the bias typically seen between Hybritech- and WHO-calibrated PSA assays.
A 2004 publication by Link et al. directly compared PSA results from the year 2000. They measured PSA values in blood samples from more than 2,300 patients with both a Hybritech-calibrated assay and two WHO-calibrated assays (4). In the group of 288 men with PSA results >2.5 ng/mL, 55 (19%) had PSA values >4.0 ng/mL with Hybritech, but not with the WHO-calibrated assays. These 55 men would have been candidates for prostate biopsy based on the traditional method of calibration but not with the WHO-calibrated assays.
What’s the Appropriate Cutoff for PSA?
While standardizing assays to a uniform calibration material can be beneficial, as discussed above, PSA standardization efforts have not eliminated the disparity among assays produced by various diagnostic companies. Although it has been more than 15 years since efforts were initiated to create a PSA standard, differences among manufacturer’s assays still exist today that affect a man’s PSA value. For example, a man may receive a PSA test result of 4.0 ng/mL with a Hybritech-calibrated assay, but his PSA value would be approximately 3 ng/mL when measured with an assay calibrated to the WHO IRP 96/670 material (Figure 2).
Effect of Standardization Bias Using a Fixed Cutoff
The top curve represents an example of using a fixed decision limit to help separate a distribution of subjects into a “normal” group and one that requires further investigation. When a 20% analytical bias is applied to the assay results as in the bottom curve, some subjects will now be below the decision limit and possibly not be considered for the care administered to those above the limit.
This difference can affect follow-up testing, because men with PSA values of 4 ng/mL are usually managed differently than men with PSA values of 3 ng/mL. Furthermore, some aggressive prostate cancer detection programs are adopting a PSA cutoff of 2.5 ng/mL for follow-up testing (5). WHO-standardization of a PSA kit has a proportional affect on this cutoff too, yielding a WHO-adjusted cutoff of ~2.0 ng/mL.
A Case of Watchful Waiting
There is even more potential for confusion and misdiagnosis when a physician closely monitors a man at high risk for prostate cancer, the so-called “watchful-waiting” approach. By definition, watchful-waiting data is accumulated over time, so it’s possible that successive PSA tests may be run with assays from different manufacturers that have been aligned to different standards.
Consider a man who is tested at age 55 using a PSA assay calibrated to the Hybritech material. His result with this assay is 3.2 ng/mL. Many physicians do not consider this value high enough to trigger a biopsy recommendation. Suppose he returns at age 56 for his second PSA test, and the result is similar to his first one. However, this time the assay used for his second PSA was calibrated with the WHO standard, and his physician is not aware of this difference in the second PSA method. As the physician reviews the two PSA results, he would not recognize that the man’s PSA value has actually increased by approximately 20% in one year. If his second test had been analyzed with a PSA assay calibrated to the same standard material as the first, the result would have been >4 ng/mL. Further, the man’s annual change in PSA of >0.75 ng/mL exceeds the National Comprehensive Cancer Network’s PSA interpretation guideline (5) and would most likely prompt the physician to recommend a prostate biopsy. But because the PSA assays used to analyze the man’s annual PSA value were calibrated to different standard materials, the physician in this case erroneously concludes that there is no change in the man’s PSA level and does not recommend a biopsy. It’s no wonder that mens’ rising and falling PSA results have bewildered physicians who are unaware of how the two different standard materials can affect an individual’s results.
Meeting the Needs of Physicians and Patients
To make sound treatment decisions, physicians need PSA assays for detection and monitoring prostate cancer that exhibit excellent consistency over time. Stable PSA test performance was made possible 20 years ago largely due to advances in monoclonal antibody production technology.However, today physicians continue to see PSA results that differ significantly from one lab to another. Physicians seldom know which manufacturer’s PSA test the lab uses and why that might matter. Moreover, very few physicians are aware that efforts to standardize PSA assays have potentially lowered their patients’ PSA results by as much as 20%, while cutoffs have somehow remained unchanged.
At the same time, highly motivated men are becoming savvy healthcare consumers and tracking their own laboratory data. As these men partner with their physicians and take greater responsibility for their own lab test results, they also need clarity in PSA testing and interpretation.
Unfortunately, standardization efforts sometimes gain momentum before the clinical impact is understood, resulting in unintended consequences for patient care. The well-recognized 4.0 ng/mL cutoff criteria for interpreting a man’s PSA concentration was established with the Hybritech-calibrated assay. Currently, manufacturers of PSA assays have aligned their results to either the original Hybritech assay or the WHO IRP96/670 material to calibrate their assays.
Laboratorians are in the best position to mitigate the impact of the disconnect between standardization and interpretive criteria for PSA assays. Until manufacturers and the medical community come to a consensus on how to resolve the issues described here, the onus is on laboratorians to determine how accurately their current PSA testing cutoff reflects the cancer detection performance physicians expect.
To meet physicians’ expectations, laboratorians need to understand how the manufacturer calibrated the PSA assay used by the lab. With this knowledge, the laboratory director can make an informed decision about the appropriateness of the lab’s PSA cutoff value and respond to physicians’ questions about their patients’ PSA test results. While physicians’ ability to correctly interpret a man’s PSA levels may have been complicated by the standardization difference in PSA assays, with proper communication, patient care need not be compromised.
Catalona WJ, Smith DS, Ratliff TL, Dodds DM, Coplen DE, Yuan JJ, et al. Measurement of prostate-specific antigen in serum as a screening test for prostate cancer. N Engl J Med 1991;324:1156–1161.
Graves HCB, Kamarei M, Stamey TA. Identity of prostate specific antigen and the semen protein p30 purified by a rapid chromatography technique. J Urol 1990;144:1510–1515.
Klee GG, Schryver PG, Kisabeth RM. Analytic bias specifications based on the analysis of effects on performance of medical guidelines. Scand J Clin Lab Invest 1999;59:509–512.
Link RE, Shariat SF, Nguyen CV, Farr A, Weinberg AD, et al. Variation in prostate specific antigen results from 2 different assay platforms: clinical impact on 2,304 patients undergoing prostate cancer screening. J Urol 2004;171:2234–2238.
Babaian R, et al. National Comprehensive Cancer Network. Clinical Practice Guidelines in Oncology 2004.
Bernard C. Cook, PhD, DABCC, FACB, is Group Manager of Scientific Professional Relations for the Immunoassay Business Group of Beckman Coulter, Inc., in Chaska, Minn.
* Hybritech, Inc. is a subsidiary of Beckman Coulter, Inc.