Calibration Verification and Linearity: Regulatory Requirements and Application to Coagulation Assays

SLIDES

TRANSCRIPTS

Slide 1:

Hello, my name is Lauren Pearson. I am an Assistant Professor of Pathology at the University Of Utah Department Of Pathology and Medical Director of clinical laboratories at University of Utah Hospital and Clinics. Welcome to this Pearl of Laboratory Medicine on “Calibration Verification & Linearity: Regulatory Requirements and Application to Coagulation Assays.”

Slide 2:

We will first begin this Pearl with some relevant definitions. Calibration verification is the process of testing materials of a known concentration in the same manner as patient specimens to assure the test system is accurately measuring samples throughout the reportable range. It is different from calibration, which is the process of establishing a correlation between the measurement signal generated by the instrument and the true concentration of the analyte in the sample. Samples can be tested in duplicate for calibration verification which may be slightly different than the process for testing patient samples.

Slide 3:

Linearity refers to the relationship between the final analytical result for a measurement and the concentration of the analyte being measured. This distinction is relevant because a plot of analyte concentration versus measurement signal from the instrument may not be linear. The concept of “linearity” is not separately designated by CLIA. Related to linearity is the concept of the analytical measurement range (AMR).

Slide 4:

The AMR is the range of concentrations of an analyte that a method can directly measure without any dilution, concentration, or other pretreatment. AMR validation is a process used to verify the linear relationship between the analytical results of a method and the concentration of analyte over the entire measurement range.

Slide 5:

CLIA regulations require that laboratories perform calibration verification at least every six months. Calibration verification is also indicated in the following situations: whenever there is a complete change in the set of reagents to a new lot, there is major preventative maintenance or replacement of critical parts of the instrument, relocation of the instrument, quality control data show a trend, shift, or are outside of acceptable limits. College of American Pathologists (CAP) checklist requirements break this down into calibration verification and AMR validation (linearity).

Slide 6:

How can labs meet the regulatory requirements? By performing a linearity experiment! The minimum requirement is to analyze three samples in duplicate that span the AMR of the assay. The samples must include a minimal value near the lower limit, a mid-point value, and a maximum value near the upper limit of the AMR. The source of materials as well as the acceptability criteria for accepting or rejecting tests during calibration verification are determined by the laboratory director. Patient samples may be used, so long as they sufficiently challenge the upper and lower ends of the AMR and are of acceptable quality and stability. Commercial kits, control materials, calibrators of a different lot than the current calibration, proficiency testing materials, and reference materials are an alternative to using patient samples, and are available for purchase from a number of vendors. It is important to ensure that samples of the appropriate matrix are used.

Slide 7:

Re-calibration of a test more frequently than every 6 months meets calibration verification requirements if the calibration includes samples with low, mid, and high values near the AMR.

Slide 8:

Calibration verification is required by CLIA, but why else is it important? Calibration verification is helpful for monitoring assay performance over time and maintaining quality results. If the calibration changes, patient results will change. It may also detect accuracy and precision problems earlier than quality control or proficiency testing data. If the assay is shown to be non- linear within the AMR, the laboratory is alerted to possible problems with reagents, specimen handling, or the instrument itself. The range of values reported on patient specimens may need to be changed accordingly.

Slide 9:

These concepts are comfortable and familiar to many laboratorians in clinical chemistry, but are newly applied to other areas of laboratory medicine, such as thrombosis and hemostasis testing. This is because in the past, coagulation testing was primarily clot-based testing using instruments that were not calibrated to measure the concentration of an analyte. Methodology has evolved since then and many coagulation laboratories use methods which may be calibrated and measure a concentration of an analyte. Hence, the requirements for calibration verification now apply in the coagulation laboratory.

Slide 10:

Examples of assays which meet this criteria include EIA methods, immunoturbidity methods, and chromogenic methods. This slide shows many examples of such applicable assays, some of which are often available in routine or stat laboratory settings as well as reference laboratory settings.

Slide 11:

Not all coagulation assays are calibratable, and thus these requirements will not apply. Examples of exempt assays include clot-based assays and platelet function tests.

Slide 12:

We will now transition to applying these concepts to a specific example, quantitative D-dimer. In this example, the AMR of the assay is 0.27-4.0 micrograms per milliliter. The linearity experiment I will show in the following slides consisted of analyzing five samples spanning the AMR, each measured in triplicate. Linear regression analysis was performed and slope and intercept were calculated. In this example, the source of the samples was a commercially produced kit.

Slide 13:

Here is a table listing the mean observed values of the raw data for the measurements for D- dimer obtained for each sample. Notice that for each sample, the mean observed measurement is close to, or equal to the expected value.

Slide 14:

Here is a scatter plot of the data. Each of the individual measurements for each sample are plotted. The x-axis is the expected concentration of D-dimer for each sample, and the y-axis is

the measured concentration. A linear regression line with a slope of 0.992 and intercept of - 0.001 was fit to the points. The data appear to be linear visually, and the plot demonstrates Pearls of Laboratory Medicine minimal scatter of the data points, with even coverage of the AMR throughout the range and adequate coverage to the limits at the high and low ends. All differences between the observed values and the expected values are within allowable error limits. The slope and intercept indicate minimal proportional and constant bias.

Slide 15:

We will now discuss what to do if you observe that an assay is not linear over its AMR, or if unexpected bias or imprecision is present.

Slide 16:

If the results show that the assay is non-linear over the full range or even a partial range, there are three areas to focus your troubleshooting steps. First, review specimen handling steps. Were the samples used for testing stored appropriately? If a kit was used, were the kit’s instructions followed? If patient samples were used, were they processed according to standard operating procedure prior to testing to ensure adequate mixing, centrifugation, or were other necessary processing steps were taken? Next, examine the analytic phase of testing. Were standard operating procedures followed appropriately? Was instrument maintenance performed as applicable? Were quality control results acceptable? Were reagents used within stability? Were any flags or errors generated by the instrument during testing? Was testing performed by an individual deemed competent to perform testing? Lastly, consider the possibility of clerical errors if results from the instrument were transcribed into another file for data analysis.

Slide 17:

It is fairly common to encounter situations where an assay is linear over the tested range, however, the samples tested at the low end or the high end of the AMR are problematic. Trouble at the low and high end is observed when the samples don’t come close enough to the limits of the AMR, or when samples do adequately challenge the ends but the observed values are different than expected. For the former, the lab may need to acquire additional samples near the low end and the high end for analysis. If the lower or upper end of the presumed AMR cannot be verified, then labs have the option of using a narrower AMR. If the observed values are different than expected, it could be the case that the analyte concentrations of the samples were not within the AMR of the instrument, so this should be verified as well. For other problems with high or low specimens, assess pre-analytic variables including sample handling and degradation. Consider errors due to recovery of the analyte, dilution protocols, etc.

Slide 18:

An assay may be proven to be linear but show unacceptable bias. Bias is evident when the linear regression analysis produces a slope that is not equal to 1, a non-zero intercept, or differences on a bias plot. What constitutes acceptable bias is at the discretion of the laboratory director. Investigate possible sources of bias by examining quality control results, instrument Pearls of Laboratory Medicine maintenance records, recent calibration data, standard operating procedures, reagent lot-to-lot
comparisons, and sample quality.

Slide 19:

An assay may be proven to be linear but show unacceptable imprecision. Possible manifestations include unexpected increased scatter in the data, large differences between replicates for specimens, or a standard deviation which exceeds allowable error. Begin the investigation by reviewing specimen handling steps and quality control data. If the source of imprecision is not evident, you may elect to perform a simple precision study using a set of samples, preferably patient samples, to further investigate.

References:

There are numerous useful resources available for assistance with meeting regulatory requirements for calibration verification and linearity, many of which are listed on this slide. Additionally (not cited here), there is a Clinical Laboratory Standards Institute document, EP-6, which may be useful.

Slide 19: Disclosures

Dr. Pearson is employed by the University of Utah and ARUP Laboratories.

Slide 20:

Thank You from www.TraineeCouncil.org
Thank you for joining me on this Pearl of Laboratory Medicine on “Calibration Verification & Linearity: Regulatory Requirements and Application to Coagulation Assays.”