Dockets Management Branch (HFA-305)
Food and Drug Administration
5630 Fishers Lane, Room 1061
Rockville, Maryland 20850
The American Association for Clinical Chemistry (AACC) welcomes the opportunity to comment on the criteria used by the Food and Drug Administration (FDA) to determine whether a test is waived under the Clinical Laboratory Improvements of 1988 (CLIA'88). We support FDA's effort to clarify the regulatory language, ensuring that the criteria are completely consistent with the CLIA'88 statutory language. However, we believe that it is impossible to discuss such criteria without also discussing federal oversight of those facilities performing waiver tests. Although HCFA is responsible for such oversight, we believe this is an appropriate venue for addressing both issues. Our comments follow:
Congress mandated that waived tests "...have an insignificant risk of an erroneous result." AACC believes that this requirement is most effectively met when devices are simple to operate, reliable under both intended and unintended conditions (which can reasonably be anticipated to occur), and accurate under a wide range of environmental and analytical conditions.
AACC believes that "insignificant risk," as used in the CLIA'88 statute, was intended to mean "infrequent" or "rare," and that the term is intended to be applied under the conditions of actual use. That is, "insignificant risk" is measured by the rate of error when operators who typically perform such testing perform the test. We believe that the "acceptable error rate" is entirely dependent upon how "error rate" is defined. AACC recommends that manufacturers be required to: (1) explain how they measured error frequency; (2) justify the standard used for classifying results as erroneous; and (3) document the error rate observed with the system or device proposed for waived status. In the case of electronic devices, the product should be so simple, accurate and user friendly that it does not report an answer if the test is performed improperly.
AACC believes that the words "erroneous result," as used in the CLIA'88 statute, were chosen broadly to include both analytical accuracy and clinical applicability. In other words, when a result is not erroneous, it will measure quantitatively or qualitatively the substance or property it purports to measure, and it will provide clinically relevant information applicable to the condition of the patient being tested.
We believe analytical accuracy can be demonstrated by (a) comparing the test to an established, approved method, and/or (b) by analyzing well-characterized reference materials. We recommend that the manufacturer be required to make a claim of accuracy against (a) or (b) above, explaining why the approach chosen is appropriate. Such assertions should include claims regarding reliability and precision, using statistical or other appropriate descriptions. We suggest that the manufacturer, who is most familiar with the product and its application, be given reasonable latitude in selecting the appropriate method(s) to support its claims.
Further, AACC believes that clinical accuracy data should be presented for devices being considered for waived status. For some devices, this may be straightforward (e.g., fecal occult blood or devices measuring a substance that is abnormal at any detectable concentration), while for others, more complex data may be required (e.g., quantitative measurements or tests that measure substances relevant to different clinical situations). In all cases, the criteria for interpretation of the result, and classification of the patient (normal vs. abnormal, pregnant vs. non-pregnant) must be provided. In addition, we believe the manufacturer should provide to the FDA a clear explanation of the risk of harm to the patient if the test is performed incorrectly.
This requirement for clinical accuracy requires consideration of the intended purpose of the test and the use of test results in evaluating the proposed device for waiver status. We do not believe that tests intended for use in the following situations should be granted waived status: (1) when the test result is substantially or entirely the basis for a clinical decision; (2) when a physical examination and/or other tests do not offer a prompt and accurate means to confirm or reject the test result in question; and (3) when the consequences of the clinical decision creates a risk of significant physical or emotional harm to the patient, to other individuals, or to the public at large.
Ease of Operation
The analytical and clinical accuracy data described above provide the operating characteristics of a device under ideal circumstances and typically yield results when the device is operated perfectly. Equally important is an assessment of device performance when it is used under actual operating conditions, including performance characteristics when tests are done by well-trained operators, as well when the manufacturer's recommendations are not followed. The following are examples of such conditions:
- Operation under increased or decreased ambient temperature;
- Storage of device components under increased or decreased storage temperatures;
- Use of outdated components or calibrators;
- Mixing of lot specific kit components;
- Use of non-recommended specimens; and
- Use of excess or insufficient specimen amount.
AACC believes that the FDA should treat screening tests as any other laboratory test. Failure to correctly assess the need for medical intervention is not test classification dependent. Thus, we do not believe their needs to be a higher threshold for placing screening tests in the waiver category.
AACC supports the existing requirement that manufacturers demonstrate ease-of-use through field studies using individuals with a 7th grade comprehension level, including individuals with limited English language skills. In performing these studies, manufacturers should design studies that emulate, as closely as possible, the potential end users of the product.
Currently, manufacturers seeking waiver status are required to submit data from three different sites, using 20 lay users, who test 2-3 samples at appropriate decision points. We recommend that FDA grant manufacturers greater flexibility in meeting this requirement, thereby placing the burden on the manufacturer for determining whether the system is safe. Given the amount of time and effort manufacturers invest in developing new products, they are more likely to submit more data than is necessary to enhance the chances their device will be categorized as waived.
We recommend that interference, environmental and flex (stress) studies done by manufacturers to establish the performance of waived tests, be done in-house and reported as part of a request for waived status. We do not believe additional studies, beyond those already mentioned, are necessary for evaluating qualitative tests under consideration for waiver status. However, the "cut point" should be quantitatively defined in manufacturers' submission to FDA. Test data should be obtained at realistic levels of an analyte, as well as on true negatives. And, in regard to evaluating quantitative tests for waiver, the manufacturer should be free to choose realistic levels, but should also be free to submit any data that supports the value of their product.
Another issue that must be considered when discussing the placement of tests in the waiver category is whether there is adequate oversight of such testing. In recent years, technological advances have allowed manufacturers to develop new and simpler devices, which make it easier for individuals with less training to accurately perform tests that were previously performed in more sophisticated laboratories. This technology-based trend is likely to accelerate in the near future.
There are great benefits to point-of-care testing, such as the potential for diagnosing and treating the patient earlier and reducing overall health care costs. However, it is critical to ensure that POCT and other laboratory tests performed in waiver facilities are done accurately. Unfortunately, because of implementation and resource issues, there has been little effective federal oversight of these laboratories. AACC believes it is essential, given the likely expansion of the waiver category, and recent media reports about CLIA noncompliance by waiver labs, that this concern be adequately addressed.
We believe that many of the noncompliance issues can be addressed within the existing regulatory framework. For example, AACC recommends that the Health Care Financing Administration (HCFA) (the agency responsible for inspecting CLIA facilities):
- randomly inspect a percentage of waiver labs annually to evaluate program compliance (e.g., the laboratory is only performing those tests on its certificate; the facility's personnel are following manufacturers instructions; and verifying the accuracy of the laboratory's reported results);
- use their discretionary authority to conduct follow-up inspections, when deemed necessary, on facilities with serious problems (the costs of the follow-up inspection should be borne by the waiver facility);
- develop a self-assessment tool for waiver facilities to identify, correct and report problems; and
- require that the owner/authorized representative attest in writing that the individuals doing the tests can competently perform them.
AACC believes that these changes, if implemented, will assure safe testing and improve the overall quality of testing in waiver facilities without significantly increasing program costs. We look forward to working with the FDA and HCFA to maintain the quality of laboratory testing, while improving the effectiveness of the CLIA program.
By way of background, AACC is the principal association of professional laboratory scientists--including MDs, PhDs and medical technologists. AACC's members develop and use chemical concepts, procedures, techniques and instrumentation in health-related investigations and work in hospitals, independent laboratories and the diagnostics industry nationwide. The AACC's objectives are to further the public interest and educational activities and to help maintain high professional standards.
If you have any questions or we may be of any assistance, please call me at (919) 684-8724 or Vince Stine, Director, Government Affairs, at (202) 835-8721.
Frank A. Sedor, PhD