Earlier this month, the Food and Drug Administration (FDA) answered industry calls for improved device submission instructions by releasing a draft guidance on preparing and reviewing premarket approval (PMA) applications and 510(k) submissions for pharmacogenetic and genetic tests. Though aimed primarily at manufacturers, the draft guidance features some important information for clinical laboratory directors about accuracy, as well as the best methods for validating the results of these tests. This issue of Strategies examines the parts of the draft guidance that are most important to laboratory directors, as well as what additional genetic testing guidance the FDA is planning to release in the future.
In vitro diagnostic manufacturers have long criticized the FDA for not keeping pace with technological innovations in the realm of genetic testing. While these manufacturers are committing more resources to genomic research and development, many are frustrated by the lack of clarity from FDA officials on what they are looking for to approve assays—an important step in making pharmacogenetic and genetic tests a part of medical practice. However, with the agency's recent release of the Draft Guidance for Industry and FDA Staff: Pharmacogenetic Tests and Genetic Tests for Heritable Markers , industry officials now have more definitive information on what the agency wants to see in PMA or 510(k) submissions for these tests. This new document replaces FDA's Feb. 27, 2003 release, Multiplex Tests for Heritable DNA Markers, Mutations and Expression Patterns: Draft Guidance for Industry and FDA Reviewers .
“The audience for this document is industry, or a laboratory that is planning a product that would come to the FDA at some point,” said Steven Gutman, MD, Director of FDA's Office of In Vitro Diagnostic Device Evaluation and Safety, Center for Devices and Radiological Health (CDRH). “This draft guidance is trying to provide information on both the analytical and the clinical issues that might be on the table when such a submission comes in. While it's hard to create a guidance that will predict every nuance and every twist and turn for products, it tries to capture elements of performance that would demonstrate the accuracy and repeatability of the test.”
While this draft is aimed at industry, it's also important for clinical lab directors, especially because it underscores FDA's commitment to confront the scientific issues surrounding this technology—such as accuracy—while moving these tests out of research labs and into clinical labs. “This shows that the FDA is very serious about moving forward with clearance of new diagnostic tests for molecular and pharmacogenomics,” said Alan H.B. Wu, PhD, Chief of Clinical Chemistry and Toxicology at San Francisco General Hospital and Professor of Laboratory Medicine, University of California , San Francisco . “They anticipate there will be more submissions in the very near future from companies that produce these kits, and they want to make sure that the process can be streamlined and not have any misunderstanding of what the FDA is looking for.”
This document raises two key issues surrounding genetic testing: what's the analytical performance of the test, and once that's been determined, what does that mean? But the FDA is also looking for feedback from both industry and the clinical lab community on the relevance of these and other issues mentioned—or ignored—in the document. “The intent of the guidance is to provide insight into the kinds of questions that would expedite the FDA review process,” said Gutman. “I would like to think that the questions we put in this guidance are questions that anyone in their right mind would want answers to before they would use the test, pay for the test, or want the test ordered on a loved one. If people think that we are asking questions that no one is interested in, then they need to let us know that we are asking the wrong questions.” However, Gutman pointed out that this draft guidance is not intended to be a roadmap to reimbursement, another regulatory hurdle currently facing genetic tests, since reimbursement is under the jurisdiction of the Centers for Medicare and Medicaid Services. “I certainly hope that this guidance helps in moving towards that direction, but it's not intended to fix that problem,” he stated.
While this draft guidance might appear truncated on clinical issues, Gutman explained that the document contains important information for laboratories on testing accuracy and validation—reiterating many of the principles outlined in the Standards for Reporting Diagnostic Accuracy, also known as the STARD Initiative which was published in the BMJ ( British Medical Journal) , Clinical Chemistry, and Annals of Internal Medicine in January 2003. “The STARD initiative is a road map for determining the accuracy of a clinical laboratory test, and creates the best practices for determining accuracy of a new test that not's been clinically carried throughout,” Gutman added.
Along with accuracy guidelines, this draft document contains important information for laboratorians about validating steps, according to Dennis O'Kane, PhD, Director, Nucleotide Polymorphism Laboratory, Dept. Laboratory Medicine and Pathology for the Mayo Clinic in Rochester , Minn.
“I think that laboratories need to revalidate a lot of the assays in their own environment to make certain that they are working properly,” said O'Kane. “The FDA has gone through what they consider the minimum acceptable criteria for submitting a 510(k) diagnostic assay in which there is already a predicate device out there. But for some of these unique tests, they may require a PMA instead. The question is, how much additional validation is required?”
Answers to the validation question will likely vary throughout the laboratory community, he predicted. “I think that you need to be looking at the accuracy—for false positives and false negatives—probably at a level that will get you down to less than 1% errors for heterozygotes,” he said. In his Mayo lab, O'Kane and his team have their own validating system: culling 100 samples for each genotype, and then performing bidirectional DNA sequencing, where feasible, to verify that the genotypes are correct. “In a bi-allelic situation, you are sequencing between 100 and 200 samples for each allele to verify whether the test is accurate,” he explained. “But that gets difficult when you have rare alleles.”
O'Kane is concerned, however, that the document discourages selecting populations, which would be done to assess accuracy, because it would alter the allele frequencies that are being used for other calculations, such as predictive values. “But that's precisely what I think you have to do in some of these scenarios where you've got a rare allele that may be critical and needs to be measured accurately,” he explained. “Therefore, you may have to enrich the population to look at the accuracy within a laboratory, but that might not be an acceptable policy to the FDA for a device submission.” This draft guidance is a great step forward, but no doubt will change over time to address the issues laboratories may bring forward for clarification, especially in the area of platform validation for pharmacogenetic testing, O'Kane added.
Just the Start
Gutman is aware that this recent draft guidance will likely make both the in vitro diagnostics manufacturers and clinical lab directors hungry for additional information from the FDA on genetic testing and pharmacogenetics. CDRH officials are continuing to work with officials from the agency's Center for Drug Evaluation and Research on the Drug-Diagnostic Co-Development Concept Paper . A draft of this paper, which will lay the foundation for guiding industry through simultaneous development of drug and diagnostics, was released in April 2005. ( www.fda.gov/cder/genomics/pharmacoconceptfn.pdf ).
“While I would never predict timelines, the concept paper is certainly a high priority,” said Gutman. Also expected in the future is a follow-up to this recent draft guidance, which will focus on expression arrays. In the meantime, however, the clinical laboratory community should familiarize itself with the FDA's draft guidance on voluntary pharmacogenomic data submissions, www.fda.gov/cder/guidance/6400fnl.pdf , as well as submit comments on all of these draft documents to the agency. “We are committed to working with companies,” said Gutman. “We learn from our review experiences, and we do learn from early interactions with companies and protocol reviews, so there's a lot of preliminary work going on in this area. The question for me is when, not if, this technology will come to primetime.”
For more information:
Comments on the draft guidance, The Draft Guidance for Industry and FDA Staff: Pharmacogenetic Tests and Genetic Tests for Heritable Markers , are due by May 10. The full document can be found on the FDA Web site: ( www.fda.gov/cdrh/oivd/guidance/1549.html )
Additional information on the STARD Initiative can be found on the Clinical Chemistry Journal's online site: www.clinchem.org/cgi/content/full/49/1/1 .
Julie McDowell is the Editor of Strategies. She can be reached by email.