Listen to the Clinical Chemistry Podcast
Article
E.S. Boja, S.A. Jortani, J. Ritchie, A.N. Hoofnagle, Z. Tezak, E. Mansfield, P. Keller, R.C. Rivers, A. Rahbar, N.L. Anderson, P. Srinivas, H. Rodriguez. The Journey to Regulation of Protein-Based Multiplex Quantitative Assays Clin Chem 2011; 57: 560–567.
Guest
Dr. Emily Boja and Dr. Henry Rodriguez are from the Office of Cancer Clinical Proteomics Research at the National Cancer Institute, National Institutes of Health
Transcript
[Download pdf]
Bob Barrett:
This is a podcast from Clinical Chemistry. I am Bob Barrett.
The discovery that proteins and peptides, originating from diseased organs or by tumors can be found in blood and urine have led to the active investigation of a variety of biomarkers for diagnosing cancer or monitoring the response to therapy. Presently, more than 1200 protein biomarker candidates for cancer have been described in the scientific literature. Unfortunately, the rate of the US Food and Drug Administration Approval has remained relatively flat over the past 15 years. Several factors have been implicated in leading to this discrepancy.
In a report published in the April issue of Clinical Chemistry, Dr. Emily Boja, a Program Manager in the Office of Cancer Clinical Proteomics Research at the National Cancer Institute, National Institutes of Health along with the Director of the Office Dr. Henry Rodriguez provided the research community with a resource that explains the regulatory processes for translating biomarker candidates discovered in research laboratories into multiplex protein-based assays for clinical use. Dr. Boja and Dr. Rodriguez are our guests in this podcast.
Dr. Boja, what is the current status of applying proteomics-based discoveries and assays into clinical settings?
Emily Boja:
Well, this is a really great question, Bob. As you know, clinical proteomics holds great promise for personalized medicines because of its ability to detect and quantify disease-relevant biomarkers in clinical bio-specimens. And if you look at the scientific literature out there for cancer alone, there are over 1200 biomarker candidates described in peer-reviewed journals.
However, if you look at what's being approved by the FDA to be used in a clinical setting, the approval rate is averaging about 1.5 per year over the last 15 years. I think currently there are a little over 109 protein-based biomarkers being used in the clinic and almost an equal amount of laboratory developed tests using protein biomarkers that are not FDA approved. So this clearly points to a deficiency in the translation of proteomics from the bench to the bedside.
Through rounds of discussions in the past, several factors have been implicated in this huge disconnect between what's being discovered on the front-end using modern proteomic technologies versus what's being used in the clinic. One of which that our office use is very important is the inability to credential these biomarker candidates that had been discovered on the front-end to be used in the downstream large- scale clinical studies of what we term the Clinical Validation or Qualification.
And we think there is a missing link in the current biomarker development pipeline, and by restructuring that pipeline by introducing a middle stage, what we termed verification using targeted proteomics approach or on hundreds of samples is a very good way to triage biomarker candidates that are thousands and thousands of them out there from discovery into the downstream large-scale clinical qualification. And this is where our office is advocating right now in our current biomarker pipeline.
Bob Barrett:
Okay, now Dr. Rodriguez, in 2008 there was a Regulatory Science Workshop, which led to the development of publicly accessible Mock 510(k) pre- submission documents in clinical chemistry last year. Is your article a follow-up to this workshop?
Henry Rodriguez:
Actually, it is. I mean one of the things that we did here is we have been very proactive in actually working with the regulatory agencies and with the research community.
So in that context, what we did in late of October 2008, we actually did what I think is a real groundbreaking and forward-thinking workshop, and we did it in collaboration with the investigators of NCI's clinical proteomic technologies assessment for cancer network.
It also involved clinical laboratories, the clinical chemistry community, instrument manufacturers, the research community at large, and also representatives from the regulatory agency, and the whole basis was to really get together and to try and get an understanding on what are the processes that's involved in moving these sorts of analytical technologies and platforms to the regulatory agencies. Because the reality today, that there is really no guidance when it comes to multiplex proteomic assays.
And what came out of it in addition to a workshop report, which subsequently that got published in Clinical Chemistry, we actually came up with these very unique first of its kind public 510(k) pre- submissions. Now these are marked documents, but the eloquence that comes out of it is that today they serve as review documents for the community at large and it really illustrates the details that's involved, if one were to develop 510(k) submissions, which is commonly done by assay developers in order to provide the necessary information to the proteomics community prior to their actual submissions to the FDA.
It allows them now to better understand, it’s correctly designed on how to address not just the analytical but also the clinical questions that's asked by the FDA.
Bob Barrett:
Doctor, who participated in that workshop?
Henry Rodriguez:
So as I illustrated earlier or that I mentioned earlier, it was a multitude of individuals. But it's really the key individuals I think that's involved in this science. So not only was it open to the research community, it really was driven by our investigators. But at the same time it absolutely involved the clinical chemistry community. So we worked closely with the American Association for Clinical Chemistry. We also worked with the instrument manufacturers, and of course, we also worked with the FDA.
Bob Barrett:
Dr. Boja, can you provide some insight on FDA's In Vitro Diagnostics Classification criteria for these multiplex assays?
Emily Boja:
Sure Bob. The FDA classifies their in vitro diagnostic products based on the risk to health outcomes of patients, and these risks really depend upon the claimed intended use of that device or that test system.
Currently there are three classes namely I, II, and III that are in the system. Class I is a low-risk based device, most of which are exempt from pre-market review requiring only general controls and GMP compliance. In the old days, the old generation mass spectrometers for example fall under that category.
Class II is based on moderate risk, which requires 510(k) level pre-market review, and the test performance of that device is compared against a pre-existing predicate device. And if it's deemed substantially equivalent to the predicate device then it's granted a 510(k) clearance status.
Class III is high-risk based device or test. It poses significant risk or unreasonable risk to the patients, but it also may have substantial significance for the prevention of the impairment of health. So this is the highest bar one can achieve for the test which requires a pre-market approval or PMA review status.
So there is an additional, de novo route that an assay sponsor can take to which their device has no predicate device to compare the performance with. So the FDA will automatically assign a de novo device as a Class III device to which after review they can choose to downgrade or down-classify it to a Class I or a Class II if the intended use of that device is deemed low or moderate risk. So to use an example to put into context of the effect of intended use for classification, the cystic fibrosis transmembrane receptor genotyping test, when it's used to aiding the diagnosis of cystic fibrosis it's considered a Class II requiring only 510(k) clearance.
However, when you use it for fetal screening, which poses a much higher risk, than it's considered as a Class III device requiring PMA approval.
Bob Barrett:
What is your view regarding the regulation of these multiplex protein tests, and how does this compare with the FDA's perspective?
Emily Boja:
Well, as Dr. Rodriguez alluded to earlier, these multiplex protein tests as described in this article including MRM or multiple reaction monitoring mass spectrometry and immunological array not really including the well established ELISA-based multiplex assays as examples in the previous mock document, are really uncharted territories and the FDA is still exploring in that space in terms of how to regulate these devices.
So it is very, very important for us to collaborate with the regulatory agency and the research community on how to understand the validation criteria required by the FDA, so that when these tests become mature and their submissions coming through the door to the FDA, then the researchers are better prepared for what the FDA is looking for.
Now these tests are very, very complex involving multiple analytes, interpretive software, and intricate operation of instrumentation requiring experienced personnel and simplification and automation eventually to streamline the whole sampled preparation process. So at the current time there needs to be a standardized evaluation paradigm that everybody can agree on in order to understand what the FDA is looking for, again when these submissions come through the door.
Bob Barrett:
From your own perspective, can you comment on how the National Cancer Institute is leveraging multiple reaction monitoring mass spectrometry-based assays to the field of proteomics, and are these multiplex assays something new to most clinical laboratories?
Emily Boja:
As a matter of fact, these multiple reaction monitoring mass spectrometry-based assays have been used in clinical labs on small molecules and metabolites for decades, an example being neonatal screening for metabolic disorders. So there is a very strong instrument base of these triple quadruple mass spectrometers already deployed into clinical labs.
So no, this is not new science, but when it comes to quantitative proteomics using MRM mass spec technology, it really has not been demonstrated across multiple labs previously. And the NCI was very interested in expanding this science from small molecule to the protein and peptide world, and as a result of the Clinical Proteomic Technology Assessment Centers conducted an inter-laboratory study in 2009 using 11 peptides from seven proteins by into a depleted human plasma, it involved eight individual labs with SOPs, how to calibrate instruments, heavy isotope labels, peptide standards. And it was the first time that the team has demonstrated that the MRM technology when used appropriately can achieve accurate results with the reproducibility on average using one single transition of less than 23%.
Now we realized that this is not optimal for clinical use; ideally, you will want CBs(ph) typically in the 5%-10% range to be used in clinical labs, but this is a first one of its kind preliminary pre-clinical study that has ever demonstrated the reproducibility of this technology across different instrument platforms in multiple laboratories.
So we anticipate that with improvement of the technology, optimization, automation and sample preparation, and potentially reducing the cost of the assays being developed and improvement in analytical variability, this technology maybe some day will have a place in clinical labs, especially when well-established methodologies such as ELISA will fail in a subset population of patients.
Bob Barrett:
Doctor, in your opinion, what are some helpful tips and resources that an assay sponsor could use to submit their data from multiplex protein test to the FDA for approval?
Emily Boja:
Well, there are actually some very useful tips and resources out there. For example, prior to launching a large scale clinical study, the FDA encourages the assay sponsors to engage in a communication with them early on through what’s called the pre- Investigational Device Exemption process or pre-IDE process.
This process will allow the assay sponsor to submit their analytical or clinical protocols for the FDA for review and comment again prior to the study, and then will also allow them to gain feedback from the FDA about establishing their protocol and explore possible regulatory pathways, especially when the technology they are using are cutting edge or the data analysis and bio-statistical analysis are very complex.
Now when they are performing the studies, it has to meet all the analytical and clinical criteria that are required by the FDA, and one of the very useful documents out there are the guidance documents published by Clinical Laboratory Standards Institute (CLSI). CLSI is very highly regarded by the clinical professionals and their documents go through rigorous rounds of review prior to approval by consensus, by all the stakeholders involved in the discussions and sometimes including FDA representatives.
For example, even though currently there are no CLSI documents existing for multiplex protein-based assays because the technology is still relatively new, there are several documents that we think that are very helpful for proteomic researches. One is the EP17-A which is Protocol for the Determination of Limits of Detection and Limits of Quantitation, which is highly cited for quantitative assays.
Another one is MM17A, which is Verification and Validation of Multiplex Nucleic Assays. Even though it’s for nucleic acid, you can still draw some conclusions from the DNA world. So that may be very helpful for the proteomics community to look at.
Bob Barrett:
Your article underscores the importance of engaging clinical chemists in the proteomics development process. Now within this context, how can clinical chemists and other clinical laboratory scientists contribute to the process?
Emily Boja:
Well, the clinical chemists are trained scientists to handle both pre-analytical and analytical issues everyday in their clinical labs. So having them involved in every stage of the biomarker development pipeline is very important.
NCI currently has a Memorandum of Understanding with the American Association of Clinical Chemistry Proteomics Division, and that illustrates how important we see the clinical chemists can play a role in our biomarker development pipeline.
For example, they are very familiar with the pre- analytical variables and their assessment including sample handling and processing. So having them involved early on in proteomic discovery will really add tremendous value to ensure that the discovered results are meaningful, and because they are also very familiar with the QA/QC measures, the validation methods including LDL-q, reproducibility, precision, linearity and all those analytical parameters involved in assay developments. Having them involved in the entire process of verification or even clinical validation will be very, very valuable.
Bob Barrett:
Well finally, are there any new activities or workshops that are being planned in the area of proteomics regulatory science that you can share with us?
Emily Boja:
Yes, there are actually several activities going on right now. First of all, our office is planning on a workshop in collaboration with the FDA and the AACC community on how to properly design experiments using statistical power and proteomics. So that’s tentatively to be held in August of this year. Now at the NIH level, the Common Fund has issued a RFA last year on regulatory science in order to advance the new knowledge innovation and technology for medical products.
Just last week, FDA just held a public meeting to gain public insight on their medical device innovation initiative in order to improve their 510(k) medical device oversight, which involves the de novo of 510(k) filing procedures.
So these are all of the very exciting activities that’s going on right now. So stay tuned.
Bob Barrett:
Dr. Emily Boja and Dr. Henry Rodriguez are from the Office of Cancer Clinical Proteomics Research at the National Cancer Institute, National Institutes of Health, and they’ve been our guests in this podcast from Clinical Chemistry. I am Bob Barrett. Thanks for listening!