As the SARS-CoV-2 virus hit U.S. shores, the nation suddenly needed widescale, reliable diagnostic testing infrastructure in place—and fast. It turned out to be harder than non-laboratorians might think to manufacture out of thin air 5 million validated, accurate polymerase chain reaction (PCR)-based tests for a novel virus. The early quality control plan, which involved central testing available only at the Centers for Disease Control and Prevention (CDC), with no scope for laboratory-developed tests, proved to be woefully inadequate to meet demand, and the CDC’s tests were plagued with technical concerns (1).
In late January the U.S. declared a formal Public Health Emergency (2), freeing up some of the regulatory burden and allowing public and private laboratories to develop tests to meet demand. Normally, of course, we can’t use something in the care of actual human patients unless we show it works and is safe. That means a device, test, or drug has to be approved by the Food and Drug Administration (FDA)—a yearslong process in which the agency independently verifies that a test developer has correctly demonstrated that its assay works.
In an official Emergency-with-a-capital-E, though, FDA takes at face value the test and internal safety data that manufacturers submit; the agency then grants an emergency use authorization (EUA) and, with several disclaimers, the manufacturer can offer its test on the market. However, the EUA only lasts as long as the Emergency, after which it melts away like leprechaun gold, and we’re back to the “prove your data” standards.
During the early days of the COVID-19 pandemic, this process allowed a few high-quality diagnostic tests to trickle onto the market. Testing was somewhat regionally based (for example, an early testing program at the University of Washington), and market saturation was minimal. Supply was ravenously outstripped by demand. The lack of wide scale diagnostic testing led to outrageous bottlenecks and, in many locations, a near-complete lack of information about the shape of the epidemic. Concomitantly, of course, researchers and commercial actors furiously attempted to develop serology tests for the virus—with the same problems.
Welcome to the Wild, Wild West
By March 16, FDA removed the brakes: All developers need to do is bring something to market and submit an EUA to FDA within 15 days. In the ensuing Wild, Wild West, unscrupulous vendors could a) misrepresent how good their test was, and no one would catch them; and b) they could offer their tests to the public—to human patients—without anyone checking to see if these tests worked.
FDA would start to apply the brakes again eventually, but the market was a free-for-all until then.Within a month, at least 90 serology tests were floating around in the U.S., with no fewer than 275 registered globally by the World Health Organization (3), many exhibiting the expected abysmal quality. For example, the city of Laredo, Texas, bought 20,000 rapid antibody tests for $500,000, only to discover that they were just 20% reliable—considerably worse than flipping a coin. The U.K. bought 17.5 million serology tests from China, but sent them back because they only detected antibodies at screamingly high titers (4). Pop-up drive-throughs in parking lots, operated by people who bought tests on the internet and who had no medical or laboratory qualifications, offered equally worthless fingerprick tests.
Amid mounting concerns and near chaos, in late April the FDA tightened up the rules considerably, not all the way to pre-Emergency standards, but closer. Now serology testing can only be performed in an actual laboratory that holds a CLIA certificate for high-complexity testing unless the EUA states otherwise, and manufacturers of serology assays must submit EUA requests to FDA within 10 days of launching a test. The agency also prioritized independently verifying the validation data submitted for all of the tests to which it had granted an EUA (5).
Diagnostic PCR testing fared much better in terms of test performance and reliability, but with two problems. The first involves a persistent rumor, based on an inaccurate understanding of a small case series from China (6), that “PCR tests have a 30% false negative rate.” This is incorrect. The clinical sensitivity of PCR tests is certainly variable since some patients simply don’t harbor virus in the nasopharynx, but these tests’ analytical sensitivity is essentially perfect.
The second issue is that some rapid tests are significantly less sensitive than the gold-standard PCR tests. In three separate studies, the Abbott ID NOW COVID-19 test missed 12%–48% of positives, showing diminished sensitivity even after optimizing sample collection and transport (7–>9). Other, slightly less rapid tests from Cepheid and GenMark had substantially better, though not perfect, performance (8, 9). Rapid tests are quickly forming a cornerstone of perioperative care and workplace screening, which should trigger skepticism from the laboratory community.
City and national governments were not alone in feeling ill-equipped to navigate the new testing landscape. Our clinical colleagues are reflexively used to trusting, implicitly, the quality of the tests we offer them. Never in their lives did they stop and think when looking at a test result, “But is the serum sodium really 140? How can I know?” Suddenly, though, COVID-19 brought into sharp relief issues of testing quality, quality control, sensitivity, specificity, limit of detection, predictive values, and the like. We ourselves must select only reliable tests, and also explain in easy-to-grasp terms a quick method to assess the quality and reliability of a particular assay.
Never before have clinical laboratories been so prominent, or so crucial. Never have all the countless hours we put into assuring reliable, high-quality, actionable results for our patients been so appreciated. This is the little rainbow in our COVID-19 thunderclouds.
- The Washington Post. Contamination at CDC lab delayed rollout of coronavirus tests. https://www.washingtonpost.com/investigations/contamination-at-cdc-lab-delayed-rollout-of-coronavirus-tests/2020/04/18/fd7d3824-7139-11ea-aa80-c2470c6b2034_story.html (Accessed June 3, 2020).
- U.S. Department of Health and Human Services. Secretary Azar declares public health emergency for United States for 2019 novel coronavirus. https://www.hhs.gov/about/news/2020/01/31/secretary-azar-declares-public-health-emergency-us-2019-novel-coronavirus.html (Accessed June 3, 2020).
- The New York Times. Antibody test, seen as key to reopening country, does not yet deliver. https://www.nytimes.com/2020/04/19/us/coronavirus-antibody-tests.html (Accessed June 3, 2020).
- The New York Times. U.K. paid $20 million for new coronavirus tests. They didn’t work. https://www.nytimes.com/2020/04/16/world/europe/coronavirus-antibody-test-uk.html (Accessed June 3, 2020).
- U.S. Food and Drug Administration. Emergency use authorizations. https://www.fda.gov/medical-devices/emergency-situations-medical-devices/emergency-use-authorizations (Accessed June 3, 2020).
- Zou L, Ruan F, Huang M, et al. SARS-CoV-2 viral load in upper respiratory specimens of infected patients. N Engl J Med 2020;382:1177‐9.
- Harrington A, Cox B, Snowdon J, et al. Comparison of Abbott ID Now and Abbott m2000 methods for the detection of SARS-CoV-2 from nasopharyngeal and nasal swabs from symptomatic patients. J Clin Microbiol 2020; doi:10.1128/JCM.00798-20.
- Basu A, Zinger T,Inglima K,et al. Performance of the rapid nucleic acid amplification by Abbott ID NOW COVID-19 in nasopharyngeal swabs transported in viral media and dry nasal swabs, in a New York City academic institution. bioRxiv 2020; doi:https://doi.org/10.1101/2020.05.11.089896.
- Zhen W, Smith E, Manji R, et al. Clinical evaluation of three sample-to-answer platforms for the detection of SARS-CoV-2. J Clin Microbiol 2020; doi:10.1128/JCM.00783-20.
Jennifer Kasten, MD, FASCP, is an assistant professor of pathology at the University of Cincinnati and a staff pathologist at Cincinnati Children’s Hospital Medical Center. +Email: [email protected]