Listen to the Clinical Chemistry Podcast
Article
Christopher W Farnsworth, Neil W Anderson. SARS-CoV-2 Serology: Much Hype, Little Data Clin Chem 2020; 66.
Guests
Dr. Christopher Farnsworth is an instructor in the Department of Pathology and Immunology at Washington University School of Medicine and a Co-Director of the Core Laboratory at Barnes-Jewish Hospital. Dr. Neil Anderson is an Assistant Professor of Pathology and Immunology at Washington University School of Medicine and an Assistant Medical Director of the Microbiology Laboratory at Barnes-Jewish Hospital in St. Louis.
Transcript
[Download pdf]
Bob Barrett:
This is a podcast from Clinical Chemistry, sponsored by the Department of Laboratory Medicine at Boston Children’s Hospital. I am Bob Barrett.
As we record this podcast in early May 2020, much of the entire globe is in lockdown with heavily restricted travel and government-issued stay-at-home edicts due to the COVID-19 pandemic. The severe acute respiratory syndrome coronavirus 2, or SARS-CoV-2, the causative agent of COVID-19, has already led to significant morbidity and mortality throughout the world.
Laboratory testing plays an important role in both treatment and epidemiological responses to this disease. Initially, testing for viral RNA was instituted to detect presence of the virus itself. Now that a growing percentage of the population has been exposed to the SARS-2 coronavirus, serum or blood measurements of antibodies to the virus have increasing importance. In record time, both commercial and laboratory-developed serological tests for these antibodies have flooded the market and literature and had gained attention as well as generated criticism in both the scientific community and the lay press.
A cautionary opinion piece on the current situation entitled “SARS CoV-2 Serology: Much Hype, Little Data” by Drs. Christopher Farnsworth and Neil Anderson has been published in Clinical Chemistry. The paper is available online now and will appear in the July 2020 print edition of the journal. Christopher Farnsworth is an instructor in the Department of Pathology and Immunology at Washington University School of Medicine and a Co-Director of the Core Laboratory at Barnes-Jewish Hospital. Neil Anderson is an Assistant Professor of Pathology and Immunology at Washington University School of Medicine and an Assistant Medical Director of the Microbiology Laboratory at Barnes-Jewish Hospital in St. Louis. We are fortunate to have both of them as guests on this podcast and we will start with you, Dr. Farnsworth. There has been much hype regarding serological testing for COVID-19. What is the purpose of this type of testing?
Chris Farnsworth:
Yes. So, they have really been three proposed uses for serologic testing. The first is for diagnosing patients with active infection and this is generally discouraged given the superior performance of molecular testing early in disease. However, the Infectious Disease Society of America recently released guidelines that serologic testing may be useful in a niche group of individuals that present later with onset of symptoms consistent with COVID, but are negative by molecular testing.
The second is identifying convalescent plasma donors who were previously infected, but have recovered and now have high titers of potentially neutralizing antibodies. The third is for population screening and within this group, there are really two subgroups: those that want to screen the population to determine who’s protected and can go back to work and those that want to do epidemiologic studies to determine the true prevalence of COVID-19 in the population.
Bob Barrett:
And Dr. Anderson, we will go to you now, what types of serologic assays are available for COVID-19 testing and what is known about their performance?
Neil Anderson:
So, there are quite a few assays available. Currently, there is greater than 100 serologic assays available for the detection of antibodies against SARS-CoV-2. They come in a wide variety of formats. Some detect IGG, some detect IGM, and some detect IGA. They have also been designed to perform in a variety of different settings. Some can be run in point-of-care whereas others are made to be run in very large batches in large in-lab analyzers.
One of the reasons why there is such a wide variety of serologic tests available is the approach the FDA took to their regulation. In order to make the testing more available, the FDA actually removed requirements for FDA review of SARS-CoV-2 serologic assays. So, this has actually been a double-edged sword. While serology assays are certainly more available, since many of them have not been properly vetted, they have quite variable performance. Some have decent performance, whereas some are actually quite inaccurate. If these assays are widely deployed, particularly inaccurate ones, they can have very negative consequences for public health.
One area in which these inaccuracies can become downright dangerous actually, is when serologic assays are relied upon for diagnosis. In fact, I would argue solely relying on COVID-19 serology for diagnosis of acute infection is always a risky proposition even for some of the better assays. One reason for this is that antibody response to SARS-CoV-2, takes up to two weeks to develop. So, patients will often have falsely negative results early in infection. Data from our institution actually shows that IGG testing has a sensitivity of less than 10% within three days of symptomatology and less than 44% at less than 14 days.
False positives are also quite a big issue with these assays. Many of them have cross-reactivity to antibodies against common circulating pathogens, such as seasonal coronaviruses. Patients with seasonal coronaviruses may present in a similar way to those with SARS-CoV-2, so that this prospect of a false positive result is way more than just a theoretical risk. The bottom line here, is that patients and providers should avoid using serology alone to diagnose SARS-CoV-2. Molecular techniques are far more accurate in patients with acute disease. If serology is used as an adjunct, it needs to be interpreted with extreme caution.
Bob Barrett:
We have now heard how such serologic testing has helped better define prevalence in places such as New York City. This has revealed a much larger number of previously unknown infected individuals than anticipated. So, shouldn’t we be doing this throughout the country as soon as possible?
Chris Farnsworth:
So, one of the advantages of serologic testing is, it can help determine previous exposure to a virus. So it may have utility in epidemiologic studies that are trying to define the prevalence in a population. However, we do have some concerns about the studies that have been mentioned in the press. The primary issues are prevalence of the disease and the specificity of the laboratory tests that are being used.
One of the basic laboratory medicine principles that we teach our residents is that, when you are screening a population with a low prevalence of disease, the screening method must have a low false positive rate, also known as a high specificity. If 1% of the population has the disease, even a test with a high specificity, say 98%, would potentially give you one false positive result for every true positive if you screened everyone. As a result, a study that used this test could claim a prevalence that is two times higher than the actual true prevalence.
We actually outlined this concept in our paper using New York City as an example. In New York, the estimated prevalence based on molecular testing is about 1.69%, of about 8,500,000 people. Therefore, if a serologic test has a specificity of 99.5%, the positive predictive value or ratio of true positives to false positives is about 75%, or three in four positives will be true positives. However, if the specificity is lowered to 98%, the positive predictive value plummets to 45% or 1 and 2 positive results is a false positive.
Our institution is actually based in Missouri, where the prevalence of COVID infections is likely tenfold lower at least than New York City. Again, just using simple math, we see that in our population even a serologic test with the specificity of 99.5% will only give us a true positive 16% of the time. Meaning that, in low prevalence cities and states, which is likely most of the country at this point, screening the entire population is likely to result in a considerable number of false positive results, and give people that were never exposed to the virus a false sense of protection from future infections.
Bob Barrett:
Well, since we are all eager to end the social distancing, should everyone get a serology tests to confirm whether or not they are immune?
Neil Anderson:
So, the desire to identify those who are immune to SARS-CoV-2 is one of the applications of serologic testing that has gained very much attention in the lay press and has garnered a lot of excitement from the general public. While this is quite understandable, it is based on a misconception that serologic positivity means immunity. There are currently are no serologic assays available that can reliably predict immunity to SARS-CoV-2.
Currently, available assays are designed to detect past exposure to SARS-CoV-2. Many infer that this past exposure equates to future immunity, though the fact of the matter is that, we simply do not know if this is true. In order to answer this question definitively, we need prospective outcome data, which is still lacking giving how relatively new this pathogen is. Those looking for an antibody test to give them a peace of mind about their immune status should really keep appropriately tempered expectations. They may have had the disease and test falsely negative or they may have never had the disease and test falsely positive. In fact, those looking for peace of mind may find the opposite, particularly if they are relying on a bad test.
Bob Barrett:
Well, finally doctors, what steps should laboratory experts take to responsibly offer serologic testing to their patient populations?
Chris Farnsworth:
So, this is a really important question, and where we as a laboratory community have an opportunity to provide expertise, especially since a large proportion of these tests have not gone through the emergency use authorization process, and have not been truly vetted by the manufacturer or by independent labs.
But it is not a one-size-fits-all approach. The validation you perform will really depend almost entirely on how you plan to use the test. As a generalization, when we validate a lab test, we will analyze a minimum of 50 positive samples from patients with PCR-confirmed infection. For negative samples, using specimens have been frozen prior to the emergence of SARS-CoV-2 is a common approach. We also have used samples from patients that have COVID-like symptoms, but are negative by molecular testing.
We also recommend that labs use specimens from patients with other confirmed seasonal coronaviruses, the flu, and other viruses like EBV and CMV to assess cross-reactivity. The number of negative specimens that you analyze is entirely dependent though on how you plan to use the test. Again, if used for screening in a likely low prevalence community, a high specificity and considerable confidence in that specificity is a must, especially for tests that have not been formally validated. This will require hundreds or potentially thousands of negative specimens to truly understand how the test performs. In contrast, if your facility is using serologic testing to aid in diagnosis, knowing how many days post symptom onset your patient population becomes positive is key, and far more positive patients will become necessary.
Finally, if you plan to use your assay for determining antibody titers in convalescent plasma donors, it is important that you validate your tests for this use.
Neil Anderson:
Another important aspect to consider prior to implementation of SARS-CoV-2 serology testing is education. Given all the hype and misconceptions about SARS-CoV-2 serology testing, even the most knowledgeable providers may be misinformed. Prior to offering serologic testing, laboratory experts really should discuss with their providers exactly how the testing should be utilized in their setting and what needs to be done to prevent misuse. Efforts to educate ordering providers should be made which may range from educational CME presentations all the way to electronic clinical decision support.
Finally, laboratory experts should be prepared to perhaps step outside their hospital walls and their comfort zones to engage in local and national testing policies. The drive for COVID serology is very real and there is a real chance that without appropriate guidance, these tests could be used in quite frankly, harmful ways. Laboratory experts can only prevent this by acknowledging the hype, educating about the perils, and guiding the way forward in the way we were all trained to, by using an evidence- and data-based approach.
Bob Barrett:
That was Dr. Neil Anderson and he was joined by Dr. Christopher Farnsworth. They are both from the Department of Pathology and Immunology at Washington University School of Medicine in St. Louis. Their opinion piece entitled “SARS-CoV-2 Serology: Much Hype, Little Data” is available online now and appears in July 2020 print edition of the journal Clinical Chemistry. I am Bob Barrett. Thanks for listening.