A New Study Evaluated the Accuracy of More Than 60 Coronavirus Antibody Tests. 13 Were a Cut Above the Rest.
By Aria Bendix
- A new review ranked the top 13 coronavirus antibody tests out of more than 60 on the market.
- The best tests had very few instances of false positive or false negative results.
- But other tests had a wide range of accuracy levels, and some were less accurate than their manufacturers claimed.
Hopes were high when the first coronavirus antibody tests debuted a few months ago. There was talk of issuing immunity passports based on the results. The tests were so coveted in the US that the Food and Drug Administration allowed some to go straight to market.
Antibody tests detect our body's natural defense against a foreign pathogen. That means they can tell whether someone has ever been infected with COVID-19 — even if that person wasn't able to get tested initially. They can also give researchers a picture of how much the virus has spread within a population.
But more than three months later, research shows many antibody tests don't offer consistently reliable results, and some are far more accurate than others. Plus, many mysteries remain when it comes to coronavirus immunity and how long it lasts.
A new, independent review published in the journal Diagnostics evaluated more than 60 serological (blood) antibody tests on the market.
"Unfortunately, the results were not good," Ali Ardakani, the founder of life-science advisory group Novateur Ventures and a co-author of the review, told Business Insider.
The tests often targeted different antibodies and were administered at different points in the course of a person's infection, making it difficult to compare performance, even for a single manufacturer. But the researchers found that most antibody tests didn't perform as well as their manufacturers claimed.
"We're six months in. There are hundreds of tests out there dispersing all sorts of information. Most of these tests are not even meeting their own specs when they're independently validated," Ardakani said.
Scientists use two basic measures to determine the accuracy of an antibody test: sensitivity and specificity. Sensitivity refers to how well a test detects antibodies when they're actually present (i.e. a true positive). Specificity refers to a test's ability to produce a true negative result.
Ardakani's team found that most antibody tests had consistently high specificity, meaning false negatives weren't likely. But sensitivity varied significantly.
At their best, some antibody tests had 100% sensitivity, meaning all positive results should be accurate. Others had sensitivity rates as low as 3.3% for certain attempts, according to raw data reviewed by Business Insider.
"The false positive is very serious," Ardakani said. "They're basically giving you the wrong sense of security."
The researchers also highlighted the 13 highest-performing antibody tests on the market. A few FDA-approved tests — from Beijing Wantai Biological Pharmacy, Healgen, and Abbott — made the cut. However, the first FDA-approved antibody test, from the biotech company Cellex, did not.
High-performing tests had very few false positives
The top 13 antibody tests in the independent review had both high sensitivity and specificity.
The rapid test from Healgen was one of the few tests with 100% sensitivity and 100% specificity — but those results were only for IgM antibodies.
Coronavirus antibody tests target three types of antibodies: immunoglobulin M (IgM), immunoglobulin G (IgG), and immunoglobulin A. Our bodies typically make IgM first in response to a viral infection, so these antibody levels tend to peak earlier than the rest. IgG is the most common antibody found in blood and usually a better indicator of long-term immunity. Most coronavirus antibody tests focus on these two antibodies as opposed to IgA, which is found mainly in the respiratory and digestive tracts.
The Healgen test for IgG antibodies had 96.7% sensitivity and 97.5% specificity. Both of the company's test results match the percentages listed on the FDA website.
The rapid test from Beijing Wantai Biological Pharmacy also performed well, with 97.5% sensitivity and 95.2% specificity, according to the review. The results listed on the FDA website are slightly better, though: 100% sensitivity and 98.8% specificity.
One of the most popular antibody tests in the US, the Abbott Architect test, had 97.2% sensitivity and 100% specificity when administered at least two weeks after a person's symptoms started, the review found. That means false positives were more likely than false negatives. But the results from the manufacturer indicate the opposite: that the test is 100% sensitive and 99.6% specific, according to the FDA.
As of Friday, Abbott has shipped around 12.6 million antibody tests across the the country.
Other tests that didn't rank highly had much wider discrepancies between their manufacturers' claims and the independent review. Cellex says its rapid antibody test for both IgG and IgM antibodies is 93.8% sensitive. According to the researcher's raw data, the test was most sensitive (96%) when administered after a person had been sick for 14 days, but its sensitivity dropped significantly under different parameters.
Biomerica's rapid diagnostic test (which hasn't been approved by the FDA) performed best — 100% sensitivity — when it targeted IgG antibodies and was administered during the second week of infection. But again, the test's sensitivity was much lower under different circumstances (like when it targeted IgM antibodies). Allen Barbieri, vice chairman of the board at Biomerica, said in April that the test was more than 90% sensitive, according to the MIT Technology Review.
Different tests may need to be administered at different times
The FDA has approved nearby 30 coronavirus antibody tests through its expedited emergency-use process. The rush to develop and manufacture these tests may have come at the expense of accuracy, Ardakani said.
"A lot of these companies that have been developing these diagnostics were not in the business of developing clinical-grade medical diagnostics before," he said.
Ardakani compared the process to building a car while driving it.
"Usually when you want to do proper development, you start with something called a target product profile —that means you say, this is my ideal diagnostic for this disease," he said. "Unfortunately, a guidance was not provided for any of these [tests] from any of the health authorities originally."
But evaluating antibody tests, he added, can sometimes be like comparing apples to oranges. A test's performance depends on when it's administered and the type of antibody being measured.
Since IgG and IgM levels are higher during the later stages of an infection, doctors usually advise patients to wait a few weeks before getting an antibody test.
German researchers determined in April that coronavirus patients typically develop IgM and IgG antibodies after five to 14 days. Half the patients in that study had detectable levels of antibodies in their blood after one week of symptoms. A May study from Chinese researchers also found that IgM and IgG antibodies were detectable after five days. After 32 days of symptoms, all patients in that study had developed antibodies.
How long do antibodies last?
People who suspect they had COVID-19 might not want to wait too long to get an antibody test.
A July study from UK researchers — though still awaiting peer review — showed that coronavirus antibodies may disappear less than three months after symptoms arrive. IgG antibodies may be detectable for longer than IgM — around 49 days compared to roughly 35 days — according to a March study that's also not yet peer reviewed.
There's also recent evidence that patients with severe COVID-19 cases may have significantly higher levels of antibodies than people with mild cases. One study even found that asymptomatic coronavirus patients may test negative for antibodies during the early stages of recovery.
That means any antibody test results should be taken with a grain of salt. Those receiving them should also inquire about the maker.
"A lot of times you cannot translate what you did in your perfect lab, the way you perfectly performed the procedure, when you give it to a nurse that's rushed in front of a patient in a busy hospital," Ardakani said. "We can't even replicate these tests back to back."
Correction: A previous version of this article cited average test sensitivity. The article has been amended to include the highest sensitivity level for specific antibody tests and offer more detail about how the review was conducted.
