Editor's note: A previous version of this story referred to the Logix test as an antibody test. It is not; it is an in vitro test. The story has been updated.
Beyond the masks and protective gear used to prevent a disease's spread, vaccines to prevent those who are exposed from becoming ill, and therapies to treat those who do become sick, one key component of combating an epidemic is actually testing for the disease.
With countries around the world gradually allowing their economies to reopen up after weeks and months of lockdowns initiated to stem the spread of COVID-19, in vitro tests will be in high demand as health officials race to figure out where the coronavirus has been, and to gauge whether a sufficient enough level of herd immunity has been achieved in various populations to protect seniors and the immunocompromised from the risk of infection. One company that claims to have developed an effective in vitro test (called Logix) is Co-Diagnostics (CODX -0.86%).
Investors seem to have appreciated that news: Co-Diagnostics shares were up a staggering 1,810% year to date as of May 15. However, before new investors take a position in the stock, I believe there is an important caveat to be considered about the company's test kits that many have overlooked.
Image Source: Getty Images
What's so special about Co-Diagnostics?
In May, Co-Diagnostics announced its COVID-19 in vitro test had been found to have 100% accuracy, 100% specificity (likelihood of preventing a false-negative error), and 100% sensitivity (likelihood of preventing a false-positive error), as per independent verification in laboratories across the world. The test has been granted an emergency use authorization by the Food and Drug Administration, but it has neither been approved nor fully vetted by the regulating agency.
To date, Co-Diagnostics has manufactured more 6 million test kits, placed orders for materials to make 20 million more, and generated more than $16.5 million in quarterly revenue -- just six weeks into its second quarter. At an annualized rate, that's about $143 million in revenue. For a company with a market cap of about $500 million, this is impressive.
Meanwhile, Co-Diagnostics has about $17 million of cash on hand, and it only burned through about $1 million in expenses in the past quarter. The company's stable financial position should put potential investors' minds at ease about the risk of stock dilution.
At this point, readers may wonder why they shouldn't go ahead and buy the stock. Allow me to explain.
The devil is in the details
To start off, Co-Diagnostics came to the conclusion that its test was 100% effective on all three diagnostic dimensions (specificity, accuracy, and sensitivity) based on studies with small sample sizes. For example, laboratory testing of the Logix test kit conducted in Australia involved about 100 COVID-19-positive patients and 100 COVID-19-negative patients.
With a sample size that small, a low error rate, say 1% to 2%, could be really hard to detect. In fact, the study itself explicitly stated that the test could in fact be between 96% to 98% effective, rather than 100%.
In addition, the testing environment is by no means indicative of the actual prevalence of COVID-19 in the population at this point in the pandemic. Among the test samples, 50% contained SARS-CoV-2, and obviously, at this point, nowhere near half the people in the world have been exposed to the coronavirus.
"But wait a minute!" the intelligent reader might say. "Nothing in the world is perfect, so who cares if a test's results are off by 1% or 3%? Effectiveness of 97% is still nothing short of an A-plus. You're just being a devil's advocate, Zhiyuan!"
Unfortunately, this is one of the cases where it is critical to pay attention to the devil in the details. In fact, a 1% or 3% error rate can render a in vitro test almost useless. Here's why.
Image Source: Oxford Academic
Let us assume, for the sake of argument, the true sensitivity of Logix is 98%, and its true specificity is also 98%. In other words, the probability of the test delivering a false positive is 2%, and the probability of the test returning a false negative is also 2%. Both of these values are directly stated as being probable in studies citing Logix's range of effectiveness, and they are valid assumptions given that the test has not been fully vetted by the FDA or other regulators.
It is also common knowledge that because there are not enough viral tests for the COVID-19, the number of people who have the virus is likely to be significantly higher than official figures. For example, it is estimated that up to 4.1% of the residents of Los Angeles County have COVID-19 antibodies.
Let's use that 4.1% figure in our calculations as a measure of prevalence of COVID-19 (a lower prevalence would hurt the test even more). Assuming 1 million people are given the Logix test, 41,000 should test positive for an ongoing SARS-CoV-2 infection. However, if the test provides a false negative 2% of the time, only 98% of those 41,000 -- 40,180 -- would show up as positives.
On the other hand, out of the 959,000 people who were actually negative for the virus, a 2% error rate would yield 19,180 cases of false positives -- individuals who don't have the disease despite the test saying they do. All told, that makes 59,360 people getting positive results, but only 40,180 of them would actually be positive. That yields a predictive value of 67.7%.
In other words, if the Logix test only works as well as it does in this scenario -- and it's right 98% of the time -- there's still a 1-in-3 chance that the test will indicate you have COVID-19 even though you don't!
As one can see, a 32.3% false-positive error rate isn't very good at all. This problem gets worse if we assume the same prevalence, but lower Logix's potential sensitivity and specificity estimates to 95% for both. In this scenario, the probability of getting a false positive increases to 55.2%! While the results are surprising, they nonetheless use the basics of conditional probability; here is a calculator in case you want to try it out for yourself.
Furthermore, a recent New York University study on COVID-19 in vitro tests developed by Abbott Laboratories (ABT 1.91%) found them to be widely inaccurate and unacceptable for use in patients. Keep in mind, those tests were also promoted as having 100% sensitivity and 99.9% specificity in earlier investigations. Unfortunately, this just serves to highlight how difficult it is to develop an accurate test for diseases with a low rate of prevalence like COVID-19.
So what should investors do?
Let's get right to it: Is Co-Diagnostics' stock in a bubble because of excessive optimism about the value of Logix? In my opinion, absolutely not, because of the independent laboratory studies worldwide that cautiously confirm Logix's effectiveness.
Additionally, Co-Diagnostics is growing its test-kit revenue by a wide margin compared with last quarter. For speculators, there is no doubt a lot of money is ready to be made (or lost) on the stock just because of the sheer magnitude of its volatility.
However, healthcare investors with a more long-term outlook should stay well away from the stock until the FDA has in fact vetted Logix to confirm whether it's 100% effective. There is just too much at stake; even a small mistake in stating its true sensitivity and specificity (S&S) can snowball into massive errors for the experimental test kit. Investors will want to avoid taking a stake in the stock, then waking up one morning to see the FDA found Logix was actually 99% accurate in terms of S&S, rather than 100% -- a 19% chance of hitting a false positive -- and watching the stock collapse as a result.
