Why Cases Are a Terrible Metric for Tracking COVID-19

2023-07-18 NOTE: A bunch of links on this page are dead.

As COVID concern creeps back up again, I thought I’d point something out that’s been bugging me since Donald Trump started bragging about testing last year. There’s a big problem with tracking “cases” (which is basically positive tests) to monitor the spread of COVID-19.

Now, I know I’m sort of stepping into the deep end on this one, so I think it’s important to give some qualifiers. First, COVID is real, and COVID is real bad. My concern is with poor interpretation of the data. Second, if you’re looking for my qualifications, I do have a degree in math, but either way, the math involved in this is actually very simple and straightforward.

Bayes’ Theorem and Tracking Cases

The only thing we need to understand what’s going on with case numbers is Bayes’ Theorem:

The probability of A given B is equal to the product of the probability of B given A and the probability of A divided by the probability of B.

I’m not going to go into detail on this because the Wikipedia page is actually very good, but here’s a simple example of why this is important.

Say I have a group of 1000 people. One hundred of them have the flu, and I have a flu test available that yields 90% true positives. If I test all one thousand people, I would expect to get 90 true positives. But 10% of the time, a person negative for the flu will produce a positive result which means we’ll get 90 false positives as well.

What’s the moral of the story? Case numbers will be wildly overinflated if widespread testing is done in the general population.

So think about what happened when this all ramped up in the spring of last year. Chances are that your circle of friends and family did your own informal contact tracing. When one person tested positive, he notified every person he came in contact with for several days. Then, several of them got tested. So all of a sudden, because one person was symptomatic, you have groups of five or ten people getting tested for COVID. In reality, COVID testing was (and is) not targeted to those with symptoms, it’s done among something that approximates the general population.

Case in point: our current “positive test rate” is sitting at only about 11%. That means lots of nonsymptomatic people are being tested, and we’re probably seeing a high instance of false positives.

What This Means for You

So what does this mean for you? Not very much.

Case tracking is a problem of scale, so on the individual level, a positive or negative COVID test is still really helpful. Of course, these tests aren’t definitive, but your doctor knows that already. All the test does is provide you and your healthcare provider with another piece of information to help make decisions about your particular situation. Chances are you’ll be told to quarantine and especially avoid vulnerable people.

That’s good because that’s what tests are for.

When you’re administered a test for any ailment, the purpose of that test is to determine whether you have whatever they’re testing for. They’re not intended to help policymakers make important decisions about lockdowns and mandates.

So I’m not discouraging testing at all; instead, I’m discouraging case numbers gained from testing as a metric for public policy decisions.

A Better Way

So if not cases, then what?

There are several other metrics we can and do look at. For example, we measure deaths and hospitalizations. The problem with these, however, is that they skew toward the vulnerable. Certainly, these are the most serious cases, but they are not representative of the impact of COVID in general.

Another option (and the one that I would advocate for) is symptomatic cases. This way, we can track the size of a core group of people who meet more than one criteria for a COVID diagnosis. This wouldn’t give us an accurate number of all active COVID cases (because you’d have false negatives and symptom-free carriers), but it would give us an accurate trend. For public policy decisions, this is what you need.

Conclusion

Feel free to listen to cable news and government organizations, but understand that they don’t know you. They have an agenda, and they’re looking to garner support in Washington for their proposed plan of action. That’s fine; it’s their job (well…maybe not cable news, but you get the point).

If you’re concerned about COVID, what you should do is trust your doctor. Your doctor knows COVID and your personal health situation. He or she is in the best position to give you guidance on how to handle yourself while we all try to get out of this mess.