How Coronavirus Case Counts Can Severely Throw off Conclusions on Crisis Without Right Context | Eastern North Carolina Now

Publisher's note: This informational nugget was sent to me by Ben Shapiro, who represents the Daily Wire, and since this is one of the most topical news events, it should be published on BCN.

The author of this post is James Barrett.

    The unprecedented state-mandated shutdowns that have brought the U.S. economy to a screeching halt and sent jobless claims skyrocketing at a trajectory that even the most grim projections did not foresee are ultimately based on models from experts attempting to project how many people will be infected, hospitalized, and die as a result of the coronavirus pandemic. The number most frequently promoted by mainstream media outlets has been the total number of cases of COVID-19, but statistician Nate Silver warns that, without the proper context, that inevitably ever-climbing number is largely "meaningless," and if hyperfocused on without enough testing data - as it has been - can lead people to false conclusions.

    In a lengthy analysis published on his FiveThirtyEight blog, the much-cited statistician addresses in detail an issue that many have been pointing out from the beginning: "the number of COVID-19 cases is not a very useful indicator of anything - unless you also know something about how tests are being conducted." In the analysis, Silver presents four testing scenarios that demonstrate how differently the number of tests being conducted impact the interpretation of the total number of cases.

    Unlike in sports, where all the key data is recorded, Silver explains, crucial information about COVID-19 is glaringly missing thus far, making it very difficult to properly determine its spread, threat, and how effectively it's being combatted. What is most problematic is the paucity of tests, particularly random tests that give us a clearer picture of how many people have actually been infected.

    "The data, at best, is highly incomplete, and often the tip of the iceberg for much larger problems," writes Silver. "And data on tests and the number of reported cases is highly nonrandom. In many parts of the world today, health authorities are still trying to triage the situation with a limited number of tests available. Their goal in testing is often to allocate scarce medical care to the patients who most need it - rather than to create a comprehensive dataset for epidemiologists and statisticians to study."

    "But if you're not accounting for testing patterns, it can throw your conclusions entirely out of whack," he emphasizes. "You don't just run the risk of being a little bit wrong: Your analysis could be off by an order of magnitude. Or even worse, you might be led in the opposite direction of what is actually happening."

    For example, case counts going up in a country that is increasing its testing might seem like a bad thing on the surface, when it is in fact a positive sign that the country is getting the epidemic "under control," Silver writes. And the opposite could be true where countries are behind in testing. Silver estimates that the U.S. is "probably somewhere in the middle of the pack" on its testing rate.

    How many people are actually infected is not just unclear in many countries, the potential range is stunning. In the U.S., Silver notes, a recent survey suggests the U.S. could be underestimating the true number of cases "by anywhere from a multiple of two times to 100 times" - which impacts every other metric, most importantly the fatality rate and the rate of serious cases vs. mild or asymptomatic cases. If far more people are infected than experts currently estimate, then both would be far lower than we think, or vice versa.

    In his lead-in to his hypothetical scenarios, Silver provides a helpful discussion about the "most important number in any epidemiological model," R, or reproduction ratio. The ultimate goal for pandemic responses is "to get R below 1, which means that a disease begins to die out in a population," Silver explains. The more the rate is below 1, the faster the disease will die out. Additionally, "if a disease has spread very widely throughout the population, R may eventually fall because of herd immunity," he notes.

    Silver goes on to present four different testing scenarios - robust growth in testing; sudden, one-time increase in testing; high test floor, low test ceiling; and testing decrease - which he notes are based on "hypothetical data, because we don't know all the parameters we'd need to properly estimate a model anyway." His scenarios assume an R rate of 2.6 without mitigation measures, 1.4 during the "intermediate" social-distancing phase, where companies are having people work from home but governments have not imposed lockdowns, and 0.7 for full lockdowns. His scenarios also assume the following still unverifiable percentages: that 10% of cases are severe, 60% are mild, and 30% are asymptomatic.