Yes, Virginia, the Models are Wrong
“All models are wrong. But some are useful.” This is a saying by the famed statistician, George Edward Pelham Box, also known as “one of the great statistical minds of the 20th century”. His saying is a principle in the field.
But how can this be? How can the modelers themselves know that the models they make are wrong and yet still tout them as true? Well, because models are supposed to be wrong. At least, in the case of the Coronavirus pandemic, that is the hope…
Let’s put it this way: Suppose you go to the doctor and you are overweight and eating junk and your bloodwork has diabetes written all over it. The doctor’s prognosis of your prospects is that if you continue on this path, odds are you will probably get sick and die. But then, if you don’t die, was the doctor wrong? Yes, absolutely he was. But was his prognosis useful? Yes, absolutely it was. Especially if you heeded the advice (even a little bit), took better care of yourself and lost some weight.
Even though this simplistic analogy pays little justice to the complexity behind George Box’s aphorism, a statistical model is essentially a prognosis, based on all the available evidence at the time. However, anything that happens subsequently will influence the outcome, and thereby the accuracy of the model. No one can predict the future. We can only guess as smartly as the available evidence allows.
Earlier this year, the prognosis was not good for our ability to withstand this pandemic. The earliest models suggested that millions would die. These were based in large part on what has happened before because that is what we know. That is our evidence, our bloodwork if you will. The last time a novel lethal pathogen hit us with this degree of ferocity (the Spanish flu, 1918), 675,000 Americans perished. A century later, we are three times the population and vastly more interconnected, so the prospects now are even more daunting than before.
These earliest warnings about Coronavirus were not heeded in the United States, but as the number of infections quickly escalated, and as a dangerous number of those people became critically ill and died, authorities began to take note. They had to. In fact, the models were proving to be correct (not wrong, in our favor, as we would hope in this case).
So, the first strategy was to limit travel in and out of the country, inform the public about how this virus spreads, and then encourage the sort of personal hygiene that can help to prevent it. Subsequent models showed that these efforts alone would have an impact if taken early enough, and the death count prognosis was lowered from millions to hundreds of thousands, but this was still not enough.
Were the earlier models wrong? Absolutely. But were they useful? Absolutely.
Hundreds of thousands of deaths are where we would very likely be today if we did not enact social distancing and economic lockdown. Without a vaccine for the virus or a remedy for the disease, these tactics were literally our only remaining course of action. This scourge had the upper hand. But we changed that and as a result, those earlier models have been readjusted downwards from hundreds of thousands to tens of thousands. Good work!
“All models are wrong. But some are useful.” Right again!
The models now say that we are in the very same place as before this all began. There is a lethal pathogen among us, we have no immunity to it, no vaccine, and we are a large and interconnected society. IOW, those millions of deaths that were first estimated are still on the table. All we need do is drop our guard. The current models say maybe 60,000 will die if we keep doing what we are doing, but if we stop now, all bets are off. We will need to dust-off those first models.
If that happens, will the current models be wrong? Absolutely. But will they be useful? We shall see…
