Should we vaccinate for COVID-19 ahead of Phase III trial results?

Warning: This is speculative back-of-the-envelope discussion of public policy outside of my fields of expertise.

For some time I’ve been thinking that perhaps, now that the Phase II safety trials of some coronavirus vaccines have been completed, we should just start vaccinating prior to completion of the Phase III trials. But of course, it’s not my field! Yesterday, however, I was pleased to note an article in Forbes by a biostatistician advocating the same thing.

While it’s not my field, I am a decision theorist, so here is a back-of-the-envelope utility calculation. The Oxford vaccine has had Phase I/II safety testing on 543 participants. Of course, there is about a 37% chance that, if there were an adverse effect that afflicts one in 543 participants, it would be missed by that sample. So, let’s suppose that there is a 40% chance of a 1 in 600 users lethal effect. So, counting vaccine-related death, the disutility of giving the vaccine to someone is (0.4)(1/600)=0.0007 deaths.

Given that Phase I/II does reveal some evidence of effectiveness, albeit evidence not sufficient for knowledge, we might say that we now have about 70% probability that the vaccine works, and let’s say that if it works, it works for 90% of the users (of course, these numbers are made-up). Moreover, by the best CDC estimate, each COVID-19 infection has a 0.0065 chance of resulting in death. So, counting death alone, the utility of giving the vaccine to someone who would otherwise have become infected is (0.7)(0.9)(0.0065)=0.004, counting only their life or death.

But here is the rub. We don’t actually know if a given individual would be infected if they did not receive the experimental vaccine. Eventually there will probably be a fully tested vaccine and/or a treatment. So far, pretty much the worst major location in the US has been New York City, and only 25% of the city has been infected. The utility of giving the vaccine to one person in New York City at the right time would thus have been something (0.25)(0.004)=0.001, again counting only their life or death.

So, the disutility is 0.0007 deaths and the utility is 0.001 lives, and these numbers are pretty close, so close that the uncertainties in all the back-of-the-envelope estimates likely swamp the differences. So by the above numbers, the idea of giving not fully tested vaccines probably wouldn’t be a good one, since as a matter of general policy we should keep to established testing protocols unless there is a compelling case to the contrary, and the case so far does not seem compelling.

However, a number of things can affect the above calculations, items 1–5 in favor of the not fully tested vaccines and 6 and 7 against it:

1. very early data from some Phase III trials—e.g., the lack of deaths in the week following the initial injection—could be used to significantly lower the probability of lethal effect

2. perhaps past data from other vaccine trials, and/or from medical theory, justifies one in thinking that the chance of a 1 in 600 lethal effect is much smaller than 40%; it’s not my field, so I have no idea if this is the case

3. infection of others: the above only counts the benefit to the person being vaccinated; but if that person got COVID-19, they would on average have also infected about one other person; so there is a significant benefit to society from a successful vaccination

4. some people are at higher risk for COVID-19 death; giving them the untested vaccine might make sense even if it doesn’t make sense for the average person; however, this is muddied by the fact that intuitively those in higher risk categories may also be at higher risk for vaccine complications

5. behavior shifts: people who receive a vaccine are likely to take fewer precautions against infecting themselves and others, so if the vaccine doesn’t work, their chance of infection is likely to go up if they are vaccinated

6. in many locations, the infection rate between now and whenever there is a fully tested successful vaccine or treatment may be rather lower than in New York City

At least the following is true: the option should be taken seriously, and careful calculations should be done.