An experimental Covid-19 vaccine from Pfizer and German drug developer BioNTech (BNT) seems to be working very well, according to early trial data. The announcement, made on Monday, has enormous implications. If we end up with a few vaccines that are as effective as this one appears to be, then very widespread vaccination could be a steamroller for public health, slowly crushing the pandemic.
The details, though, are complicated, and the press coverage to this point has often been off target. Some stories underplay the good news with spurious caveats, such as the claim that the numbers are drawn from “a very small number” of cases. Others have overplayed it—wrongly claiming, for example, that the vaccine has been shown to be more than 90 percent effective at preventing infections, or that it has “displayed no side effects.” Here’s what we do know: Pfizer and BNT have just hiked up the bar for Covid-19 vaccines; and while it’s not yet clear exactly how high that bar now sits, there’s every reason to think it’s pretty high.
First, was the number of infections really that small? Long before any results came in, the protocol for this trial calculated what sort of data—including how many people ended up getting sick—would need to be tallied in order to know that the vaccine had cleared the minimum requirement of 50-percent effectiveness set by the Food and Drug Administration. (A vaccine that prevents about half of illnesses would be more like a flu shot than a measles immunization—not enough to squash the pandemic on its own, but still much better than nothing.) It’s tricky, because there was no telling where the virus might surge in the first few months of the trial. To get an answer fast, you would need to have a lot of people in both the vaccinated and control groups who were sitting in the virus’ possible path—far more people than you would need if you were running your trial on, say, a giant cruise ship about to have an out-of-control outbreak. That’s why it was necessary to have tens of thousands of wonderful people as volunteers, but the required number of actual “events”—in this case, volunteers getting symptomatic Covid-19—would be small, if the vaccine were to end up making a really big difference.
An independent data monitoring group was scheduled to look at the data at several points along the way, to check whether it was having a dramatically good or bad impact. That’s a process with a lot of planning and safeguards. BNT and Pfizer originally planned to have this group check in with four interim analyses, once certain numbers of events had been recorded. The companies were criticized, though, for allowing themselves too many opportunities to stop the trial early, and for planning to have the first analysis when there were just 32 events. “Take the time, the extra weeks,” one expert told The New York Times. “No shortcuts. Nobody will regret it.”
Monday’s press release reports that there had been negotiations with the FDA to drop the earliest analysis, but by the time the new protocol was signed off, there were already enough events to skip ahead to what would have been the third analysis. Now, with the virus surging in the US, the full number of events required for this trial—a total of 164 volunteers getting sick—may even be reached in the coming weeks. For the moment, the companies are only reporting on the first 94 events.
Read all of our coronavirus coverage here.
So there really isn’t a reason to be concerned about the number of events included in this analysis. What about the results—have BNT and Pfizer really shown that the vaccine is “more than 90 percent effective at preventing infections”? No. For one thing, we have no information on how many people in the study were infected with the virus—only a number who actually ended up getting sick and diagnosed with Covid-19. The “90 percent” figure is also up in the air, for the time being. Because there’s more uncertainty in the smaller numbers that emerge early in the trial, the hurdle for declaring success must be set higher than 50 percent. The companies had calculated ahead of time that if they found the vaccine to be 62.7 percent effective at around this point, they could be confident the vaccine would turn out to be at least 50 percent effective overall. On the other hand, if that number were lower than 38.6 percent, it would mean the trial was never going to be able to clamber over the bar. When the data-monitoring board made its examination, the effectiveness rate was more than 90 percent for people who had definitely been Covid-free before the trial started. (Wow.) That rate could drop when the next 70 events join this unlucky group, but it should still be well clear of the FDA’s minimum requirement. On the other hand, it could end up considerably higher than 90 percent, too.
There’s still a lot to be determined. Note that they’ve only told us the result for people who were definitely Covid-free when the trial started. When a vaccine is eventually rolled out to the public, some people who get it will already have had Covid-19 without knowing it. The same is true for trial participants. We don’t yet know the effectiveness rate for everyone in the trial, and it’s not clear how much data is missing because volunteers didn’t check back in with the people running the study, for example. Nor do we know whether the vaccine prevents asymptomatic infections or protects against severe illness among those who do get sick. The latter question is very important, because a vaccine that only prevents mild illness might help reduce the spread of the virus, but it won’t save as many lives, and as quickly. We also need to be sure being vaccinated doesn’t increase the risk of severe illness, which is theoretically possible.
We need to know more about people who are at higher risk too, including older people and people of color. No word yet on that, either. The trial was designed to ensure that at least 40 percent of its participants were 55 or older, and it was extended to include children as young as 12 and people with some chronic diseases. So we can expect a trove of critical data.
Everything You Need to Know About the Coronavirus
Here's all the WIRED coverage in one place, from how to keep your children entertained to how this outbreak is affecting the economy.
By Eve Sneider
As for possible side effects, it’s wildly wrong to say that “none were shown.” While the press release said there were “no serious safety concerns,” no data was offered to support this statement. So far, the data safety monitoring board hasn’t sent up any flares, though. A few weeks ago, the scientist leading BNT, Uğur Şahin, said that the adverse effects in this big trial were in line with those reported earlier on in the vaccine’s development. His company had developed 20 candidate vaccines and picked four of them for initial clinical trials. Of the two that were most effective in those early tests, the company proceeded with the one that had produced fewer adverse effects, especially in older people. But there’s not a lot of data publicly available on that: Only 72 people were injected with this particular vaccine, and just two dozen of those received the same dose as the volunteers whose data was reported this week. Among the 24 people getting the most relevant dose, there were zero “serious adverse events,” defined as those in which a volunteer needed hospitalization, for example, or experienced a life-threatening illness or death. As for “severe adverse events” that restrict a person’s usual activities, including fever of between 102.1 and 104.0°F or needing an intravenous drip, the BNT-Pfizer vaccine did better than some other vaccines that are being tested: Only two out of 24 people (8 percent) reported experiencing such symptoms. But it’s a tiny group of people, and many of them nevertheless had a rough couple of days, with symptoms that included milder fevers, headaches, chills, fatigue, and/or pain at the injection site. That kind of trade-off is worth it, when the effectiveness is so high.
It’s important to get the details right, but the basic message here should be very clear: After safety, the big hurdle in this process was vaccine effectiveness—and it looks like we’re passed it. Now we just have to protect as many people as we can by other means, until vaccines can be widespread.