On November 9th, 2020, Pfizer announced that its vaccine “was found to be more than 90% effective in preventing COVID-19 in participants without evidence of prior SARS-CoV-2 infection” (Ref 1). That was the first such announcement from any of the number of trials trying to develop a vaccine for the virus SARS-CoV-2, which can cause the disease Covid-19. The press release reported that 43,538 participants had been enrolled and that 94 cases of Covid-19 had been confirmed among the participants (Note 2). On December 10th, 2020, an academic paper followed the Pfizer announcement (Ref 3). It was called “Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine”.
The Oxford/AstraZeneca announcement followed a month later and it was accompanied by an academic paper published in The Lancet on December 8th, 2020 (Ref 4). The paper in The Lancet was a follow-up to another paper in the same journal published on July 20th, 2020 (Ref 5). The Oxford/AstraZeneca team also published a paper on November 18th, 2020, which reported on a trial specifically undertaken in younger and older adults than those studied in the July and December papers (Ref 6).
This note primarily reviews the Lancet publications about the safety and efficacy of ChAdOx1 nCoV-19, the Oxford/AstraZeneca vaccine. References to the Pfizer vaccine are also made where appropriate. The two vaccines are different and both use new techniques. ChAdOx1 nCoV-19 is a chimpanzee adenovirus vaccine vector. Coronaviruses have spikes on their outer coats (which is why they look like a crown and which is where they get their “corona” name from). The Oxford/AstraZeneca vaccine contains the genetic sequence of the surface spike protein. When the vaccine enters cells in the body, it uses this genetic code to produce the surface spike of the coronavirus to induce an immune response (Ref 7). The first adenovirus-based vaccine (a vaccine for Ebola, made by Johnson & Johnson) was approved for general use in July 2020 (Ref 8).
The Pfizer is an mRNA vaccine. It is the first mRNA vaccine to be approved (for emergency use). RNA vaccines introduce into the body a messenger RNA (mRNA) sequence that contains the genetic instructions for the vaccinated person’s own cells to produce the vaccine antigens and generate an immune response (Ref 9).
The ChAdOx1 nCoV-19 Trials
As the US Centers for Disease Control and Prevention (CDC), explains, there are three phases in a vaccine trial. During Phase I, small groups of people receive the trial vaccine as a test for safety. In Phase II the study is expanded and the vaccine is given to people who have characteristics (such as age and physical health) similar to those for whom the new vaccine is intended. In Phase III, the vaccine is given to thousands of people and tested for efficacy and safety (Ref 10).
The July paper for the Oxford/AstraZeneca trial reported that “We did a phase 1/2, single-blind, randomised controlled trial in five trial sites in the UK of a chimpanzee adenovirus-vectored vaccine (ChAdOx1 nCoV-19) expressing the SARS-CoV-2 spike protein compared with a meningococcal conjugate vaccine (MenACWY) as control.” Phase 1 and 2 were thus run and reported on alongside. The five sites were at university hospitals/clinical facilities in Oxford, Southampton, Imperial college London, St Georges London, and Bristol.
Describing the trial as single-blind usually means that one of either the participants or the researchers didn’t know who received the vaccine and who received the placebo. The narrative in the paper showed that it was more complex than this. The control (the meningitis vaccine) was deliberately chosen so that participants wouldn’t know if they received the vaccine or the placebo, as both would likely cause similar side effects. The participants were thus ‘blind’ to which group they had been randomly allocated to (with the exception of 10 people – see below). However, many of the researchers were also unaware of who was in each group and thus also ‘blind.’ Hence many parts of the trial were more than single-blind. (The Pfizer placebo was a saline solution and so would be unlikely to elicit any side effects, but would have been less easy to ‘blind’).
Between April 23 and May 21, 1,077 participants were enrolled and assigned to either the ChAdOx1 nCoV-19 or the meningitis vaccine. There were four initial groups. Group 1 was the phase 1 component of the study. It comprised 88 people – 44 who were assigned to each vaccine. Participants in Group 2 were examined in much more detail (for immunogenicity) than those in Group 4. Group 2 contained 412 people and there were 567 in Group 4. Group 3 consisted of 10 people who were enrolled in a non-randomised, non-blinded group. They knew they were getting the ChAdOx1 nCoV-19 vaccine and they received a booster jab 28 days after the first dose. They were monitored closely for safety. Participants in all groups will be followed up at days 184 and 364, as the medium-term effects are not yet known/reported on.
The dose protocol has become an issue for the Oxford/AstraZeneca trial. The July paper reported that all except the 10 participants in Group 3 received one single dose of 5 × 10¹⁰ viral particles. The 10 people in Group 3 received a two-dose schedule, with the booster vaccine administered 28 days after the first dose. We’ll come back to this.
Across all groups, in the first UK trial, 543 had the ChAdOx1 nCoV-19 vaccine and 534 received the meningitis vaccine. The median age was 35. Participants were approximately 50/50 male/female. The majority (91%) were white. The participants were healthy adults aged 18-55 years with no history of laboratory confirmed SARS-CoV-2 infection or of COVID-19-like symptoms.
The co-primary outcomes were to assess efficacy (measured by cases of symptomatic virologically confirmed COVID-19) and to assess safety (as measured by the occurrence of serious adverse events).
Safety was assessed over 28 days after vaccination and was reported in the July Lancet paper. Reactions were more common in the ChAdOx1 nCoV-19 group (pain, fever, chills, muscle ache, headaches, and malaise). Paracetamol alleviated the reactions to an extent. The paper reported that “No serious adverse reactions to ChAdOx1 nCoV-19 occurred.”
Prophylactic paracetamol (i.e. as a preventative measure) was given to 56 people in the ChAdOx1 nCoV-19 group and 57 in the MenACWY group. Of the (majority) of participants who didn’t receive paracetamol, 67% of ChAdOx1 nCoV-19 vaccine recipients reported pain vs 38% of meningitis vaccine recipients. Tenderness was reported in 83% of ChAdOx1 nCoV-19 participants and 58% of meningitis vaccine recipients (both with no paracetamol). Paracetamol reduced these numbers to 77% and 46% respectively. The July paper similarly reported incidence of fatigue, headaches, and other reactions. Such effects are of little concern if a vaccine is found to be safe longer term and efficacious overall. However, safety beyond seven to eight months (the first vaccine having been administered on April 23rd, 2020) is not yet known.
On September 9th, 2020, it was reported that the Oxford/AstraZeneca trial had been put on hold while a UK patient event was investigated (Ref 11). The event was reported to be a case of transverse myelitis. This is a rare inflammation of the spinal cord, which interrupts the messages that the spinal cord nerves send throughout the body. This can cause pain, muscle weakness, bowel/bladder dysfunction, and/or paralysis. The longer of the two supplementary appendices (129 pages) to the July paper reported that acute disseminated encephalomyelitis was an adverse event of special interest. The longer of the two supplementary appendices (562 pages) to the December paper reported that encephalomyelitis, myelitis, and transverse myelitis were adverse events of special interest (Table 9). i.e. those are events that are important to observe during the trial.
The December paper gave further updates on transverse myelitis. It confirmed that “A case of transverse myelitis was reported 14 days after ChAdOx1 nCoV-19 booster vaccination as being possibly related to vaccination, with the independent neurological committee considering the most likely diagnosis to be of an idiopathic, short segment, spinal cord demyelination.” The paper went on to say, “There were two additional cases of transverse myelitis that were originally reported as potentially related but later determined to be unlikely to be related to vaccination by an independent committee of neurological experts.” (One in each group – see Note 12 for further details).
Figure S1, in the shorter of the two supplementary appendices to the December paper, reported that safety analysis was based on 12,021 participants in the ChAdOx1 nCoV-19 group and 11,724 in the meningitis vaccine group. The usual incidence of transverse myelitis is approximately 1 to 8 new cases per 1 million people per year (Ref 13). That’s between 1 in 125,000 and 1 in 1,000,000 per year. In a few months, the incidence in the ChAdOx1 nCoV-19 group was approximately 1 in 6,000 and in the meningitis vaccine group it was approximately 1 in 12,000.
For efficacy of ChAdOx1 nCoV-19, we turn to the November and December papers. Let’s start with the December paper as that reports on the original 1,077 UK participants and the groups of people that have been added to the trial since then. The initial Phase 1/2 UK trial (of 1,077 people, which started on April 23rd) was called COV001. Three further randomised controlled trials were initiated across the UK (COV002 – 10,673 participants), Brazil (COV003 – 10,002 participants), and South Africa (COV005 – 2,096 participants). (There doesn’t seem to be a COV004). There are 23,848 participants in total. All trials are ongoing.
For various exclusion reasons (the main one being “less than 15 days of follow up since second dose”), the number of people included in Tables 1 and 2 in the December paper was 11,636 and not the 23,848 total participants. The vast majority of these 11,636 participants were in the original age range 18-55. COV002 (UK) had 428 participants over 70 years old (8.9% of participants) and COV003 (Brazil) had 16 participants over 70 years old (0.4%) of participants.
The first Pfizer press release reported 90% efficacy, based on 94 cases among 43,538 people. The Pfizer December paper updated these numbers to report that there were 8 cases among 18,198 people who received the vaccine and 162 cases among 18,325 people who received the placebo. (Ref 3) The difference between 8 and 162 represented 95% efficacy ((162-8)/162).
The tiny number of people who tested positive is striking. The revised Pfizer numbers could also be reported as 99.9% of people in the vaccine group and 99.1% of people in the placebo group didn’t test positive. The Oxford/AstraZeneca numbers were similar. Out of the 11,636 people included in Tables 1 and 2 (December paper), there were 131 total cases. That means 98.9% of people overall did not test positive. That was split between the vaccine and placebo groups as follows: 99.5% of the vaccine group and 98.3% of the placebo group didn’t test positive. In May 2020, there were discussions about the ethics of whether or not vaccine volunteers should be deliberately infected with the virus because so few people were catching it naturally, and vaccine trials were at risk of not having enough cases to study (Ref 14).
The ChAdOx1 nCoV-19 efficacy was reported as 70% overall. This was the difference between 30 cases in the vaccine group and 101 in the placebo group and so this was based on 131 people.
It is important to note in this efficacy section that a case was defined in both the Oxford/AstraZeneca and Pfizer trials as a positive Nucleic acid amplification test (NAAT) for Covid-19 and at least one symptom, such as cough, shortness or breath, fever, loss of sense of smell or taste. NAAT is a collective name for PCR and similar tests that use a cycle/amplification technique, which have significant issues with false positives (Ref 15).
The dose issue in the ChAdOx1 nCoV-19 trials
COV001 was a UK trial, with 1,077 participants, which is still ongoing. The December paper reported “This study was originally planned as a single-dose study and 88 participants remain recipients of a single dose. However, the protocol was modified to a two-dose regime, following an amendment on July 30, 2020.” The single-dose study thus became a two-dose regime “with the booster dose given at the earliest possible time.”
COV002 was described in the December paper as a single-blind phase 2/3 study in the UK, which began on May 28th, 2020. It enrolled participants in England, Wales and Scotland, particularly targeting professionals working in health and social care to recruit those with “high possible exposure to SARS-CoV-2”. Two dosage groups were included in COV002. There has been some controversy about this. The December paper reported “Two dosage groups were included in COV002: participants who received a low dose of the vaccine (2·2 × 10¹⁰ viral particles) as their first dose and were boosted with a standard dose (in the LD/SD group), and subsequent cohorts who were vaccinated with two standard-dose vaccines (SD/SD group).”
This was reported in the BMJ (and the media) as a “dosing error” (Ref 16). It appears that the low dose was given in error, resulting in a number of people who received a low dose followed by a standard dose. However, this group appeared to achieve the best efficacy results and hence this (unplanned) protocol is being examined further by the UK Medicines and Healthcare products Regulatory Agency (MHRA).
The numbers in Table 2 of the December Lancet paper revealed something interesting, which I have not seen mentioned. The numbers in the table below are extracted from that Table 2. Participants and cases are split by trial (COV002 and COV003) and by dose (LD/SD = low dose/standard dose and SD/SD = standard/standard) for the vaccine and placebo. The table shows that, among 11,636 participants, there were 131 cases, of which 30 were in the vaccine group (incident rate 0.5%) and 101 in the meningitis (placebo) group (incident rate 1.7%). The final right hand column called “efficacy” was calculated by person years of follow-up but it is very close to the simple calculation of (101-30)/101.
Look at the LD/SD line and look at the highlighted numbers. The efficacy of the LD/SD regime was reported as 90%. While the LD/SD ChAdOx1 nCoV-19 rate of 0.2% is lower than the rest of the rates in that column, so the placebo rate of 2.2% is higher than the rest of the rates in its column. But the placebo dose did not change and so this rate should not have changed. This also needs investigating.
On October 21st, 2020, associate editor of the BMJ, Peter Doshi, wrote an insightful article about the aims of the trials that were at phase 3 stage in the race for a vaccine (Ref 17). Doshi quoted Peter Hotez, dean of the National School of Tropical Medicine at Baylor College of Medicine in Houston, who said, “Ideally, you want an antiviral vaccine to do two things… first, reduce the likelihood you will get severely ill and go to the hospital, and two, prevent infection and therefore interrupt disease transmission.”
Doshi concurred that we want the vaccine to prevent serious illness and transmission. However, he explained that the current trials are not designed to test either. None of the Pfizer, Oxford, Moderna, Janssen, Sinopharm and Sinovac trials are designed to test reduction in severe Covid-19 (hospital admission, ICU, or death) or interruption of transmission. They are all focused on detection of a positive test and at least one symptom of Covid-19 and the comparison of those results between the vaccine and placebo arm.
Doshi also noted that the trials have not included immunocompromised people. The November Lancet paper reported on 240 people over 70 years old, who were included in the UK COV002 trial. Those with severe or uncontrolled comorbidities (respiratory, heart, kidney, liver, gastrointestinal disease, diabetes – HbA1c >58, cancer, autoimmune disease, vascular disease, and other conditions) were excluded. The vaccine roll-out is starting with the oldest and most vulnerable people when the trials have been undertaken on the young and healthy (Note 18).
The first results from the Pfizer and Oxford/AstraZeneca trials on that efficacy outcome (positive test and at least one symptom) have been reported above. The December Pfizer paper reported 95% efficacy based on 170 cases and the Oxford/AstraZeneca December paper reported 70% efficacy based on 131 cases. The numbers 95% and 70% are relative differences. The absolute difference was 0.84% for Pfizer and 1.2% for the Oxford/AstraZeneca results. We don’t know for how long any protection might last.
The Number Needed to Treat (NNT) is an important measure of the value of an intervention. The NNT is the number of patients, on average, who would need to receive experimental treatment (instead of control treatment) for one additional patient to not have the study outcome (Ref 19). The NNT for the Pfizer drug was 119 people. (It was 286 people at the time of the press release when 94 cases had been reported on.) The NNT for the Oxford/AstraZeneca drug was 83 people.
If you stopped 100 intelligent people in the street and asked them what 90% vaccine efficacy means, they will likely say that it means 90% of people vaccinated won’t get the virus. The fact is that 99% of people who were given the placebo didn’t get the virus. The intelligent person in the street won’t realise that it means approximately 100 people need to be vaccinated to avoid 1 case. They might think those vaccinated won’t have a bad outcome and can no longer transmit the virus. They probably won’t realise that the trials were not even designed to test these key issues. On the one hand, getting to vaccine approval in the same number of months that normally takes years is an extraordinary feat. On the other hand, the gap between what these vaccines can do and what people think and hope they can do is tragically wide.
Post Script (Jan 2021)
Peter Doshi has published some further research in the BMJ (Ref i). Further data have become available to him (although not all the data that he would like). Doshi reported in this updated post that the 95% efficacy was based on 170 PCR confirmed Covid-19 cases, split 8 to 162 between vaccine and placebo groups. Doshi noted that “these numbers were dwarfed by a category of disease called ‘suspected covid-19’ – those with symptomatic covid-19 that were not PCR confirmed.” According to the FDAs report on the Pfizer vaccine there were “3410 total cases of suspected, but unconfirmed covid-19 in the overall study population, 1594 occurred in the vaccine group vs. 1816 in the placebo group” (Ref ii)
When these suspected cases are added to the PCR confirmed cases, this works out at a relative risk reduction of 19% (Note iii). Doshi then did another calculation. He removed cases occurring within 7 days of vaccination (409 on Pfizer’s vaccine vs. 287 on placebo), “which should include the majority of symptoms due to short-term vaccine reactogenicity” (that is interesting in itself) and the relative risk then became 29%. Both 19% and 29% would fail the bar set (at 50%) by the regulation agencies for vaccine efficacy.
If the numbers are recalculated on the suspected and confirmed cases added together – using the 19% relative difference numbers – the absolute difference is 1.99% which gives a NNT of 50. The NNT improves, but this is not a measurement flagged by the trial, while the relative efficacy is lower.
Ref i) https://blogs.bmj.com/bmj/2021/01/04/peter-doshi-pfizer-and-modernas-95-effective-vaccines-we-need-more-details-and-the-raw-data/
Ref ii) https://www.fda.gov/media/144245/download#page=42
Note iii) Placebo cases become 162 + 1,816 = 1,978. Vaccine cases become 8 + 1,594 = 1,602. Then (1,978-1,602)/1,978 is 19%.
Ref 1: https://www.pfizer.com/news/press-release/press-release-detail/pfizer-and-biontech-announce-vaccine-candidate-against
Note 2: Simultaneous equations (A + B = 94 and (A-B)/A = 0.9) enabled us to calculate that (out of the 43,538 participants) 85 people in the placebo group tested positive vs 9 people in the vaccine group. The 90% is the difference between 85 and 9.
Ref 3: https://www.nejm.org/doi/full/10.1056/NEJMoa2034577
Ref 4: https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(20)32661-1/fulltext
Ref 5: https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(20)31604-4/fulltext
Ref 6: https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(20)32466-1/fulltext
Ref 7: https://www.research.ox.ac.uk/Article/2020-07-19-the-oxford-covid-19-vaccine
Ref 8: https://www.nytimes.com/interactive/2020/health/oxford-astrazeneca-covid-19-vaccine.html
Ref 9: https://www.pfizer.co.uk/behind-science-what-mrna-vaccine
Ref 10: https://www.cdc.gov/vaccines/basics/test-approve.html
Ref 11: https://www.nature.com/articles/d41586-020-02594-w
Note 12: Extract from the December paper: “One case that occurred 10 days after a first vaccination with ChAdOx1 nCoV-19 was initially assessed as possibly related, but later considered unlikely to be related by the site investigator when further investigation revealed preexisting, but previously unrecognised, multiple sclerosis. The second case was reported 68 days after MenACWY vaccination. While considered possibly related by the site investigator at the time of reporting, an independent panel of neurological experts considered this to be unlikely. All trial participants have recovered, or are in a stable or improving condition.”
Ref 13: Simone & Emmady. Transverse Myelitis. August 2020. https://www.ncbi.nlm.nih.gov/books/NBK559302/
Ref 14: https://www.theguardian.com/society/2020/may/24/uk-scientists-want-to-infect-volunteers-with-covid-19-in-race-to-find-vaccine
Ref 15: https://www.thelancet.com/journals/lanres/article/PIIS2213-2600(20)30453-7/fulltext
Ref 16: https://www.bmj.com/content/371/bmj.m4670
Ref 17: https://www.bmj.com/content/371/bmj.m4037
Note 18: The November paper had no results for efficacy as in number of infections in the over 70s by ChAdOx1 nCoV-19 vs placebo, but it did report that the vaccine was well tolerated by over 70s with fewer reports of pain, soreness etc. Such a stoic generation would probably not report anything so trifling as a tender arm!
Ref 19: https://clincalc.com/Stats/NNT.aspx