* A paper published in the BMJ in October 2020 reported that none of the Covid-19 vaccine trials were designed to test severity of outcome (hospitalisation, intensive care (ICU) or death) or transmission of the virus.
* A study has been undertaken in Scotland, to investigate a current unknown – whether vaccination can reduce transmission of Covid-19. The paper on the study has been published in pre-print, while undergoing peer review.
* The study examined approximately 150,000 healthcare workers and their approximately 200,000 household members for a period of 86 days following the start of the vaccination programme in Scotland on December 8th, 2020.
* During that period, approximately 80% of the healthcare workers got at least one dose of either the Pfizer or AstraZeneca vaccine. The study reviewed the number of cases (positive PCR test) in healthcare workers while unvaccinated and then ≥ 14 days post vaccination. It matched this to cases among household members during the period when the healthcare worker family member was unvaccinated and then vaccinated.
* The study reported that the positive test rate per 100 person years was 9.40 for household members during the period when the healthcare workers in the household were unvaccinated and 5.93 for household members during the period when the healthcare workers in the household were ≥ 14 days post vaccination. The conclusion was thus that “Vaccinating healthcare workers for SARS-CoV-2 reduces documented cases and hospitalisation in both those individuals vaccinated and members of their households.” (The claims made for hospitalisations were not statistically significant and should not have been made).
* The cases claim came from a particular table, which reported cases in healthcare workers and household members in the unvaccinated and vaccinated periods. The table provided an alternative conclusion. For every 100 healthcare worker cases in the unvaccinated period there were 64 household cases and for every 100 healthcare worker cases in the vaccinated period there were 94 household cases. We could conclude, therefore, that transmission was higher from healthcare workers to household members post vaccination.
* Even if the 9.40 vs. 5.93 case comparison is valid, there are many other factors that could have impacted these numbers that were not mentioned, let alone adjusted for: i) infections in Scotland generally were much higher in the unvaccinated relative to vaccinated period; ii) lockdown policy was much tighter in the vaccinated relative to unvaccinated period; iii) people were excluded from further analysis as soon as they tested positive for Covid-19, so there were fewer people in the vaccinated period to be able to test positive; and iv) people would be less likely to think they had Covid-19 post vaccination and thus less likely to test for it.
* There were a number of other interesting findings that were not highlighted in the paper:
– Healthcare workers being vaccinated had no significant impact on hospitalisation in household members.
– 27% of the positive tests among healthcare workers occurred after being vaccinated and 35% of household cases occurred after healthcare workers had been vaccinated.
– There was no significant difference in the likelihood of household members over the age of 65 testing positive after healthcare workers had been vaccinated. There was, therefore, no evidence that healthcare worker vaccination helped the most vulnerable (elderly) in the household.
* This study of healthcare workers and their households in Scotland, during varying lockdown restrictions, lacks generalisability beyond these circumstances.
* This study fails to provide convincing evidence that living with a vaccinated person makes the household member less likely to test positive for Covid-19 and it provides no evidence for serious outcomes (hospitalisation being the lowest rung of that ladder) being less likely.
In December I reviewed two papers published that month (Ref 1). One was in the New England Journal of Medicine by Polack et al. It was called “Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine” (Ref 2). The other was the Voysey et al paper in The Lancet called “Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK” (Ref 3). The first presented the initial safety and efficacy findings for the Pfizer Covid-19 vaccine; the second presented the same for the Oxford AstraZeneca vaccine.
An important BMJ article was referenced in that post (Ref 4). It was written by Dr Peter Doshi and it summarised some aspects of the vaccine trials of which I had not been aware, and I suspect many others had not been aware either:
“None of the trials currently under way are designed to detect a reduction in any serious outcome such as hospital admissions, use of intensive care, or deaths. Nor are the vaccines being studied to determine whether they can interrupt transmission of the virus.”
Table 1 in Doshi’s paper summarised characteristics of ongoing Phase 3 Covid-19 vaccine trials for Moderna, Pfizer, AstraZeneca (US), AstraZeneca (UK), Janssen, Sinopharm, and Sinovac. All the trials excluded children, adolescents, immunocompromised patients, pregnant or breastfeeding women. None were designed to evaluate reduction in severe cases (hospitalisation, intensive care (ICU), or death) and none were designed to evaluate interruption of transmission (person to person spread.) This does not seem to be widely known.
The trials were set up to measure cases, defined as a positive PCR test with at least one symptom of Covid-19. Cases were the primary outcome reported in the results, which led to the medications being approved for emergency use, while trials are ongoing. (The Pfizer trial is not due for completion until April 6th, 2023 (Ref 5) and the Oxford/AstraZeneca trial is not due for completion until February 14th, 2023 (Ref 6).)
Ideally the vaccine trials would have been designed to test the important issues – serious outcomes and transmission. We want to know if injections can prevent hospitalisation, ICU and/or death and transmission. Given that the trials didn’t test for these, we will need to rely on retrospective population studies, which provide a lower level of evidence.
A paper has been published as a preprint, which means that it has not yet been peer reviewed (Ref 7). The paper contains a caution: “NOTE: This preprint reports new research that has not been certified by peer review and should not be used to guide clinical practice.”
The paper is called “Effect of vaccination on transmission of COVID-19: an observational study in healthcare workers and their households.” As the issue of transmission following vaccination is so important, I thought it would be worth reviewing this paper while it is going through peer-review.
As I was deep into the analysis for this pre-print, a peer-reviewed paper on the same topic was published based on England data (Ref 8). Both papers reviewed healthcare workers and members of their household, but the approaches were different, (and the period examined was longer for the Scotland study) so I proceeded with the Scotland paper.
The Scotland (transmission) paper
This preprint paper was undertaken on data from Scotland. The starting study population included healthcare workers employed as of March 1st, 2020 (the first positive reported case of Covid-19 in Scotland) and still employed by the NHS on November 1st, 2020. Any of these healthcare workers with a positive Covid-19 PCR test before December 8th, 2020 (the date when the vaccination programme was initiated) were excluded from the analysis.
This produced the first interesting finding – the number of people excluded from the study because they had had a positive test between March 1st, 2020 and December 8th, 2020. Originally 152,299 healthcare workers were available to be included. Only 5% of these needed to be excluded from the study because they had tested positive for Covid-19 during the nine and a bit months before December 8th, 2020 (Ref 9). That struck me as a low infection rate for NHS staff working through a pandemic.
Excluding these people left 144,525 healthcare workers (average age 44) and 194,362 household members (average age 31) who lived with them. The study examined the vaccination status of the healthcare workers to see if this had an impact on the people in their household.
The period examined was between 8th December 2020 and 3rd March 2021 (vaccination started in Scotland on December 8th). The primary endpoint of interest was a positive test for Covid-19 ≥ 14 days following the first dose of either the Pfizer (BNT162b2) mRNA vaccine or the Oxford/AstraZeneca (ChAdOx1 nCoV-19) vaccine.
The paper reported that, out of the 144,525 healthcare workers 78.3% (113,253) received at least one dose of a vaccine; 25∙1% (36,227) received a second dose. Table 1 reported that 114,257 healthcare workers had been vaccinated (adding up Dec/Jan/Feb vaccinations) and Table 2 reported 109,074 healthcare workers as “post first dose” (this should reflect those excluded for having tested positive, but this doesn’t add up either). This is in pre-print, so I hope the numerical discrepancies get sorted. I have been in correspondence with the senior author and they plan to check these and resolve the differences.
There were 3,123 and 4,343 documented Covid-19 cases and 175 and 177 Covid-19 hospitalisations in household members of healthcare workers and healthcare workers, respectively. The paper summarised the results as:
“Household members of vaccinated healthcare workers had a lower risk of COVID-19 case compared to household members of unvaccinated healthcare worker (rate per 100 person-years 9∙40 versus 5∙93; HR 0∙70, 95% confidence interval [CI] 0∙63 to 0∙78). The effect size for COVID-19 hospitalisation was similar, with the confidence interval crossing the null (HR 0∙77 [95% CI 0∙53 to 1∙10]).”
We’ll look at where those results came from shortly. The second part of that statement is disingenuous. If the confidence interval crosses the null (i.e., includes 1.0), the finding is not significant and should not be implied as such. The key findings then become; vaccination made a significant difference to transmission of cases but no significant difference to hospitalisation. (I will ignore hospitalisation for the rest of this note, as there were no significant differences). Let’s explore further.
The process was difficult to follow at times and so I put it into familiar context. I know a family where the mother of the household is a healthcare worker (Jane) and the other household members are Rob (husband) and Tom (son). This study was trying to examine the impact of Jane being vaccinated on Rob and Tom. If Rob (and/or Tom – less likely) was also vaccinated, “person-time was censored on the day of their vaccination.” (i.e., they stopped being included on the day they were vaccinated.) Correspondence with the authors revealed that “only about 10% of household members were vaccinated” during the period being examined.
We started on December 8th, 2020 with 144,525 ‘Janes’ and 194,362 ‘Robs’ and ‘Toms’. Between that date and March 3rd, 2021 (which is a period of 86 days), Table 1 told us that 79% of Janes got at least one vaccine. The number of household members must have been evenly distributed because Table 1 also told us that 79% of Robs and Toms counted as living with a vaccinated Jane during that period. There were 21% of Janes unvaccinated during the period and 21% of Robs and Toms who remained living with an unvaccinated Jane during that period.
I’ve extracted the following numbers from Table 2, as the best presentation of that period. Author correspondence confirmed that the 0–13-day period from vaccination until considered post first dose was not included in Table 2 and thus not included in this table below (Note 10).
I also clarified with the authors that the numbers below present the events among household members during the healthcare workers’ unvaccinated and vaccinated period – not the household members’ own unvaccinated and vaccinated period. To spell it out I asked, “Is 148,366 the number of household people who could get infected post the first healthcare worker vaccine and is 1,086 the number of cases among that group post the healthcare worker first vaccine?” The answer was “That is correct.”
The above extracted numbers (from Table 2) show where the claimed 9.40 vs. 5.93 came from (numbers in red). Before we look at these, there are a couple of important findings from this table, which were not highlighted in the paper:
i) We started off with 144,525 unvaccinated Janes. On average, the Janes were unvaccinated for 40 days and then post their first dose for 45 days. Cases were a positive test for Covid-19, so 3,191 Janes had a positive test while unvaccinated and 1,152 Janes had a positive test while vaccinated. That’s another interesting finding – 27% of cases occurring in healthcare workers happened ≥ 14 days after vaccination (Note 11).
ii) Similarly, in Robs and Toms, 2,037 cases occurred while Janes were unvaccinated. And then 1,086 cases occurred after Janes had been vaccinated. So, 35% of household cases occurred after Janes had been vaccinated.
Back to the numbers in red. Reading down the above table, we have people, then cases and then average days in the period, so we can calculate average person days (not shown) and then the rate per 100 person years. The table reported that the rate per 100 person years was 9.40 for household members during the period when Janes were unvaccinated and 5.93 for household members during the period when Janes were ≥ 14 days post vaccination.
The paper thus concluded: “Vaccinating healthcare workers for SARS-CoV-2 reduces documented cases and hospitalisation in both those individuals vaccinated and members of their households.” There’s the disingenuous hospitalisation claim again. But is it right to conclude that vaccinating healthcare workers reduced cases in healthcare workers and/or their household members? Could anything else explain the reduced case rates?
Confounders not adjusted for
Yes, in short. There were a number of key confounders that were not mentioned, let alone adjusted for:
1) Infection rates.
Vaccination started on December 8th, 2020 and the period reviewed ended on March 3rd, 2021. There were 40-41 days on average in the unvaccinated period and 45 on average in the vaccinated period. On average, the unvaccinated period ran until approximately January 16th. Using very detailed and helpful official Scottish government data, I calculated that average cases per day were 1,517 in the unvaccinated period and 942 cases per day in the vaccinated period (Ref 12). Household members may have experienced fewer positive tests in the vaccinated period because everyone in Scotland was experiencing fewer positive tests. (We need to take care here too, because cases are a function of tests, so there are confounders on top of confounders).
There is an argument that cases declined because of the general Scotland vaccination programme and then we’re into a circular argument of – did vaccination reduce transmission in the house or outside the house, but the idea that vaccination reduced transmission anywhere can be challenged with wider data. As of March 3rd, 2021, 30.9% of the UK had been vaccinated compared with 5.5% in Germany and 0.14% in South Africa (Ref 13). Until March 3rd, cases and deaths in both those countries followed a similar (and lower) trajectory than the UK (Ref 14). One black swan and the hypothesis is over – there’s two without looking any further. (Graphs at the end of this note).
2) Lockdown rules.
These were different in Scotland during the 86-day period (Ref 15). At and following December 8th, Scotland was in three levels – most places in level 3 – some in levels 1 & 2. On December 20th, travel between Scotland and the rest of the UK was prohibited. On December 26th, all of Scotland moved into level 4. On January 4th, 2021, a complete lockdown was imposed. On January 15th, all travel corridors were suspended. On February 15th, hotel quarantine was introduced following any type of travel. On March 3rd, furlough was extended. Hence lockdowns were tighter in the vaccinated period than the unvaccinated period. Household members may have experienced fewer positive tests in the vaccinated period because they couldn’t meet anyone outside the household.
UK Prime Minister, Boris Johnson, thinks this. On April 13th he said, “Of course the vaccination programme has helped, but the bulk of the work in reducing the disease has been done by the lockdown” (Ref 16). In case it is thought that this has made the case for lockdown, it is important to remember that, from the outset (March 2020), SAGE (the UK Scientific Advisory Group for Emergencies) reported that “measures seeking to completely suppress spread of Covid-19 will cause a second peak” i.e., lockdowns have a temporary effect (Ref 17). (They flatten the curve – the don’t change the area under the curve).
3) You can only get it once.
Having had 5,228 positive tests in the unvaccinated period (when cases were higher and restrictions were fewer), there were fewer people to test positive in the vaccinated period.
4) Less likely to test.
Once people had been vaccinated, they would be less likely to think that they had Covid-19 and thus less likely to test for Covid-19. That would apply to the household members too.
The bigger issue
The main claim – 9.40 vs. 5.93 cases per 100 person years – came from reading down the table. The confounders above show that there were a number of reasons that could explain the differences in these two numbers, other than the vaccination status of the healthcare worker in that household. Reading across the table gives a different conclusion…
In the unvaccinated period there were 3,191 Jane cases and 2,037 Rob and Tom cases. In the vaccinated period there were 1,152 Jane cases and 1,086 Rob and Tom cases. This means that, for every 100 Jane cases in the unvaccinated period there were 64 Rob & Tom cases and for every 100 Jane cases in the vaccinated period there were 94 Rob & Tom cases. We could conclude, therefore, that transmission was higher from healthcare workers to household members post vaccination (Note 18).
A couple of other points
Figure 1 in the paper reported risk ratios for different subgroups of people. The different age groups were interesting. Figure 1 reported that there was no significant difference in the likelihood of household members over the age of 65 testing positive after healthcare workers had been vaccinated. There was, therefore, no evidence that healthcare worker vaccination helped the people in the household most in need of help.
There was no comparison of transmission in households where the healthcare worker remained unvaccinated during the 86-day period. It would have been useful to compare transmission in unvaccinated with vaccinated people – not just unvaccinated and vaccinated periods in the same person.
This study only applies to Scottish healthcare workers and their households and thus may lack generalisability beyond this. More importantly, the study was undertaken during periods of non-pharmaceutical interventions (lockdown measures) and thus cannot be assumed to apply to periods of normality (if that returns.)
This study fails to provide convincing evidence that living with a vaccinated person makes the household member less likely to test positive for Covid-19 and it provides no evidence for serious outcomes (hospitalisation being the lowest rung of that ladder) being less likely.
Vaccines, cases and deaths for the UK, Germany & South Africa between December 8th (27th for vaccines) 2020 and March 3rd, 2021.
Ref 1: https://www.zoeharcombe.com/2020/12/chadox1-ncov-19-the-lancet-papers
Ref 2: Polack et al. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. NEMJ. December 2020. https://www.nejm.org/doi/full/10.1056/NEJMoa2034577
Ref 3: Voysey et al. Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. The Lancet. December 2020.
Ref 4: Peter Doshi. Will covid-19 vaccines save lives? Current trials aren’t designed to tell us. BMJ October 2020. https://www.bmj.com/content/371/bmj.m4037
Ref 5: https://clinicaltrials.gov/ct2/show/NCT04368728
Ref 6: https://clinicaltrials.gov/ct2/show/NCT04516746
Ref 7: Shah et al. Effect of vaccination on transmission of COVID-19: an observational study in healthcare workers and their households. Pre-Print. March 21st, 2021. https://www.medrxiv.org/content/10.1101/2021.03.11.21253275v1
Ref 8: https://www.gov.uk/government/news/ one-dose-of-covid-19-vaccine-can-cut-household-transmission-by-up-to-half
Ref 9: Supplementary material (https://www.medrxiv.org/content/10.1101/2021.03.11.21253275v1. supplementary-material)
Note 10: I also asked the authors about this, as I couldn’t see how the 0–14-day period hd been accounted for. In a period of 86 days, healthcare workers were unvaccinated on average for 40 days and vaccinated on average for 45 days, so where did the 14-day period go?! The authorsa replied that “days 0-13 post first dose were not included in this table.” But I don’t see how.
Note 11: Table 2 reported that only 4% of the vaccines administered to the healthcare workers were Oxford/AstraZeneca; 96% were Pfizer. Pfizer’s efficacy rate is claimed to be 95%.
Ref 12: https://data.gov.scot/coronavirus-covid-19/detail.html#cases
Ref 13: https://ourworldindata.org/covid-vaccinations
Ref 14: https://ourworldindata.org/covid-cases
Ref 15: https://www.scotsman.com/health/coronavirus/ when-did-lockdown-start-in-scotland-key-dates-and-milestones-as-country-marks-anniversary-of-covid-restrictions-3175303
Ref 16: https://www.standard.co.uk/news/uk/lockdown-covid-19-cases-pandemic-boris-johnson-b929418.html
Ref 17: Meeting 15 March 13th, 2020. Point 24.
( https://assets.publishing.service.gov.uk/government/uploads/system/uploads/ attachment_data/file/888783/S0383_Fifteenth_SAGE_meeting_on_Wuhan_
Note 18: This holds if it is argued that cases as a proportion of people should be taken. In the unvaccinated period, 2.2% of healthcare workers and 1% of household members tested positive. In the vaccinated period these numbers were 1.1% and 0.7%. Again, the ratio was greater in the vaccinated period.