Mortality involving and not involving COVID-19 among vaccinated vs. unvaccinated in England between Apr 21 and May 23

Dear Referee 2,

I greatly appreciate the time and effort you took to provide critical, yet constructive, feedback on my previous version of the paper. In the following, you will read how I have addressed your comments. For your information, I have also made minor corrections and edits in the text to improve accuracy and readability. 

Sincerely,

The author.


Comment #1

The new version of this report includes several changes that improve the clarity and highlight some limitations. However, the report contains several errors and does not do a convincing job of demonstrating the validity of its methods or through reasoning for its conclusions. In short, it concludes too much from too little.

Response #1

In the following, you will read my responses to the specific issues you have raised.

Comment #2

Major Issues with methodology:

The new section in the methods is incorrect. Assume that the rate of deaths is Mv (total mortality in the vaccinated), Mu (mortality in the unvaccinated), Nv (non-COVID in the vaccinated), Nu, Cv (COVID mortality in the vaccinated), Cu (COVID mortality in the unvaccinated). Then let Mu be 60% greater than Mv (Mu/Mv = 1.6). Let the non-COVID mortality be (1-k)%, then Nu/Nv = k. We also have that Mv = Nv + Cv and Mu = Nu + Cu. The relative mortality of COVID in the unvaccinated is Cu/Cv. Now, if non-COVID mortality is 60% greater in the unvaccinated (k = 1.6,) then we get Cu/Cv = (Mu - Nu)/(Mv - Nv) = (1.6Mv - kNv)/(Mv - Nv) = 1.6. Hence, it does not imply that vaccinations don’t provide protection. If we solve for k, we find that k = (0.6Mv + Nv)/Nv for the COVID mortality in the two groups to be the same. The other numbers given in the example also do not hold.

Response #2

I acknowledge your math. Therefore, I have omitted the discussion you referred to in the revision. Additionally, I have done my best to address your comment in the revised version, taking it into account when presenting my findings. Please see my responses below. 

Comment #3

This also undermines the point made when referring to this argument with Fig 2A; however, this is also not that relevant since these rates (or odds ratios) are not directly compared that way.

Response #3

Related to Fig. 2A (p. 6), I write as follows: “At the beginning of the period, the ORs of all-cause mortality (marked in green) among unvaccinated were approximately between 2 and 2.5 compared to vaccinated (significant at the 95% CIs), and mortality not involving COVID-19 (marked in orange) shows a similar pattern.” I assume we can agree on that statement.

Next, I write: “In parallel, Figure 3 shows that the mortality rate involving COVID-19 was low at the beginning of the period for both vaccinated and unvaccinated (A and B are identical, except for different scaling).” [Figure 3 was Figure 5 in the previous version]. Also, I assume we can agree on that statement.

Drawing an implication of what I write above, I continue as follows: “Therefore, I conclude that unvaccinated had between 2 and 2.5 times higher ORs of all-cause mortality and mortality not involving COVID-19 compared to vaccinated at the beginning of the period, largely due to inferior health at the outset, and not vaccine protection since the overall mortality involving COVID-19 during that period was low. The argument is grounded in the assumption that the vaccine unlikely protects against mortality not involving COVID-19. 26 That is, if close to zero people died from COVID-19, I cannot see any logical reason why the mortality pattern observed at the beginning of the period has another explanation than unvaccinated having inferior health at the outset.” I do hope we can agree on the arguments that I have addressed here.

Also, I hope we can agree on the following statement, which, from my point of view does not contradict your mathematical explanation: “Between the last half of 21 and the beginning of 22, on the other hand, the ORs were higher for all-cause mortality than for mortality not involving COVID-19 (Figure 2A), which may indicate a temporal preventive vaccine effect. Figure 3 supports that assumption as it particularly shows an uptick in the mortality rate involving COVID-19 among unvaccinated during that period. However, we cannot rule out that the uptick may not be due to vaccine protection, but instead high vulnerability at the outset among unvaccinated to die from the virus infection. Yet an argument countering that assumption is that the ORs of mortality involving COVID-19 among unvaccinated (compared to vaccinated as a reference group), although significant during the whole period (95% CIs), were down from about 10 at the beginning to about 2 at the end (Figure 4). The decrease may either indicate temporal but declining vaccine protection, potentially because of the rise of the Omicron variant, or the relative increase in mortality among vaccinated may indicate a detrimental health effect, which I address below [in the article].”

Concerning ORs, please see #5.

Comment #4

The reasoning around when non-COVID mortality rates are equal does not make sense either; you could assume that the unvaccinated are unhealthy in a way that greatly increases the risk of COVID-19 death and not non-COVID deaths (though crudely this would be indistinguishable from a vaccine effect).

Response #4

(i) I agree “that you could assume that the unvaccinated are unhealthy in a way that greatly increases the risk of COVID-19 death”, which I particularly illustrate in Figure 3 (Figure 5 in the previous version) and address in the manuscript (please see #3). In the revision, I cannot see that it contradicts your argument.

(ii) You also state that “you could assume that the unvaccinated are unhealthy in a way that [does not increase] non-COVID deaths”. Theoretically, your statement may be correct, even though I do not find it very plausible from a medical point of view. Nonetheless, I argue that the data I analyze falsify your statement, the reason being that all-cause mortality and mortality not involving COVID-19 were much higher among the unvaccinated compared to the vaccinated at the beginning of the period, when registered COVID-19-related deaths were very low (please see #3).

Finally, from my understanding, you argue that the above issues crudely “would be indistinguishable from a vaccine effect”. First, from my point of view, I show that (ii) is distinguishable from a vaccine effect as all-cause mortality and mortality not involving COVID-19 were much higher among unvaccinated compared to vaccinated at the beginning of the period when registered COVID-19 related deaths were very low. Concerning (i), I acknowledge that you have a valid point. In the revision, I therefore write as follows (p. 6): “we cannot rule out that the uptick [in mortality involving COVID-19, particularly among unvaccinated] may not be due to vaccine protection, but instead high vulnerability at the outset among unvaccinated to die from the virus infection. Yet an argument countering that assumption is that the ORs of mortality involving COVID-19 among unvaccinated (compared to vaccinated as a reference group), although significant during the whole period (95% CIs), were down from about 10 at the beginning to about 2 at the end (Figure 4). The decrease may either indicate temporal but declining vaccine protection, potentially because of the rise of the Omicron variant, or the relative increase in mortality among vaccinated may indicate a detrimental health effect, which I address below. To summarize, the vaccine may have provided temporary but declining protection, but we cannot rule out an increasingly detrimental health effect among vaccinated as an alternative or complementary explanation.”

Comment #5

I still do not see how the rate transformations are necessary. The transformed data would still not give “correct” logistic ORs since they are constructed using age-adjusted rates. Why can’t you just compare the rates using the person-years given, without the transformations?

Response #5

I agree that the transformation I report on in Figure 2B may be redundant. In the revision, I accordingly write as follows: “Substantially, Figure 2A and Figure 2B provide the same information …, but in my opinion, the latter illuminates the contrast between all-cause mortality and mortality not involving COVID-19 better...”

Stating that “The transformed data would still not give ‘correct’ logistic ORs since they are constructed using age-adjusted rates” in my opinion would imply that the age-adjusted rates are also invalid. On the other hand, if the age-adjusted rates provide a valid picture, then the transformed logistic ORs would also provide an equally valid picture. Moreover, from my reading, it appears that age-adjusted ORs ratios have also been reported in other research (e.g., https://cardiab.biomedcentral.com/articles/10.1186/s12933-020-01159-5). In itself, that does not suffice to defend my approach, but I cannot see how my approach provides substantially uninformative ORs. If yes, from my understanding, the mortality ratios would be equally uninformative.

In addition, from my perspective, the ORs in Figure 2A provide more precise information about mortality (all-cause and mortality not involving COVID-19) among the unvaccinated compared to the vaccinated, which is not as evident in Figures 1A and 1B. Similarly, I argue that ORs in Figure 4 give more precise information about mortality involving COVID-19 among unvaccinated compared vaccinated than what we observe in Figure 3. E.g., ORs being reduced from about 10 to 2 in Figure 4 is not easily observable in Figure 3.

Comment #6

There is still almost no consideration of other factors that might explain the observed trends. The author concludes that the limited duration of vaccine protection (against COVID deaths) is evidence of the unhealthy vaccine effect but that could equally be due to the rise of COVID variants (such as Omicron that arose shortly before the relevant period, with evidence of reduced mortality and reduced vaccine efficacy) or other changes to behaviour over this time.

Response #6

First, at least in the revision, I do not conclude “that the limited duration of vaccine protection (against COVID deaths) is evidence of the unhealthy vaccine effect”. Instead, I conclude “that vaccination, despite a potential temporary protection, may have increased mortality” (p. 1). In other words, a very tentative conclusion. I do not use the phrase “evidence” a single time. Nor, as far as I can see, do I use similar phrases. Concerning the “consideration of other factors that might explain the observed trends”, I believe I make sober reflections in “Limitations and future research”. Also, I address similar issues in Note 3. The arise of Omicron may address the fall ORs in Table 4, which I address in the revision, writing in relationship to Figure 4 (p. 6): “The decrease [in ORs of mortality involving COVID-19 among unvaccinated (compared to vaccinated as a reference group)] may either indicate temporal but declining vaccine protection, potentially because of the rise of the Omicron variant, or the relative increase in mortality among vaccinated may indicate a detrimental health effect, which I address below [in the paper].” Concerning Figure 5 (Figure 3 in the previous version), I cannot see that the Omicron variant may have had a substantial impact, as it includes mortality data not involving COVID-19 only.

Comment #7

The author makes some attempt to dismiss the limitation that the unvaccinated population could have started unhealthy, but on an aggregate level, improved in health (due to deaths, vaccination or behaviour change). In general, rates in risk groups may be lower, but maybe not on the scale at which this data is presented (i.e. by month), but the author does not explore this data.

Response #7

I assume you here refer to the data I present in Figure 5 (Figure 3 in the previous version) of mortality not involving COVID-19, and addressing the limitation concerning that “During the study period, a share of people in the unvaccinated group were transferred to the vaccinated” (p. 11). However, I cannot see that I attempted “to dismiss the limitation that the unvaccinated population could have started unhealthy”. Instead, I showed (in Figure 5) “that while mortality not involving COVID-19 decreased among unvaccinated (marked in red) compared to the first observation month, it remained high among vaccinated (marked in blue)”. In my opinion, this is an undisputable empirical observation, and as long as accounting for potential limitation concerning the dynamics in the unvaccinated vs. unvaccinated cohorts (which I believe I addressed adequately on p. 8 and in Note 5), one may therefore conclude as I do: “the data show a relatively high and relative increase in mortality not involving COVID-19 among vaccinated. An interpretation may be that vaccination, despite temporary protection, increased mortality. Strengthening the interpretation was relatively high mortality among vaccinated not involving COVID-19 counterintuitively following periods of excess mortality (Figure 6) …. Further strengthening the interpretation was the relatively high mortality not involving COVID-19 among the vaccinated, corresponding with excess mortality during much of the same period (ibid.) ….”  


Comment #8

The fact that apparent high-risk groups have higher rates of non-vaccination is alluded to in this paper with no in-depth exploration. While the author acknowledges that improper adjustment for confounders can bias results further, they make a worse mistake by adjusting for these variables in their mental model in a way that cannot be quantified and hence cannot be examined. This holds for the fact that they use age-adjusted death rates; is that not exactly the partial confounding that the author cautions against in the introduction?

Response #8

You state that “The fact that apparent high-risk groups have higher rates of non-vaccination is alluded to in this paper with no in-depth exploration.” In my opinion, I addressed the issue adequately. First, I related the statement by the UK Office for National Statistics, “rates for COVID-19 unvaccinated adults in England ‘were higher for Black Caribbean, Black African and White Other ethnic groups. Rates were also higher for those living in deprived areas, who have never worked or are long-term unemployed, who are limited a lot by a disability, … or who are male’”, to vaccine hesitancy research (with proper references). Then, I state that the above citation from the UK Office for National Statistics “indicates that unvaccinated have inferior health at the outset compared to vaccinated, inducing biased comparisons as the groups are not randomly assigned.” In the paper’s empirical section, I further address the issue related to the likely difference between non-randomized groups in much detail, as far as I can see.

Then you state that “While the author acknowledges that improper adjustment for confounders can bias results further, they make a worse mistake by adjusting for these variables in their mental model in a way that cannot be quantified and hence cannot be examined. This holds for the fact that they use age-adjusted death rates; is that not exactly the partial confounding that the author cautions against in the Introduction?” Ok, it seems that we agree on the issue “that improper adjustment for confounders can bias results further”, but York states that “unless all potential confounding factors are included in an analysis (which is unlikely to be achievable with most real-world data-sets), adding control variables to a model in many circumstances can make estimated effects … less accurate” (cited on p. 2 in my paper). He does say that adding any control variable will, but can, make the estimates “less accurate”. As the data I apply match for age, theoretically, we can therefore assume that the estimates are less accurate, but I cannot see any logical reason for that. However, on the contrary, assuming that matching for age were to increase bias, I would still argue that the way I interpret the data would yield a similar conclusion, the reason being that, whether matching or not matching for age, one could nonetheless expect that vaccinated and non-vaccinated are dissimilar at the outset concerning health profile.

Comment #9

In general, the report is unconvincing and concludes too much from its data. The author should indicate what their hypothesis is and what we would expect to see in the observations based on this. They should also indicate where these observations would contradict other, more common explanations. However, I don’t believe this is viable from such a limited set of data (particularly with no exploration of potential confounders), the author should greatly limit their conclusions.

Response #9

I hope that my revisions, on which I have commented above, clarify the study’s contribution. What I can conclude from the data, and what I cannot. I agree that the previous version may have drawn too many conclusions from the data. In the revision, I have accordingly applied wordings such as follows:

“First, I found that all-cause mortality among unvaccinated was higher than among vaccinated. [I believe that statement is indisputable.] But, as the pattern was similar concerning mortality not involving COVID-19, the discrepancy may be attributed mainly to unvaccinated having inferior health at the outset” (p. 1) [Note that I write “may”, but having said that, I believe the finding has relatively strong empirical support due to my findings, and om which I report (p. 6): I conclude that unvaccinated had between 2 and 2.5 times higher ORs of all-cause mortality and mortality not involving COVID-19 compared to vaccinated at the beginning of the period, largely due to inferior health at the outset, and not vaccine protection since the overall mortality involving COVID-19 during that period was low. The argument is grounded in the assumption that the vaccine unlikely protects against mortality not involving COVID-19. That is, if close to zero people died from COVID-19, I cannot see any logical reason why the mortality pattern observed at the beginning of the period has another explanation than unvaccinated having inferior health at the outset.”]

Then I write that “There were nonetheless indications of significant protection for vaccinated between July 21 and Jan 22” (p. 1) [Note that I write “indications of significant protection…”. My statement is grounded in how I address the presentation and discussion of the relevant data (p. 6): “Between the last half of 21 and the beginning of 22 … the ORs were higher for all-cause mortality than for mortality not involving COVID-19 (Figure 2A), which may indicate a temporal preventive vaccine effect. Figure 3 supports that assumption as it particularly shows an uptick in the mortality rate involving COVID-19 among unvaccinated during that period. However, we cannot rule out that the uptick may not be due to vaccine protection, but instead high vulnerability at the outset among unvaccinated to die from the virus infection. Yet an argument countering that assumption is that the ORs of mortality involving COVID-19 among unvaccinated (compared to vaccinated as a reference group), although significant during the whole period (95% CIs), were down from about 10 at the beginning to about 2 at the end (Figure 4). The decrease may either indicate temporal but declining vaccine protection, potentially because of the rise of the Omicron variant, or the relative increase in mortality among vaccinated may indicate a detrimental health effect…. To summarize, the vaccine may have provided temporary but declining protection, but we cannot rule out an increasingly detrimental health effect among vaccinated as an alternative or complementary explanation.”]

Finally, I write (p. 1): “while mortality not involving COVID-19 decreased among unvaccinated compared to the first observation month, it was high among vaccinated, indicating a relative increase”. I hope we can agree on that statement.

From my point of view, in the revision, I have done my utmost not to draw more conclusions from the data than what is reasonably plausible.

You state that “The author should indicate what their hypothesis is and what we would expect to see in the observations based on this.” I agree that postulating one or more hypotheses in a deductive setting has advantages, which are tested in, for instance, regression models providing significant or non-significant empirical findings. However, I cannot see how I can study my research question using that approach, as I consider the examination of my research question to be more of a puzzle where different analyses (graphed in the different figures) in totality provide information about what the overall empirical landscape looks like. My general research question is as follows (p. 3): “how do the mortality patterns differ in England from Apr 21 to May 23 between COVID-19 vaccinated and unvaccinated?”, and I argue that my study illuminates that research question fairly well. “The study’s major contribution is to illustrate how contrasting all-cause mortality with mortality not involving COVID-19 may indicate valid estimates between non-randomized groups of vaccinated and unvaccinated.” To do that, I further discuss data on mortality involving COVID-19. From my writings above (particularly #3 and #4), in my opinion, I argue that my empirical data and the explanations of them sensibly address the research question, but without overexplaining the interpretations. 

You continue writing, “They [the hypotheses] should also indicate where these observations would contradict other, more common explanations.” In my opinion, I argue that my research question and postulated contribution explain how I contribute to the current research literature. In the Discussion (p. 8), I explain in detail how studying my research question aligns with and contributes to the existing research literature.

Finally, you write that “I don’t believe this is viable from such a limited set of data (particularly with no exploration of potential confounders), the author should greatly limit their conclusions.” In my opinion, I argue that this revised version, in particular, provides a sober interpretation of the available and analyzed data. Moreover, the issue of “confounders”, which I admit were available for the data I analyzed, can, even in their presence, be challenging concerning validity, as I address in the paper’s Introduction.

Comment #10

Finally, a couple of minor points:

Wouldn’t it be viable to do a sensitivity analysis, including the other ICD codes, and see how that impacts the results?

Response #10

Unfortunately, I do not have access to the data you refer to. Addressing limitations, I write as follows: “The validity of the finding indicating that vaccinated had significant protection between July 21 and Jan 22 hinges on non-systematic skewness in classifying false positives concerning mortality involving COVID-19 and false negatives concerning mortality not involving COVID-19.” I further elaborate on that topic in the manuscript.

Comment #11

Figures 5/6 are only mentioned in notes, which is confusing; they could be addressed in the discussion instead.

Response #11

In the revision, I have particularly addressed Figure 5 (Figure 3 in the revision) when presenting the results. The same goes for Figure 6 (Figure 4 in the revision). Please also see #3.

Dear Referee 2,

I highly appreciate your time and efforts in giving constructive feedback on my previous version of the paper. In the following, you will read how I have addressed your comments. For your information, I have also made minor corrections and editions in the text to improve accuracy and readability. Hopefully, the revised version will be uploaded shortly.  

I look forward to hearing from you.

Sincerely,

The author.


This report investigates the impact of COVID-19 vaccination on mortality in England between mid-2021 and mid-2023. The author observes that the age-stratified non-COVID mortality rates in the vaccinated population increase or remain stable over this period, whilst those in the unvaccinated population decrease. The author states that this observation is consistent with a vaccination-related decline in health. The author also observes periods where the COVID-19-related mortality rates in the unvaccinated are higher than those in the vaccinated, potentially showing a protective effect of vaccination against COVID-19-related disease. However, the author posits an alternative theory based on the unvaccinated population being generally more "unhealthy" (i.e. healthy vaccinee effect) as evidenced by higher rates of all-cause and non-covid related mortality in the unvaccinated population at the study start. The author claims that their approach can adjust for unobserved variables that explain the differences in health between the two comparison groups.

Response: Below, I will address the particular issues you have raised in detail.

The author has mixed his methods/reasoning into the report's introduction and results sections. It would be better to explore the approach in the methods section and highlight any potential limitations.

Response: I agree with you, and in the revised version, I have removed the methodological approach from the Introduction, but mention the following: “To address the research gap [explained above in the Introduction], using English data covering 26 months from Apr 21 to May 23, 5 I elaborate an achievable approach by comparing all-cause mortality among COVID-19 vaccinated and unvaccinated with mortality not involving COVID-19. In the Methods section, I explain it in full detail.”
Also, I highlight more extensively the potential limitations of the approach in the latter part of the Discussion, writing as follows: “The validity of the finding that vaccinated had significant protection between July 21 and Jan 22 hinges on non-systematic skewness in classifying false positives concerning mortality involving COVID-19 and false negatives concerning mortality not involving COVID-19. A relevant issue in this regard is that the English data excluded ICD10 death certificate codes U09.9 (Post-COVID condition, where the acute COVID had ended before the condition immediately causing death occurred) and U10.9 (Multisystem inflammatory syndrome associated with COVID-19) as criteria when classifying mortality involving COVID-19, but as this was the case only when the U07.1 (COVID-19, virus identified) or U07.2 (COVID-19, virus not identified) were not mentioned, I cannot see substantial skewness in false positives and negatives between vaccinated and unvaccinated. The potential limitation may nonetheless induce cautiousness when interpreting the data, which I encourage future research to address. The validity of the finding that vaccinated had non-significant protection from Feb 22 also has limitations, as relatively low mortality involving COVID-19 can be an alternative explanation. However, in Note [3], I elaborate extensively on the issue, concluding that the alternative explanation is not very likely, but I nonetheless encourage cautiousness when interpreting the data.”
Please note that the revised text is an extension and further elaboration of the previous text addressing limitations.
Please also see #4, which addresses revisions I have carried out in the Introduction by following advice from the other referee.

The results should describe any major observations and the theorising should be limited to the discussion. Alternatively, the author could be more explicit about the theories he wants to test in the methods section; either way, the presentation should be improved.

Response: In the revision, I have added a paragraph at the end of the Methods section where I argue in detail how distinctions between all-cause mortality and mortality not involving COVID-19 among vaccinated and unvaccinated, absent of control variables in populations with potentially different health statuses at the outset, can assess eventually genuine health effects. Please see #18. I agree with the referee that extensive discussions of empirical findings should not be conducted in the Results section, but presenting them without any interpretation will make it more difficult for the reader to interpret the text, I argue. Therefore, I point to findings, and briefly explain their potential meaning. In the revision, I have excluded some figures and included them in the Notes section (please see #6). As such, I have aimed to reduce the complexity of presenting the data and hope that the results are more interpretable. Also, a couple of places in the Results section, I refer to my explanation at the end of the Methods section.

In Figure 3, the author should highlight how this relates to the other figures by overlaying the data or plotting on the same time scale.

Response: I agree with your point, but unfortunately, it is challenging to carry out as the time scales are different; the English data I apply in my study use monthly observations, while the Our World in Data uses weekly ones. I find it challenging to convert the different time scales into one, as there is no distinct overlap in weekly and monthly observations. Moreover, in the revision, I have edited the text in the Results section and Abstract writing, “Further strengthening the interpretation was the relatively high mortality not involving COVID-19 among the vaccinated, corresponding with excess mortality during much of the same period”, as it more precisely reflects the genuine interpretation of the data.  

A third of the methods section describes how the author converted the ONS's age-stratified mortality rates (per 10000 person-years) to "mortality probability." The author should know that this process does not calculate a probability and rescales the given mortality rates. It is the equivalent of dividing the age-stratified mortality rates by 12*10000, calculating the age-stratified mortality rate per person-month. The report should compare the ONS rates as these are already at a more sensible scale.

Response: In the revision, I use the term monthly mortality rate per 100,000. (Of course, I could have used a yearly rate, but in my opinion, a monthly rate is more logical in the current context since I analyze monthly data.) I carry out the exercise, as I do, to assess how many died or survived of a population in a given month, vaccinated or unvaccinated, to estimate as statistically correct standard errors as possible using logistic regression.

The report should also consider explicitly how the ONS definition of COVID-19-related death would impact these results. Excluding ICD10 codes U09.9 and U10.9 as COVID-related could bias these findings.

Response: Thanks for this comment. In the revised version, I address the issue in the revision writing as follows: “The validity of the finding that vaccinated had significant protection between July 21 and Jan 22 hinges on non-systematic skewness in classifying false positives concerning mortality involving COVID-19 and false negatives concerning mortality not involving COVID-19. A relevant issue in this regard is that the English data excluded ICD10 death certificate codes U09.9 (Post-COVID condition, where the acute COVID had ended before the condition immediately causing death occurred) and U10.9 (Multisystem inflammatory syndrome associated with COVID-19) as criteria when classifying mortality involving COVID-19, but as this was the case only when the U07.1 (COVID-19, virus identified) or U07.2 (COVID-19, virus not identified) were not mentioned, I cannot see substantial skewness in false positives and negatives between vaccinated and unvaccinated. The potential limitation may nonetheless induce cautiousness when interpreting the data, which I encourage future research to address.”

The author should clearly explain the reasoning around how the assumption that COVID-19 vaccination does not prevent non-COVID-19 deaths supports the theory that the difference in COVID-19 death rates (between unvaccinated and vaccinated) is explainable by inferior health at the onset.

Response: At the end of the Methods section, I write as follows in the revision: “Assuming a 60% higher all-cause mortality rate among unvaccinated compared to vaccinated, in the absence of other information, can have two explanations: (i) the unvaccinated have inferior health at the outset compared to the vaccinated or (ii) vaccination protects against mortality. In addition, there can be a combination of (i) and (ii). If the mortality not involving COVID-19 is also 60% higher among unvaccinated, explanation (i) has more validity. The reason is that COVID-19 vaccination unlikely protects against mortality not involving COVID-19. ¹⁶ Conversely, if the mortality rate not involving COVID-19 is equal between unvaccinated and vaccinated, explanation (ii) has higher validity. The reason is that there is no other likely explanation than a vaccine effect as to why the all-cause mortality among unvaccinated compared to unvaccinated is higher than the mortality not involving COVID-19. Finally, if the mortality not involving COVID-19 is 20% higher among unvaccinated compared to the vaccinated, a combination of explanations (i) and (ii) has more validity. I.e., 20% higher mortality not involving COVID-19 among unvaccinated can be explained as inferior health status at the outset, while vaccination protection can explain 33% higher mortality among unvaccinated (((1.6/1.2)-1)*100). The explanations hinge on the assumption of non-systematic skewness in classifying false positives concerning mortality involving COVID-19 and false negatives concerning mortality not involving COVID-19, which I address in the Discussion. Further, the explanations hinge on the assumption that the mortality involving COVID-19 is not zero, which I address in Note 3.”

The report does not sufficiently consider alternative explanations for the observed data. While the healthy-vaccinee effect might be strong in clinical trials (since these tend to recruit healthy volunteers), this effect might not be so strong in mass vaccination campaigns, particularly ones like COVID-19 that specifically target vulnerable populations. If we assume that this effect does explain the initial difference in non-COVID-19 mortality rates and that many of the unvaccinated (but not all, i.e. the vaccine-hesitant) are acutely ill, then we would expect to see a trend towards parity in the non-covid mortality rates of the two. As the acutely ill expire (or recover and get vaccinated), the mortality rates in the non-vaccinated would reduce in future months. This trend would be strong if the vulnerable and very elderly were targeted first for vaccination as they are at higher risk of becoming ill later (thus contributing to the mortality rate in the vaccinated population). This is to say nothing about the countless other confounding variables that could explain temporal differences in mortality across these groups, such as different temporal vaccine uptake in different ethnic or SES groups and different rates of adherence to restrictions. These alternative theories do not disprove the theory put forward in this report. However, they highlight that the methodology here cannot convincingly adjust for the potential health differences between the two comparison groups. While improper adjustment for confounding can increase bias, that is no excuse to ignore potential confounding. This report must focus on the actual observation it is theorising around (i.e. a decrease in the non-covid health gap between the vaccinated and the unvaccinated) and more convincingly explore/counter alternative explanations and consider sensitivities to their results.

Response: You mention that “[w]hile the healthy-vaccinee effect might be strong in clinical trials (since these tend to recruit healthy volunteers), this effect might not be so strong in mass vaccination campaigns, particularly ones like COVID-19 that specifically target vulnerable populations.” Considering that statement, I cannot see that it aligns with the UK Office for National Statistics stating that “rates for COVID-19 unvaccinated adults in England “were higher for Black Caribbean, Black African and White Other ethnic groups. Rates were also higher for those living in deprived areas, who have never worked or are long-term unemployed, who are limited a lot by a disability, … or who are male.” Nor does it align with vaccine hesitancy (to which I refer in the revision), and Norwegian data showing much higher mortality among young unvaccinated in a population where practically zero young people died of COVID-19. Also, in the revised version, I explain in detail why vaccination cannot explain the difference in mortality not involving COVID-19. From my reading off the comment, it seems that the referee points to the dynamic of the group of people being transferred from the group of unvaccinated to the group of vaccinated during the time studied. That is definitely a relevant issue, which I have addressed in the Discussion, writing as follows (the text in the paper includes relevant references): “During the study period, a share of people in the unvaccinated group were transferred to the vaccinated. Assuming they had inferior health status at the outset, it may explain the relative increase (decrease) in mortality among the vaccinated (unvaccinated). However, those who remained unvaccinated, on the contrary, had inferior health status at the outset, making the above reasoning implausible. Ceteris paribus, one may even oppositely conclude that it would decrease (increase) relative mortality among vaccinated (unvaccinated). (In Note 7, I add: “People in England under 70 years old but clinically extremely vulnerable were prioritized vaccination with those aged between 70-74. Hence, they were prioritized early.”) Since most elderly candidates had been offered vaccine before Apr 21, I nonetheless assume the estimates were not substantially skewed over the study period, as relatively few people die in younger age cohorts.”

In conclusion, this report needs considerable reworking regarding its statistical and epidemiological content.

Response: Above, you will read how I have addressed the issues raised.