Drug effectiveness for COVID-19 inpatients inferred from Japanese medical claim data using propensity score matching

Shingo Mitsushima; Hiromasa Horiguchi; Kiyosu Taniguchi

doi:10.12688/f1000research.131102.2

Home Browse Drug effectiveness for COVID-19 inpatients inferred from Japanese...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Research Article

Revised

Drug effectiveness for COVID-19 inpatients inferred from Japanese medical claim data using propensity score matching

[version 2; peer review: 2 approved]

Shingo Mitsushima ¹, Hiromasa Horiguchi², Kiyosu Taniguchi^3,4

PUBLISHED 22 Jan 2024

Author details Author details

¹ Center for Field Epidemic Intelligence, Research and Professional Development, National Institute of Infectious Diseases, Shinjuku-ku, Tokyo, 1620052, Japan
² Department of Clinical Data Management and Research, Clinical Research Center, National Hospital Organization, Meguro-ku, Tokyo, 1528621, Japan
³ Research Director, The Tokyo Foundation for Policy Research, Minato-ku, Tokyo, 1066234, Japan
⁴ General Director, National Hospital Organization Mie National Hospital, Tsu, Mie, 5140125, Japan

Shingo Mitsushima
Roles: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review & Editing

Hiromasa Horiguchi
Roles: Conceptualization, Data Curation, Funding Acquisition, Resources, Software, Supervision, Validation, Writing – Review & Editing

Kiyosu Taniguchi
Roles: Conceptualization, Data Curation, Funding Acquisition, Methodology, Project Administration, Resources, Supervision, Validation, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

This article is included in the Emerging Diseases and Outbreaks gateway.

Abstract

Background

Earlier studies and clinical trials of Coronavirus 2019 (COVID-19) showed that drugs such as antiviral drugs, antibody cocktails, and steroids and anti-inflammatory drugs can prevent severe outcomes and death.

Methods

Observational data in Japan assess drug effectiveness against COVID-19. We applied the average treatment effect model, particularly propensity scoring, which can treat the choice of administered drug as if administration were randomly assigned to inpatients. Data of the Medical Information Analysis Databank, operated by National Hospital Organization in Japan, were used. The outcome was defined as mortality. Subjects were all inpatients, inpatients with oxygen administration, and inpatients using respiratory ventilation, classified by three age classes: all ages, 65 years old or older, and younger than 65 years old. Information about demographic characteristics, underlying disease, administered drug, the proportions of Alpha, Beta and Omicron variant strains, and vaccine coverage were used as explanatory variables for logistic regression.

Results

Estimated results indicated that only one antibody cocktail (sotrovimab, casirivimab and imdevimab) was associated with raising the probability of survival consistently and significantly. By contrast, other drugs, an antiviral drug (remdesivir), a steroid (dexamethasone), and an anti-inflammatory drug (baricitinib and tocilizumab) were related to reduce the probability of survival. However, propensity score matching method might engender biased results because of a lack of data such as detailed information related to intervention and potential confounders. Therefore, the effectiveness of some drugs might not be evaluated properly in this study.

Conclusions

Results indicate high likelihood that antibody cocktails were consistently associated with high probability of survival, although low likelihood was found for other drugs for older patients with mild to severe severity and all age patients with moderate severity. Further study is necessary in light of the lack of available data.

Keywords

COVID-19, mutated strain, underlying diseases, antibody cocktail, antiviral drug, mortality, steroid and anti-inflammatory drug

Corresponding author: Shingo Mitsushima

Competing interests: No competing interests were disclosed.

Grant information: This work was supported by the Ministry of Health, Labour, and Welfare [grant number 20HA1005].
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2024 Mitsushima S et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Mitsushima S, Horiguchi H and Taniguchi K. Drug effectiveness for COVID-19 inpatients inferred from Japanese medical claim data using propensity score matching [version 2; peer review: 2 approved]. F1000Research 2024, 12:398 (https://doi.org/10.12688/f1000research.131102.2) First published: 13 Apr 2023, 12:398 (https://doi.org/10.12688/f1000research.131102.1) Latest published: 22 Jan 2024, 12:398 (https://doi.org/10.12688/f1000research.131102.2)

Revised Amendments from Version 1

We have made revisions to the manuscript in the following aspects:

Added a rationale for the propensity score matching method.
Provided an explanation regarding the reliability and representativeness of the observational data in this study.
Included a rationale for the effectiveness or ineffectiveness of certain drugs.
Added figures 1 and 2 to present a clearer timeline of COVID-19 events in Japan.
Recapitulated and added limitations to the manuscript.

Reorganized and re-proofread the manuscript for clarity and consistency.

See the authors' detailed response to the review by Ramesh Kandimalla

Introduction

Coronavirus disease 2019 (COVID-19) is caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). COVID-19 caused pandemic throughout the world from December 2019. In Japan, the first case was detected in January 2020; it then spread across the country. As characteristics of COVID-19 were revealed and treatments developed, the mortality rate attributable to COVID-19 has declined.¹

Earlier studies and clinical trials have shown that drugs used against COVID-19, such as antiviral drugs (remdesivir),² antibody cocktails (casirivimab/imdevimab and sotrovimab),³^,⁴ steroids (dexamethasone),⁵ anti-inflammatory drugs (baricitinib and tocilizumab),⁶^–⁸ statins,⁹^,¹⁰ RNA-dependent RNA polymerase inhibitor (molnupiravir),¹¹^,¹² and protease inhibitor (nirmatrelvir/ritonavir),¹¹^,¹³ are expected to prevent severe COVID-19 outcomes and death.

The National Hospital Organization (NHO) in Japan, an organization of regional core hospitals accounting for about 3.4% of all beds in Japan,¹⁴ provides a database of medical claims from 60 representative NHO hospitals: the Medical Information Analysis Databank (MIA). NHO is one of the largest hospital organizations. Each prefecture has one or more NHO hospitals. MIA includes accumulated data of medical insurance claims for outpatients and inpatients. It includes inpatients’ demographic information, underlying diseases, medical interventions including oxygen administration, use of respiratory ventilation, administration of drugs, and outcomes such as discharge or death.¹⁵

We evaluated the top five most commonly used drugs in MIA: remdesivir, dexamethasone, baricitinib, tocilizumab, and antibody cocktails (casirivimab/imdevimab and sotrovimab). In earlier studies, all these drugs were associated with improved survival or reduced the risk of disease progression, but antibody cocktails were effective only for mildly ill patients; dexamethasone was only effective for moderately to severely ill patients.²^–⁸ We did not divide the antibody cocktails sotrovimab and casirivimab/imdevimab because the numbers of cases with antibody cocktails were insufficiently large. We considered the overall effects of antibody cocktail treatments.

Whether a drug is administered or not depends on the patient condition. In other words, drug administration is probably not a random assignment. Generally speaking, patients who have a higher likelihood of developing an illness have a higher probability of being administered drugs, which implies, because of selection bias in the observational data, a negative association between drug administration and outcomes. Non-random choice of drug administration must be considered for drug effectiveness estimation. Propensity score matching might resolve this difficulty statistically but not experimentally.¹⁶ Propensity score matching is used widely in social sciences to evaluate programs: participants choose to join a program spontaneously, such as a job training program or an unemployment payment program. However, in medical sciences, a researcher can perform experiments and thereafter delete selection bias in the choice of subjects. Nevertheless, such experiments are expensive and require a longer period. These limitations cause the number of subjects in experiments to be constrained. Moreover, outcomes tend to be evaluated with less severity.¹⁷^,¹⁸ Moreover, after approval, experiments with random assignment are expected to be difficult to conduct in the real world because of ethical considerations: some patients might be unable to receive a potentially effective drug. Therefore, pseudo-experimental method should be applied to evaluate drug effectiveness in the real world. An average treatment effect model including propensity matching was applied to create these situations. This procedure predicts the likelihood of receiving drugs initially. Thereafter, outcomes of a drug-administered group and a drug not-administered group are compared. The members of those groups have an almost equal likelihood of having been administered some drugs. We then examined drug effectiveness using this method.

The objective of this study was evaluating the “real-world effectiveness of drugs against COVID-19 in Japan.” “Real-world effectiveness of drugs” means the effectiveness of drugs considering several factors including population characteristics, examinations, vaccinations, and healthcare system such as hospitalization and treatment protocols. Therefore, the results of this study might not be applicable to other countries.

Method

Data sources

This study used MIA for confirmed inpatients: age, sex, underlying diseases, hospitalization date, administered drug, outcome and whether they received oxygen therapy and/or ventilation. We created an overview of inpatients in MIA and patients in Japan based on National data and MIA data.

We also used data for vaccine administration published by the Cabinet Secretariat. Moreover, prevalence of Alpha, Delta and Omicron variant strains were referred from a monitoring meeting in Tokyo because MIA included no information about a patient’s vaccine status or sublineage in SARS-CoV-2.

The study period was January 2020 through March 2022, using data recorded as of May 2022. This study area was the entirety of Japan.

Definitions of variables

We defined demographical conditions as age and sex, underlying diseases as cancer (C00–C90 in ICD10), asthma (J45), chronic obstructive pulmonary disease (COPD) (J44), hypertension (HT) (I10), heart failure (HF) (I50), and diabetes mellitus (DM) (E10). We examined the effects of an antiviral drug (remdesivir),² an antibody cocktail (sotrovimab and casirivimab/imdevimab),³^,⁴ a steroid (dexamethasone),⁵ and anti-inflammatory drugs (baricitinib and tocilizumab),⁶^–⁸ which were proven effective against COVID-19 by earlier studies. We assumed vaccination coverage as the rate of the second dose of vaccine received two weeks prior by age class, as younger than 65 years old, and 65 years old or older. Mutated strains, Alpha, Delta, and Omicron variant strains, were measured by percentage at one week before admission. The Alpha, Delta, and Omicron variant strains were defined as the respective proportions of Alpha, Delta, and Omicron variant strains. Omicron included BA.2 or a later sublineage. Alternatively, we used a dummy variable during the 4th–6th wave instead of the proportion of the mutated strains as an explanatory variable, to check robustness. By this specification, the Alpha variant strain emerged and then dominated in the 4th wave, defined as from 1 March through 20 June 2021. Similarly, the Delta variant strain emerged and then dominated in the 5th wave, defined as 21 June through 21 November 2021. Omicron BA.1 strain emerged and then dominated from the 6th wave, defined as 22 November 2021 to the end of the study period. Death during hospitalization was defined as the outcome.

Subjects

Subjects were all inpatients confirmed from MIA data as having SARS-CoV-2. For this study, mild symptoms were defined as a patient who did not require medical intervention such as oxygen therapy or respiratory ventilation, moderate symptoms were defined as a patient who required oxygen administration, and severe symptoms were defined as a patient who required respiratory ventilation.

In Japan, during the Delta variant epidemic, patients with mild severity were not admitted to hospitals because medical resources such as hospital beds had reached full capacity, although they could have been hospitalized before the period of the Delta variant epidemic. Aside from purely medical criteria, the criteria of hospitalization for asymptomatic patients or patients with mild symptoms who did not require oxygen therapy were probably affected strongly by medical resource scarcity or social situations such as support for their staying at home and recuperation at home. For that reason, we also limited our study to subjects who were expected to be inpatients with oxygen therapy or respiratory ventilation.

Statistical analysis

We conducted estimates separately by drug type: remdesivir, antibody cocktail (sotrovimab, casirivimab and imdevimab), dexamethasone, baricitinib, and tocilizumab.

The first step to assess whether the patient was administered a type of drug or not was performed through logistic regression on their age, sex, underlying diseases, pharmaceutical therapy, vaccine coverage, and prevalence in the mutated strains as explanatory variables. The second step was comparison of outcomes among administered patients and non-administered patients with almost identical likelihood to that of the prediction done in the first step.

All statistical analyses were conducted using software (Stata SE 17.0; Stata Corp.). We inferred significance at the 5% level.

Ethical considerations

Individual informed consent was not required to conduct this study because the dataset was provided as anonymized data by NHO.¹⁵ This study was approved by the Ethics Committee of Mie Hospital (Approval No. 2020-89) on 23 October 2020. Specific permission to use MIA data was obtained from the NHO (Registration No. 1201003) on 1 December 2020.

Results

Table 1 shows the number of COVID-19 patients from National data and the numbers of inpatients, inpatients with oxygen therapy, and inpatients with respiratory ventilation in MIA data. These data show similar trends. Most notably, several surges occurred from January 2020 through March 2022, as shown in Figure 1. Figure 2 presents the COVID-19 fatality rate in National data and the rate of deaths among COVID-19 inpatients in MIA data. These data were less than 10%. They followed a similar trend. A supplementary table (see Extended data²¹), presents a summary of the estimation results obtained for the first step logistic regression for drug administration. Table 1 presents estimation results obtained for all inpatients, inpatients with oxygen therapy, and for inpatients who used a respiratory ventilation in three age classes. Results show 24 significance estimators of 90 estimators in all, but 20 estimators were positive, which indicates that drug administration was associated significantly with low probability of survival.

Table 1. Estimation results of propensity score matching method.

Age class	All		65 years old and older		Younger than 65 years old
Age class	difference	p-value	difference	p-value	difference	p-value
Remdesivir
Proportion
All	0.034	0.000	0.064	0.000	0.019	0.068
Oxygen administration	0.015	0.315	0.045	0.056	0.013	0.311
Respiratory ventilator used	0.012	0.768	0.127	0.080	N.A.	N.A.
Period
All	0.031	0.000	0.067	0.004	0.010	0.126
Oxygen administration	0.012	0.348	0.044	0.087	0.011	0.337
Respiratory ventilator used	-0.006	0.893	0.047	0.477	N.A.	N.A.
Dexamethasone
Proportion
All	0.027	0.000	0.079	0.000	0.003	0.253
Oxygen administration	0.020	0.013	0.052	0.000	0.006	0.234
Respiratory ventilator used	-0.021	0.548	-0.055	0.296	0.044	0.319
Period
All	0.029	0.000	0.076	0.000	0.004	0.119
Oxygen administration	0.014	0.090	0.038	0.011	0.006	0.131
Vent	-0.051	0.158	-0.083	0.125	0.008	0.857
Tocilizumab
Proportion
All	0.091	0.000	0.214	0.000	0.019	0.057
Oxygen administration	0.079	0.001	0.126	0.021	N.A.	N.A.
Respiratory ventilator used	0.117	0.227	N.A.	N.A.	-0.077	0.030
Period
All	0.103	0.000	0.223	0.000	0.025	0.030
Oxygen administration	0.086	0.001	0.126	0.020	N.A.	N.A.
Respiratory ventilator used	0.090	0.375	N.A.	N.A.	-0.062	0.079
Baricitinib
Proportion
All	0.023	0.100	0.411	0.114	N.A.	N.A.
Oxygen administration	-0.016	0.220	-0.033	0.900	-0.008	0.023
Respiratory ventilator used	-0.149	0.586	0.384	0.000	N.A.	N.A.
Period
All	N.A.	N.A.	N.A.	N.A.	N.A.	N.A.
Oxygen administration	N.A.	N.A.	N.A.	N.A.	N.A.	N.A.
Respiratory ventilator used	N.A.	N.A.	N.A.	N.A.	N.A.	N.A.
Sotrovimab or casirivimab/imdevimab
Proportion
All	N.A.	N.A.	-0.105	0.000	N.A.	N.A.
Oxygen administration	-0.085	0.000	N.A.	N.A.	N.A.	N.A.
Respiratory ventilator used	N.A.	N.A.	N.A.	N.A.	N.A.	N.A.
Period	N.A.	N.A.	N.A.	N.A.	N.A.	N.A.
All	N.A.	N.A.	N.A.	N.A.	N.A.	N.A.
Oxygen administration	N.A.	N.A.	N.A.	N.A.	N.A.	N.A.
Respiratory ventilator used	N.A.	N.A.	N.A.	N.A.	N.A.	N.A.

Figure 1. COVID-19 patients of National data and COVID-19 inpatients of MIA data.

Notes: Bars show the number of COVID-19 patients in National data (left scale). Orange, gray, and yellow lines respectively show the numbers of COVID-19 inpatients in MIA, COVID-19 inpatients with oxygen therapy, and COVID-19 inpatients with respiratory ventilation (right scale).

Figure 2. Fatality rate for National data and Rate of deaths among COVID-19 inpatients in MIA.

Notes: Blue and orange lines respectively denote the rate of deaths among COVID-19 inpatients in MIA and the COVID-19 fatality rate in National data. Data for COVID-19 patients and deaths in 0–18 week in National data were unavailable.

Four estimators indicating a drug as probably related to high probability of survival were the following: tocilizumab among younger inpatients with ventilation; baricitinib among younger inpatients with oxygen therapy; and antibody cocktails among all older inpatients and among all inpatients with oxygen therapy. Of the three drugs, only the antibody cocktail had no positive difference, but association with an increased probability of survival. Some treatment effects of tocilizumab and baricitinib were positive, especially for all older inpatients or for older inpatients with moderate severity. Therefore, the results obtained for these two drugs were mixed.

Discussion

We estimated drug effectiveness separately. Results showed that antibody cocktails might contribute to survival. The estimated results for tocilizumab and baricitinib were mixed, but these drugs were presumed to provide some probability of survival among younger but severely ill patients. Nevertheless, we were unable to find any evidence that remdesivir or dexamethasone contributes to survival at all. We must assess the drugs’ effectiveness after approval using observational data because factors such as the emergence of mutated strains, vaccination, and development of therapies can affect drug efficacy.

These counterintuitive findings, which are inconsistent with results of earlier studies indicating the effectiveness of remdesivir and dexamethasone, might be attributable to several limitations. First, worse matching at the first step could engender biased results.²^,⁵ The propensity score matching method could engender biased results because of lack of data such as detailed information related to intervention and potential confounders. This bias results from insufficient data such as potential confounders we did not measure and from a lack of detailed information related to interventions (high flow nasal canula or extracorporeal membrane oxygenation) and underlying diseases recognized as risk factors for severe COVID-19 (obesity or immunosuppression). Second, we examined the data, particularly using Japanese vaccine coverage instead of the vaccination history of the patients. We also used data of the prevalence of mutated strains in Tokyo as causative strains. If the vaccine history and causative strains of the patients themselves were available, then the results might be different. Third, this study was an observational study that was able to show only association between considered drugs and death, not necessarily a causal association. Fourth, endogeneity might not be controlled well in decisions to use drugs. Drugs to be administered against COVID-19 depend on the severity of illness of the patient. For instance, an earlier report described dexamethasone as effective for moderate or severe COVID-19, but as ineffective for mild cases.⁵ Actually, more than antibody cocktails,³^,⁴ remdesivir² and dexamethasone⁵ tend to be administered more for severely ill patients. Finally, to simplify the analysis used for this study, we did not incorporate the combination of drugs or change from one considered drug to another considered drug. That choice might bias the results to some degree. Such a process or pattern of drug administration might be important to evaluate drug effectiveness. Intercorrelation among drugs should be incorporated into the estimation model in future studies.

In our results, only antibody cocktails were found to be associated with high probability of survival. An earlier study indicated that casirivimab/imdevimab, an antibody cocktail, lowered the viral load.³ Typically, these antibody cocktails were given to patients with mild severity, which means that they were administered before a marked increase in viral load. This pattern of administration might be one reason why antibody cocktails were effective for survival. The antibody cocktail results included many “not available” entries, although drug effectiveness was confirmed in well-matched cases. The approval of antibody cocktails was late. The number of prescriptions was simply low, which might affect many cases for which “not available” was reported. In Japan, remdesivir, baricitinib, casirivimab/imdevimab, sotrovimab, and tocilizumab were approved for treatment of COVID-19, respectively, in May 2020, April 2021, July 2021, September 2021, and January 2022.¹⁹ Data accumulation might resolve that shortcoming.

We used MIA data of inpatients to verify drug effectiveness. In our earlier study, although we confirmed the representativeness of fatality cases using MIA data, the data on the numbers of patients with oxygen therapy and respiratory ventilation in Japan have not been published, yet.²⁰ We have not verified the representativeness of patients with oxygen therapy and respiratory ventilation. Moreover, because MIA is a database of medical claims, data of the prior few weeks might change during a few months. For this study, we collected and analyzed data from January 2020 through the end of March 2022, recorded as of May 2022. If the study period was extended, the estimation results might change over time because the accumulated data engender more accurate results. Emerging new mutated strains, immunization, and development of new therapies might affect the results. Inclusion of additional data and extending the study period remain as challenges for future research.

Conclusion

The obtained results demonstrated that antibody cocktails for all older patients and all inpatients with moderate severity, tocilizumab for severely ill younger inpatients, and baricitinib for younger inpatients with moderate severity might contribute to survival. For the latter two drugs, tocilizumab and baricitinib, some treatment effects suggest a reduction of survival for all or older inpatients. We infer that the effects of these drugs were mixed. In fact, remdesivir and dexamethasone might not contribute to survival. However, more information, including test results, is needed for better matching and for the achievement of definitive conclusions.

Data availability

Underlying data

The data used for this study were provided by the National Hospital Organization (NHO), but the availability of these data was restricted because of privacy policies of NHO. These data were used under license for this study and are therefore not available to the public. Authors were permitted to use these data by the Ethical Committee of Mie Hospital and NHO Committee.

Extended data

Figshare: Drug effectiveness for COVID-19 inpatients inferred from Japanese medical claim data using propensity score matching_supplement data.csv https://doi.org/10.6084/m9.figshare.22102016.v1.²¹

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

References

1. Miyashita K, Hozumi H, Furuhashi K, et al.: Changes in the characteristics and outcomes of COVID-19 patients from the early pandemic to the delta variant epidemic: a nationwide population-based study. Emerg. Microbes. Infect. 2023; 12(1): 2155250. PubMed Abstract | Publisher Full Text | Free Full Text
2. Mozaffari E, Chandak A, Zhang Z, et al.: Remdesivir Treatment in Hospitalized Patients With Coronavirus Disease 2019 (COVID-19): A Comparative Analysis of In-hospital All-cause Mortality in a Large Multicenter Observational Cohort. Clin. Infect. Dis. 2022; 75(1): e450–e458. PubMed Abstract | Publisher Full Text | Free Full Text
3. Weinreich DM, Sivapalasingam S, Norton T, et al.: REGEN-COV Antibody Combination and Outcomes in Outpatients with Covid-19. N. Engl. J. Med. 2021; 385(23): e81. Epub 20210929. PubMed Abstract | Publisher Full Text | Free Full Text
4. Gupta A, Gonzalez-Rojas Y, Juarez E, et al.: Early Treatment for Covid-19 with SARS-CoV-2 Neutralizing Antibody Sotrovimab. N. Engl. J. Med. 2021; 385(21): 1941–1950. PubMed Abstract | Publisher Full Text
5. Horby P, Lim WS, Emberson JR, et al.: Dexamethasone in Hospitalized Patients with Covid-19. N. Engl. J. Med. 2021; 384(8): 693–704. PubMed Abstract | Publisher Full Text
6. Marconi VC, Ramanan AV, de Bono S , et al.: Efficacy and safety of baricitinib for the treatment of hospitalised adults with COVID-19 (COV-BARRIER): a randomised, double-blind, parallel-group, placebo-controlled phase 3 trial. Lancet Respir. Med. 2021; 9(12): 1407–1418. PubMed Abstract | Publisher Full Text | Free Full Text
7. Salama C, Han J, Yau L, et al.: Tocilizumab in Patients Hospitalized with Covid-19 Pneumonia. N. Engl. J. Med. 2021; 384(1): 20–30. PubMed Abstract | Publisher Full Text | Free Full Text
8. Gordon AC, Mouncey PR, Al-Beidh F, et al.: Interleukin-6 Receptor Antagonists in Critically Ill Patients with Covid-19. N. Engl. J. Med. 2021; 384(16): 1491–1502. PubMed Abstract | Publisher Full Text
9. Umakanthan S, Senthil S, John S, et al.: The Effect of Statins on Clinical Outcome Among Hospitalized Patients With COVID-19: A Multi-Centric Cohort Study. Front. Pharmacol. 2022; 13: 742273. Epub 20220705. PubMed Abstract | Publisher Full Text | Free Full Text
10. Bergqvist R, Ahlqvist VH, Lundberg M, et al.: HMG-CoA reductase inhibitors and COVID-19 mortality in Stockholm, Sweden: A registry-based cohort study. PLoS Med. 2021; 18(10): e1003820. Epub 20211014. PubMed Abstract | Publisher Full Text | Free Full Text
11. Wong CKH, Au ICH, Lau KTK, et al.: Real-world effectiveness of molnupiravir and nirmatrelvir plus ritonavir against mortality, hospitalisation, and in-hospital outcomes among community-dwelling, ambulatory patients with confirmed SARS-CoV-2 infection during the omicron wave in Hong Kong: an observational study. Lancet. 2022; 400(10359): 1213–1222. Publisher Full Text
12. Jayk Bernal A, Gomes da Silva MM, Musungaie DB, et al.: Molnupiravir for Oral Treatment of Covid-19 in Nonhospitalized Patients. N. Engl. J. Med. 2022; 386(6): 509–520. PubMed Abstract | Publisher Full Text | Free Full Text
13. Hammond J, Leister-Tebbe H, Gardner A, et al.: Oral Nirmatrelvir for High-Risk, Nonhospitalized Adults with Covid-19. N. Engl. J. Med. 2022; 386(15): 1397–1408. PubMed Abstract | Publisher Full Text | Free Full Text
14. 令和2 (2020) 年医療施設 (静態・動態) 調査 [Survey of Medical Institutions. 2020]. Tokyo: Ministry of Health, Labour, and Welfare; 2020 [cited 2022 Nov 10]. Reference Source
15. Kanazawa N, Tani T, Imai S, et al.: Existing Data Sources for Clinical Epidemiology: Database of the National Hospital Organization in Japan. Clin. Epidemiol. 2022; 14: 689–698. Publisher Full Text
16. Heckman JJ, Ichimura H, Todd PE: Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme. Rev. Econ. Stud. 1997; 64(4): 605–654. Publisher Full Text
17. Kane LT, Fang T, Galetta MS, et al.: Propensity Score Matching: A Statistical Method. Clin. Spine Surg. 2020; 33(3): 120–122. Publisher Full Text
18. Deb S, Austin PC, Tu JV, et al.: A Review of Propensity-Score Methods and Their Use in Cardiovascular Research. Can. J. Cardiol. 2016; 32(2): 259–265. PubMed Abstract | Publisher Full Text
19. Ministry of Health, Labour, and Welfare: Clinical Management of Patients with COVID-19 ver. 8.1. Tokyo: Ministry of Health, Labour, and Welfare (in Japanese); [cited 2022 Nov 8]. Reference Source
20. Mitsushima S, Horiguchi H, Taniguchi K: Alpha and Delta variants and vaccination effectiveness against severity in COVID-19 inpatients based on medical claims in Japan. GHM Open. 2023; 3(1): 28–36. Publisher Full Text
21. Mitsushima S, Horiguchi H, Taniguchi K: Drug effectiveness for COVID-19 inpatients inferred from Japanese medical claim data using propensity score matching_supplement data.csv. Dataset. figshare. Publisher Full Text

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 13 Apr 2023

Author details Author details

Hiromasa Horiguchi
Roles: Conceptualization, Data Curation, Funding Acquisition, Resources, Software, Supervision, Validation, Writing – Review & Editing

Kiyosu Taniguchi
Roles: Conceptualization, Data Curation, Funding Acquisition, Methodology, Project Administration, Resources, Supervision, Validation, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

This work was supported by the Ministry of Health, Labour, and Welfare [grant number 20HA1005].
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (2)

version 2

Revised

Published: 22 Jan 2024, 12:398

https://doi.org/10.12688/f1000research.131102.2

version 1

Published: 13 Apr 2023, 12:398

https://doi.org/10.12688/f1000research.131102.1

© 2024 Mitsushima S et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Mitsushima S, Horiguchi H and Taniguchi K. Drug effectiveness for COVID-19 inpatients inferred from Japanese medical claim data using propensity score matching [version 2; peer review: 2 approved]. F1000Research 2024, 12:398 (https://doi.org/10.12688/f1000research.131102.2)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 2

VERSION 2

PUBLISHED 22 Jan 2024

Revised

Views

Reviewer Report 01 Aug 2024

Junko Kurita, Department of Nursing, Faculty of Sports & Health Science, Daitobunka University, Higashimatsuyama-shi, Japan

Approved

https://doi.org/10.5256/f1000research.161283.r305927

"Drug effectiveness for COVID-19 inpatients inferred from Japanese medical claim data using propensity score matching" was an excellent study investigating drug effectiveness in the real world. Even though it was approved through rct, effectiveness in the real world might be different from rct. So as to realize an experimental situation in the observational clinical data, they used propensity score matching procedure.
I congratulate the author's clear analysis in this field. However, I have some minor points to revise.

Minor points:

1) Figure 2 shows that there were COVID-19 patients and died in the first or second week, 2020, in MIA dara. I understand that the initial COVID-19 patient was confirmed on January 15, 2020 and thus the third or fourth week of 2020. Was the initial patient in MIA true ? Or had these patients administered in the first week of 2020 before COVID-19 infection, but been infected in COVID-19 in the hospital much later and died ?

2) Because the study period was from January 2020, was the period before approve mentor recommendation in guideline for COVID-19 included in the study period ? If so, it might cause some bias to the results. It might be just a comparison before and after approvement.

3) Though the yellow marks were very clear, but it might not be the usual presentation.
I recommended to remove yellow marks.

4) "Proportion" or "period" in Table 1 were unclear. Could you add words in Table and/or add more explanation about them in legend.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: statistical and/or mathematical analysis for medical data

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Respond or Comment

Views

Reviewer Report 11 Apr 2024

Ramesh Kandimalla, Biochemistry, Kakatiya Medical College, Warangal, Telengana, India

Approved

https://doi.org/10.5256/f1000research.161283.r239585

All the comments were addressed appropriately. ... Continue reading

CITE

Report a concern

Respond or Comment

Version 1

VERSION 1

PUBLISHED 13 Apr 2023

Views

Reviewer Report 10 Nov 2023

Ramesh Kandimalla, Biochemistry, Kakatiya Medical College, Warangal, Telengana, India

Approved with Reservations

https://doi.org/10.5256/f1000research.143911.r214688

The methods section is detailed, but it could benefit from additional clarity in explaining the rationale behind using propensity scoring and the choice of variables in logistic regression. This could help readers better understand the study design.

The methods section is detailed, but it could benefit from additional clarity in explaining the rationale behind using propensity scoring and the choice of variables in logistic regression. This could help readers better understand the study design.
It would be helpful to provide more information about the reliability and representativeness of the observational data from Japan. Mentioning any limitations or potential biases in the data source would enhance the transparency of the study.
The abstract implies causal relationships between drug administration and outcomes, but the observational nature of the study might only establish correlations. It's essential to acknowledge the limitations in inferring causation from observational data.
The abstract mentions that the logistic regression might have "insufficient explanatory power." It would be beneficial to elaborate on this limitation and discuss potential sources of bias or confounding that could affect the study's validity.
When discussing the drugs, providing a brief rationale for why certain drugs might be effective or ineffective could enhance the comprehensibility of the results.
The mention of the Omicron variant is timely, but given the evolving nature of the pandemic, it's important to note the potential for changes in outcomes as new variants emerge. This could affect the generalizability of the findings.
The abstract could include a brief section discussing the limitations of the study, such as potential confounders not accounted for in the analysis or any challenges in accurately measuring certain variables.
The use of phrases like "in Japan" and "against COVID-19" is repeated frequently. Adding variety to the language could enhance the readability of the introduction.
The timeline of events related to COVID-19 in Japan could be presented more clearly. For example, mentioning the initial surge in cases, the development of treatments, and the subsequent decline in mortality rate in a more chronological order could improve the flow.
The introduction references earlier studies and clinical trials but doesn't provide specific citations for the mentioned data or findings. Including citations within the text could enhance the credibility of the information presented.
While the introduction mentions the decline in the mortality rate due to COVID-19 in Japan, it might be beneficial to include specific statistics or percentages to quantify this change and emphasize the significance.
The term "real-world effectiveness" is used, but a brief explanation or definition could be included to clarify its meaning for readers who may not be familiar with the term.
The limitations of the MIA data, such as the absence of vaccination history or causative strain information, are mentioned but could be discussed more explicitly. Acknowledging these limitations upfront adds transparency to the study.
The transition from providing background information to discussing the methods is somewhat abrupt. Adding a sentence or two to smoothly transition into the methods section could improve the overall flow of the introduction.
While the introduction mentions propensity score matching, a brief explanation of how it works and its purpose in addressing selection bias could enhance reader understanding.
The introduction could touch upon the external validity of the study, acknowledging that findings may not be universally applicable due to variations in healthcare systems, populations, or treatment protocols.
The introduction could explicitly state the research question or hypothesis being addressed by the study to guide readers in understanding the study's purpose from the outset.
The text repeats certain points, such as the mention of "propensity score" and "saving life," multiple times. Reducing redundancy could improve the overall clarity and conciseness of the text.
Some sentences are quite complex, which might make it challenging for readers to follow the arguments. Simplifying the language without compromising precision could enhance comprehension.
The text occasionally uses language that implies causation, such as "might contribute to saving life." It's important to be cautious about making causal claims based on observational data. Acknowledging the limitations in establishing causation would strengthen the study's validity.
The text mentions "not available" entries in the results for antibody cocktails, attributing it to a small sample size. Providing a more explicit explanation of why certain data is missing and its potential impact on the results would enhance transparency.
The limitations section is placed towards the end of the text. Moving it closer to the beginning or integrating limitations as they become relevant in the discussion could improve the flow and help readers contextualize the findings.
The use of terms like "severely ill," "moderate severity," and "less–severe illness" could benefit from clearer operational definitions or references to specific criteria used for classification.
The discussion section covers a range of topics, including limitations and future research, without clear organization. Structuring the discussion into subsections could make it more reader-friendly.

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Partly
Are sufficient details of methods and analysis provided to allow replication by others?

Partly
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Partly

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: COVID-19, Diabetes, Alzheimer's, Mitochondria, Carbon dots, Nanospheres

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response 22 Jan 2024

Shingo Mitsushima, Center for Field Epidemic Intelligence, Research and Professional Development, National Institute of Infectious Diseases, Shinjuku-ku, 1620052, Japan

22 Jan 2024

Author Response

1. The methods section is detailed, but it could benefit from additional clarity in explaining the rationale behind using propensity scoring and the choice of variables in logistic regression. This ... Continue reading 1. The methods section is detailed, but it could benefit from additional clarity in explaining the rationale behind using propensity scoring and the choice of variables in logistic regression. This could help readers better understand the study design.

Response: We added the following sentence to Introduction.
“Moreover, after approval, experiments with random assignment are expected to be difficult to conduct in the real world because of ethical considerations: some patients might be unable to receive a potentially effective drug. Therefore, pseudo-experimental method should be applied to evaluate drug effectiveness in the real world.”

2. It would be helpful to provide more information about the reliability and representativeness of the observational data from Japan. Mentioning any limitations or potential biases in the data source would enhance the transparency of the study.

Response: We added the following sentence to Introduction.
“In our earlier study, although we confirmed the representativeness of fatality cases using MIA data, the data on the numbers of patients with oxygen therapy and respiratory ventilation in Japan have not been published, yet. We have not verified the representativeness of patients with oxygen therapy and respiratory ventilation.”

3. The abstract implies causal relationships between drug administration and outcomes, but the observational nature of the study might only establish correlations. It's essential to acknowledge the limitations in inferring causation from observational data.

Response: We revised sentences in Abstract as below.
OLD: “raised the probability to save life” “contribute to saving life”
NEW: “was associated with raising the probability of survival” “were related to reduced probability of survival”
In addition, we added the following sentence to Discussion.
“this study was an observational study that was able to show only association between considered drugs and death, not necessarily a causal association.”

4. The abstract mentions that the logistic regression might have "insufficient explanatory power." It would be beneficial to elaborate on this limitation and discuss potential sources of bias or confounding that could affect the study's validity.

Response: We added the following sentence to Abstract.
“However, propensity score matching method might engender biased results because of a lack of data such as detailed information related to intervention and potential confounders. Therefore, the effectiveness of some drugs might not be evaluated properly in this study.”

5. When discussing the drugs, providing a brief rationale for why certain drugs might be effective or ineffective could enhance the comprehensibility of the results.

Response: We added the following sentences to Discussion.
“An earlier study indicated that casirivimab/imdevimab, an antibody cocktail, lowered the viral load. Typically, these antibody cocktails were given to patients with mild severity, which means that they were administered before a marked increase in viral load. This pattern of administration might be one reason why antibody cocktails were effective for survival.”

6. The mention of the Omicron variant is timely, but given the evolving nature of the pandemic, it's important to note the potential for changes in outcomes as new variants emerge. This could affect the generalizability of the findings.

Response: We added the following sentences to Discussion.
“If the study period was extended, the estimation results might change over time because the accumulated data engender more accurate results. Emerging new mutated strains, immunization, and development of new therapies might affect the results.”

7. The abstract could include a brief section discussing the limitations of the study, such as potential confounders not accounted for in the analysis or any challenges in accurately measuring certain variables.

Response:
We added the following sentences to Abstract.
“However, propensity score matching method might engender biased results because of a lack of data such as detailed information related to intervention and potential confounders. Therefore, the effectiveness of some drugs might not be evaluated properly in this study.”

8. The use of phrases like "in Japan" and "against COVID-19" is repeated frequently. Adding variety to the language could enhance the readability of the introduction.

Response: We replaced some words “Japan” and “COVID-19” with alternative words.

9. The timeline of events related to COVID-19 in Japan could be presented more clearly. For example, mentioning the initial surge in cases, the development of treatments, and the subsequent decline in mortality rate in a more chronological order could improve the flow.

Response: We added Figure 1 and Figure 2 to show the timeline of COVID-19 events. We also added the following sentences to Result.
“Table 1 shows the number of COVID-19 patients from National data and the numbers of inpatients, inpatients with oxygen therapy, and inpatients with respiratory ventilation in MIA data. These data show similar trends. Most notably, several surges occurred from January 2020 through March 2022, as shown in Figure 1. Figure 2 presents the COVID-19 fatality rate in National data and the rate of deaths among COVID-19 inpatients in MIA data. These data were less than 10%. They followed a similar trend.”

10. The introduction references earlier studies and clinical trials but doesn't provide specific citations for the mentioned data or findings. Including citations within the text could enhance the credibility of the information presented.

Response: We added the following sentence to Introduction.
“In earlier studies, all these drugs were associated with improved survival or reduced the risk of disease progression, but antibody cocktails were effective only for mildly ill patients; dexamethasone was only effective for moderately to severely ill patients.”

11. While the introduction mentions the decline in the mortality rate due to COVID-19 in Japan, it might be beneficial to include specific statistics or percentages to quantify this change and emphasize the significance.

Response: We drew Figure 2, which shows fatality rates in National data and rate of deaths among COVID-19 inpatients with specific percentage.

12. The term "real-world effectiveness" is used, but a brief explanation or definition could be included to clarify its meaning for readers who may not be familiar with the term.

Response: We added the following sentences to Introduction.
“The objective of this study was evaluating the “real-world effectiveness of drugs against COVID-19 in Japan.” “Real-world effectiveness of drugs” means the effectiveness of drugs considering several factors including population characteristics, examinations, vaccinations, and healthcare system such as hospitalization and treatment protocols.”

13. The limitations of the MIA data, such as the absence of vaccination history or causative strain information, are mentioned but could be discussed more explicitly. Acknowledging these limitations upfront adds transparency to the study.

Response: We added the following sentences about the limitation of vaccination history and causative strain to Discussion.
“Second, we examined the data, particularly using Japanese vaccine coverage instead of the vaccination history of the patients. We also used data of the prevalence of mutated strains in Tokyo as causative strains. If the vaccine history and causative strains of the patients themselves were available, then the results might be different.”
We also added the sentences about the limitation of MIA data to Discussion.
“In our earlier study, although we confirmed the representativeness of fatality cases using MIA data, the data on the numbers of patients with oxygen therapy and respiratory ventilation in Japan have not been published, yet. We have not verified the representativeness of patients with oxygen therapy and respiratory ventilation. Moreover, because MIA is a database of medical claims, data of the prior few weeks might change during a few months. For this study, we collected and analyzed data from January 2020 through the end of March 2022, recorded as of May 2022. If the study period was extended, the estimation results might change over time because the accumulated data engender more accurate results. Emerging new mutated strains, immunization, and development of new therapies might affect the results.”

14. The transition from providing background information to discussing the methods is somewhat abrupt. Adding a sentence or two to smoothly transition into the methods section could improve the overall flow of the introduction.

Response: We added the following sentence to Introduction.
“An average treatment effect model including propensity matching was applied to create these situations.”

15. While the introduction mentions propensity score matching, a brief explanation of how it works and its purpose in addressing selection bias could enhance reader understanding.

Response: We added the following sentence to Abstract.
“We applied the average treatment effect model, particularly propensity scoring, which can treat the choice of administered drug as if administration were randomly assigned to inpatients.”

16. The introduction could touch upon the external validity of the study, acknowledging that findings may not be universally applicable due to variations in healthcare systems, populations, or treatment protocols.

Response: We added the following sentences to Introduction.
““Real-world effectiveness of drugs” means the effectiveness of drugs considering several factors including population characteristics, examinations, vaccinations, and healthcare system such as hospitalization and treatment protocols. Therefore, the results of this study might not be applicable to other countries.”

17. The introduction could explicitly state the research question or hypothesis being addressed by the study to guide readers in understanding the study's purpose from the outset.

Response: We added the following sentence to Introduction.
“The objective of this study was evaluating the “real-world effectiveness of drugs against COVID-19 in Japan.””

18. The text repeats certain points, such as the mention of "propensity score" and "saving life," multiple times. Reducing redundancy could improve the overall clarity and conciseness of the text.

Response: We did the English proofreading.

19. Some sentences are quite complex, which might make it challenging for readers to follow the arguments. Simplifying the language without compromising precision could enhance comprehension.

Response: We did the English proofreading.

20. The text occasionally uses language that implies causation, such as "might contribute to saving life." It's important to be cautious about making causal claims based on observational data. Acknowledging the limitations in establishing causation would strengthen the study's validity.

Response: We added the following sentence to Discussion.
“Third, this study was an observational study that was able to show only association between considered drugs and death, not necessarily a causal association.”

21. The text mentions "not available" entries in the results for antibody cocktails, attributing it to a small sample size. Providing a more explicit explanation of why certain data is missing and its potential impact on the results would enhance transparency.

Response: The approval of antibody cocktails was relatively late, and the number of prescriptions was simply low, so there is no potential impact. We added the following sentence in Discussion.
“The antibody cocktail results included many “not available” entries, although drug effectiveness was confirmed in well-matched cases. The approval of antibody cocktails was late. The number of prescriptions was simply low, which might affect many cases for which “not available” was reported.”

22. The limitations section is placed towards the end of the text. Moving it closer to the beginning or integrating limitations as they become relevant in the discussion could improve the flow and help readers contextualize the findings.

Response: We integrated limitations to Discussion and changed each limitation to understand easily for the reader.

23. The use of terms like "severely ill," "moderate severity," and "less–severe illness" could benefit from clearer operational definitions or references to specific criteria used for classification.

Response: We added the following sentence to Method.
“For this study, mild symptoms were defined as a patient who did not require medical intervention such as oxygen therapy or respiratory ventilation, moderate symptoms were defined as a patient who required oxygen administration, and severe symptoms were defined as a patient who required respiratory ventilation.”

24. The discussion section covers a range of topics, including limitations and future research, without clear organization. Structuring the discussion into subsections could make it more reader-friendly.

Response: We have organized discussion structure by issue.
1. The methods section is detailed, but it could benefit from additional clarity in explaining the rationale behind using propensity scoring and the choice of variables in logistic regression. This could help readers better understand the study design.

Response: We added the following sentence to Introduction.
“Moreover, after approval, experiments with random assignment are expected to be difficult to conduct in the real world because of ethical considerations: some patients might be unable to receive a potentially effective drug. Therefore, pseudo-experimental method should be applied to evaluate drug effectiveness in the real world.”

2. It would be helpful to provide more information about the reliability and representativeness of the observational data from Japan. Mentioning any limitations or potential biases in the data source would enhance the transparency of the study.

Response: We added the following sentence to Introduction.
“In our earlier study, although we confirmed the representativeness of fatality cases using MIA data, the data on the numbers of patients with oxygen therapy and respiratory ventilation in Japan have not been published, yet. We have not verified the representativeness of patients with oxygen therapy and respiratory ventilation.”

3. The abstract implies causal relationships between drug administration and outcomes, but the observational nature of the study might only establish correlations. It's essential to acknowledge the limitations in inferring causation from observational data.

Response: We revised sentences in Abstract as below.
OLD: “raised the probability to save life” “contribute to saving life”
NEW: “was associated with raising the probability of survival” “were related to reduced probability of survival”
In addition, we added the following sentence to Discussion.
“this study was an observational study that was able to show only association between considered drugs and death, not necessarily a causal association.”

4. The abstract mentions that the logistic regression might have "insufficient explanatory power." It would be beneficial to elaborate on this limitation and discuss potential sources of bias or confounding that could affect the study's validity.

Response: We added the following sentence to Abstract.
“However, propensity score matching method might engender biased results because of a lack of data such as detailed information related to intervention and potential confounders. Therefore, the effectiveness of some drugs might not be evaluated properly in this study.”

5. When discussing the drugs, providing a brief rationale for why certain drugs might be effective or ineffective could enhance the comprehensibility of the results.

Response: We added the following sentences to Discussion.
“An earlier study indicated that casirivimab/imdevimab, an antibody cocktail, lowered the viral load. Typically, these antibody cocktails were given to patients with mild severity, which means that they were administered before a marked increase in viral load. This pattern of administration might be one reason why antibody cocktails were effective for survival.”

6. The mention of the Omicron variant is timely, but given the evolving nature of the pandemic, it's important to note the potential for changes in outcomes as new variants emerge. This could affect the generalizability of the findings.

Response: We added the following sentences to Discussion.
“If the study period was extended, the estimation results might change over time because the accumulated data engender more accurate results. Emerging new mutated strains, immunization, and development of new therapies might affect the results.”

7. The abstract could include a brief section discussing the limitations of the study, such as potential confounders not accounted for in the analysis or any challenges in accurately measuring certain variables.

Response:
We added the following sentences to Abstract.
“However, propensity score matching method might engender biased results because of a lack of data such as detailed information related to intervention and potential confounders. Therefore, the effectiveness of some drugs might not be evaluated properly in this study.”

8. The use of phrases like "in Japan" and "against COVID-19" is repeated frequently. Adding variety to the language could enhance the readability of the introduction.

Response: We replaced some words “Japan” and “COVID-19” with alternative words.

9. The timeline of events related to COVID-19 in Japan could be presented more clearly. For example, mentioning the initial surge in cases, the development of treatments, and the subsequent decline in mortality rate in a more chronological order could improve the flow.

Response: We added Figure 1 and Figure 2 to show the timeline of COVID-19 events. We also added the following sentences to Result.
“Table 1 shows the number of COVID-19 patients from National data and the numbers of inpatients, inpatients with oxygen therapy, and inpatients with respiratory ventilation in MIA data. These data show similar trends. Most notably, several surges occurred from January 2020 through March 2022, as shown in Figure 1. Figure 2 presents the COVID-19 fatality rate in National data and the rate of deaths among COVID-19 inpatients in MIA data. These data were less than 10%. They followed a similar trend.”

10. The introduction references earlier studies and clinical trials but doesn't provide specific citations for the mentioned data or findings. Including citations within the text could enhance the credibility of the information presented.

Response: We added the following sentence to Introduction.
“In earlier studies, all these drugs were associated with improved survival or reduced the risk of disease progression, but antibody cocktails were effective only for mildly ill patients; dexamethasone was only effective for moderately to severely ill patients.”

11. While the introduction mentions the decline in the mortality rate due to COVID-19 in Japan, it might be beneficial to include specific statistics or percentages to quantify this change and emphasize the significance.

Response: We drew Figure 2, which shows fatality rates in National data and rate of deaths among COVID-19 inpatients with specific percentage.

12. The term "real-world effectiveness" is used, but a brief explanation or definition could be included to clarify its meaning for readers who may not be familiar with the term.

Response: We added the following sentences to Introduction.
“The objective of this study was evaluating the “real-world effectiveness of drugs against COVID-19 in Japan.” “Real-world effectiveness of drugs” means the effectiveness of drugs considering several factors including population characteristics, examinations, vaccinations, and healthcare system such as hospitalization and treatment protocols.”

13. The limitations of the MIA data, such as the absence of vaccination history or causative strain information, are mentioned but could be discussed more explicitly. Acknowledging these limitations upfront adds transparency to the study.

Response: We added the following sentences about the limitation of vaccination history and causative strain to Discussion.
“Second, we examined the data, particularly using Japanese vaccine coverage instead of the vaccination history of the patients. We also used data of the prevalence of mutated strains in Tokyo as causative strains. If the vaccine history and causative strains of the patients themselves were available, then the results might be different.”
We also added the sentences about the limitation of MIA data to Discussion.
“In our earlier study, although we confirmed the representativeness of fatality cases using MIA data, the data on the numbers of patients with oxygen therapy and respiratory ventilation in Japan have not been published, yet. We have not verified the representativeness of patients with oxygen therapy and respiratory ventilation. Moreover, because MIA is a database of medical claims, data of the prior few weeks might change during a few months. For this study, we collected and analyzed data from January 2020 through the end of March 2022, recorded as of May 2022. If the study period was extended, the estimation results might change over time because the accumulated data engender more accurate results. Emerging new mutated strains, immunization, and development of new therapies might affect the results.”

14. The transition from providing background information to discussing the methods is somewhat abrupt. Adding a sentence or two to smoothly transition into the methods section could improve the overall flow of the introduction.

Response: We added the following sentence to Introduction.
“An average treatment effect model including propensity matching was applied to create these situations.”

15. While the introduction mentions propensity score matching, a brief explanation of how it works and its purpose in addressing selection bias could enhance reader understanding.

Response: We added the following sentence to Abstract.
“We applied the average treatment effect model, particularly propensity scoring, which can treat the choice of administered drug as if administration were randomly assigned to inpatients.”

16. The introduction could touch upon the external validity of the study, acknowledging that findings may not be universally applicable due to variations in healthcare systems, populations, or treatment protocols.

Response: We added the following sentences to Introduction.
““Real-world effectiveness of drugs” means the effectiveness of drugs considering several factors including population characteristics, examinations, vaccinations, and healthcare system such as hospitalization and treatment protocols. Therefore, the results of this study might not be applicable to other countries.”

17. The introduction could explicitly state the research question or hypothesis being addressed by the study to guide readers in understanding the study's purpose from the outset.

Response: We added the following sentence to Introduction.
“The objective of this study was evaluating the “real-world effectiveness of drugs against COVID-19 in Japan.””

18. The text repeats certain points, such as the mention of "propensity score" and "saving life," multiple times. Reducing redundancy could improve the overall clarity and conciseness of the text.

Response: We did the English proofreading.

19. Some sentences are quite complex, which might make it challenging for readers to follow the arguments. Simplifying the language without compromising precision could enhance comprehension.

Response: We did the English proofreading.

20. The text occasionally uses language that implies causation, such as "might contribute to saving life." It's important to be cautious about making causal claims based on observational data. Acknowledging the limitations in establishing causation would strengthen the study's validity.

Response: We added the following sentence to Discussion.
“Third, this study was an observational study that was able to show only association between considered drugs and death, not necessarily a causal association.”

21. The text mentions "not available" entries in the results for antibody cocktails, attributing it to a small sample size. Providing a more explicit explanation of why certain data is missing and its potential impact on the results would enhance transparency.

Response: The approval of antibody cocktails was relatively late, and the number of prescriptions was simply low, so there is no potential impact. We added the following sentence in Discussion.
“The antibody cocktail results included many “not available” entries, although drug effectiveness was confirmed in well-matched cases. The approval of antibody cocktails was late. The number of prescriptions was simply low, which might affect many cases for which “not available” was reported.”

22. The limitations section is placed towards the end of the text. Moving it closer to the beginning or integrating limitations as they become relevant in the discussion could improve the flow and help readers contextualize the findings.

Response: We integrated limitations to Discussion and changed each limitation to understand easily for the reader.

23. The use of terms like "severely ill," "moderate severity," and "less–severe illness" could benefit from clearer operational definitions or references to specific criteria used for classification.

Response: We added the following sentence to Method.
“For this study, mild symptoms were defined as a patient who did not require medical intervention such as oxygen therapy or respiratory ventilation, moderate symptoms were defined as a patient who required oxygen administration, and severe symptoms were defined as a patient who required respiratory ventilation.”

24. The discussion section covers a range of topics, including limitations and future research, without clear organization. Structuring the discussion into subsections could make it more reader-friendly.

Response: We have organized discussion structure by issue.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 22 Jan 2024

Shingo Mitsushima, Center for Field Epidemic Intelligence, Research and Professional Development, National Institute of Infectious Diseases, Shinjuku-ku, 1620052, Japan

22 Jan 2024

Author Response

1. The methods section is detailed, but it could benefit from additional clarity in explaining the rationale behind using propensity scoring and the choice of variables in logistic regression. This ... Continue reading 1. The methods section is detailed, but it could benefit from additional clarity in explaining the rationale behind using propensity scoring and the choice of variables in logistic regression. This could help readers better understand the study design.

Response: We added the following sentence to Introduction.
“Moreover, after approval, experiments with random assignment are expected to be difficult to conduct in the real world because of ethical considerations: some patients might be unable to receive a potentially effective drug. Therefore, pseudo-experimental method should be applied to evaluate drug effectiveness in the real world.”

2. It would be helpful to provide more information about the reliability and representativeness of the observational data from Japan. Mentioning any limitations or potential biases in the data source would enhance the transparency of the study.

Response: We added the following sentence to Introduction.
“In our earlier study, although we confirmed the representativeness of fatality cases using MIA data, the data on the numbers of patients with oxygen therapy and respiratory ventilation in Japan have not been published, yet. We have not verified the representativeness of patients with oxygen therapy and respiratory ventilation.”

3. The abstract implies causal relationships between drug administration and outcomes, but the observational nature of the study might only establish correlations. It's essential to acknowledge the limitations in inferring causation from observational data.

Response: We revised sentences in Abstract as below.
OLD: “raised the probability to save life” “contribute to saving life”
NEW: “was associated with raising the probability of survival” “were related to reduced probability of survival”
In addition, we added the following sentence to Discussion.
“this study was an observational study that was able to show only association between considered drugs and death, not necessarily a causal association.”

4. The abstract mentions that the logistic regression might have "insufficient explanatory power." It would be beneficial to elaborate on this limitation and discuss potential sources of bias or confounding that could affect the study's validity.

Response: We added the following sentence to Abstract.
“However, propensity score matching method might engender biased results because of a lack of data such as detailed information related to intervention and potential confounders. Therefore, the effectiveness of some drugs might not be evaluated properly in this study.”

5. When discussing the drugs, providing a brief rationale for why certain drugs might be effective or ineffective could enhance the comprehensibility of the results.

Response: We added the following sentences to Discussion.
“An earlier study indicated that casirivimab/imdevimab, an antibody cocktail, lowered the viral load. Typically, these antibody cocktails were given to patients with mild severity, which means that they were administered before a marked increase in viral load. This pattern of administration might be one reason why antibody cocktails were effective for survival.”

6. The mention of the Omicron variant is timely, but given the evolving nature of the pandemic, it's important to note the potential for changes in outcomes as new variants emerge. This could affect the generalizability of the findings.

Response: We added the following sentences to Discussion.
“If the study period was extended, the estimation results might change over time because the accumulated data engender more accurate results. Emerging new mutated strains, immunization, and development of new therapies might affect the results.”

7. The abstract could include a brief section discussing the limitations of the study, such as potential confounders not accounted for in the analysis or any challenges in accurately measuring certain variables.

Response:
We added the following sentences to Abstract.
“However, propensity score matching method might engender biased results because of a lack of data such as detailed information related to intervention and potential confounders. Therefore, the effectiveness of some drugs might not be evaluated properly in this study.”

8. The use of phrases like "in Japan" and "against COVID-19" is repeated frequently. Adding variety to the language could enhance the readability of the introduction.

Response: We replaced some words “Japan” and “COVID-19” with alternative words.

9. The timeline of events related to COVID-19 in Japan could be presented more clearly. For example, mentioning the initial surge in cases, the development of treatments, and the subsequent decline in mortality rate in a more chronological order could improve the flow.

Response: We added Figure 1 and Figure 2 to show the timeline of COVID-19 events. We also added the following sentences to Result.
“Table 1 shows the number of COVID-19 patients from National data and the numbers of inpatients, inpatients with oxygen therapy, and inpatients with respiratory ventilation in MIA data. These data show similar trends. Most notably, several surges occurred from January 2020 through March 2022, as shown in Figure 1. Figure 2 presents the COVID-19 fatality rate in National data and the rate of deaths among COVID-19 inpatients in MIA data. These data were less than 10%. They followed a similar trend.”

10. The introduction references earlier studies and clinical trials but doesn't provide specific citations for the mentioned data or findings. Including citations within the text could enhance the credibility of the information presented.

Response: We added the following sentence to Introduction.
“In earlier studies, all these drugs were associated with improved survival or reduced the risk of disease progression, but antibody cocktails were effective only for mildly ill patients; dexamethasone was only effective for moderately to severely ill patients.”

11. While the introduction mentions the decline in the mortality rate due to COVID-19 in Japan, it might be beneficial to include specific statistics or percentages to quantify this change and emphasize the significance.

Response: We drew Figure 2, which shows fatality rates in National data and rate of deaths among COVID-19 inpatients with specific percentage.

12. The term "real-world effectiveness" is used, but a brief explanation or definition could be included to clarify its meaning for readers who may not be familiar with the term.

Response: We added the following sentences to Introduction.
“The objective of this study was evaluating the “real-world effectiveness of drugs against COVID-19 in Japan.” “Real-world effectiveness of drugs” means the effectiveness of drugs considering several factors including population characteristics, examinations, vaccinations, and healthcare system such as hospitalization and treatment protocols.”

13. The limitations of the MIA data, such as the absence of vaccination history or causative strain information, are mentioned but could be discussed more explicitly. Acknowledging these limitations upfront adds transparency to the study.

Response: We added the following sentences about the limitation of vaccination history and causative strain to Discussion.
“Second, we examined the data, particularly using Japanese vaccine coverage instead of the vaccination history of the patients. We also used data of the prevalence of mutated strains in Tokyo as causative strains. If the vaccine history and causative strains of the patients themselves were available, then the results might be different.”
We also added the sentences about the limitation of MIA data to Discussion.
“In our earlier study, although we confirmed the representativeness of fatality cases using MIA data, the data on the numbers of patients with oxygen therapy and respiratory ventilation in Japan have not been published, yet. We have not verified the representativeness of patients with oxygen therapy and respiratory ventilation. Moreover, because MIA is a database of medical claims, data of the prior few weeks might change during a few months. For this study, we collected and analyzed data from January 2020 through the end of March 2022, recorded as of May 2022. If the study period was extended, the estimation results might change over time because the accumulated data engender more accurate results. Emerging new mutated strains, immunization, and development of new therapies might affect the results.”

14. The transition from providing background information to discussing the methods is somewhat abrupt. Adding a sentence or two to smoothly transition into the methods section could improve the overall flow of the introduction.

Response: We added the following sentence to Introduction.
“An average treatment effect model including propensity matching was applied to create these situations.”

15. While the introduction mentions propensity score matching, a brief explanation of how it works and its purpose in addressing selection bias could enhance reader understanding.

Response: We added the following sentence to Abstract.
“We applied the average treatment effect model, particularly propensity scoring, which can treat the choice of administered drug as if administration were randomly assigned to inpatients.”

16. The introduction could touch upon the external validity of the study, acknowledging that findings may not be universally applicable due to variations in healthcare systems, populations, or treatment protocols.

Response: We added the following sentences to Introduction.
““Real-world effectiveness of drugs” means the effectiveness of drugs considering several factors including population characteristics, examinations, vaccinations, and healthcare system such as hospitalization and treatment protocols. Therefore, the results of this study might not be applicable to other countries.”

17. The introduction could explicitly state the research question or hypothesis being addressed by the study to guide readers in understanding the study's purpose from the outset.

Response: We added the following sentence to Introduction.
“The objective of this study was evaluating the “real-world effectiveness of drugs against COVID-19 in Japan.””

18. The text repeats certain points, such as the mention of "propensity score" and "saving life," multiple times. Reducing redundancy could improve the overall clarity and conciseness of the text.

Response: We did the English proofreading.

19. Some sentences are quite complex, which might make it challenging for readers to follow the arguments. Simplifying the language without compromising precision could enhance comprehension.

Response: We did the English proofreading.

20. The text occasionally uses language that implies causation, such as "might contribute to saving life." It's important to be cautious about making causal claims based on observational data. Acknowledging the limitations in establishing causation would strengthen the study's validity.

Response: We added the following sentence to Discussion.
“Third, this study was an observational study that was able to show only association between considered drugs and death, not necessarily a causal association.”

21. The text mentions "not available" entries in the results for antibody cocktails, attributing it to a small sample size. Providing a more explicit explanation of why certain data is missing and its potential impact on the results would enhance transparency.

Response: The approval of antibody cocktails was relatively late, and the number of prescriptions was simply low, so there is no potential impact. We added the following sentence in Discussion.
“The antibody cocktail results included many “not available” entries, although drug effectiveness was confirmed in well-matched cases. The approval of antibody cocktails was late. The number of prescriptions was simply low, which might affect many cases for which “not available” was reported.”

22. The limitations section is placed towards the end of the text. Moving it closer to the beginning or integrating limitations as they become relevant in the discussion could improve the flow and help readers contextualize the findings.

Response: We integrated limitations to Discussion and changed each limitation to understand easily for the reader.

23. The use of terms like "severely ill," "moderate severity," and "less–severe illness" could benefit from clearer operational definitions or references to specific criteria used for classification.

Response: We added the following sentence to Method.
“For this study, mild symptoms were defined as a patient who did not require medical intervention such as oxygen therapy or respiratory ventilation, moderate symptoms were defined as a patient who required oxygen administration, and severe symptoms were defined as a patient who required respiratory ventilation.”

24. The discussion section covers a range of topics, including limitations and future research, without clear organization. Structuring the discussion into subsections could make it more reader-friendly.

Response: We have organized discussion structure by issue.
1. The methods section is detailed, but it could benefit from additional clarity in explaining the rationale behind using propensity scoring and the choice of variables in logistic regression. This could help readers better understand the study design.

Response: We added the following sentence to Introduction.
“Moreover, after approval, experiments with random assignment are expected to be difficult to conduct in the real world because of ethical considerations: some patients might be unable to receive a potentially effective drug. Therefore, pseudo-experimental method should be applied to evaluate drug effectiveness in the real world.”

2. It would be helpful to provide more information about the reliability and representativeness of the observational data from Japan. Mentioning any limitations or potential biases in the data source would enhance the transparency of the study.

Response: We added the following sentence to Introduction.
“In our earlier study, although we confirmed the representativeness of fatality cases using MIA data, the data on the numbers of patients with oxygen therapy and respiratory ventilation in Japan have not been published, yet. We have not verified the representativeness of patients with oxygen therapy and respiratory ventilation.”

3. The abstract implies causal relationships between drug administration and outcomes, but the observational nature of the study might only establish correlations. It's essential to acknowledge the limitations in inferring causation from observational data.

Response: We revised sentences in Abstract as below.
OLD: “raised the probability to save life” “contribute to saving life”
NEW: “was associated with raising the probability of survival” “were related to reduced probability of survival”
In addition, we added the following sentence to Discussion.
“this study was an observational study that was able to show only association between considered drugs and death, not necessarily a causal association.”

4. The abstract mentions that the logistic regression might have "insufficient explanatory power." It would be beneficial to elaborate on this limitation and discuss potential sources of bias or confounding that could affect the study's validity.

Response: We added the following sentence to Abstract.
“However, propensity score matching method might engender biased results because of a lack of data such as detailed information related to intervention and potential confounders. Therefore, the effectiveness of some drugs might not be evaluated properly in this study.”

5. When discussing the drugs, providing a brief rationale for why certain drugs might be effective or ineffective could enhance the comprehensibility of the results.

Response: We added the following sentences to Discussion.
“An earlier study indicated that casirivimab/imdevimab, an antibody cocktail, lowered the viral load. Typically, these antibody cocktails were given to patients with mild severity, which means that they were administered before a marked increase in viral load. This pattern of administration might be one reason why antibody cocktails were effective for survival.”

6. The mention of the Omicron variant is timely, but given the evolving nature of the pandemic, it's important to note the potential for changes in outcomes as new variants emerge. This could affect the generalizability of the findings.

Response: We added the following sentences to Discussion.
“If the study period was extended, the estimation results might change over time because the accumulated data engender more accurate results. Emerging new mutated strains, immunization, and development of new therapies might affect the results.”

7. The abstract could include a brief section discussing the limitations of the study, such as potential confounders not accounted for in the analysis or any challenges in accurately measuring certain variables.

Response:
We added the following sentences to Abstract.
“However, propensity score matching method might engender biased results because of a lack of data such as detailed information related to intervention and potential confounders. Therefore, the effectiveness of some drugs might not be evaluated properly in this study.”

8. The use of phrases like "in Japan" and "against COVID-19" is repeated frequently. Adding variety to the language could enhance the readability of the introduction.

Response: We replaced some words “Japan” and “COVID-19” with alternative words.

9. The timeline of events related to COVID-19 in Japan could be presented more clearly. For example, mentioning the initial surge in cases, the development of treatments, and the subsequent decline in mortality rate in a more chronological order could improve the flow.

Response: We added Figure 1 and Figure 2 to show the timeline of COVID-19 events. We also added the following sentences to Result.
“Table 1 shows the number of COVID-19 patients from National data and the numbers of inpatients, inpatients with oxygen therapy, and inpatients with respiratory ventilation in MIA data. These data show similar trends. Most notably, several surges occurred from January 2020 through March 2022, as shown in Figure 1. Figure 2 presents the COVID-19 fatality rate in National data and the rate of deaths among COVID-19 inpatients in MIA data. These data were less than 10%. They followed a similar trend.”

10. The introduction references earlier studies and clinical trials but doesn't provide specific citations for the mentioned data or findings. Including citations within the text could enhance the credibility of the information presented.

Response: We added the following sentence to Introduction.
“In earlier studies, all these drugs were associated with improved survival or reduced the risk of disease progression, but antibody cocktails were effective only for mildly ill patients; dexamethasone was only effective for moderately to severely ill patients.”

11. While the introduction mentions the decline in the mortality rate due to COVID-19 in Japan, it might be beneficial to include specific statistics or percentages to quantify this change and emphasize the significance.

Response: We drew Figure 2, which shows fatality rates in National data and rate of deaths among COVID-19 inpatients with specific percentage.

12. The term "real-world effectiveness" is used, but a brief explanation or definition could be included to clarify its meaning for readers who may not be familiar with the term.

Response: We added the following sentences to Introduction.
“The objective of this study was evaluating the “real-world effectiveness of drugs against COVID-19 in Japan.” “Real-world effectiveness of drugs” means the effectiveness of drugs considering several factors including population characteristics, examinations, vaccinations, and healthcare system such as hospitalization and treatment protocols.”

13. The limitations of the MIA data, such as the absence of vaccination history or causative strain information, are mentioned but could be discussed more explicitly. Acknowledging these limitations upfront adds transparency to the study.

Response: We added the following sentences about the limitation of vaccination history and causative strain to Discussion.
“Second, we examined the data, particularly using Japanese vaccine coverage instead of the vaccination history of the patients. We also used data of the prevalence of mutated strains in Tokyo as causative strains. If the vaccine history and causative strains of the patients themselves were available, then the results might be different.”
We also added the sentences about the limitation of MIA data to Discussion.
“In our earlier study, although we confirmed the representativeness of fatality cases using MIA data, the data on the numbers of patients with oxygen therapy and respiratory ventilation in Japan have not been published, yet. We have not verified the representativeness of patients with oxygen therapy and respiratory ventilation. Moreover, because MIA is a database of medical claims, data of the prior few weeks might change during a few months. For this study, we collected and analyzed data from January 2020 through the end of March 2022, recorded as of May 2022. If the study period was extended, the estimation results might change over time because the accumulated data engender more accurate results. Emerging new mutated strains, immunization, and development of new therapies might affect the results.”

14. The transition from providing background information to discussing the methods is somewhat abrupt. Adding a sentence or two to smoothly transition into the methods section could improve the overall flow of the introduction.

Response: We added the following sentence to Introduction.
“An average treatment effect model including propensity matching was applied to create these situations.”

15. While the introduction mentions propensity score matching, a brief explanation of how it works and its purpose in addressing selection bias could enhance reader understanding.

Response: We added the following sentence to Abstract.
“We applied the average treatment effect model, particularly propensity scoring, which can treat the choice of administered drug as if administration were randomly assigned to inpatients.”

16. The introduction could touch upon the external validity of the study, acknowledging that findings may not be universally applicable due to variations in healthcare systems, populations, or treatment protocols.

Response: We added the following sentences to Introduction.
““Real-world effectiveness of drugs” means the effectiveness of drugs considering several factors including population characteristics, examinations, vaccinations, and healthcare system such as hospitalization and treatment protocols. Therefore, the results of this study might not be applicable to other countries.”

17. The introduction could explicitly state the research question or hypothesis being addressed by the study to guide readers in understanding the study's purpose from the outset.

Response: We added the following sentence to Introduction.
“The objective of this study was evaluating the “real-world effectiveness of drugs against COVID-19 in Japan.””

18. The text repeats certain points, such as the mention of "propensity score" and "saving life," multiple times. Reducing redundancy could improve the overall clarity and conciseness of the text.

Response: We did the English proofreading.

19. Some sentences are quite complex, which might make it challenging for readers to follow the arguments. Simplifying the language without compromising precision could enhance comprehension.

Response: We did the English proofreading.

20. The text occasionally uses language that implies causation, such as "might contribute to saving life." It's important to be cautious about making causal claims based on observational data. Acknowledging the limitations in establishing causation would strengthen the study's validity.

Response: We added the following sentence to Discussion.
“Third, this study was an observational study that was able to show only association between considered drugs and death, not necessarily a causal association.”

21. The text mentions "not available" entries in the results for antibody cocktails, attributing it to a small sample size. Providing a more explicit explanation of why certain data is missing and its potential impact on the results would enhance transparency.

Response: The approval of antibody cocktails was relatively late, and the number of prescriptions was simply low, so there is no potential impact. We added the following sentence in Discussion.
“The antibody cocktail results included many “not available” entries, although drug effectiveness was confirmed in well-matched cases. The approval of antibody cocktails was late. The number of prescriptions was simply low, which might affect many cases for which “not available” was reported.”

22. The limitations section is placed towards the end of the text. Moving it closer to the beginning or integrating limitations as they become relevant in the discussion could improve the flow and help readers contextualize the findings.

Response: We integrated limitations to Discussion and changed each limitation to understand easily for the reader.

23. The use of terms like "severely ill," "moderate severity," and "less–severe illness" could benefit from clearer operational definitions or references to specific criteria used for classification.

Response: We added the following sentence to Method.
“For this study, mild symptoms were defined as a patient who did not require medical intervention such as oxygen therapy or respiratory ventilation, moderate symptoms were defined as a patient who required oxygen administration, and severe symptoms were defined as a patient who required respiratory ventilation.”

24. The discussion section covers a range of topics, including limitations and future research, without clear organization. Structuring the discussion into subsections could make it more reader-friendly.

Response: We have organized discussion structure by issue.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 13 Apr 2023

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 2 (revision) 22 Jan 24	read	read
Version 1 13 Apr 23	read

Ramesh Kandimalla, Kakatiya Medical College, Warangal, India
Junko Kurita, Daitobunka University, Higashimatsuyama-shi, Japan

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

4 Views

01 Aug 2024 | for Version 2

Junko Kurita, Department of Nursing, Faculty of Sports & Health Science, Daitobunka University, Higashimatsuyama-shi, Japan

4 Views Cite this report Responses(0)

Approved

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

statistical and/or mathematical analysis for medical data

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

9 Views

11 Apr 2024 | for Version 2

Ramesh Kandimalla, Biochemistry, Kakatiya Medical College, Warangal, Telengana, India

9 Views Cite this report Responses(0)

Approved

All the comments were addressed appropriately. No more additional comments for me

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

COVID-19, Diabetes, Alzheimer's, Mitochondria, Carbon dots, Nanospheres

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

37 Views

10 Nov 2023 | for Version 1

Ramesh Kandimalla, Biochemistry, Kakatiya Medical College, Warangal, Telengana, India

37 Views Cite this report Responses(1)

Approved With Reservations

The methods section is detailed, but it could benefit from additional clarity in explaining the rationale behind using propensity scoring and the choice of variables in logistic regression. This could help readers better understand the study design.
It would be helpful to provide more information about the reliability and representativeness of the observational data from Japan. Mentioning any limitations or potential biases in the data source would enhance the transparency of the study.
The abstract implies causal relationships between drug administration and outcomes, but the observational nature of the study might only establish correlations. It's essential to acknowledge the limitations in inferring causation from observational data.
The abstract mentions that the logistic regression might have "insufficient explanatory power." It would be beneficial to elaborate on this limitation and discuss potential sources of bias or confounding that could affect the study's validity.
When discussing the drugs, providing a brief rationale for why certain drugs might be effective or ineffective could enhance the comprehensibility of the results.
The mention of the Omicron variant is timely, but given the evolving nature of the pandemic, it's important to note the potential for changes in outcomes as new variants emerge. This could affect the generalizability of the findings.
The abstract could include a brief section discussing the limitations of the study, such as potential confounders not accounted for in the analysis or any challenges in accurately measuring certain variables.
The use of phrases like "in Japan" and "against COVID-19" is repeated frequently. Adding variety to the language could enhance the readability of the introduction.
The timeline of events related to COVID-19 in Japan could be presented more clearly. For example, mentioning the initial surge in cases, the development of treatments, and the subsequent decline in mortality rate in a more chronological order could improve the flow.
The introduction references earlier studies and clinical trials but doesn't provide specific citations for the mentioned data or findings. Including citations within the text could enhance the credibility of the information presented.
While the introduction mentions the decline in the mortality rate due to COVID-19 in Japan, it might be beneficial to include specific statistics or percentages to quantify this change and emphasize the significance.
The term "real-world effectiveness" is used, but a brief explanation or definition could be included to clarify its meaning for readers who may not be familiar with the term.
The limitations of the MIA data, such as the absence of vaccination history or causative strain information, are mentioned but could be discussed more explicitly. Acknowledging these limitations upfront adds transparency to the study.
The transition from providing background information to discussing the methods is somewhat abrupt. Adding a sentence or two to smoothly transition into the methods section could improve the overall flow of the introduction.
While the introduction mentions propensity score matching, a brief explanation of how it works and its purpose in addressing selection bias could enhance reader understanding.
The introduction could touch upon the external validity of the study, acknowledging that findings may not be universally applicable due to variations in healthcare systems, populations, or treatment protocols.
The introduction could explicitly state the research question or hypothesis being addressed by the study to guide readers in understanding the study's purpose from the outset.
The text repeats certain points, such as the mention of "propensity score" and "saving life," multiple times. Reducing redundancy could improve the overall clarity and conciseness of the text.
Some sentences are quite complex, which might make it challenging for readers to follow the arguments. Simplifying the language without compromising precision could enhance comprehension.
The text occasionally uses language that implies causation, such as "might contribute to saving life." It's important to be cautious about making causal claims based on observational data. Acknowledging the limitations in establishing causation would strengthen the study's validity.
The text mentions "not available" entries in the results for antibody cocktails, attributing it to a small sample size. Providing a more explicit explanation of why certain data is missing and its potential impact on the results would enhance transparency.
The limitations section is placed towards the end of the text. Moving it closer to the beginning or integrating limitations as they become relevant in the discussion could improve the flow and help readers contextualize the findings.
The use of terms like "severely ill," "moderate severity," and "less–severe illness" could benefit from clearer operational definitions or references to specific criteria used for classification.
The discussion section covers a range of topics, including limitations and future research, without clear organization. Structuring the discussion into subsections could make it more reader-friendly.

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Partly
Are sufficient details of methods and analysis provided to allow replication by others?

Partly
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Partly

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

COVID-19, Diabetes, Alzheimer's, Mitochondria, Carbon dots, Nanospheres

Respond to this report

Responses (1)

Author Response

22 Jan 2024

Shingo Mitsushima, Center for Field Epidemic Intelligence, Research and Professional Development, National Institute of Infectious Diseases, Shinjuku-ku, 1620052, Japan

1. The methods section is detailed, but it could benefit from additional clarity in explaining the rationale behind using propensity scoring and the choice of variables in logistic regression. This could help readers better understand the study design.

Response: We added the following sentence to Introduction.
“Moreover, after approval, experiments with random assignment are expected to be difficult to conduct in the real world because of ethical considerations: some patients might be unable to receive a potentially effective drug. Therefore, pseudo-experimental method should be applied to evaluate drug effectiveness in the real world.”

2. It would be helpful to provide more information about the reliability and representativeness of the observational data from Japan. Mentioning any limitations or potential biases in the data source would enhance the transparency of the study.

Response: We added the following sentence to Introduction.
“In our earlier study, although we confirmed the representativeness of fatality cases using MIA data, the data on the numbers of patients with oxygen therapy and respiratory ventilation in Japan have not been published, yet. We have not verified the representativeness of patients with oxygen therapy and respiratory ventilation.”

3. The abstract implies causal relationships between drug administration and outcomes, but the observational nature of the study might only establish correlations. It's essential to acknowledge the limitations in inferring causation from observational data.

Response: We revised sentences in Abstract as below.
OLD: “raised the probability to save life” “contribute to saving life”
NEW: “was associated with raising the probability of survival” “were related to reduced probability of survival”
In addition, we added the following sentence to Discussion.
“this study was an observational study that was able to show only association between considered drugs and death, not necessarily a causal association.”

4. The abstract mentions that the logistic regression might have "insufficient explanatory power." It would be beneficial to elaborate on this limitation and discuss potential sources of bias or confounding that could affect the study's validity.

Response: We added the following sentence to Abstract.
“However, propensity score matching method might engender biased results because of a lack of data such as detailed information related to intervention and potential confounders. Therefore, the effectiveness of some drugs might not be evaluated properly in this study.”

5. When discussing the drugs, providing a brief rationale for why certain drugs might be effective or ineffective could enhance the comprehensibility of the results.

Response: We added the following sentences to Discussion.
“An earlier study indicated that casirivimab/imdevimab, an antibody cocktail, lowered the viral load. Typically, these antibody cocktails were given to patients with mild severity, which means that they were administered before a marked increase in viral load. This pattern of administration might be one reason why antibody cocktails were effective for survival.”

6. The mention of the Omicron variant is timely, but given the evolving nature of the pandemic, it's important to note the potential for changes in outcomes as new variants emerge. This could affect the generalizability of the findings.

Response: We added the following sentences to Discussion.
“If the study period was extended, the estimation results might change over time because the accumulated data engender more accurate results. Emerging new mutated strains, immunization, and development of new therapies might affect the results.”

7. The abstract could include a brief section discussing the limitations of the study, such as potential confounders not accounted for in the analysis or any challenges in accurately measuring certain variables.

Response:
We added the following sentences to Abstract.
“However, propensity score matching method might engender biased results because of a lack of data such as detailed information related to intervention and potential confounders. Therefore, the effectiveness of some drugs might not be evaluated properly in this study.”

8. The use of phrases like "in Japan" and "against COVID-19" is repeated frequently. Adding variety to the language could enhance the readability of the introduction.

Response: We replaced some words “Japan” and “COVID-19” with alternative words.

9. The timeline of events related to COVID-19 in Japan could be presented more clearly. For example, mentioning the initial surge in cases, the development of treatments, and the subsequent decline in mortality rate in a more chronological order could improve the flow.

Response: We added Figure 1 and Figure 2 to show the timeline of COVID-19 events. We also added the following sentences to Result.
“Table 1 shows the number of COVID-19 patients from National data and the numbers of inpatients, inpatients with oxygen therapy, and inpatients with respiratory ventilation in MIA data. These data show similar trends. Most notably, several surges occurred from January 2020 through March 2022, as shown in Figure 1. Figure 2 presents the COVID-19 fatality rate in National data and the rate of deaths among COVID-19 inpatients in MIA data. These data were less than 10%. They followed a similar trend.”

10. The introduction references earlier studies and clinical trials but doesn't provide specific citations for the mentioned data or findings. Including citations within the text could enhance the credibility of the information presented.

Response: We added the following sentence to Introduction.
“In earlier studies, all these drugs were associated with improved survival or reduced the risk of disease progression, but antibody cocktails were effective only for mildly ill patients; dexamethasone was only effective for moderately to severely ill patients.”

11. While the introduction mentions the decline in the mortality rate due to COVID-19 in Japan, it might be beneficial to include specific statistics or percentages to quantify this change and emphasize the significance.

Response: We drew Figure 2, which shows fatality rates in National data and rate of deaths among COVID-19 inpatients with specific percentage.

12. The term "real-world effectiveness" is used, but a brief explanation or definition could be included to clarify its meaning for readers who may not be familiar with the term.

Response: We added the following sentences to Introduction.
“The objective of this study was evaluating the “real-world effectiveness of drugs against COVID-19 in Japan.” “Real-world effectiveness of drugs” means the effectiveness of drugs considering several factors including population characteristics, examinations, vaccinations, and healthcare system such as hospitalization and treatment protocols.”

13. The limitations of the MIA data, such as the absence of vaccination history or causative strain information, are mentioned but could be discussed more explicitly. Acknowledging these limitations upfront adds transparency to the study.

Response: We added the following sentences about the limitation of vaccination history and causative strain to Discussion.
“Second, we examined the data, particularly using Japanese vaccine coverage instead of the vaccination history of the patients. We also used data of the prevalence of mutated strains in Tokyo as causative strains. If the vaccine history and causative strains of the patients themselves were available, then the results might be different.”
We also added the sentences about the limitation of MIA data to Discussion.
“In our earlier study, although we confirmed the representativeness of fatality cases using MIA data, the data on the numbers of patients with oxygen therapy and respiratory ventilation in Japan have not been published, yet. We have not verified the representativeness of patients with oxygen therapy and respiratory ventilation. Moreover, because MIA is a database of medical claims, data of the prior few weeks might change during a few months. For this study, we collected and analyzed data from January 2020 through the end of March 2022, recorded as of May 2022. If the study period was extended, the estimation results might change over time because the accumulated data engender more accurate results. Emerging new mutated strains, immunization, and development of new therapies might affect the results.”

14. The transition from providing background information to discussing the methods is somewhat abrupt. Adding a sentence or two to smoothly transition into the methods section could improve the overall flow of the introduction.

Response: We added the following sentence to Introduction.
“An average treatment effect model including propensity matching was applied to create these situations.”

15. While the introduction mentions propensity score matching, a brief explanation of how it works and its purpose in addressing selection bias could enhance reader understanding.

Response: We added the following sentence to Abstract.
“We applied the average treatment effect model, particularly propensity scoring, which can treat the choice of administered drug as if administration were randomly assigned to inpatients.”

16. The introduction could touch upon the external validity of the study, acknowledging that findings may not be universally applicable due to variations in healthcare systems, populations, or treatment protocols.

Response: We added the following sentences to Introduction.
““Real-world effectiveness of drugs” means the effectiveness of drugs considering several factors including population characteristics, examinations, vaccinations, and healthcare system such as hospitalization and treatment protocols. Therefore, the results of this study might not be applicable to other countries.”

17. The introduction could explicitly state the research question or hypothesis being addressed by the study to guide readers in understanding the study's purpose from the outset.

Response: We added the following sentence to Introduction.
“The objective of this study was evaluating the “real-world effectiveness of drugs against COVID-19 in Japan.””

18. The text repeats certain points, such as the mention of "propensity score" and "saving life," multiple times. Reducing redundancy could improve the overall clarity and conciseness of the text.

Response: We did the English proofreading.

19. Some sentences are quite complex, which might make it challenging for readers to follow the arguments. Simplifying the language without compromising precision could enhance comprehension.

Response: We did the English proofreading.

20. The text occasionally uses language that implies causation, such as "might contribute to saving life." It's important to be cautious about making causal claims based on observational data. Acknowledging the limitations in establishing causation would strengthen the study's validity.

Response: We added the following sentence to Discussion.
“Third, this study was an observational study that was able to show only association between considered drugs and death, not necessarily a causal association.”

21. The text mentions "not available" entries in the results for antibody cocktails, attributing it to a small sample size. Providing a more explicit explanation of why certain data is missing and its potential impact on the results would enhance transparency.

Response: The approval of antibody cocktails was relatively late, and the number of prescriptions was simply low, so there is no potential impact. We added the following sentence in Discussion.
“The antibody cocktail results included many “not available” entries, although drug effectiveness was confirmed in well-matched cases. The approval of antibody cocktails was late. The number of prescriptions was simply low, which might affect many cases for which “not available” was reported.”

22. The limitations section is placed towards the end of the text. Moving it closer to the beginning or integrating limitations as they become relevant in the discussion could improve the flow and help readers contextualize the findings.

Response: We integrated limitations to Discussion and changed each limitation to understand easily for the reader.

23. The use of terms like "severely ill," "moderate severity," and "less–severe illness" could benefit from clearer operational definitions or references to specific criteria used for classification.

Response: We added the following sentence to Method.
“For this study, mild symptoms were defined as a patient who did not require medical intervention such as oxygen therapy or respiratory ventilation, moderate symptoms were defined as a patient who required oxygen administration, and severe symptoms were defined as a patient who required respiratory ventilation.”

24. The discussion section covers a range of topics, including limitations and future research, without clear organization. Structuring the discussion into subsections could make it more reader-friendly.

Response: We have organized discussion structure by issue.

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. Miyashita K, Hozumi H, Furuhashi K, et al.: Changes in the characteristics and outcomes of COVID-19 patients from the early pandemic to the delta variant epidemic: a nationwide population-based study. Emerg. Microbes. Infect. 2023; 12(1): 2155250. PubMed Abstract | Publisher Full Text | Free Full Text

[2] 2. Mozaffari E, Chandak A, Zhang Z, et al.: Remdesivir Treatment in Hospitalized Patients With Coronavirus Disease 2019 (COVID-19): A Comparative Analysis of In-hospital All-cause Mortality in a Large Multicenter Observational Cohort. Clin. Infect. Dis. 2022; 75(1): e450–e458. PubMed Abstract | Publisher Full Text | Free Full Text

[3] 3. Weinreich DM, Sivapalasingam S, Norton T, et al.: REGEN-COV Antibody Combination and Outcomes in Outpatients with Covid-19. N. Engl. J. Med. 2021; 385(23): e81. Epub 20210929. PubMed Abstract | Publisher Full Text | Free Full Text

[4] 4. Gupta A, Gonzalez-Rojas Y, Juarez E, et al.: Early Treatment for Covid-19 with SARS-CoV-2 Neutralizing Antibody Sotrovimab. N. Engl. J. Med. 2021; 385(21): 1941–1950. PubMed Abstract | Publisher Full Text

[5] 5. Horby P, Lim WS, Emberson JR, et al.: Dexamethasone in Hospitalized Patients with Covid-19. N. Engl. J. Med. 2021; 384(8): 693–704. PubMed Abstract | Publisher Full Text

[6] 6. Marconi VC, Ramanan AV, de Bono S , et al.: Efficacy and safety of baricitinib for the treatment of hospitalised adults with COVID-19 (COV-BARRIER): a randomised, double-blind, parallel-group, placebo-controlled phase 3 trial. Lancet Respir. Med. 2021; 9(12): 1407–1418. PubMed Abstract | Publisher Full Text | Free Full Text

[7] 7. Salama C, Han J, Yau L, et al.: Tocilizumab in Patients Hospitalized with Covid-19 Pneumonia. N. Engl. J. Med. 2021; 384(1): 20–30. PubMed Abstract | Publisher Full Text | Free Full Text

[8] 8. Gordon AC, Mouncey PR, Al-Beidh F, et al.: Interleukin-6 Receptor Antagonists in Critically Ill Patients with Covid-19. N. Engl. J. Med. 2021; 384(16): 1491–1502. PubMed Abstract | Publisher Full Text

[9] 9. Umakanthan S, Senthil S, John S, et al.: The Effect of Statins on Clinical Outcome Among Hospitalized Patients With COVID-19: A Multi-Centric Cohort Study. Front. Pharmacol. 2022; 13: 742273. Epub 20220705. PubMed Abstract | Publisher Full Text | Free Full Text

[10] 10. Bergqvist R, Ahlqvist VH, Lundberg M, et al.: HMG-CoA reductase inhibitors and COVID-19 mortality in Stockholm, Sweden: A registry-based cohort study. PLoS Med. 2021; 18(10): e1003820. Epub 20211014. PubMed Abstract | Publisher Full Text | Free Full Text

[11] 11. Wong CKH, Au ICH, Lau KTK, et al.: Real-world effectiveness of molnupiravir and nirmatrelvir plus ritonavir against mortality, hospitalisation, and in-hospital outcomes among community-dwelling, ambulatory patients with confirmed SARS-CoV-2 infection during the omicron wave in Hong Kong: an observational study. Lancet. 2022; 400(10359): 1213–1222. Publisher Full Text

[12] 12. Jayk Bernal A, Gomes da Silva MM, Musungaie DB, et al.: Molnupiravir for Oral Treatment of Covid-19 in Nonhospitalized Patients. N. Engl. J. Med. 2022; 386(6): 509–520. PubMed Abstract | Publisher Full Text | Free Full Text

[13] 13. Hammond J, Leister-Tebbe H, Gardner A, et al.: Oral Nirmatrelvir for High-Risk, Nonhospitalized Adults with Covid-19. N. Engl. J. Med. 2022; 386(15): 1397–1408. PubMed Abstract | Publisher Full Text | Free Full Text

[14] 14. 令和2 (2020) 年医療施設 (静態・動態) 調査 [Survey of Medical Institutions. 2020]. Tokyo: Ministry of Health, Labour, and Welfare; 2020 [cited 2022 Nov 10]. Reference Source

[15] 15. Kanazawa N, Tani T, Imai S, et al.: Existing Data Sources for Clinical Epidemiology: Database of the National Hospital Organization in Japan. Clin. Epidemiol. 2022; 14: 689–698. Publisher Full Text

[16] 16. Heckman JJ, Ichimura H, Todd PE: Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme. Rev. Econ. Stud. 1997; 64(4): 605–654. Publisher Full Text

[17] 17. Kane LT, Fang T, Galetta MS, et al.: Propensity Score Matching: A Statistical Method. Clin. Spine Surg. 2020; 33(3): 120–122. Publisher Full Text

[18] 18. Deb S, Austin PC, Tu JV, et al.: A Review of Propensity-Score Methods and Their Use in Cardiovascular Research. Can. J. Cardiol. 2016; 32(2): 259–265. PubMed Abstract | Publisher Full Text

[19] 19. Ministry of Health, Labour, and Welfare: Clinical Management of Patients with COVID-19 ver. 8.1. Tokyo: Ministry of Health, Labour, and Welfare (in Japanese); [cited 2022 Nov 8]. Reference Source

[20] 20. Mitsushima S, Horiguchi H, Taniguchi K: Alpha and Delta variants and vaccination effectiveness against severity in COVID-19 inpatients based on medical claims in Japan. GHM Open. 2023; 3(1): 28–36. Publisher Full Text

[21] 21. Mitsushima S, Horiguchi H, Taniguchi K: Drug effectiveness for COVID-19 inpatients inferred from Japanese medical claim data using propensity score matching_supplement data.csv. Dataset. figshare. Publisher Full Text

Drug effectiveness for COVID-19 inpatients inferred from Japanese medical claim data using propensity score matching

Abstract

Background

Methods

Results

Conclusions

Keywords

Revised Amendments from Version 1

Introduction

Method

Data sources

Definitions of variables

Subjects

Statistical analysis

Ethical considerations

Results

Table 1. Estimation results of propensity score matching method.

Figure 1. COVID-19 patients of National data and COVID-19 inpatients of MIA data.

Figure 2. Fatality rate for National data and Rate of deaths among COVID-19 inpatients in MIA.

Discussion

Conclusion

Data availability

Underlying data

Extended data

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated