We thank the reviewers for their comments which we attempt to answer in detail below.  Please note that, as advised by the journal, our proposed changes to the paper have not, at this stage, been made, pending comments from other reviewers on our paper.  Please also note that, as explained in more detail below, some of the points raised by the reviewers will be dealt with more fully in a further paper we are currently preparing based on data from Waves 1 to 4, as mentioned in the penultimate paragraph of the conclusions section.   As will be evident, the reviewers original comments are in normal type and our replies and suggested changes to the paper are in bold.  We hope that our reply and the changes will affect the reviewers’ opinion of our work.

In the background of the abstract, the authors state that their goal is to address the research question whether e-cigarettes encourage smokers to quit using a quality nationally representative cohort study from the United States.

The introduction presents a somewhat limited review of the literature.

The introduction was never intended to be a fully detailed and comprehensive review of the extensive literature relating e-cigarettes to quitting, the main purpose of the paper being to describe the results of our own analyses.  Nevertheless the introduction includes as many as 39 references, and we feel gives a relatively succinct summary of the findings from the literature available at the time of writing.

In the discussion of the randomized trials the authors infer that these trials reached statistically significant findings where e-cigarettes were favored. This is a misrepresentation of the literature.

We made no reference to statistical significance and had not intended to imply this.  We were merely summarizing the direction of the differences reported in the papers cited, virtually all of which indicated a higher quit rate in the nicotine e-cigarette group.  However to avoid confusion, the second sentence of the second paragraph of the discussion will be amended so that it ends

‘…generally indicates higher quit rates in the nicotine e-cigarette group, although not all the differences cited were statistically significant (at p<0.05), and one study did not find such higher quit rates  (Halpern et al., 2018) .’

The authors note the following problems that have been identified with observational studies addressing the target research question: studies in which the exposure measure does not precede the outcome measure (as happens in cross-sectional studies), the use of non-representative populations, and failure to fully account for the many potential confounding factors. While these are appropriate criticisms of some of the literature, they are far from complete. For example, in 2018, the US National Academies of Science, Engineering and Medicine put out a major report on e-cigarettes where they expressed concern about the quality of some of the papers addressing whether these products were cessation aids:

We had already stated in the first sentence of paragraph 4 of the discussion that’…a recent review considered the evidence is inconclusive due to the low quality of the research’ citing Malas et al. (2016) .  We will amend this to start ‘…recent reviews have considered the evidence…’ and additionally cite the reference to the major report in 2018 that the reviewers mention.

In addition to these three concerns, this report noted the importance of the study focusing on quit attempts with a comparable control group – this issue has been highlighted in the 2019 paper by Pierce et al. in Nicotine and Tobacco Research ¹ .

In the second sentence of paragraph 4 of the discussion we will add ‘the use of non-comparable control groups’ after ‘the use of unrepresentative populations’.

This analysis showed that the likelihood of a quit attempt was much higher in e-cigarette users in the PATH study (many who started using e-cigarettes did so because they were trying to quit) than it was in the control population. Thus, the higher probability of making a quit attempt could explain the difference in smoking cessation and so the difference should not be attributed to the use of e-cigarettes.

We already do adjust for aspects of smoking related to quitting in our analyses, as shown in Tables 1 and 3.

However, the authors go further and argue “ Problems involve the …failure to limit attention to established e-cigarette users,” (Para 4 introduction). It is not clear what the authors mean by this? Do they mean that the study population need to have developed a consistent pattern of dual use of cigarettes and e-cigarettes? What is their justification for such a limitation when addressing the stated research question? It is apparent that they do not place such a limitation on their own analyses.

It does not seem sensible to include among e-cigarette users those who have in the past tried them once or twice and abandoned them as they did not like them.  The reviewers state that we did not limit attention to established users.  This is untrue; that we do limit attention to established users is very clearly stated in the third paragraph of the methods section.

Indeed, the authors have not laid out specific research questions or hypotheses that guide the analyses that they undertake in this paper. This is an important omission.

In the introduction, the first sentence of the final paragraph is ‘Here we describe results from a prospective study aimed at avoiding such weaknesses.’ which might be regarded as a statement of our objectives.  However we will insert a sentence after this to read: 

‘The main objective of our analyses is to quantify the relationship between e-cigarette use in smokers and subsequent cessation of smoking, with detailed adjustment for the multitude of factors that may differ between e-cigarette users and non-users.’  We will also amend the next sentence to start ‘Our analyses are based on’ rather than ‘It is based on…’.

In the methods, the authors state that analyses are based on individuals with relevant data available at Waves 1, 2 and 3 on smoking and e-cigarette use. They do not mention that the population needed to have made a recent quit attempt. As noted above, unless they demonstrate that the populations that they choose are comparable on this variable, then their study is confounded.

We did not say that the population needed to have made a recent quit attempt, as this was not a requirement in our study, and indeed was not a requirement of a number of the other studies we had cited in our discussion  (Berry et al., 2019; Kalkhoran et al., 2019; Verplaetse et al., 2019) , though it was in others  (Benmarhnia et al., 2018; Watkins et al., 2019) .  Those who used e-cigarettes at baseline may not at that time have used them intending to quit, but may have found during follow-up that they could meet their nicotine needs without smoking cigarettes.  Our analyses compared quit rates between e-cigarette users and non-users to try to answer the simpler, and highly relevant, question ‘are e-cigarette users more likely to quit?’  We prefer to present results based on the whole population of baseline smokers, thus using a larger sample size than if we restricted attention to quit attempters,  and to try to avoid bias by taking into account a range of predictor variables related to quitting. 

However we will add a second paragraph in the methods section as follows.

‘Some studies of data from Waves 1 to 3 of the PATH study  (Benmarhnia et al., 2018; Pierce et al., 2020a; Watkins et al., 2019) have limited analyses to quit attempters, but others  (Berry et al., 2019; Kalkhoran et al., 2019; Verplaetse et al., 2019) have not.  Although such a limitation more closely mimics randomized control trials  (Pierce et al., 2020b) we prefer not to do so, and to avoid bias by adjusting for aspects of quitting in analyses.  Our approach uses a larger sample size and provides results that are more representative of the whole population of baseline smokers’.

As already noted, we intend to carry out some additional analyses limited to quit attempters in the paper being prepared based on data from Waves 1 to 4, though we do note for the reviewers’ information that a preliminary analysis we conducted, based on data from Waves 1 and 2 and limiting attention to those who at Wave 2 had ever made a quit attempt, produced an adjusted odds ratio of 1.20 (95%CI 0.91-1.57) which is very similar to the estimate we give in Table 2 of our paper of 1.23 (0.94-1.61).

The methods are not at all clear about the outcome measure used in this analysis. In their section outlining their main analyses, the authors use the term “quitting during follow-up”. What does this mean? A logical interpretation is that a quit attempt was made in the year prior to the follow-up survey. However, the authors have not addressed the quit attempt data in the methods. Do they use a point prevalence of former smoking at the follow-up survey? If so, how can this be described as successful smoking cessation? This is one of the most critical points in the paper. Most other papers discussing successful quitting require abstinence for either 6 of 12 months at follow-up (see, Gilpin et al. 1997 ² ). This is particularly important in an observational study when a person quit at follow-up could have quit only for the day before the survey, for example.

We certainly thought that we had made it clear enough in the methods that we were defining smoking status as at each Wave, and defining quitting based on being a current established smoker at baseline and a former established smoker at follow-up.  While we agree that some former smokers may only have quit for a relatively short time, we believe that our method still provides meaningful results.  However, we will extend the first sentence of the old fourth paragraph of the methods to read

‘The main analysis for each period relates ever e-cigarette use at baseline to the probability of being an established former smoker at follow-up (referred to subsequently as either “quitting” or “quitting during follow-up”), with adjustment for predictor variables measured at baseline.’

Again, for the reviewers’ information, we repeated our Wave 1 to 2 analysis, redefining quitting as having quit for at least 30 days.  This produced an adjusted estimate of 1.26 (95%CI 0.93-1.70) which is very similar to the estimate we give in Table 2 of our paper of 1.23 (0.94-1.61).

The potential confounders in this analysis include those variables that are associated with use of e-cigarettes as well as variables that are associated with successful cessation. The authors omit any discussion of variables associated with the probability of a smoker using an e-cigarette, thus seriously confounding their analyses. At least in a supplement, there should be a summary table of these predictors outlining how they are associated with e-cigarette use.

We will extend the fifth paragraph of the methods section to read as follows:

‘The specific predictor variables are listed in the Results section, with fuller details of their definition given in the Extended data.  While the variables were chosen as being suggested by the literature as being related to smoking, the Extended data also provides information, based on Wave 1, of their association with ever e-cigarette use.  As shown there, ever use was highly significantly (p < 0.001) more frequent in the young and in females, and after adjustment for age and sex, was also highly significantly related to a range of the predictor variables considered, being less frequent in Hispanics and Blacks, and more frequent in those with more income or education, those who ever use other tobacco products, those who have a perceived greater need for tobacco, those who have tried to quit more often, those who plan to quit, those who find it hard to stop smoking and those who have used quitting aids.  Users were also clearly more likely to have significant problems more recently with sleeping, anxiety and distress, to see a doctor more often, to use the internet often, and to use various different types of drugs (but not cocaine or crack).  At most weak relationships were seen with smoking by family and friends, awareness of the hazards of smoking, use of alcohol, body mass index, or self-perception of physical health or quality of life.  Little relationship was also  seen between ever e-cigarette use and daily cigarette consumption, a finding which was reported earlier  (Lee et al., 2020) , where it was suggested that it was explained by smokers taking up e-cigarettes having higher consumption initially, reduced by partial replacement of cigarettes by e-cigarettes.’  

We will also, in due course, add to the Extended data a section describing and presenting the output from which the above results are summarized.

Analytic Plan: The analysis plan lays out a stepwise forward selection logistic regression, adjusted for selected covariates. What is unclear is how the authors use step 5 & 6 of their analytic plan in drawing their inferences.

Our inferences are drawn from the results of the step 5 and 6 analyses which, reassuringly, produced the same models.  The results in Table 2 compare unadjusted odds ratios with those adjusted for the final (step 5/6) models shown in Table 3.

There is a problem with the use of study weights. For their Wave 2 to Wave 3 analysis, it would seem that the appropriate weights would be the Wave 2 weights,

In fact we did use Wave 2 weights for the Wave 2 to Wave 3 analysis.  We will correct the end of the second sentence of the first paragraph of the Methods section to read

‘, and take account of the person-based weights of the baseline population’.

A figure laying out the different analyses with samples sizes (including loss to follow-up) would be very helpful to the reader.

Results: The unadjusted quitting rates by ever use of e-cigarettes should be presented in a table in the results section.

Also, the authors should report the percentage of subjects excluded in the multiple regressions.

We already present in Tables 1 and 2 sample sizes in each of the main analyses.  We will add information on the proportion of the baseline population that were not followed up, but do not think that this merits a figure.  We will also extend Table 2 to include additional lines to cover the other points made by the reviewers.  Thus the revised tables will start with the following lines:

Table 1.

Table 2.

Any reworking of this manuscript should discuss the four recent analyses that address e-cigarettes and smoking using the PATH data. All these papers use appropriate analytic procedures and arrive at opposite conclusions to those of the authors ^{3 -  6} .

Of the four references which the reviewers cite, two  (Dai and Leventhal, 2019; Everard et al., 2020) relate to relapse, which is outside the scope of the paper, and not quitting, and one of these  (Dai and Leventhal, 2019) in any case states that ‘Baseline e-cigarette use was not associated with smoking relapse at follow-up after covariate adjustment.’  We do not propose to cite these two references, as our paper concerns quitting.

One of the other two references the reviewers cite  (Pierce et al., 2020a) , not published at the time our paper was prepared, describes analyses based on Waves 1 to 3 of the PATH study.  We have, in the discussion in paragraphs 3 to 7, already summarized findings from five other analyses based on the PATH study which are consistent with e-cigarette use increasing the probability of quitting cigarettes.  To take into account this paper, and to provide further detail in the discussion, we propose to make the following two changes.

First, we will extend the second paragraph of the discussion to read as follows:

‘Six other related analyses based on the first three Waves 1 of the PATH study have previously been published.  The first five analyses summarized below  (Benmarhnia et al., 2018; Berry et al., 2019; Kalkhoran et al., 2019; Verplaetse et al., 2019; Watkins et al., 2019) are consistent with e-cigarette use increasing the probability of quitting cigarettes, despite variation in whether Wave 3 data has been used or not, whether analyses are restricted to those attempting quitting at baseline, the definition of abstinence used, the confounding variables adjusted for, and other analytical details.  However, the final analysis  (Pierce et al., 2020a) only reported a small and non-significant increase in quitting related to e-cigarette use.’

Second, we will add a new paragraph about the latest study at the end of the paragraphs describing the results of the first five analyses as follows:

‘A third analysis based on data from Waves 1, 2 and 3  (Pierce et al., 2020a) restricted attention to adult (ages 18+) smokers identified at Wave 1 who reported a quit attempt before Wave 2 and completed Wave 3.  12 month abstinence at Wave 3 among e-cigarette users was slightly but non-significantly reduced as compared both to users of pharmacotherapy to quit or no product.’

The other reference cited by the reviewers  (Chen et al., 2020) refers to analyses based on Waves 1 to 4 of the PATH study.  Partly because our analyses relate to Waves 1 to 3, and partly because we are currently developing analyses based on Waves 1 to 4 for a further publication, we strongly prefer not to consider analyses that include Wave 4 data in our discussion. 

We will also add two extra sentences at the end of the penultimate paragraph of our discussion section, which refers to the planned further paper, as follows:

‘That paper might also consider different definitions of quitting, such as at least 30 day quitting, and investigate the effect of restricting attention to those attempting quitting at baseline.  It will also describe and comment on other publications that have used data up to Wave 4’.

Is the work clearly and accurately presented and does it cite the current literature?

Partly  

It was never intended to consider literature based on analyses using PATH Wave 4 data, or considering relapse, and the huge literature on e-cigarettes from other studies is covered in fair detail.

< >Is the study design appropriate and is the work technically sound?Are sufficient details of methods and analysis provided to allow replication by others?If applicable, is the statistical analysis and its interpretation appropriate? Are all the source data underlying the results available to ensure full reproducibility?Are the conclusions drawn adequately supported by the results?

We confirm that we have read this submission and believe that we have an appropriate level of expertise to state that we do not consider it to be of an acceptable scientific standard, for reasons outlined above.

We hope that our submission, when the revisions described are made, will now be considered of an acceptable scientific standard.

References

Benmarhnia, T., Pierce, J.P., Leas, E., White, M.M., Strong, D.R., Noble, M.L., Trinidad, D.R., 2018. Can e-cigarettes and pharmaceutical aids increase smoking cessation and reduce cigarette consumption? Findings from a nationally representative cohort of American smokers. Am. J. Epidemiol. 187, 11, 2397-2404. DOI:10.1093/aje/kwy129.

Berry, K.M., Reynolds, L.M., Collins, J.M., Siegel, M.B., Fetterman, J.L., Hamburg, N.M., Bhatnagar, A., Benjamin, E.J., Stokes, A., 2019. E-cigarette initiation and associated changes in smoking cessation and reduction: the Population Assessment of Tobacco and Health Study, 2013-2015. Tob. Control. 28, 1, 42-49. DOI:10.1136/tobaccocontrol-2017-054108.

Chen, R., Pierce, J.P., Leas, E.C., White, M.M., Kealey, S., Strong, D.R., Trinidad, D.R., Benmarhnia, T., Messer, K., 2020. E-cigarette use to aid long-term smoking cessation in the US: Prospective evidence from the PATH Cohort Study. Am. J. Epidemiol. Published online ahead of print July 27, 2020 (doi: 10.1093/aje/kwaa161). DOI:10.1093/aje/kwaa161. Erratum appears in Am J Epidemiol. 2020 Sep 12;kwaa193. doi: 10.1093/aje/kwaa193.

Dai, H., Leventhal, A.M., 2019. Association of electronic cigarette vaping and subsequent smoking relapse among former smokers. Drug Alcohol Depend. 199, 10-17. DOI:10.1016/j.drugalcdep.2019.01.043.

Everard, C.D., Silveira, M.L., Kimmel, H.L., Marshall, D., Blanco, C., Compton, W.M., 2020. Association of electronic nicotine delivery system use with cigarette smoking relapse among former smokers in the United States. JAMA Netw. Open. 3, 6, e204813. DOI:10.1001/jamanetworkopen.2020.4813.

Halpern, S.D., Harhay, M.O., Saulsgiver, K., Brophy, C., Troxel, A.B., Volpp, K.G., 2018. A pragmatic trial of e-cigarettes, incentives, and drugs for smoking cessation. N. Engl. J. Med. 378, 24, 2302-2310. DOI:10.1056/NEJMsa1715757.

Kalkhoran, S., Chang, Y., Rigotti, N.A., 2019. E-cigarettes and smoking cessation in smokers with chronic conditions. Am. J. Prev. Med. 57, 6, 786-791. DOI:10.1016/j.amepre.2019.08.017.

Lee, P., Fry, J., Forey, B., Coombs, K.J., Thornton, A.J., 2020. Cigarette consumption in adult dual users of cigarettes and e-cigarettes: a review of the evidence, including new results from the PATH study [version 1; peer review: awaiting peer review] F1000Res. 9, 630. DOI:10.12688/f1000research.24589.1.

Malas, M., van der Tempel, J., Schwartz, R., Minichiello, A., Lightfoot, C., Noormohamed, A., Andrews, J., Zawertailo, L., Ferrence, R., 2016. Electronic cigarettes for smoking cessation: A systematic review. Nicotine Tob. Res. 18, 10, 1926-1936. DOI:10.1093/ntr/ntw119.

Pierce, J.P., et al., 2020a. Role of e-cigarettes and pharmacotherapy during attempts to quit cigarette smoking: The PATH Study 2013-16. PLoS One. 15, 9, e0237938. DOI:10.1371/journal.pone.0237938.

Pierce, J.P., Messer, K., Leas, E.C., Kealey, S., White, M.M., Benmarhnia, T., 2020b. A source of bias in studies of e-cigarettes and smoking cessation. Nicotine Tob. Res. 22, 5, 861-862. DOI:10.1093/ntr/ntz143.

Verplaetse, T.L., Moore, K.E., Pittman, B.P., Roberts, W., Oberleitner, L.M., Peltier, M.K.R., Hacker, R., Cosgrove, K.P., McKee, S.A., 2019. Intersection of e-cigarette use and gender on transitions in cigarette smoking status: Findings across Waves 1 and 2 of the Population Assessment of Tobacco and Health Study. Nicotine Tob. Res. 21, 10, 1423-1428. DOI:10.1093/ntr/nty187.

Watkins, S.L., Thrul, J., Max, W., Ling, P.M., 2019. Real-world effectiveness of smoking cessation strategies for young and older adults: Findings from a nationally representative cohort. Nicotine Tob. Res. 22, 9, 1560-1568. DOI:10.1093/ntr/ntz223.