Heterogeneity and other problems in a pooled analysis of snus use and mortality

Introduction

A recent analysis by Byhamre et al. of Swedish snus use and mortality combined eight Swedish datasets and found that exclusive Swedish male snus users have statistically significant increased mortality from all causes (adjusted hazard ratio, aHR = 1.28, 95% confidence interval, CI = 1.20 – 1.35), cardiovascular diseases (aHR = 1.27, CI = 1.15 – 1.41) and other causes (aHR = 1.37, CI = 1.24 – 1.52), as well as a 12% elevation in cancer mortality (aHR = 1.12, CI = 1.00 – 1.26).¹ These findings, from the Swedish Collaboration on Health Effects of Snus Use, are in sharp contrast with previous pooled results from the same group that found no significant increases in incidence and/or mortality from acute myocardial infarction,² stroke,³ atrial fibrillation,⁴ pancreatic cancer,⁵ oral cancer⁶ and colorectal cancer.⁷ The latter study did report a significant association of exclusive current snus use and rectal cancer (aHR = 1.38, CI = 1.07 – 1.77).⁷

We are concerned that the discrepant results between the latest analysis and previous published studies may be indicative of unresolved statistical problems that haven’t been addressed by the collaboration authors in any of their studies. These problems include unresolved heterogeneity among the eight cohorts, for which the random effects method the authors used was not a sufficient remedy. In addition, the tables in the article are uninformative, and the exclusion of smokers in the analysis was not validated and eliminated important information. There are straightforward and easily implemented corrective measures that the authors could undertake.

The pooled cohorts are highly heterogeneous

First, the cohorts in the study are heterogeneous, as our Table 1 demonstrates. The eight cohorts were all described as “population-based,” but only four fit that description, and two of those were duplicative. The population-based cohorts were from: (a) the most populous Swedish County – the Stockholm Public Health Cohort (StPHC); (b) a Diet and Cancer Study in the third most populous city, Malmö; (MDCS) (c) the Scania Public Health Cohort (ScPHC), which contains Malmö; and (d) the Multinational Monitoring of Trends and Determinants in Cardiovascular Disease (MONICA) cohort, from rural Västerbotten and Norbotten Counties, which make up 34% of Sweden’s land area but only 5% of its population. The other four cohorts were convenience samples (the first three are from throughout Sweden). They consisted of: (a) the construction worker cohort (CWC); (b) participants in a charity walk, the National March Cohort (NMC); (c) twins born from 1926 to 1958 (Screening Across the Lifespan Twin, or SALT, Study); and (d) employees in three counties (Work, Lipids and Fibrinogen, WOLF study), which included Stockholm.

The heterogeneity of the samples is further illustrated by other characteristics described in Table 1, including the mean age at death. It was 61 years among employees, which was much lower compared not only with the other cohorts, but with men in Sweden during the same recruitment and follow-up periods.⁸ Other diverse variables were the prevalence of smoking and snus use. As we discuss in detail later, Byhamre et al.¹ excluded smokers from their analysis, but didn’t justify their choice, which is not innocuous and contributes to the estimates that the authors conclusions are based upon. They also, misleadingly, included these omitted observations in their table describing their cohorts. So, we point out that smoking and snus use prevalence varied from very low (7% and 9%, respectively) among charity walkers to very high (46% and 27%) among construction workers.

The construction workers account for 72-73% of both the sample and the number of deaths. Their mean enrollment age was 34 years, which was vastly different than four other cohorts that recruited men aged 50+ years. The first mortality follow-up study of snus use involving this cohort, published in 1994,⁹ had an unusual and unresolved distribution of excess deaths.¹⁰

Pooled analyses are intended to yield results that are generalizable to the pool. However, the vast differences in the characteristics of the cohorts that have been disclosed raise a concern whether the findings are interpretable as population estimates. As discussed, only four cohorts were from population-based samples, and three enrolled very small numbers. The participant selection process in the vastly larger convenience sample cohorts is the dominant influence of this study’s results and, given that the employee cohort is by far the largest and makes up a large majority of the pooled sample, the very different snus use and mortality in this cohort is reason alone to question the external validity of the results even if they are correct, which we do not think is the case.

The methodology was flawed

Byhamre et al.¹ used a random-effects method to account for the variation in mortality across cohorts, in which they assumed that there is no correlation between individual cohorts and snus use. However, that method also assumes that unmeasured factors specific to a cohort are unrelated to either snus use or mortality, which is almost surely unlikely to hold in practice. For example, as we show in Table 1, the mean age at enrollment in the four population-based cohorts was 48 to 59 years, compared with 34 years in the construction workers, the latter accounting for over 70% of the combined sample. Furthermore, the mean age at death was highly variable, ranging from 61 years in the employee sample and 66 years in construction workers to 70+ years in the other cohorts.

At a minimum, the authors could have addressed these problems by other methods^11,12 providing the following information:

The tables are uninformative

Byhamre et al. provided the “Characteristics of included cohorts...” in Table 1 of their publication, and we have used some of this information to illustrate heterogeneity and other problems.¹ However, this table does not reflect Byhamre et al.’s actual analytic sample (n = 169,103), because it includes the characteristics of all male participants from the eight cohorts (n = 383,025), including ever regular smokers and others that the authors excluded. Simply put, including information on 214,000 participants that aren’t in the analysis is not only uninformative, it is misleading and must be revised.

Although Byhamre et al.’s Table 2 provides information on some baseline characteristics in the analytic sample, it presents the pooled characteristics, which is uninformative because it further conceals large differences between the cohorts. For example, variations in enrollment ages across the cohorts have substantial impact on BMI, physical activity, and alcohol use that may overwhelm and obfuscate any differences due to snus use. The authors should have reported the baseline characteristics according to cohorts, and cohorts by snus use.

Furthermore, the authors did not provide adequate definitions for snus use, the sole exposure of interest in the study. Thus, questionnaires containing actual questions about snus use from each cohort is a prerequisite. According to the authors, baseline snus use was not consistent throughout the cohorts. Some studies did not have information on former snus use, while others did not have information on amount and/or duration.

The exclusion of smokers was not validated and eliminates important information

The authors excluded participants “reporting ever regular use of cigarettes” (emphasis added), which was intended “to eliminate potential residual effects”, a vague reference to an unspecified statistical problem. The authors provided no foundation for this choice and no evidence that their objective was met. Furthermore, snus users who were less-than-regular cigarette smokers – those with a smoking history and some dual users – were still in the pooled analytic cohort.¹³ Less-than-regular smoking is likely because of the lower social acceptability of smoking vis-à-vis snus in Sweden.^14,15

The authors could validate whether their analytic tactic excluded smoking by reporting the results for two disease categories: cancer of the trachea, bronchus, and lung (ICD-10 codes C33–C34) and chronic lower respiratory diseases (J40-J47). If they truly eliminated smokers from their analyses, they should not have found any significantly increased hazard ratios for these diseases among snus users.

Excluding “ever regular” smokers is a routine practice in Swedish follow-up studies of snus use, but it also eliminates a lot of valuable information and may bias the results. First, partial exclusion of an exposure closely associated with snus use precludes assessment of interactions. Second, the findings on snus use in this and similar studies completely lack context with respect to the effects of smoking on mortality.¹⁶

Summary

In conclusion, Byhamre et al.¹ published an analysis with several flaws that render their results uninformative and possibly misleading. The authors should publish revised tables and re-estimate their analyses as described above to address these problems. These measures are readily achievable, and the validity of the results will be considerably enhanced.

Data availability

No data is associated with this article.