Mortality risk factors among National Football League players: An analysis using player career data

Not sure if this comment is necessary, but the authors have made a technical amendment to the final version-- I certainly "approve" the amendment, as I approved the revised ms. some weeks ago.

[I would still welcome comment on the points I raised in my re-review, but not expecting this, as the points are relatively minor].

Very good updates.

This was a frustrating paper to review: on the one hand, I definitely applaud any analysis of athletes that compares athletes TO athletes and thus avoids the healthy worker effect HWE). (See Nguyen et al. 2019¹ for a good recent example of avoiding this pitfall; see footnote 2 in this article for a good put-down of the HWE fallacy²). On the other hand, I don’t think this analysis actually involves the HWE at all; as I will discuss, it actually purports to find a “frailty” issue within this worker population, which is a very different phenomenon and one with rather different research and policy implications than a true HWE finding.
 
In this context, a true HWE occurs when reach the mistaken conclusion that X is not riskier than not-X (as in “football players don’t die more often of cancer than general population”), or even that X is safer than not-X, but what’s really happening is that being in class X shows you were less likely to have the risk than general to begin with. This is important to find and point out, because then you either have to adjust for it (see Choi et al.³) or do a different study that compares apples to apples. But what these researchers may have found (see below) is that within subpopulation of NFL players, some are less frail than others. This is heterogeneity-dynamics at work (see Manton, K.G., E. Stallard, and J.W. Vaupel. 1986. Alternative models for the heterogeneity of mortality risks among the aged. J. Am. Stat. Assoc. 81:635-644⁴). Heterogeneity-dynamics means that you may need to adjust the slope of an observed dose-response function to account for the fact that there are 2 or more subpopulations within the subpopulation—everyone can have an individual positive association between seasons and risk, but the group may have a flat (null) slope simply because those who played the most seasons were not “lucky” but more immune.
 
The difference between a finding of an HWE between football and the general public, versus the finding of two of more differentially-susceptible subgroups within the NFL population, is far from merely semantic, because the practical implications of the two situations are so different. Finding an HWE allows the researcher to correct for it, either by discarding a flawed analysis in favor of an apples-to-apples comparison, or by taking the existing analysis and trying to re-estimate the odds ratio (see, for example, Joffe, 2012⁵). Even without correction, researchers who appreciate the pitfalls of the HWE can simply say that it leads to false negatives preferentially: comparing, for example, the cognitive performance of 60-year-old retired NFL players against the general population of that age would tend to divert attention away from the consequences of repeated head trauma (RHT) simply because the general population includes many persons who were never fit enough to work, let alone work in this taxing occupation.
 
But finding a negative-then-positive relationship between length of (NFL) career and age at death, because the cohort under study has, by definition, more fit persons remaining as more and more of the cohort dies early, may have no empirical or policy implications whatsoever. The authors say nothing about how we might even identify who among incoming NFL players might “benefit” from longer careers and whose dose-response is the most steeply negative—if we could, and IF we had the means and the will to discourage the latter group from choosing this occupation, THEN perhaps we could make use of the “finding” that some players are not at risk to the extent that others are (there is an extensive literature about the gap between identifying a powerful interindividual risk factor in a working population—these are usually genetic factors, such as “slow acetylators” who are more at risk from certain occupational chemicals—and the wisdom of trying to exclude these people from the workplace. Generally, policy analysts prefer interventions that can make the workplace safe for everyone who participates—imagine a policy of not allowing people with hemophilia to become carpenters, as opposed to OSHA regulations that mandate guards on saws so that no one will be cut by them). 
 
So this article, at most, finds an “expected curiosity”—that once you have had a long career, you probably are revealing to an epidemiologist that you have been more “immune” to the harmful effects of that career than the average person—and then says nothing about how we could use the finding to adjust scientific conclusions or policy responses. By invoking the HWE throughout the paper, the authors are not only using the wrong term, but inviting statistical corrections or policy responses that have already been made or that would not respond to what they may have actually found.
 
But all the foregoing assumes that the authors have actually found evidence of a “resistant subpopulation” within the NFL cohort, and I’m not sure that’s the case. The authors don’t mention alternative explanations for the slight upslope in Figures 1a and 1b, including: (1) reverse causation—if a significant number of players died on the field or soon after sustaining football-related injuries early in their careers, then of course the remaining population would not have “negative death residuals” that large; and (2) effect modification—similarly, if physical inactivity leads to earlier death, then players who sustained career-ending injuries early on would die earlier than others.
 
More problematic is the use of rote statistics without also applying “common sense” to the finding. How robust, in particular, is the slight upslope obtained by regression to the presence of outliers? The interactive Figures show that the five players with the longest careers were Sammy Baugh, Johnny Unitas, George Blanda, Earl Morrall, and YA Tittle. Some of these (Morrall) I believe played long careers but were backups much of the time, so an index based on number of games rather than seasons might have shown something different. It would be important to explore what happens to the upslope if outliers were trimmed—I say this in part because visual exploration of the Figures does not present a compelling “common sense” picture of a positive slope among the longer careers; I believe the numbers, but the visual impression, especially excluding outliers, is one of a rather FLAT dose-response that one might be convinced is slightly negative and then very slightly positive. Similarly, I accept (p. 3) that the quadratic model fits slightly better than the linear one, but the authors don’t let on how well the linear model fit in the first place. Could they be “finding” something more sophisticated simply by over-fitting? Also, I dispute that the fixed-effects model is correct, with respect to the kinds of effects (CTE, dementia) the authors clearly are trying to shed light on. The three categories they use seem out-of-place here: other studies have shown that the three positions with the greatest cumulative amount of RHT (instances times g-forces) are tight ends, quarterbacks, and defensive linemen, and yet these are in three different categories in this model!
 
Finally, the authors focus on mortality, which is fine, but completely ignore quality of life. Just consider the case of Earl Morrall, who lived to age 79 (death residual of > 10) but who reportedly had Stage 4 CTE and a reduced QOL. Someone who comes away from the paper concluding that “once you play 11 seasons, you may live longer than your cohort” may not realize that age at death is not the most relevant outcome…
 
In summary, the authors should replace “HWE” with “interindividual variability in susceptibility,” explore the implications of THAT analysis, consider how robust their analyses of statistical significance actually are, and ground this paper in terms of the other more ambitious studies that have already explored the relationship between better indices of lifetime intensity of play and mortality/morbidity (especially Montenigro et al., 2017⁶). They might also consider this recent paper by Mez et al.⁷ and explore if and how their quadratic model might better explain (or be contradicted by) these prior findings.