Measurement properties of the translations of instruments evaluating the subjective effects of tobacco- and nicotine-containing products:  a systematic review of the literature

Search strategy

Embase and MEDLINE databases were searched (in March 2018) with no limitation in timeframe, by using the following keywords and Boolean operators: (1) translation OR language OR version or cross-cultural valid* OR internal consistency OR Cronbach’s alpha OR reliability OR validation OR responsiveness OR validity, combined (AND) with (2) QSU OR Questionnaire on Smoking Urges OR Cigarette Evaluation Questionnaire OR Fagerström Test for Nicotine Dependence OR FTND OR Fagerström Test for Cigarette Dependence OR Minnesota Nicotine Withdrawing Scale OR MNWS. The combination of (1) and (2) was limited to Abstract, Human research, and English. We screened reference lists to identify supplemental pertinent studies.

Selection criteria

Abstracts retrieved through the search strategy were reviewed and excluded if they (1) did not refer to the instruments of interest; (2) referred to the original version of the instruments of interest; or (3) referred to a translation used (a) in an epidemiological or behavioral context (i.e., not reporting measurement properties) or (b) for validating another measure and not for assessing/reporting the internal consistency or structural validity of the instruments of interest. Conference abstracts were excluded.

The reference lists of the papers considered for inclusion were reviewed, and articles of interest were included if they (a) referred to a translation for which internal consistency or structural validity was assessed at minimum or (b) provided additional information on an existing translation identified through the first round of review.

Two independent reviewers performed the selection. Initial data were extracted by one reviewer and then reviewed (and complemented if needed) by another.

Measurement properties

We used the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) categorization (Mokkink et al., 2010a; Mokkink et al., 2010b; Mokkink et al., 2010c; Mokkink et al., 2018), to classify the measurement properties as follows: reliability, validity, and responsiveness to change. A fourth category, sensitivity and specificity, was added where appropriate (e.g., when the instrument was used for screening).

Reliability is described as the overall consistency of a measure, i.e., the degree to which scores for subjects who have not changed are the same when the measurement is repeated over time [test–retest reliability], is done with different evaluators on the same occasion [inter-rater reliability] or with the same evaluator on different occasions [intra-rater reliability]). As for internal consistency reliability, this estimate assesses the consistency of scores across items within a measurement instrument.

Validity is the degree to which an instrument measures what it is supposed to measure and includes the following:

Responsiveness to change (Hays & Hadorn, 1992) is the ability of an instrument to detect change over time in the construct to be measured. Responsiveness to change is considered an aspect of validity in a longitudinal context.

Sensitivity and specificity are used to evaluate the screening performance of a measure. Sensitivity is the proportion of true positives that are exactly identified, whereas specificity relates to the proportion of true negatives correctly identified. Generally, an optimal cutoff point for the score is selected to reduce the sum of false-positive and false-negative results.

FTND results

The search retrieved 34 FTND studies and one review. We identified 25 different FTND translations (Table 3), including two different Arabic versions for Yemeni speakers [Yemeni immigrants in the UK (Kassim, et al., 2012) and inhabitants of Yemen (Nakajima et al., 2012)], two different Chinese versions [Taiwanese (Huang et al., 2009: Huang et al., 2006) and Chinese immigrants in the US (Yamada et al., 2009)], two different Dutch versions (Breteler et al., 2004; Vink et al., 2005), two different Farsi versions (Iran) (Robabeh et al., 2017; Sarbandi et al., 2015), and two different Portuguese versions for Brazil (Carmo & Pueyo, 2002; de Meneses-Gaya et al., 2009).

In case of different studies exploring the same language (e.g., Japanese versions reported in two independent studies or French versions reported in French and Swiss studies), we contacted the authors for clarification about the version used (personal communications by email). Authors confirmed using either an existing translation [Japanese (Kawada et al., 2010)] or their own version [French (Chabrol et al., 2003); Portuguese for Brazil (de Meneses-Gaya et al., 2009); Spanish for Mexico (Moreno-Coutiño & Villalobos-Gallegos, 2017); and Dutch (Vink et al., 2005)]. In cases where we did not receive any answer, we counted only one version. We did not include the study by de Leon et al. (2003), as the results were based on a combined sample of Spanish and American smokers [study reported in the review by Meneses-Gaya et al. (2009)].

The main objective of the majority of the studies was to investigate the psychometric properties of the FTND in a specific target language and population. In some instances, the studies aimed to develop other dependence scales by using the FTND as a reference instrument [Spanish (Becoña et al., 2010); French (Etter, 2005); Italian (Grassi et al., 2014); Japanese (Kawada et al., 2010); and Arabic (Salameh et al., 2013; Salameh & Khayat, 2014)] while also exploring the properties of the FTND (Table 3).

Supplementary Material 2 describes the sociodemographic/design characteristics and targeted country/language of each study retrieved (Table S2.1), translation process (steps and people involved if mentioned) as described in the paper (Table S2.2), and measurement properties of the translations (Table S2.3). Combustible cigarettes were the TNP evaluated in all studies. Studies in India had evaluated bidis (cigarettes made locally by wrapping coarse tobacco in dried temburni leaf) [Malayalam (Jayakrishnan et al., 2012) and Hindi (Jhanjee & Sethi, 2010)].

A wide range of populations was investigated (Table S2.1). Participants were recruited from the general population in 24% of the studies (Becoña & Vázquez, 1998; Etter, 2005; John et al., 2004; Klinsophon et al., 2017; Nakajima et al., 2012; Salameh & Khayat, 2014; Stavem et al., 2008; Yamada et al., 2009). While 12% of the studies investigated youth samples, such as students (de Meneses-Gaya et al., 2009; Etter et al., 1997; Nakajima et al., 2012; Salameh & Jomaa, 2014), 20% studied patients with comorbidities or drug dependence (Becoña et al., 2010; de Meneses-Gaya et al., 2009; Osório Fde et al., 2013; Jhanjee & Sethi, 2010; Mikami et al., 1999; Park et al., 2004; Robabeh et al., 2017). The sex ratio was balanced, except in some languages or countries with predominantly male samples [e.g., Iran (Robabeh et al., 2017; Sarbandi et al., 2015), India (Jayakrishnan et al., 2012; Jhanjee & Sethi, 2010), Japan (Kawada et al., 2010; Mikami et al., 1999), Korea (Park et al., 2004), Malaysia (Yee et al., 2011), Taiwan (Huang et al., 2006, Huang et al., 2009), Thailand (Klinsophon et al., 2017), and the Yemeni population in the U.K. (Kassim et al., 2012)]. With regard to cigarette consumption, participants in most countries were light to moderate smokers, except in India (Jhanjee & Sethi, 2010), Italy (Ferketich et al., 2008; Grassi et al., 2014; Svicher et al., 2018), Japan (Kawada et al., 2010; Mikami et al., 1999), Spain (Becoña & Vázquez, 1998), Taiwan (Huang et al., 2006, Huang et al., 2009), and Turkey (Uysal et al., 2004; Uysal et al., 2015), where the samples included a mix of moderate to heavy smokers. The original version of the FTND (Heatherton et al., 1991) was developed with a sample of 254 adult visitors (male, 111; female, 143) at the Ontario Science Centre, who ranged in age from 17 to 77 years (mean age, 33.5 ± 12.7 years) and smoked an average of 20.7 cigarettes per day.

Translation process. Of the translations, 60% (15 out of 25) were documented with a description of the translation process used to develop each of them (Table S2.2). Out of these 15 translations, only three (Kassim et al., 2012; Uysal et al., 2004; Yee et al., 2011) were presented with a brief report of the difficulties encountered and solutions found. References to guidelines or recommendations were given for six translations (Becoña & Vázquez, 1998; Kassim et al., 2012; Klinsophon et al., 2017; Sarbandi et al., 2015; Yamada et al., 2009; Yee et al., 2011). Descriptions of the translation process were either minimal, with only a mention of the steps performed (Becoña & Vázquez, 1998; Jayakrishnan et al., 2012; Jhanjee & Sethi, 2010; Mikami et al., 1999; Nakajima et al., 2012; Robabeh et al., 2017), or more detailed, with information about the people involved in the process (Etter et al., 1999; Kassim et al., 2012; Klinsophon et al., 2017; Park et al., 2004; Salameh et al., 2013; Salameh & Khayat, 2014; Sarbandi et al., 2015; Uysal et al., 2004; Yamada et al., 2009; Yee et al., 2011). Except for one translation, i.e., French for Switzerland (Etter et al., 1999), all teams included in their process a backward translation step (i.e., the translation of the target language version back to the source language, English).

Measurement properties All translations were assessed using the classical test theory (CTT), except for the Chinese version developed for immigrants to the US (Yamada et al., 2009), which was assessed only by an item response theory (IRT)-based approach. The Dutch (Breteler et al., 2004) and Italian versions (Svicher et al., 2018) were assessed by IRT supplemented with CTT. Table S2.3 provides detailed information on all properties.

Measurement equivalence was explored for only one translation—Chinese for immigrants to the US (Yamada et al., 2009)—for which DIF was examined by using IRT. Question (Q) 2 (difficult to refrain) showed a significantly substantial DIF, indicating that users of the Chinese version were more likely to support this item and to report more difficulty in refraining from smoking at several public places even after controlling for the nicotine-dependence level. As this DIF item in the Chinese version contributed minimally at the aggregate level, the authors concluded that its impact was negligible on scale scores. Neither unidimensional nor multidimensional results showed DIF for Q1 (time to first cigarette) or Q3 (cigarette hated most to give up), indicating that these two items are DIF-free. Authors concluded that these two items should be retained in the FTND to enable comparison between Chinese- and English-speaking smokers.

Structural validity was documented for 72% of the translations (18 of 25): Arabic for Lebanon (Salameh et al., 2013; Salameh & Khayat, 2014); Arabic for Yemenites living in the UK (Kassim et al., 2012) and Yemen (Nakajima et al., 2012); Chinese for Taiwan (Huang et al., 2006) and the US (Yamada et al., 2009); Dutch (Breteler et al., 2004); both Farsi versions (Robabeh et al., 2017; Sarbandi et al., 2015); French for France (Chabrol et al., 2003) and Switzerland (Etter, 2005; Etter et al., 1999); German (John et al., 2004); Hindi (Jhanjee & Sethi, 2010); Italian (Grassi et al., 2014; Svicher et al., 2018); Korean (Park et al., 2004); Malay (Yee et al., 2011); Portuguese for Brazil (de Meneses-Gaya et al., 2009); Spanish for Mexico (Moreno-Coutiño & Villalobos-Gallegos, 2017); and Turkish (Uysal et al., 2004; Uysal et al., 2015). Six of the translations had a monofactorial structure, similar to their original versions (Heatherton et al., 1991): Arabic (Lebanon) (Salameh et al., 2013); Farsi (Sarbandi et al., 2015); French for France (Chabrol et al., 2003) and Switzerland (Etter, 2005; Etter et al., 1999)]; Italian (Svicher et al., 2018); and Spanish for Mexico (Moreno-Coutiño & Villalobos-Gallegos, 2017). The bifactorial structure, described by Haddock et al. (1999) and Radzius et al. (2003) (i.e., one factor labeled “Smoking Pattern” with Q1, Q2, Q4, and Q6 and the other factor labeled “Morning Pattern” with Q3 and Q5, without and with cross-loading of Q1), was replicated in six studies: Chinese for Taiwan (Huang et al., 2006) and the US (Yamada et al., 2009); Farsi (Robabeh et al., 2017); German (John et al., 2004); Korean (Park et al., 2004); and Portuguese for Brazil (de Meneses-Gaya et al., 2009). For the Arabic version for Lebanon (different from the original version), Salameh & Khayat (2014) reported a bifactorial structure in a general population sample but a monofactorial structure in a student sample (Salameh et al., 2013).

Internal consistency was explored for all translations. In monofactorial structures, Cronbach’s alpha ranged from 0.52 (Thai version, Klinsophon et al., 2017) to 0.82 (Portuguese for Brazil, Osório Fde et al., 2013). For the French version (France, Chabrol et al., 2003), Cronbach’s alpha was 0.86 after deletion of Q3, which led the authors to propose a revised FTND without Q3. Removing Q3 improved the Cronbach’s alpha of the French for Switzerland (Etter et al., 1999), Portuguese for Brazil (Carmo & Pueyo, 2002) (slightly), and Turkish (Uysal et al., 2004) versions as well. In the original version (Heatherton et al., 1991), the alpha value was low (0.61), and Q3 loaded less than 0.30 (i.e., 0.23).

Test–retest reliability was assessed for 40% of the translations (10 of 25)—Dutch (Vink et al., 2005); French (Switzerland) (Etter et al., 1999); Farsi (Sarbandi et al., 2015); Japanese (Mikami et al., 1999); Malay (Yee et al., 2011); Malayalam (Jayakrishnan et al., 2012); Norwegian (Stavem et al., 2008); Portuguese for Brazil (Carmo & Pueyo, 2002; de Meneses-Gaya et al., 2009; Osório Fde et al., 2013); Thai (Klinsophon et al., 2017); and Turkish (Uysal et al., 2004)—with coefficients ranging from 0.50 (Yee et al., 2011) to 0.92 (de Meneses-Gaya et al., 2009) and intervals from 1 week (Moreno-Coutiño & Villalobos-Gallegos, 2017; Yee et al., 2011) to 1.8 years (Vink et al., 2005). In comparison, test–retest reliability analyses conducted on the original version showed coefficients ranging from 0.65 (smokers with schizophrenia, interval not specified) (Weinberger et al., 2007) to 0.87 (a sample of young smokers during military training, 6-week interval) (Haddock et al., 1999).

Inter-rater reliability was investigated for only one translation—Portuguese for Brazil (de Meneses-Gaya et al., 2009)—with the authors reporting an intraclass coefficient of 0.99 (95% confidence interval [CI]: 0.98–1.0) for two raters (sample size, 40).

Correlations with biomarkers and consumption patterns were documented for 52% of the translations (13 of 25): Arabic (Lebanon) (Salameh et al., 2013; Salameh & Khayat, 2014); Chinese (Taiwan) (Huang et al., 2006; Huang et al., 2009); Dutch (Vink et al., 2005); Farsi (Sarbandi et al., 2015); French (Switzerland) (Etter et al., 1999); Hindi (Jhanjee & Sethi, 2010); Italian (Ferketich et al., 2008; Grassi et al., 2014); Japanese (Kawada et al., 2010); Korean (Park et al., 2004); Malay (Yee et al., 2011); Malayalam (Jayakrishnan et al., 2012); Norwegian (Stavem et al., 2008); and Spanish [Spain (Becoña & Vázquez, 1998) and Mexico (Moreno-Coutiño & Villalobos-Gallegos, 2017)]. Correlations with biomarkers such as exhaled carbon monoxide and salivary and urinary cotinine levels were explored, as were correlations with duration of smoking (years), packs/year, packs/day, age of regular smoking, age of first cigarette, and willingness to pay for a cigarette after a day of abstinence. Correlations with biomarkers ranged from weak to moderate [e.g., CO: r = 0.288 (Salameh & Khayat, 2014) to 0.535 (Salameh et al., 2013)], in agreement with the correlations reported in studies that had used the original FTND [CO: r = 0.210 (Steinberg et al., 2005), 0.40 (Buckley et al., 2005), and 0.59 (Burling & Burling, 2003)].

Correlations with self-reported instruments investigating similar or dissimilar constructs was explored for 24% of the translations (6 of 25): Farsi (Robabeh et al., 2017); French for Switzerland (Etter et al., 1999); Japanese (Kawada et al., 2010; Mikami et al., 1999); Norwegian (Stavem et al., 2008); and Spanish [Spain (Becoña et al., 2010) and Mexico (Moreno-Coutiño & Villalobos-Gallegos, 2017)]. Correlations with the following scales were investigated: Beck Anxiety Inventory (BAI) and Beck Depression Inventory (BDI) (r = 0.091 and 0.116, respectively) (Moreno-Coutiño & Villalobos-Gallegos, 2017); Cigarette Dependence Scale (CDS) 12 and 5 (r = 0.60 and 0.72, respectively) (Stavem et al., 2008); Diagnostic and Statistical Manual (DSM) diagnostic criteria for nicotine dependence (DSM II-R; r = 0.70) (Mikami et al., 1999); DSM-V nicotine dependence (no correlation) (Robabeh et al., 2017); Nicotine Dependence Syndrome Scale (NDSS) (r = 0.58) (Becoña et al., 2010); Structured Clinical Interview for DSM-IV (r = 0.38) (Becoña et al., 2010); Tobacco Dependence Scale (TDS) (r = 0.352) (Kawada et al., 2010); and withdrawal symptoms (irritability, sensation of relief, and embarrassment: relative validity 93, 100, and 100, respectively) (Etter et al., 1999). For the original version, Pomerleau et al. (1994) showed a correlation with the Classification of Smoking by Motives addictive factor (r = 0.53). No relationship was detected with depression measured by using the Center for Epidemiological Studies Depression Scale (CESDS) (r = -0.24).

Predictive validity was investigated for 12% of the translations (3 of 25): French for Switzerland (Etter, 2005); Italian (Ferketich et al., 2008); and Spanish for Mexico (Moreno-Coutiño & Villalobos-Gallegos, 2017). The FTND predicted abstinence only at 7 weeks (Ferketich et al., 2008) and not in the long term (Etter, 2005; Moreno-Coutiño & Villalobos-Gallegos, 2017). In contrast, Kozlowski et al. (1994) showed that the FTND could predict smoking cessation to a small degree (Study 2: 16-month follow-up, r = -0.11).

Sensitivity/specificity (Se/Sp) was explored for 28% of the translations (7 of 25): Chinese for Taiwan (Huang et al., 2009); Italian (Svicher et al., 2018); Japanese (Mikami et al., 1999); Malay (Yee et al., 2011); Portuguese for Brazil (de Meneses-Gaya et al., 2009; Osório Fde et al., 2013); and Spanish for Spain (Becoña et al., 2010). Measures of reference varied among the studies: biomarkers (Huang et al., 2009: salivary cotinine – cutoff score = +4, Se = 76.2%, Sp = 67.5%); status (Svicher et al., 2018); measures of dependence (Becoña et al., 2010: score under the curve, 0.69; Osório Fde et al., 2013: cutoff score = +2, Se = 76%, Sp = 91%; de Meneses-Gaya et al., 2009: cutoff score = +4, Se = 80%, Sp = 74%; Mikami et al., 1999: cutoff score = +5/6, Se = 75%, Sp = 80%); and others (Yee et al., 2011).

Responsiveness to change has never been assessed.

QSU/QSU-b results

We retrieved four translations for the QSU, four for the QSU-b, and one for the QSU-12 (Table 4).

Table 5 presents the general characteristics of each study. Combustible cigarettes were the TNP evaluated in all studies. Nearly all studies involved moderate smoker samples, on average. In comparison, the original QSU (Tiffany & Drobes, 1991) and QSU-b (Cox et al., 2001) were developed with heavy smoker samples. Most studies had recruited samples from the general population, excepting studies in Spain [QSU (Cepeda-Benito et al., 2004) and QSU-b (Cepeda-Benito & Reig-Ferrer, 2004)] and the Netherlands [QSU-b (Littel et al., 2011)], which had recruited students. The sex ratio was balanced in all countries except China [QSU-b (Yu et al., 2010)] and Malaysia [QSU-b (Blebil et al., 2015)], where men were predominant (98% and 99%, respectively) and Spain [QSU (Cepeda-Benito et al., 2004)], where the sample comprised a majority of women (81.5%).

In comparison, the original version of the QSU (Tiffany & Drobes, 1991) was developed in a sample of 230 daily cigarette smokers (141 men and 89 women) assigned to one of three levels of deprivation (0, 1, or 6 hours). The mean participant age in each subgroup was 20.91, 20.64, and 22.73 years (SD not shown), and the consumption rate was 23.3, 21.28, and 22.36 cig/day, respectively (SD not shown). The QSU-b (Cox et al., 2001) was developed in two populations: Study 1 included 221 continuing smokers (111 men and 110 women; mean age, 30.23 ± 10.27 years; consumption rate, 26.95 ± 10.68 cig/day), while Study 2 included 112 smokers who were contemplating quitting (49 men and 63 women; mean age, 43.15 ± 11.60 years; consumption rate, 27.82 ± 12.57 cig/day).

Translation process. The translation process (Table 6) was not described for three translations: German QSU (Müller et al., 2001); French QSU-12 (Dethier et al., 2014); and Dutch QSU-b (Littel et al., 2011). References to guidelines were provided for three translations: Brazilian (Araujo et al., 2006); Chinese (Yu et al., 2010); and Malay (Blebil et al., 2015). The description for the Chinese QSU-b was minimal. Only the Brazilian team had provided some insight into the problems that arose during translation and the solutions they found.

Measurement properties. Table 7 reports the measurement properties explored for each translation. The results of the Spanish QSU-b (Cepeda-Benito & Reig-Ferrer, 2004) are not included, as the version developed by the authors is not a translation of the QSU-b but a new version with completely different content derived from the QSU, with items being positively keyed. It, therefore, represents a new version not comparable to the original US version of the QSU-b.

Measurement equivalence using DIF was never assessed. Structural validity was explored for all translations:

Internal consistency was explored for all translations of the QSU. The alpha values for QSU F1 and F2 ranged from 0.89 (Guillin et al., 2000) to 0.96 (Araujo et al., 2006) and 0.87 (Cepeda-Benito et al., 2004) to 0.92 (Araujo et al., 2006), respectively, in line with the values of the original version, in which scores representing these two factors demonstrated strong internal consistency (Cronbach’s alpha = 0.95 and 0.93, respectively). The Cronbach’s alpha values of the QSU-b translations in Malay (Blebil et al., 2015), Dutch (Littel et al., 2011), and Chinese (Yu et al., 2010) were 0.81, 0.83, and 0.92, respectively. When scored as a 10-item scale, the original QSU-b demonstrated high reliability as a measure of global craving in both initial and follow-up sessions (alpha = 0.89 and 0.87, respectively).

Test–retest reliability was never assessed.

Correlations with biomarkers and consumption patterns were explored for all translations (QSU/QSU-b) except the German (Müller et al., 2001) and Spanish (Cepeda-Benito et al., 2004) QSU versions, with the latter focusing only on factor analysis.

Correlations with self-reported measures exploring similar or dissimilar constructs were not studied in the French (Guillin et al., 2000) or Spanish versions of the QSU (Cepeda-Benito et al., 2004) but were investigated in other translations (QSU/QSU-b):

Responsiveness to change, predictive validity, sensitivity, and specificity were not assessed.

MNWS results

Four studies were retrieved for the MNWS (Table 8), corresponding to three translations of the MNWS (nine-item version) into Chinese (China) (Yu et al., 2010), Korean (Kim et al., 2007), and Malay (Blebil et al., 2014) and one translation of the MNWS-R and MNWS (eight-item version) into Italian (same paper) (Svicher et al., 2017).

In the MNWS, there were variations in the items included in the original versions that were used as a basis for translation. The Chinese version was based on the MNWS developed by Cappelleri et al. (2005), while the Italian version was based on that by Hughes (1992). The Malay and Korean versions included “impatience” and were based on the MNWS developed by Jorenby et al. (1996) (Table 9).

Table 10 presents the general characteristics of each study. Combustible cigarettes were the TNP evaluated in all studies. Most subjects reported previous attempts to quit, except in the Malay sample (Blebil et al., 2014), where 77% of the subjects had not attempted to quit previously.

All studies were run with moderate smoker samples, on average, except for the study involving Koreans living in the US, which had recruited light smokers (Kim et al., 2007). In comparison, the original MNWS was developed with heavy smokers (Hughes & Hatsukami, 1986). Mean participant age ranged from 34 to 47.7 years. Men were predominant in all studies except in that in Italy, where women were slightly preponderant (59%) (Svicher et al., 2017).

Translation process. All four papers provided a description of the translation process used to develop each translation (Table 11). Only the Korean version (Kim et al., 2007) presented a brief report of the difficulties encountered and solutions found. References to guidelines or recommendations were given for all translations except for the Italian version (Svicher et al., 2017). Descriptions of the translation process were detailed for all translations except the Chinese version (Yu et al., 2010).

Measurement properties. Table 12 reports the measurement properties explored for each translation. All translations were assessed for structural validity, with a one-factor structure reported for the Italian MNWS eight-item version (Svicher et al., 2017) and the Malay nine-item version (Blebil et al., 2014).

A two-factor structure was reported for the Chinese version of the MNWS nine-item version (Yu et al., 2010) and the Korean nine-item version (Kim et al., 2007). The structure of the Chinese version was identical to the two-factor structure of the original version reported by Cappelleri et al. (2005): negative effect (F1, four items: depressed mood; irritability, frustration, or anger; anxiety; and difficulty concentrating), insomnia (F2, two items: difficulty going to sleep and difficulty staying asleep), and three single items (craving, restlessness, and increased appetite). A review of the items showed a slight discrepancy in those used as originals, with impatience listed in the Korean version but not in the Chinese, where insomnia represents two items (difficulty going to sleep and difficulty staying asleep). For the Korean version, F1 represented early-occurring disorders in mental functioning, and F2 represented disorders in physiological functioning and late-occurring disorders in mental functioning (i.e., increased appetite, disturbed sleep, depression, and impatience), explaining 66% of the variance. A two-factor structure was also reported in the Italian MNWS-R.

Internal consistency was explored for all translations, with Cronbach’s alpha values of 0.85 (Italian MNWS eight-item version) and 0.91 (Malay nine-item version) reported for the monofactorial structure. In comparison, the Cronbach’s alpha values of the original eight-item MNWS explored by Toll et al. (2007) were 0.80 (abstinence study), 0.83 (framing study), and 0.82 (naltrexone + patch study) at the initial time point after quitting. Internal consistency was not evaluated by Jorenby et al. (1996). With regard to the bifactorial structure, only a global alpha value was provided for the Chinese version (0.90). Cappelleri et al. (2005) reported alpha values ranging from 0.76 to 0.87 for the negative domain and 0.71 to 0.83 for the insomnia domain in the studies and time assessed.

Measurement equivalence using DIF was never assessed.

In total, 75% percent of the translations (3 of 4; Italian, Korean, and Malay) were assessed for test–retest reliability, with coefficients ranging from 0.51 (Kim et al., 2007) to 0.88 (Blebil et al., 2014) and intervals from 1 (Blebil et al., 2014; Kim et al., 2007) to 3 (Svicher et al., 2017) months.

Correlations with consumption patterns and biomarkers were reported for three translations: Chinese, Italian, and Malay. Correlations with self-reported measures of dependence, craving, and anxiety were explored for all translations (Table 12).

Responsiveness to change, predictive validity, sensitivity, and specificity were not assessed.

Underlying data

All data underlying the results are available as part of the article and no additional source data are required.

Extended data

Open Science Framework. Measurement properties of the translations of instruments evaluating the subjective effects of tobacco- and nicotine-containing products: a systematic review of the literature. https://doi.org/10.17605/OSF.IO/3Z2EV (Acquadro, 2019).

This project contains the following extended data:

Reporting guidelines

Open Science Framework: PRISMA checklist for: “Measurement properties of the translations of instruments evaluating the subjective effects of tobacco- and nicotine-containing products: a systematic review of the literature” https://doi.org/10.17605/OSF.IO/3Z2EV (Acquadro, 2019).

Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).