Systematic Review of Translation and Cultural Adaptations of Autism Spectrum Disorder’s Screening Tool: The Modified Checklist for Autism in Toddlers, Revised (M-CHAT-R)

Cultural adaptation

The process of cultural adaptation is complicated, as it goes beyond the surface level of language translation and involves adaptation of constructs and technical changes before reaching functional equivalence between the adapted instrument and the original version (DuBay and Watson, 2019). A growing body of literature has suggested several guidelines in the translation and cultural adaptation process that are needed to produce high-quality adapted versions (Beaton et al., 2000; Sousa and Rojjanasrirat, 2011; Soto et al., 2015). Generally, the cultural adaptation process involves reproducing the selected instrument in a new target language, pretesting the new instrument with a small number of individuals from the target population, and analysing the psychometric properties of the new instrument. However, producing new versions of a selected instrument with functional equivalence requires a lengthy reproduction and pretesting process with many quality assurance measures (Ware et al., 1995; Sousa and Rojjanasrirat, 2011). While most studies report that simple forward-backward translation with optional pretesting might be sufficient to determine the validity of a translated version (Perneger et al. 1999; da Mota Falcão et al. 2003), a newer study found that research that undertook more quality assurance measures resulted in higher-feasibility and higher-quality instruments (Acquadro et al., 2008).

In addition, an analysis of the psychometric properties of the translated version should be performed before concluding the validity and reliability of the instrument being used in the new target population (Rodríguez and Bernal, 2012; Al Maskari et al., 2018). Many studies have documented differences in psychometric properties between translated versions and the originals despite the extensive steps taken in the reproduction process (Fourie and Feinauer, 2005; Kondolot et al., 2016). Hence, the synthesis of the equivalence dimensions of a translated instrument cannot be assumed simply by changing the linguistic code (McKenna and Doward, 2005). Necessities in planning the complex reproduction processes should be researched, and psychometric testing should be analysed before determining the functional equivalence of the translated instrument (Beaton et al., 2000). Several guidelines on translation and adaptation processes have been published (Beaton et al., 2000; Soto et al., 2015; Tsang et al., 2017) noting the importance of forward and backward translation, alongside the qualification of translators and sample size suggestions for pilot testing. However, little cross-cultural adaptation research has documented adherence to such guidelines, nor is there any literature that has examined the evaluation of such processes (Gudmundsson, 2009).

Problem statement and study aims

Studies and findings from cross-cultural research have always been valued for their significant clinical implications among healthcare professionals who provide care for diverse populations worldwide (Sousa and Rojjanasrirat, 2011). For example, a recent systematic literature review by Bevan et al., (2020) found that most Hispanic or Spanish developmental and behavioural screening instruments were not validated despite being widely used across the United States. Although the systematic review by Bevan et al., (2020) focused only on Spanish-language screening instruments, it undoubtfully offered valuable insights into the importance of the validation and norming of translated and culturally adapted instruments for a target that differs from that of the original. The results of their review suggested that language might be an incomplete proxy, while geographic region of origin or heritage is accounted for as a more culturally relevant factor in providing health care. However, more literature is needed to provide supportive data for such a notion. Unfortunately, in the area of ASD, most of the current available systematic review publications have mainly reviewed the available ASD screening tools and their psychometric properties (Soto et al., 2015; Levante et al., 2019; Petrocchi et al., 2020), while few have examined the feasibility of the translated versions and quality assurance measures taken in the translation and cultural adaptation process. Empirically, traditional simple forward-backward translation methods are insufficient to produce a quality instrument (Gjersing et al., 2010). The cultural values that depict how we view the world often represent what is considered appropriate parenting and child developmental growth (van Kleeck, 1994). Discounting the importance of cultural adaptation and the translation process in developing a translated screening instrument often results in substantial differences in identification rates and the development of poor instruments (Soto et al., 2015). Specifically, the risk level of impairments in related developmental areas may be over- or underestimated (van Kleeck, 1994). Some guidelines have been published to address the complications of cultural adaptation and translation processes (Guillemin et al., 1993; Mokkink et al., 2010; Sousa and Rojjanasrirat, 2011; DuBay and Watson, 2019). Similarities have been noted among published guidelines in several areas, including forward and backward translation, pretesting, and psychometric analysis. However, there are also notable distinctions in the evaluation of specific criteria to assess the quality of the translated instruments. For example, requirements for a translator and the synthesis process could be different from one guideline to another (Beaton et al., 2000; Sousa and Rojjanasrirat, 2011). While the synthesis process proposed by Sousa and Rojjanasrirat (2011) recommends a third independent translator, the guidelines by Beaton et al., (2000) suggest inviting the translators who were involved in the initial forward translation process to discuss and resolve to reach a consensus rather than having another independent translator.

Perhaps the incongruities in guidelines highlight the possibility that there is more than one way to achieve the cultural validity of a translated instrument (Al Maskari et al., 2018), while methodologies for the translation and adaptation of instruments have yet to be established (Gudmundsson, 2009). Hence, examining the steps and levels of adaptation processes used by some of the published studies may extend the knowledge of determining the feasibility of the existing guidelines and offer insights into future work on cultural adaptation and validation (Bird et al., 2014). However, the instrument used for evaluating measurement properties should always be of high quality to provide a sound conclusion. Although there is no single accepted guideline or checklist for evaluating the translation and cultural adaptation process, the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist (Mokkink et al., 2010), which was developed with the involvement of an international multidisciplinary team, seems to be a promising evaluation tool. Moreover, the COSMIN checklist has the advantage of being used as a modular tool where completion of the whole checklist is not necessary. According to the manual (Mokkink et al., 2010), in the evaluation of cross-cultural validity, only Box G should be completed. Similar to the core steps listed in most translation and cultural adaptation guidelines (Beaton et al., 2000; Soto et al., 2015; Tsang et al., 2017), the COSMIN Box G checklist evaluates the quality of the translation procedures and methodological measures involved. Hence, the COSMIN checklist would seem to be justifiable as an instrument to evaluate translation and cultural adaptation studies.

In summary, despite the growing number of studies on the translation and cultural adaptation of screening instruments, there seems to be limited literature that reviews the quality of translation and cultural adaptation processes. Therefore, this systematic review paper aims to methodologically review published translation and cultural adaptation studies of the M-CHAT-R. As the aim of this systematic review study is not to evaluate the effectiveness of an autism screening instrument but to evaluate the level and quality of the cultural adaptation process, comparing different translations of the same instrument seems appropriate in helping to recognise what is feasible and what is not in the adaptation process. Hence, the M-CHAT-R, which has been documented as one of the most translated instruments (Al Maskari et al., 2018; Petrocchi et al., 2020) undoubtfully offers more informative data for evaluation than other screening instruments. Ultimately, this research aims to (1) identify translations and cultural adaptations of the M-CHAT-R for different populations, (2) report on the methodology and psychometric properties of the M-CHAT-R translations, (3) evaluate the quality of translation and cultural adaptation processes and their adherence to the recommended COSMIN checklist and, (4) and critically discuss the implications of these findings to improve future research and practice.

Population

All studies involving children aged 48 months and below who had completed the M-CHAT-R in languages other than the original English version were included.

Study selection criteria

According to the research questions of this systematic review, both predefined inclusion and exclusion criteria were specified. The predefined inclusion criteria were (1) validation studies of translation of the M-CHAT-R, (2) published papers in peer-reviewed journals, (3) papers written in English, and (4) papers published from 1 January 2009 to 31 December 2022. The predefined exclusion criteria were as follows: (1) epidemiological studies and guidelines, (2) retrospective studies and systematic reviews, (3) dissertation theses or conference papers, (4) studies that used the M-CHAT-R to detect developmental disorders other than ASD, (5) studies that compared the M-CHAT-R with other instruments, and (6) papers without the specific aim of evaluating the psychometric properties and validation properties of the M-CHAT-R.

Search and strategy

An electronic search was conducted through various databases, such as EBSCOhost, Google Scholar, PubMed, Elsevier and other resources, and a secondary manual search was performed to include papers identified through references and citations as of 25 July 2023. Keywords such as ‘M-CHAT-R,’ ‘Modified Checklist for Autism in Toddlers Revised,’ ‘validation’, and ‘translation’ were applied to all research databases during the search. Papers were filtered by publication date from 1 January 2009 to 31 December 2022. A total of 281 articles were successfully identified at the initial stage. After the removal of the duplicates of 120 articles, 161 articles were screened based on the predefined inclusion and exclusion criteria. Most articles did not meet the inclusion criterion for validation studies, as most of the articles were epidemiological studies or systematic reviews. Hence, only 13 studies were included in this systematic review paper. Figure 1 below shows the PRISMA flow diagram, which portrays the steps performed in identifying literature from the databases (Han, 2023).

Figure 1. A PRISMA flow chart showing the selection of the included studies for systemic review.

Data extraction procedure

The final 13 eligible articles were organised, and the basic information with descriptive data was extracted and presented in table format. Descriptive data included the country, sample size, sample source, cut-off point for positive screening, screening age range, and mean age of the toddlers recruited as samples. The table also included the studies' psychometric properties in terms of sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

Quality assessment of the studies

As this systematic review emphasised the methodological evaluations and characteristics of the selected papers, the COSMIN checklist was used as the basis for the evaluations, as it is consistent with the aim of this study (Mokkink et al., 2010). Although guidelines for cultural adaptation and the translation process are available, they were not used in this review study, as they serve as a template and form of advice (Beaton et al., 2000) but not as a validated evaluation checklist as the COSMIN checklist does. The COSMIN checklist is a modular tool that evaluates the measures reported in a study, including its psychometric properties and validity. The COSMIN checklist has nine boxes that identify the main measurement properties. Each box contains various items and can be used individually depending on the target assessed. In this systematic review paper, the evaluation of measures and data obtained were mainly for translation and cross-cultural validity purposes; hence, Box G of the COSMIN checklist, which explicitly evaluates cross-cultural validity, was selected.

The Box G cross-cultural validity checklist is composed of a total of 15 items. The first three items focus on the adequacy of sample size and management of missing items for the studies being reviewed. Subsequently, Items four to nine serve as a clear checklist for the procedures involved in the translation of the M-CHAT-R items. Items 10 to 11 were included in the pre-test process conducted in the studies being reviewed to ensure that the translated versions were understandable and well adapted to the respective culture. Items 12 to 13 examine the characteristics of the samples in the studies reviewed and the quality of the methods and study designs. Our systematic review concentrated exclusively on Item one to Item 13 to evaluate cultural adaptation and the quality of the translation process. In other words, this review did not focus on the statistical methods used in previous studies; hence, Items 14 and 15 were not scored (Mokkink et al., 2010).

Since there was no consensus in the translation and cultural adaptation process, items were not rated based on numbers or scores but on the availability of specific information related to the items. When the item description was available in the particular study, the study was scored with a ‘+’, and when the study did not fulfil the item description, it was scored with a ‘-‘. If a study did not provide enough detail to fulfil the item description or no information was provided, it was scored as ‘NA’ (Mokkink et al., 2010). After each study was scored, the obtained data with evaluation markers were extracted into Excel (version 2303) and presented in a table. A simple total sum of ‘+’ scores, which signified the availability of information for each item, was calculated for each paper. Table 1 shows the item descriptions of Cosmin Checklist Box G for cross-cultural validity.

Overview of the studies and measures

A total of 281 articles were successfully identified through the electronic search. During the initial review, 120 articles were removed due to duplication. Based on the predefined exclusion criteria mentioned above, 148 articles were removed. Finally, only 13 papers were included in this systematic review.

When the 13 eligible papers were identified for this systematic review, basic descriptive measures of the literature were extracted, presented in Table 2. All13 studies reported the results of the validation of the M-CHAT-R for 12 languages across13 countries. The targeted samples for all 13 studies were children aged 14 months to 48 months to whom the M-CHAT-R was administered. Table 2 shows that the highest mean age among the thirteen studies was 31.66 months, while the lowest mean age was 18.22 months (Magán-Maganto et al., 2020; Jonsdottir et al., 2021).

Out of the 13 studies reviewed, 11 studies recruited their sample population from specific healthcare settings such as hospitals, paediatric centres, or community healthcare centres. According to the basic demographic and descriptive data extracted, the number of samples recruited in each study varied widely. The lowest number of recruited samples was 13 (McClure et al., 2018), and no psychometric properties could be analysed due to the limited sample size. The largest number of recruited samples was 7,928 in China (Guo et al., 2019). Although the M-CHAT-R has been suggested for screening toddlers aged 16 to 30 months (Robins et al., 2014), not all 13 studies that were reviewed adhered to the suggested age range. A study from Spain (Magán-Maganto et al., 2020) recruited subjects as young as 14 months, while two other studies from Indonesia (Windiani et al., 2016) and South Africa (Vorster et al., 2021) recruited samples aged up to 48 months.

Psychometric properties of the translated version of the M-CHAT-R

The psychometric properties of 13 studies were extracted. The properties of the desired screening accuracy of the 13studies, namely, sensitivity, specificity, PPV, and NPV, are presented in Table 2. Only seven studies reported all screening accuracy properties. Although the PPV and NPV were reported in several studies, the prevalence rate of ASD in the study population was not reported in every paper. The prevalence rate was specified in only six studies (Brennan et al., 2016; Carakovac et al., 2016; Tsai et al., 2019; Magán-Maganto et al., 2020; Oner and Munir, 2020; Jonsdottir et al., 2021). A Chilean study (Coelho-Medeiros et al., 2019) reported sensitivity and specificity values but did not specify the PPV and NPV. Among all the studies that reported sensitivity and specificity values, only a French study (Sangare et al., 2019) reported a low sensitivity value of 0.50, while the others reported good sensitivity and specificity values. Except for the three studies with large sample sizes, namely, the Chinese study (Guo et al., 2019) with 7,928 samples, the Spanish study (Magán-Maganto et al., 2020) with 6,625 samples and the Turkish study (Oner and Munir, 2020) with 6,712 samples, four studies reported high PPVs and NPVs, similar to those of the original study (Robins et al., 2001). Interestingly, the three studies that reported low PPVs (<0.40) were the studies that recruited subject samples from the general population across different provinces or districts of a country. Although the PPV varied among studies, the NPVs reported by all studies remained consistently in the high range of 87%-100%.

Five studies did not report any screening accuracy values. An Albanian study (Brennan et al., 2016) reported only the PPV and failed to report the NPV and sensitivity and specificity values due to a high drop-out rate, while Nepali (McClure et al., 2018) and Serbian (Carakovac et al., 2016) studies could not provide sensitivity and specificity values due to small sample sizes. An Indian study (Divya et al., 2020), despite having an adequate sample size of 450 individuals, did not provide the exact values, as the authors adopted a different validation approach. These authors reported excellent reliability, with a Cronbach’s alpha score of 0.894, and validated the Tamil version of the M-CHAT-R (T-M-CHAT-R) through divergent validity. The T-M-CHAT-R was statistically compared to the Indian Scale for the Assessment of Autism (ISAA), a standardised assessment tool that serves to diagnose and measure the severity of ASD specifically for the Indian population, and yielded a high positive correlation of r=0.01. As such, the authors suggested that the results obtained in their study were similar to the psychometric properties of the original M-CHAT-R study (Robins et al., 2001).

Quality appraisal of the reviewed studies

In alignment with the research objectives, COSMIN Box G on cross-cultural validity was used as a quality appraisal tool for all 13 studies, particularly rating the translation and adaptation process, research methodology, and data handling. Table 3 presents the COSMIN checklist results of all 13 studies. Although 13 items were evaluated, the highest number of ‘+’ items, which showed the availability of information presented in each study, was only eight. None of the studies reviewed provided all information needed for each item. The studies that provided the most information were Jonsdottir et al., (2020), McClure et al. (2018), Tsai et al. (2019), and Vorster et al. (2021), while the studies by Coelho-Medeiros et al. (2019), Sangare et al. (2019), and Windiani et al. (2016) provided the least information, with only two items that could be scored ‘+’. Our results showed a lack of justification for highlighting percentages and methods in handling missing items during the validation process, as an Albanian study (Brennan et al., 2016) was the only study that disclosed the percentage of missing items out of all thirteen studies.

On the other hand, studies conducted in Serbia (Carakovac et al., 2016) and Nepal (McClure et al., 2018) were unable to recruit an adequate sample size, resulting in a failure to report psychometric properties. According to the COSMIN checklist, only nine studies described the translation process from the original English M-CHAT-R to other languages. All nine studies adopted the forward and backward translation approach. However, only six studies highlighted the involvement of experts in the relevant subject matters (Brennan et al., 2016; McClure et al., 2018; Tsai et al., 2019; Divya et al., 2020; Jonsdottir et al., 2020; Vorster et al., 2021).

In addition, among these six studies, only five stated how differences between certain items in the translated version compared to the original version were resolved. Furthermore, only four studies explicitly stated that a committee reviewed the translated versions. Only the Nepali study (McClure et al., 2018) satisfied all the checklist items regarding the translation and adaptation process; in this study, a certified Nepali translator first translated the Nepali version of the M-CHAT-R, then a modification and review was performed by expert panels with double modifications, and finally, back-translation was conducted by a bilingual native Nepali speaker. On the other hand, the Chinese and Icelandic versions of the M-CHAT-R fulfilled all checklist items similar to the Nepali version, except for the lack of committee review. In contrast, the Northern Sotho version of the M-CHAT-R (Vorster et al., 2021) was reviewed by the research committee, but the study did not mention how the difference between items was resolved.

COSMIN Box G includes four checklist items regarding pretesting, sample similarities, and methodological flaws. Although none of the studies seemed to have any major methodological flaws, only five studies performed a pre-test, and four studies specifically described the samples used for the pretesting process (McClure et al., 2018; Tsai et al., 2019; Jonsdottir et al., 2020; Oner and Munir, 2020; Vorster et al., 2021).