This study has an observational clinimetric design and involved two phases. The first part of this study consisted of the translation of the English version of the SUMAC into Dutch (SUMAC-NL). In the second part of this study, the reliability and validity evaluation of the SUMAC-NL was performed. The second part of the study was a replication study of the work by Hunter et al. (2020). ¹⁴

The SUMAC was translated into Dutch between July and August 2021. The tasks and domains in the SUMAC remained the same. The translation was done according to the Beaton guideline ¹⁵ and consisted of five phases. The first phase was the forward translation from English to Dutch by two Dutch native speakers with a medical background. The first translator knew the purpose of the instrument. The second translator did not know the instrument to keep the translation as objective as possible. The second phase was the synthesis of the two translations by the Dutch native translators. The third phase was the backward translation from Dutch to English by two English native speakers. The first translator had a medical background and the second translator did not. Both translators were unaware of the purposes of the SUMAC. The fourth phase consisted of a review of the translations by the researcher, two experts, and the developer of the SUMAC. The fifth phase consisted of getting an understanding of the items. For the feasibility of this study, the last step has been replaced by the evaluation of the content validity ( Figure 1). This last step of the Beaton guideline consists of testing the prefinal version of the translated test. This involves administering the translated test to 30 to 40 people and then conducting interviews to ask participants what they thought was meant by the questions and the chosen response. ¹⁵

Participants

Older people with dementia who walk with a walker, living in an intramural setting were recruited by their physiotherapist between 1 December 2021 and 28 February 2022 by convenience sampling. The recruiter selected potential participants from the institution’s patient database. Potential participants and their families were informed about the study through the information letter and through verbal explanations. They were then asked for their participation.

The inclusion criteria were: 1) diagnosed with dementia 2) walking with a walker, 3) Dutch-speaking, 4) able to follow simple instructions, 5) able to walk 60 meters without the supervision of another person, 6) able to perform the SUMAC-NL and 7) having someone available who can make decisions (e.g. on participation) if necessary. ¹⁷ Exclusion criteria for this study were: 1) comorbidities that limit mobility, such as Multiple Sclerosis (MS), a past Cerebro Vascular Accident (CVA), or (poly) neuropathy, and 2) people who have recently undergone surgery that limits mobility. All participants provided written informed consent. The legal representative of the participant was asked to give informed consent if the participant was deemed unable to do so.

Data collection

The demographic data requested from the participants were age, sex, amount of medication use, number of comorbidities, level of activities of daily living, and fear of falling. The 6-item Katz ADL scale was used to assess the participants’ ability to perform ADL activities and the short-FES-I is administered to determine the fear of falling of the participants. The Katz ADL (ICC = 0.82, p < 0.05) and short-FES-I (α = 0.84) are both reliable instruments to measure these parameters. ^{18 – 20}

The assessment of the performed tasks by the participants was done by a group of raters consisting of a convenience sample of five physiotherapists who were recruited by one of the researchers (JK). All physiotherapists provided written informed consent for using their demographic data at the start of data collection. The physiotherapists were all legally registered therapists working in the physical therapy primary practice. This group of raters only participated in the reliability part of this study.

The participants were asked to perform the tasks of the SUMAC-NL while being filmed. Filming took place on 25 February and 4 March 2022 in a corridor with sufficient space in the institution where the participants live. Participants were asked if they wanted to be recognised in the video. If they did not agree, their faces were blurred in the videos so that the participants are unrecognisable in the video. The videos were stored in a 2FA-protected digital location, according to the data management plan and were deleted after the raters finished rating the videos. These videos were only used for the reliability assessment.

Ethical considerations

This study was conducted following the Declaration of Helsinki and the FAIR principle and was approved by the Hanze University of Applied Sciences Groningen’s Ethical Review Board (HEAC), with approval number heac.2021.026. Participants gave informed consent for understanding the information letters and that participating in the study was voluntary. In addition, they gave permission to contact their GP in case of unexpected findings, the collection and use of data and longer storage of the data as well as whether or not they wanted to be contacted again for follow-up studies.

Reliability

Each of the raters was asked to watch the provided instruction video and to read the instruction manual of the SUMAC-NL before starting the evaluation of the participants. The instruction video consisted of a detailed instruction of the components and items within the SUMAC-NL and an example of the evaluation with the SUMAC-NL. The videos of the participants performing the SUMAC-NL were sent to the raters on 11 March 2022. The evaluations of the participants was done twice at an interval of one week. The ratings were collected by the researcher on 8 April 2022.

The order of the tasks in the videos was the same as in the SUMAC-NL. The videos were offered in a random order. This order differed between the first and second evaluation session. Each rater evaluated all participants. The raters could evaluate the videos digitally at a location of their choice.

Validity

An expert panel was established to determine content validity. This group is composed of experts other than the reliability section raters. The expert panel consisted of six master of science trained geriatric physiotherapists, one master of science geriatric physiotherapist student, and one master of science trained physiotherapist. The experts in the panel have a mean age of 31.5 ± 2.7 years. Each of them has experience in working with older people with dementia (3.5 ± 3.2 years).

Before determining content validity, the expert panel received an instructional video explaining how to score the content of the SUMAC-NL. In addition, the expert panel received a digital score sheet in Microsoft Office Word in which the assessment of the SUMAC-NL was recorded. The scores of the experts were compared with each other and the content validity ratio (CVR) and content validity index (CVI) were calculated. Scoring of the SUMAC-NL by the expert panel to determine content validity took place between 18 February 2022 and 25 March 2025.

IBM SPSS statistical package version 28 (IBM Corp, Armonk, NY) was used for descriptive analysis and reliability analysis. Microsoft Office Excel 2019 was used in the content validity calculation.

Reliability

An a priori sample size calculation for the reliability study was done using the formula of Shoukri et al. (2004). ²¹ This formula showed that a sample consisting of nine participants and five raters making two evaluations is required to determine the desired ICC of >0.9 (CI ± 0.1). An ICC of >0.90 is considered excellent reliability. ²² The demographic characteristics of the participants are presented using descriptive statistics.

Continuous data (SUMAC-NL subscores) were used and presented quantitatively. Missing data were considered ‘missing completely at random’ (MCAR) and imputed by the Expectation-Maximization (E-M) algorithm. ²³ Normality checks were done by the Shapiro-Wilk test and using histograms and Q-Q plots.

The absolute reliability was calculated using the standard error of the measurement (SEM) and minimal detectable change with a 95% confidence interval (MDC ₉₅) for the PF and EQ domains of the SUMAC-NL. A smaller SEM means higher absolute reliability. ²⁴ The SEM was estimated using the pooled standard deviation and the ICC for each group. The following formula was used: SE _measurement = s √(1- r _xx ). ^{25 , 26} The MDC ₉₅ was calculated using the SEM by the following formula: MDC ₉₅ = (1.96)*(√2)*(SEM). ^{26 , 27}

The relative values of inter-rater and test-retest reliability were calculated using ICC. An average measurement, absolute agreement, 2-way random-effects model (ICC 2, k) was used for determining the ICC. ²² An ICC value less than 0.5 is indicative of poor reliability, values between 0.5 and 0.75 indicate moderate reliability, values between 0.75 and 0.9 indicate good reliability and values greater than 0.9 indicate excellent reliability. ²²

Content validity

Content validity was determined using CVR and the CVI and was calculated separately for both domains of the SUMAC-NL. The CVR was used to show the acceptance of tasks in the instrument and the CVI was used to determine the content validity of both the entire domains. ²⁸

For CVR, the expert panel separately scored each item of the SUMAC-NL on a three-point Likert scale where a score of one is essential, a score of two is useful but not essential and a score of three is not essential. After obtaining the scores of the assessments of the experts, the CVR was calculated based on the following formula in Microsoft Office Excel 2019, where n _e represents the number of panel members who have chosen the item of ‘essential’, and N is the number of panel members: CVR = n e − N 2 N 2

The CVR varies between -1 and 1. A higher score indicates further agreement of members of the panel on the necessity of an item in the instrument. The CVR of each domain is compared with the values in Lawshe’s table, and if the CVR is equal to or higher than the value (0.75) in Lawshe’s table, the item is preserved. ²⁸

To determine the CVI, all items of the instrument have been scored on three parameters by the experts: relevance, simplicity, and clarity. Each parameter was scored on a 4-point Likert scale from disagree to totally agree. The average of the total scores was taken from the table and the CVI of each item was calculated with the following formula: CVI = ( the number of experts giving a rating of either 3 or 4 ) the number of experts

The CVI of both domains of the SUMAC-NL was determined by dividing the sum of the CVI scores per item by the number of items. ^{29 , 30} A CVI of >0.79 was considered appropriate. ³⁰

The absolute reliability values were: SEM for the PF domain was 1.20 and for the EQ domain was 2.22; the MDC ₉₅ was 3.33 for the PF domain and was 6.15 for the EQ domain ( Table 2).

A good to excellent inter-rater reliability in both evaluation moments in the PF domain (ICC = 0.85 (0.63, 0.96), p < 0.001 for assessment 1 and ICC = 0.90 (0.74, 0.97), p < 0.001 for assessment 2) and EQ domain (ICC = 0.86 (0.64, 0.96), p < 0.001 for assessment 1 and ICC = 0.75 (0.41 – 0.94), p < 0.001 in assessment 2) was found ( Table 2). In addition, we found excellent test-retest reliability in the PF (ICC = 0.94, 95%CI (0.86, 0.99), p < 0.001) and EQ (ICC = 0.91 95%IC (0.79, 0.98), p < 0.001) domains ( Table 2). The full raw data can be found under Underlying data. ³¹

The evaluation of the content validity was done by an expert panel (N =8) with experience in working with older people with dementia.

In the evaluation of the content validity, we calculated a CVI of 0.86 on the PF domain and 0.96 on the EQ domain. In addition, we calculated the CVR of 0.53 on the PF domain and 0.78 on the EQ domain ( Table 3).

The CVI of all individual tasks was >0.79. The CVR of tasks 4 and 5 in the PF domain were both 0, of task 7 it was 0.5, and of tasks 8 and 9, it was 0.25. The other tasks showed a CVR >0.75. In the EQ domain, task 4 had a CVR of 0 and task 5 had a CVR of 0.5. The other tasks had a CVR >0.75.

The translation of the English-language SUMAC into Dutch was carried out systematically using the Beaton guideline. ¹⁵ This increases the comparability between different cultures. ⁴² For the feasibility of this study, the last step in the guideline has not been reviewed. We have overcome this by determining the content validity and reliability, which also includes the comprehensibility of the instrument.

Another strength of this study is that the experts involved in the validation assessment of the SUMAC-NL were all master trained physiotherapists who are experts in the field of physical activity who have experience in geriatrics and work with people living with dementia. ⁴³

This study also has some limitations. For the power and feasibility of this study, the reliability phase of this study consisted of several assessors (N = 5) and nine participants. Although the sample size is appropriate to achieve power in the analysis, research with a larger sample size (n >50) is recommended. ⁴⁴ After data collection, there was missing data (<10%). This data was assumed to be missing completely at random (MCAR). After imputation, we did calculations on the imputed data and the raw data. The results of both calculations were similar to each other.

The variation in age of the expert group for determining content validity is small. The heterogeneity of the expert group was not explicitly taken into account when recruiting the expert group. As a result, the expert group is homogeneous in terms of age and experience. This reduces the generalizability of the results of this study to older or more experienced therapists. It may be useful to have a comparison of the content validity results with therapists with more experience in order to calibrate the results.

This study was done in older people with dementia who were able to perform the SUMAC-NL. Therefore, the results cannot be generalised to people with dementia who may not be able to complete the SUMAC-NL. The people in our sample show no severe fear of falling on the FES-I and are largely independent in ADL following the Katz ADL scores. The results are therefore not generalizable to people with severe fear of falling and are more dependent in ADL. Further research is needed on the psychometric properties of the SUMAC-NL in people who are less likely to be able to perform the SUMAC-NL, have a greater fear of falling and are more dependent in ADL.

Participants were included based on the diagnosis of dementia by a specialist in geriatrics. However, this did not include the degree of dementia. These data were not available at the time of recruitment. This makes the SUMAC-NL difficult to specify according to the cognitive state of an individual with dementia. In retrospect, this could have been remedied by taking the Montreal Cognitive Assessment (MoCA). ⁴⁵

The inter-rater reliability, test-retest reliability, and validity support the use of the SUMAC-NL in practice. The SUMAC-NL should be considered to assess the walking skills in people living with dementia who use a walker independently and/or to evaluate the training success of walking with a walker. With the help of the SUMAC-NL, more specific training, education, and advice can be given for walker use.

The results of this study show that the Dutch version of the SUMAC has similar psychometric properties to the original Canadian version. This suggests that the original instrument is robust, and resistant to cultural and linguistic variations, reinforcing its generalisability and applicability. This is important because using non-validated translations in clinical practice or research can lead to measurement errors, misinterpretations and ultimately suboptimal care.

In addition, this study fills a gap in the Dutch context. Until now, there has been no validated instrument specifically designed to observe the safe use of a rollator by people with dementia in Dutch contexts. As such, this study provides concrete tools for professionals working with people with dementia.

Future research must determine whether the use of the SUMAC-NL can improve the walking safety of older people with dementia who walk with a walker and thus can reduce the risk of falling in this population. Because of the lack of generalizability of this study to older people with dementia with more severe fear of falling and are more dependent in ADL, further research into the psychometric properties of the SUMAC-NL in people with dementia is advised.

Finally, this translation and validation also provides opportunities for international comparison and collaboration. By using standardised and validated instruments in different countries, it will be possible to compare data on a larger scale and collectively contribute to evidence-based practice in walking skills for people with dementia who walk with a walker.