Keywords
dementia, walker, geriatric evaluation, fall risk, walking, translation, validity, reliability
This article is included in the Global Public Health gateway.
This article is included in the Dignity in Aging collection.
dementia, walker, geriatric evaluation, fall risk, walking, translation, validity, reliability
Dementia is a neurocognitive disorder in which mental abilities as associative memory, reasoning and perceptual speed are severely reduced.1 It is estimated that there are approximately 290,000 people with dementia in the Netherlands and it is the fastest-growing cause of death in the Netherlands.1–3 Dementia is characterized by a cognitive decline related to memory and in at least one of the other domains: perceptual-motor function, language, executive function, attention, and social cognition.4,5
People with dementia have a high risk of falling and the cause is multifactorial. Intrinsic and extrinsic factors such as vision or balance problems and the use of medication can play a role in falling.6 Also, impairments in attention and executive functions are prominent risk factors for falls among people living with dementia.7 Importantly, balance and gait problems are common and progressive in dementia as well. People with dementia have a yearly risk to fall of sixty to eighty percent. This is twice as high as in their cognitively healthy peers.7,8 Moreover, people with dementia have a high risk of admission to a long-care setting.7,8
To compensate for deficits in balance, gait and strength that accompany dementia, the use of a walker is often recommended. A walker is known for improving stability during walking in cognitively healthy adults.9,10 However, the use of a walker in people with dementia is associated with a threefold increased odds of falls compared to their healthy peers who walk with a walker.6,11 Lindemann et al. identified obstacles and opening a door towards a person as the main problems for older adults using a walker.12
Many standardized scales for balance and gait developed for cognitively healthy older adults have been evaluated for reliability and validity in people living with dementia.13 The existing measures provide only partial information regarding mobility and few target elements that become relevant as dementia progresses.13 In addition, none of these instruments can assess the functional skills for safe walking when using a walker in people with dementia.
The high frequency of falls among people with dementia and the absence of a measurement instrument to assess walking skills in walker use led Hunter et al. to develop the Safe Use of Mobility Aid Checklist (SUMAC) in 2020.14 The SUMAC was generated by a focus group (N = 12) consisting of one geriatrician, two registered nurses, five physiotherapists, and four occupational therapists.14 The generated tasks were scored for relevance by an independent panel of five healthcare professionals. This resulted in a final selection of nine tasks in two domains (physical functioning (PF) and interaction with equipment (EQ) that form the SUMAC. The SUMAC can evaluate the skills of walking with a walker in people with dementia who walk with a walker independently or assess the success of training when the walker has been identified as an appropriate level of gait aid to compensate for deficits.14 The SUMAC has a good test-retest and inter-rater reliability (ICC = 0.89 for PF and ICC = 0.88 for EQ) and shows good construct validity (rs = 0.92 for the PF score and rs = 0.82 for the EQ score).14 This makes it a promising tool to give specific advice and training in walker use and to reduce the risk of falling in people with dementia.
Currently, there is no Dutch assessment instrument that can assess walking skills in people with dementia who need a walker. Therefore, a Dutch measuring instrument must be developed that can assess the safety of walking with a walker in people with dementia. This will lead to better advice and training regarding the use of a walker and thereby reduce the risk of falls in people with dementia in the Netherlands.
Therefore, the primary objective of this study is to 1) translate the SUMAC from English to Dutch and 2) determine the inter-rater reliability, the test-retest reliability, and the content validity of the SUMAC-NL to assess walking skills in older people with dementia living in the Netherlands who walk with a walker.
This study has an observational clinimetric design and involved two phases. The first part of this study consisted of the translation of the English version of the SUMAC into Dutch (SUMAC-NL). In the second part of this study, the reliability and validity evaluation of the SUMAC-NL was performed. The second part of the study was a replication study of the work by Hunter et al. (2020).14
The SUMAC was translated into Dutch between July and August 2021. The tasks and domains in the SUMAC remained the same. The translation was done according to the Beaton guideline15 and consisted of five phases. The first phase was the forward translation from English to Dutch by two Dutch native speakers with a medical background. The first translator knew the purpose of the instrument. The second translator did not know the instrument to keep the translation as objective as possible. The second phase was the synthesis of the two translations by the Dutch native translators. The third phase was the backward translation from Dutch to English by two English native speakers. The first translator had a medical background and the second translator did not. Both translators were unaware of the purposes of the SUMAC. The fourth phase consisted of a review of the translations by the researcher, two experts, and the developer of the SUMAC. The fifth phase consisted of getting an understanding of the items. For the feasibility of this study, the last step has been replaced by the evaluation of the content validity (Figure 1). This last step of the Beaton guideline consists of testing the prefinal version of the translated test. This involves administering the translated test to 30 to 40 people and then conducting interviews to ask participants what they thought was meant by the questions and the chosen response.15
Participants
Older people with dementia who walk with a walker, living in an intramural setting were recruited by their physiotherapist between 1 December 2021 and 28 February 2022 by convenience sampling. The recruiter selected potential participants from the institution’s patient database. Potential participants and their families were informed about the study through the information letter and through verbal explanations. They were then asked for their participation.
The inclusion criteria were: 1) diagnosed with dementia 2) walking with a walker, 3) Dutch-speaking, 4) able to follow simple instructions, 5) able to walk 60 meters without the supervision of another person, 6) able to perform the SUMAC-NL and 7) having someone available who can make decisions (e.g. on participation) if necessary.16 Exclusion criteria for this study were: 1) comorbidities that limit mobility, such as Multiple Sclerosis (MS), a past Cerebro Vascular Accident (CVA), or (poly) neuropathy, and 2) people who have recently undergone surgery that limits mobility. All participants provided written informed consent. The legal representative of the participant was asked to give informed consent if the participant was deemed unable to do so.
Data collection
The demographic data requested from the participants were age, sex, amount of medication use, number of comorbidities, level of activities of daily living, and fear of falling. The 6-item Katz ADL scale was used to assess the participants’ ability to perform ADL activities and the short-FES-I is administered to determine the fear of falling of the participants. The Katz ADL (ICC = 0.82, p < 0.05) and short-FES-I (α = 0.84) are both reliable instruments to measure these parameters.17–19
The assessment of the performed tasks by the participants was done by a group of raters consisting of a convenience sample of five physiotherapists who were recruited by one of the researchers (JK). All physiotherapists provided written informed consent for using their demographic data at the start of data collection. The physiotherapists were all legally registered therapists working in the physical therapy primary practice.
The participants were asked to perform the tasks of the SUMAC-NL while being filmed. Filming took place on 25 February and 4 March 2022 in a corridor with sufficient space in the institution where the participants live. Participants were asked if they wanted to be recognised in the video. If they did not agree, their faces were blurred in the videos so that the participants are unrecognisable in the video. The videos were stored in a 2FA-protected digital location, according to the data management plan and were deleted after the raters finished rating the videos. These videos were only used for the reliability assessment.
Ethical considerations
This study was conducted following the Declaration of Helsinki and the FAIR principle and was approved by the Hanze University of Applied Sciences Groningen’s Ethical Review Board (HEAC), with approval number heac.2021.026. Participants gave informed consent for understanding the information letters and that participating in the study was voluntary. In addition, they gave permission to contact their GP in case of unexpected findings, the collection and use of data and longer storage of the data as well as whether or not they wanted to be contacted again for follow-up studies.
Reliability
Each of the raters was asked to watch the provided instruction video and to read the instruction manual of the SUMAC-NL before starting the evaluation of the participants. The instruction video consisted of a detailed instruction of the components and items within the SUMAC-NL and an example of the evaluation with the SUMAC-NL. The videos of the participants performing the SUMAC-NL were sent to the raters on 11 March 2022. The evaluations of the participants was done twice at an interval of one week. The ratings were collected by the researcher on 8 April 2022.
The order of the tasks in the videos was the same as in the SUMAC-NL. The videos were offered in a random order. This order differed between the first and second evaluation session. Each rater evaluated all participants. The raters could evaluate the videos digitally at a location of their choice.
Validity
The expert panel consisted of six master of science trained geriatric physiotherapists, one master of science geriatric physiotherapist student, and one master of science trained physiotherapist. The experts in the panel have a mean age of 31.5 ± 2.7 years. Each of them has experience in working with older people with dementia (3.5 ± 3.2 years).
Before determining content validity, the expert panel received an instructional video explaining how to score the content of the SUMAC-NL. In addition, the expert panel received a digital score sheet in Microsoft Office Word in which the assessment of the SUMAC-NL was recorded. The scores of the experts were compared with each other and the content validity ratio (CVR) and content validity index (CVI) were calculated. Scoring of the SUMAC-NL by the expert panel to determine content validity took place between 18 February 2022 and 25 March 2025.
IBM SPSS statistical package version 28 (IBM Corp, Armonk, NY) was used for descriptive analysis and reliability analysis. Microsoft Office Excel 2019 was used in the content validity calculation.
Reliability
An a priori sample size calculation for the reliability study was done using the formula of Shoukri et al. (2004).20 This formula showed that a sample consisting of nine participants and five raters making two evaluations is required to determine the desired ICC of >0.9 (CI ± 0.1). An ICC of >0.90 is considered excellent reliability.21 The demographic characteristics of the participants are presented using descriptive statistics.
Continuous data (SUMAC-NL subscores) were used and presented quantitatively. Missing data were considered ‘missing completely at random’ (MCAR) and imputed by the Expectation-Maximization (E-M) algorithm.22 Normality checks were done by the Shapiro-Wilk test and using histograms and Q-Q plots.
The absolute reliability was calculated using the standard error of the measurement (SEM) and minimal detectable change with a 95% confidence interval (MDC95) for the PF and EQ domains of the SUMAC-NL. A smaller SEM means higher absolute reliability.23 The SEM was estimated using the pooled standard deviation and the ICC for each group. The following formula was used: SEmeasurement = s √(1-rxx).24,25 The MDC95 was calculated using the SEM by the following formula: MDC95 = (1.96)*(√2)*(SEM).25,26
The relative values of inter-rater and test-retest reliability were calculated using ICC. An average measurement, absolute agreement, 2-way random-effects model (ICC 2,k) was used for determining the ICC.21 An ICC value less than 0.5 is indicative of poor reliability, values between 0.5 and 0.75 indicate moderate reliability, values between 0.75 and 0.9 indicate good reliability and values greater than 0.9 indicate excellent reliability.21
Content validity
Content validity was determined using CVR and the CVI and was calculated separately for both domains of the SUMAC-NL. The CVR was used to show the acceptance of tasks in the instrument and the CVI was used to determine the content validity of both the entire domains.27
For CVR, the expert panel separately scored each item of the SUMAC-NL on a three-point Likert scale where a score of one is essential, a score of two is useful but not essential and a score of three is not essential. After obtaining the scores of the assessments of the experts, the CVR was calculated based on the following formula in Microsoft Office Excel 2019, where ne represents the number of panel members who have chosen the item of ‘essential’, and N is the number of panel members:
The CVR varies between -1 and 1. A higher score indicates further agreement of members of the panel on the necessity of an item in the instrument. The CVR of each domain is compared with the values in Lawshe’s table, and if the CVR is equal to or higher than the value (0.75) in Lawshe’s table, the item is preserved.27
To determine the CVI, all items of the instrument have been scored on three parameters by the experts: relevance, simplicity, and clarity. Each parameter was scored on a 4-point Likert scale from disagree to totally agree. The average of the total scores was taken from the table and the CVI of each item was calculated with the following formula:
The CVI of both domains of the SUMAC-NL was determined by dividing the sum of the CVI scores per item by the number of items.28,29 A CVI of >0.79 was considered appropriate.29
Nine participants with dementia participated in this study. The sample consisted of five women (55.6%) and four men with a mean age of 85.56 (± 7.49). See Table 1 for the demographic characteristics of the participants.
Variable | Mean ± SD, or n % |
---|---|
Age (years) | 85.56 ± 7.49 |
Sex n (% female) | 5 (55.6%) |
FES-I score | 12.11 ± 4.26 |
Katz ADL score | 4.44 ± 2.00 |
Number of prescribed Medications | 5.13 ± 1.13 |
Number of Comorbidities | 3.33 ± 1.50 |
The absolute reliability values were: SEM for the PF domain was 1.20 and for the EQ domain was 2.22; the MDC95 was 3.33 for the PF domain and was 6.15 for the EQ domain (Table 2).
A good to excellent inter-rater reliability in both evaluation moments in the PF domain (ICC = 0.85 (0.63, 0.96), p < 0.001 for assessment 1 and ICC = 0.90 (0.74, 0.97), p < 0.001 for assessment 2) and EQ domain (ICC = 0.86 (0.64, 0.96), p < 0.001 for assessment 1 and ICC = 0.75 (0.41 – 0.94), p < 0.001 in assessment 2) was found (Table 2). In addition, we found excellent test-retest reliability in the PF (ICC = 0.94, 95%CI (0.86, 0.99), p < 0.001) and EQ (ICC = 0.91 95%IC (0.79, 0.98), p < 0.001) domains (Table 2). The full raw data can be found under Underlying data.30
The evaluation of the content validity was done by an expert panel (N =8) with experience in working with older people with dementia.
In the evaluation of the content validity, we calculated a CVI of 0.86 on the PF domain and 0.96 on the EQ domain. In addition, we calculated the CVR of 0.53 on the PF domain and 0.78 on the EQ domain (Table 3).
Domains of the Dutch Safe Use of Mobility Aid Checklist | ||
---|---|---|
Physical functioning (PF) | Interaction with walker (EQ) | |
CVI | 0.86 | 0.96 |
CVR | 0.53 | 0.78 |
The CVI of all individual tasks was >0.79. The CVR of tasks 4 and 5 in the PF domain were both 0, of task 7 it was 0.5, and of tasks 8 and 9, it was 0.25. The other tasks showed a CVR >0.75. In the EQ domain, task 4 had a CVR of 0 and task 5 had a CVR of 0.5. The other tasks had a CVR >0.75.
This replication study of the study from Hunter et al.14 includes the translation of the English SUMAC into Dutch and the determination of the test-retest reliability, inter-rater reliability, and validity of the Dutch SUMAC (SUMAC-NL) for people with dementia who walk with a walker. This study found a good to excellent inter-rater reliability and test-retest reliability, and strong support for content validity.
Small differences were found between our results and the results of the study by Hunter et al.14 The inter-rater reliability of the SUMAC-NL seems to be higher than the English version. The test-retest reliability of the SUMAC-NL seems higher for both the PF domain and the EQ domain (0.94 vs. 0.89 and 0.91 vs. 0.88, respectively). The SEM is smaller for the PF domain (1.20 vs. 1.31) and higher for the EQ domain (2.22 vs. 1.93) in the SUMAC-NL. The MDC95 for the PF domain of the SUMAC-NL is smaller than that of the English-language SUMAC (3.33 vs. 3.64) and greater for the EQ domain (6.15 vs. 5.35). These differences could be explained by the differences in the way the raters were instructed. In the study by Hunter et al.,14 the raters received a one-hour one-on-one training session while, in this study, an online education via an instructional video was chosen due to the COVID-19 pandemic.14 As a result, the interpretation of the scoring method may differ slightly between the two raters’ groups.
The content validity index (CVI) shows that both the PF domain and the EQ domain are considered valid by the expert panel. However, the content validity ratio (CVR) shows that tasks 4, 5, 7, 8, and 9 are not valid (CVR <0.75) tasks for determining physical function (PF domain) for walking with a walker in people with dementia. This finding is not in line with the results from the study by Hunter et al.14 The content validity was determined by two sets of experts to include these tasks important for observing physical functioning.14 In addition, it appears that walking through a doorway causes serious problems for older people who use a walker, which is observed in tasks 7, 8, and 9.12 Because of the results from previous research and a good CVR (>0.75) for tasks 7, 8, and 9 in the EQ domain, it seems appropriate to keep these tasks in the FF domain of the SUMAC-NL.12
The expert panel in this study determined tasks 4 and 5 as not valid for both the PF and EQ domain. These tasks involve walking with a double task (horizontal head turns) and a cognitive task. The expert panel scored lower for these tasks on the simplicity parameter within the CVI. One of the reasons given is that the wording of the tasks can be interpreted in different ways. However, different studies show that gait performance decreases and fall risk increases with increasing task complexity with the addition of a second cognitive task in community-dwelling adults or cognitively impaired older adults.31–34 Therefore, it seems appropriate to observe these performances and keep tasks 4 and 5 in the SUMAC-NL but the formulation may have to be modified in the future.
Other measuring instruments can score balance and physical performance to interpret a risk of falls. A commonly used measuring instrument for evaluating physical performance and walking in older adults is the Berg Balance Scale (BBS).35,36 The BBS measures the balance of participants through transfers and static postures, without a walker. The BBS is a reliable instrument for determining the risk of falling and has good reliability in older people with dementia to assess physical functioning.35,37,38 This measuring instrument can have value in determining walking safety but is not specified for determining walking safety in older people with dementia who walk with a walker. The SUMAC-NL seems to be an appropriate tool to fill this gap because of its good reliability in people with dementia and the functional items that are included for walking with a walker. The SUMAC-NL provides professionals a standardised assessment tool to assess safe walking with a walker in people with dementia.
Previous research shows that people with dementia are well able to learn tasks both implicitly and explicitly, making possible therapy effective.39,40 The SUMAC-NL could be performed in people living with dementia who already walk with a walker or in people who have received training in walking with a walker so that the walking skills and/or the success of the training can be evaluated.
The translation of the English-language SUMAC into Dutch was carried out systematically using the Beaton guideline.15 This increases the comparability between different cultures.41 For the feasibility of this study, the last step in the guideline has not been reviewed. We have overcome this by determining the content validity and reliability, which also includes the comprehensibility of the instrument.
Another strength of this study is that the experts involved in the validation assessment of the SUMAC-NL were all master trained physiotherapists who are experts in the field of physical activity who have experience in geriatrics and work with people living with dementia.42
This study also has some limitations. For the power and feasibility of this study, the reliability phase of this study consisted of several assessors (N = 5) and nine participants. Although the sample size is appropriate to achieve power in the analysis, research with a larger sample size (n >50) is recommended.43 After data collection, there was missing data (<10%). This data was assumed to be missing completely at random (MCAR). After imputation, we did calculations on the imputed data and the raw data. The results of both calculations were similar to each other.
This study was done in older people with dementia who were able to perform the SUMAC-NL. Therefore, the results cannot be generalised to people with dementia who may not be able to complete the SUMAC-NL. The people in our sample show no severe fear of falling on the FES-I and are largely independent in ADL following the Katz ADL scores. The results are therefore not generalizable to people with severe fear of falling and are more dependent in ADL. Further research is needed on the psychometric properties of the SUMAC-NL in people who are less likely to be able to perform the SUMAC-NL, have a greater fear of falling and are more dependent in ADL.
The inter-rater reliability, test-retest reliability, and validity support the use of the SUMAC-NL in practice. The SUMAC-NL should be considered to assess the walking skills in people living with dementia who use a walker independently and/or to evaluate the training success of walking with a walker. With the help of the SUMAC-NL, more specific training, education, and advice can be given for walker use.
Future research must determine whether the use of the SUMAC-NL can improve the walking safety of older people with dementia who walk with a walker and thus can reduce the risk of falling in this population. Because of the lack of generalizability of this study to older people with dementia with more severe fear of falling and are more dependent in ADL, further research into the psychometric properties of the SUMAC-NL in people with dementia is advised.
In this study, the English-language SUMAC was translated into the Dutch SUMAC-NL. The SUMAC-NL seems to have good to excellent inter-rater reliability, test-retest reliability, and content validity. The SUMAC-NL seems to be a promising instrument for the assessment of walking with a walker in older people with dementia.
DataverseNL: Safe Use of Mobility Aid Checklist (SUMAC) for walker use in people living with dementia. https://doi.org/10.34894/RUKZQP 30
This project contains the following underlying data:
Dataset of the reliability study
Dataset of the validity study
This project contains the following extended data:
- README_ENG_ SUMAC_NL_translation_relibility_validation 27-1-2023.pdf
- informatiebrief SUMAC-NL studie 2-12-2021.docx (participant information sheet)
- Handleiding SUMAC-NL_final.docx (manual for using the SUMAC-NL)
- SUMAC-NL_final.docx (The official SUMAC-NL tool for clinical use)
- Dataset definitions.pdf
- The English version of the SUMAC can be found via: http://www.mobility-in-aging-lab.ca/#publications
Data are available under the terms of the Creative Commons Zero “No rights reserved” data waiver (CC0 1.0 Public domain dedication).
This research is part of the graduation phase of the Physiotherapy Sciences programme at Utrecht University. Our thanks go to the supervisors and fellow students who helped in the process of the research.
Views | Downloads | |
---|---|---|
F1000Research | - | - |
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
Is the work clearly and accurately presented and does it cite the current literature?
Yes
Is the study design appropriate and is the work technically sound?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Yes
If applicable, is the statistical analysis and its interpretation appropriate?
Yes
Are all the source data underlying the results available to ensure full reproducibility?
No source data required
Are the conclusions drawn adequately supported by the results?
Yes
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: aging, mobility, device evaluation and translation.
Is the work clearly and accurately presented and does it cite the current literature?
Partly
Is the study design appropriate and is the work technically sound?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Yes
If applicable, is the statistical analysis and its interpretation appropriate?
Yes
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Are the conclusions drawn adequately supported by the results?
Partly
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: rehabilitation, ageing, exercise, physiotherapy
Alongside their report, reviewers assign a status to the article:
Invited Reviewers | ||
---|---|---|
1 | 2 | |
Version 3 (revision) 30 Jun 25 |
read | |
Version 2 (revision) 15 Apr 25 |
read | read |
Version 1 14 Sep 23 |
read | read |
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Already registered? Sign in
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
If your email address is registered with us, we will email you instructions to reset your password.
If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.
Comments on this article Comments (0)