ALL Metrics
-
Views
-
Downloads
Get PDF
Get XML
Cite
Export
Track
Study Protocol
Revised

Psychometric properties of portable devices used in kinematic gait assessment after stroke: a systematic review protocol

[version 2; peer review: 2 approved with reservations]
PUBLISHED 08 Feb 2024
Author details Author details
OPEN PEER REVIEW
REVIEWER STATUS

Abstract

Background

Portable devices comprising inertial sensors are an alternative for kinematic gait assessment in people after stroke. Nevertheless, there is limited knowledge regarding the psychometric properties of these devices, specifically their reliability and measurement error, in the context of post-stroke gait assessment. Our objective is to investigate the psychometric properties of these devices in evaluating kinematic gait parameters in people after stroke. We will also analyze the reliability and measurement error of these devices and identify portable devices that can assess changes in lower limb angular movements during gait in this population.

Methods

We will search for studies in English, without publication date restriction, that have evaluated psychometric properties of portable devices utilizing inertial sensors to assess kinematic gait parameters in people after stroke. Searches will be performed in the following electronic databases: Cochrane Central Registry of Controlled Trials (CENTRAL), Medline/PubMed, EMBASE Ovid, CINAHL EBSCO, PsycINFO Ovid, IEEE Xplore Digital Library (IEEE), and Physiotherapy Evidence Database (PEDro). Gray literature will also be searched, including published and unpublished studies (dissertations and theses). The Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) risk of bias tool will be used to assess the quality of studies that analyzed reliability and measurement error of devices.

Expected results

This will be the first review to analyze reliability and measurement error of portable devices utilizing inertial sensors to evaluate kinematic gait parameters in people after stroke, using the COSMIN tool. Then, we hope to elucidate this topic and help the decision-making of clinicians regarding the use of these devices. Finally, we also hope to list the portable devices that assessed changes in angular lower limb movements during gait in this population.

Registration

The protocol was registered in Open Science Framework on May 11th 2023 (https://doi.org/10.17605/OSF.IO/7M6DA).

Keywords

stroke, gait, portable devices, inertial sensors, psychometric properties, reliability, measurement error

Revised Amendments from Version 1

According to the reviewer's comments, we adjusted some excerpts in the text to better understand the article.
Furthermore, in this version, we mainly seek to clarify the reason for choosing the psychometric properties "validity" and "measurement error". With this issue in mind, we adjusted the objective of the paper.

See the authors' detailed response to the review by Rachel Reoli

Introduction

Background

The World Health Organization defines stroke as a focal (sometimes global) neurological impairment of vascular origin and sudden onset that lasting more than 24 hours and may lead to death.1 This condition has a global impact: approximately 12.2 million cases and 6.5 million deaths reported in 2019.2

People after stroke often experience impairments including spasticity,3 sensory changes,3 and deficits in strength and muscle control,4 which commonly impact the gait cycle. The gait cycle can be divided into a stance phase, comprising initial contact, loading response, mid stance, terminal stance, pre-swing; and a swing phase further subdivided into initial swing, mid swing and terminal swing.5 Stroke sequelae contribute to several changes in this cycle.

Alterations in the angular movements of the lower limbs, such as excessive knee extension and reduced hip flexion and dorsiflexion (more prevalent in the paretic limb) can significantly impact gait dynamics. Consequently, spatial parameters (e.g., step length and stride), temporal parameters (e.g., double support duration and paretic balance time), and spatiotemporal parameters (e.g., speed and cadence) are also commonly altered.6,7

In this context, the quantitative assessment of kinematic gait using accurate systems is essential to the gait rehabilitation program after a stroke. For example, optoelectronic systems (gold standard) allow a three-dimensional gait analysis by capturing light from active (light-emitting diodes, LED) or passive markers (reflect light from infrared sources) using cameras. Nevertheless, these systems are costly and demand a controlled environment, limiting the monitoring of patients in natural settings.

Portable devices are an alternative to optoelectronic systems. They are composed of inertial sensors (accelerometers and gyroscopes) that may be used individually or combined with other sensors (magnetometers) to form inertial measurement units (IMUs).8 These devices acquire information regarding the movement trajectory of one or several body parts during gait and allow the analysis of intra- and inter-individual variations of kinematic gait parameters.8 They also have several advantages: portability, low weight, no wires, low energy consumption, easy installation and handling, and low manufacturing cost.8 For this reason, sensors are widely used for gait analysis because they provide continuous real-time data for clinicians and patients.9

Why is this review important?

To support the use of a measuring instrument in research or clinical practice, it is essential that its psychometric properties undergo testing and prove adequate.10 These properties include reliability (the proportion of total variance in measurements that is due to “true” differences between patients)11; validity (the degree to which an instrument measures it purports to measure)11; responsiveness (the ability of instrument to detect changes over time in the outcome measured)11 and measurement error (systematic and random error in a patient’s score that is not attributed to true changes in the construct being measured).11 This investigation is particularly important when evaluating specific movement disorders, such as those in people after stroke.

In this context, recent studies by Ferraris et al. (2021) and Cimolin et al. (2022) delved into the agreement between a gold standard system (instrumented 3D gait analysis) and a test system (single RGB-D camera) for gait assessment in individuals with stroke and Parkinson’s disease.12,13 Their findings underscore the importance of conducting studies of this nature. Conversely, Peters et al. (2021), in a systematic review, analyzed the quality of studies carried out with wearable devices to evaluate gait and mobility of people after stroke.14 However, this review reported few studies that included an assessment of psychometric properties. Notably, terms related to psychometrics were not incorporated into the search strategy, and the review did not specifically aim to evaluate the psychometric properties of the devices.

Therefore, we believe a review that evaluates psychometric properties of portable devices utilizing inertial sensors could facilitate a more targeted analysis of this technology, aiding clinicians in making informed decisions regarding its usage.

Another important aspect refers to angular changes in the lower limb movements in people after stroke. These changes can alter the gait dynamics, constituting an important outcome to be evaluated in this population. Currently, portable devices are used to analyze various kinematic gait parameters, such as speed, cadence, balance, asymmetry and support times (single and double).15 However, it’s noteworthy that recent reviews incorporating inertial sensors have yet to assess angular changes in the lower limb during post-stroke gait.9,14

Objective

This systematic review protocol has as main objective to investigate the psychometric properties of portable devices utilizing inertial sensors to assess kinematic gait parameters in people after stroke, and analyze the reliability and measurement error of these devices. As a secondary objective, we will investigate which portable devices have been utilized to assess changes in lower limb angular movements during gait in this population.

Methods

Study design

This systematic review protocol was developed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA-P).16 Our study question was structured based on the recommendation by Munn et al. (2018) for reviews of psychometric properties,17 which covers the following: 1) construct of interest or name of measurement instruments (kinematic gait parameters); 2) population (people after stroke); 3) type of instrument (portable devices utilizing inertial sensors); and 4) measurement properties (validity, reliability, responsiveness and measurement error).

Study registration

This protocol has been registered in Open Science Framework on May 11th 2023 (Registration DOI: https://doi.org/10.17605/OSF.IO/7M6DA).18

Inclusion criteria

Types of studies

We will encompass studies evaluating the psychometric properties, including validity, reliability, responsiveness and measurement error, of portable devices utilizing inertial sensors to assess kinematic gait parameters in people after stroke. We will not seek other measurement properties, such as internal consistency (a subtype of reliability) and construct validity (a subtype of validity), as they are not applicable to ’results measurement instruments,’ such as portable devices.

Only full-text studies, published or not (e.g., theses and dissertations), will be included.

Participants

We will include studies involving individuals aged 18 years and older, of any sex or race, affected by a stroke, irrespective of its etiology and neuroanatomical location. Participants in the acute, subacute, or chronic phase, who are capable of walking, with or without the assistance of an assistive device, will be considered for inclusion.

Instrument types

Studies that assessed kinematic gait parameters using portable devices utilizing some type of inertial sensor (accelerometers, gyroscopes, and inertial measurement units) will be included.

Primary outcomes

Psychometric properties (validity, reliability, responsiveness and measurement error) related to outcomes obtained by portable devices will be considered the primary outcome.

Secondary outcomes

Angular parameters of lower limb joints (hip, knee, and ankle) during gait and spatial, temporal or spatiotemporal parameters (i.e. step length, stride length, step width, single support time, double support time, swing time, walking speed, cadence and number of steps), will be considered secondary outcomes.

Search strategy

We will search for studies in English without publication date restriction.

Data sources

Searches will be performed in the following electronic databases: Cochrane Central Registry of Controlled Trials (CENTRAL), Medline/PubMed, EMBASE Ovid, CINAHL EBSCO, PsycINFO Ovid, IEEE Xplore Digital Library (IEEE), and Physiotherapy Evidence Database (PEDro).

A strategy was developed based on descriptors indexed in the Medical Subject Headings Database (MeSH terms) related to: condition (e.g., ‘stroke’ and ‘cerebrovascular accident’), inertial sensors (e.g., ‘inertial sensors’, ‘inertial measurement unit’, and ‘wearable devices’), and psychometric properties (e.g., ‘psychometric properties’, ‘validity’, and ‘reliability’). The terms regarding condition were combined with those for portable devices and psychometric properties using the boolean operator AND. The initial strategy was developed for the MEDLINE/PubMed database and will be translated and adapted to other databases according to codes and syntaxes. Details of the search strategy are shown in the Extended data.19

Searching other resources

We will search the reference lists of primary studies included in the review to identify relevant studies. The gray literature will also be searched, including published and unpublished studies (dissertations and theses). Moreover, we will contact authors to obtain information from relevant articles and search for additional information regarding the device on the websites of manufacturers.

Data collection and analysis

Study selection

Two independent researchers (SS and KR) will use Rayyan software20 to examine the titles and abstracts of studies and exclude irrelevant articles. Full texts of all potentially eligible articles will be retrieved. The same researchers will read the texts, identify studies, record reasons for excluding ineligible studies, and discuss disagreements. A third researcher (RS) will be consulted in case of disagreements. Duplicates will be removed, and the authors of studies will be contacted if more information is needed. The selection process will be recorded following the PRISMA flowchart.21

Data extraction

Two researchers will independently extract (SS and KR) the following data regarding methods, participants, and characteristics of inertial sensors using a predefined form (see the Extended data)22:

  • Methods: objectives, year of publication, study duration, number of study centers and location, assessment environments, psychometric properties assessed, time between repeated measurements, results, and withdrawals.

  • Participants: age, sex, anthropometric data (body weight, height, and body mass index), stroke duration and etiology (acute or hemorrhagic), most affected side (right or left), stroke stage (acute, subacute, or chronic), and use of assistive devices.

  • Inertial sensors: technology type, sensor components, reports of discomfort and inadequacy, kinematic parameters, location of sensors on the body, angular data (hip, knee, and ankle joints), cost of equipment, and feasibility of implementation.

Quality assessment

The Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) risk of bias tool23 will be used to assess the quality of studies that analyzed reliability and measurement error. The reliability box contains six items referring to study design and three regarding statistical methods. The measurement error box contains six items referring to study design and four regarding statistical methods. The checklists of the COSMIN risk of bias tool23 classify items as very good, adequate, doubtful, inadequate, or not applicable. The final score of each checklist is based on the worst score counts. Only reliability and measurement error studies will be evaluated as these properties are the only ones covered by the COSMIN tool.23 Specific tools for evaluating the risk of bias in studies assessing other properties, such as validity, are currently lacking in the literature. Two independent reviewers (RS and ST) will conduct the quality assessment, resolving any discrepancies through discussion between them.

To assess the quality of the evidence we will use the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) modified approach, cited in the COSMIN methodology for systematic reviews of Patient-Reported Outcome Measures PROMs.24,25 This GRADE version can be easily adjusted for outcome measurement instrument studies. In consonance with this approach, four factors must be considered: 1) risk of bias, 2) inconsistency, 3) indirectness and 4) imprecision (wide confidence intervals). The tool also has four levels of evidence quality: “high”, “moderate”, “low” and “very low”. In the evaluation, we will start from the highest level of evidence quality (high), which may be downgraded depending on the factors: risk of bias, inconsistency, indirectness and imprecision.24,25 The assessment of evidence quality will be conducted by the same independent evaluators who assessed the quality of studies, and any discrepancies will be resolved through discussion between them.

Data synthesis

The study results will be presented through a combination of tables and graphs. Additionally, we will provide a comprehensive narrative synthesis, offering a descriptive explanation of the findings.

The data extracted from the forms will be summarized in a table using descriptive statistics. Measures of central tendency, including mean and standard deviation, will be employed for continuous parametric data such as sample size and anthropometric characteristics. Non-parametric data will be described using median and interquartile ranges.

The results of the quality assessment of studies carried out using the COSMIN risk of bias tool will be presented in a “Table of quality and results of studies on reliability and measurement error”, according to the model provided by the COSMIN tool.23

The GRADE classification will be presented in a format adapted from the ‘Summary of Findings Table,’ as outlined in the COSMIN methodology for systematic reviews of Patient-Reported Outcome Measures (PROMs).24

Dissemination plans

A systematic review will be performed according to this protocol and published in a relevant journal in the health area. Additionally, we aim to present the results at conferences and academic events.

Study status

Currently, the review is in the phase of “study selection”.

Discussion

To our knowledge, this is one of the first reviews to investigate psychometric properties of portable devices utilizing inertial sensors to evaluate kinematic gait parameters in people after stroke. While Petraglia et al. (2019) analyzed the validity of inertial sensors used in gait analysis in both healthy and pathological adults, their review did not address ‘reliability’ and ‘measurement error.26 Consequently, the present review is the first to comprehensively analyze these properties utilizing the COSMIN tool. Furthermore, the prior review by Petraglia et al. (2019) only covered studies from 2005 to 2017.26 Thus, we believe that our review serves as a valuable update, incorporating the latest literature on this topic.

Furthermore, detailing the characteristics of portable devices that have assessed changes in angular lower limb movements during gait will offer a comprehensive overview. This is particularly significant as few studies employing portable devices have specifically addressed this question, highlighting the novelty and contribution of our review.

Finally, recent reviews have focused on assessing gait aspects in individuals post-stroke using portable devices.9,14 Consequently, we chose not to delve into certain important topics, namely the gait analysis environment and data processing and analysis techniques, to avoid redundancy. It is important to acknowledge that this decision could be considered a limitation of the current study.

Conclusion

This study aims to investigate the psychometric properties of portable devices for gait analysis in people after stroke and analyze the reliability and measurement error of these devices. Through this investigation, we aim to provide insights into this topic, ultimately contributing to informed decision-making for clinicians regarding the utilization of these devices.

Authors’ contributions

R.S.S., T.S.R., and K.M.O.B.F.R, conceived the idea of the study and contributed to the design of the research. R.S.S. wrote the initial draft. All authors contributed to the writing, editing, and approval of the final protocol.

Data availability

Underlying data

No underlying data are associated with this article.

Extended data

Figshare: Search strategy for: ‘Psychometric properties of portable devices used in kinematic gait assessment after stroke: a systematic review protocol’, https://doi.org/10.6084/m9.figshare.22787045.19

This project contains the following extended data:

  • Search strategy.pdf

Figshare: Data extraction form to: ‘Psychometric properties of portable devices used in kinematic gait assessment after stroke: a systematic review protocol’, https://doi.org/10.6084/m9.figshare.22787456.22

This project contains the following extended data:

  • Data extraction form.pdf

Reporting guidelines

Figshare: PRISMA-P checklist for ‘Psychometric properties of portable devices used in kinematic gait assessment after stroke: a systematic review protocol’. https://doi.org/10.6084/m9.figshare.22782863.27

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

Comments on this article Comments (0)

Version 2
VERSION 2 PUBLISHED 15 Jun 2023
Comment
Author details Author details
Competing interests
Grant information
Copyright
Download
 
Export To
metrics
Views Downloads
F1000Research - -
PubMed Central
Data from PMC are received and updated monthly.
- -
Citations
CITE
how to cite this article
Silva R, Silva S, Santos V et al. Psychometric properties of portable devices used in kinematic gait assessment after stroke: a systematic review protocol [version 2; peer review: 2 approved with reservations]. F1000Research 2024, 12:677 (https://doi.org/10.12688/f1000research.134161.2)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?
Key to Reviewer Statuses VIEW
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions
Version 2
VERSION 2
PUBLISHED 08 Feb 2024
Revised
Views
1
Cite
Reviewer Report 30 Aug 2025
Rita Chiaramonte, University of Catania, Catania, Italy 
Approved with Reservations
VIEWS 1
The study describes the search protocol of a future systematic review about psychometric properties of portable devices used for the kinematic gait assessment in stroke patients

INTRODUCTION

As this is a pilot study, it ... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Chiaramonte R. Reviewer Report For: Psychometric properties of portable devices used in kinematic gait assessment after stroke: a systematic review protocol [version 2; peer review: 2 approved with reservations]. F1000Research 2024, 12:677 (https://doi.org/10.5256/f1000research.160267.r400140)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Version 1
VERSION 1
PUBLISHED 15 Jun 2023
Views
31
Cite
Reviewer Report 23 Oct 2023
Rachel Reoli, University of Maryland Baltimore, Baltimore, Maryland, USA 
Approved with Reservations
VIEWS 31
The authors’ objectives were to develop a protocol for a systematic review of current literature to aid in the understanding of 1) the psychometric properties of portable inertial sensor devices for kinematic gait assessment for individuals post-stroke, and 2) how ... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Reoli R. Reviewer Report For: Psychometric properties of portable devices used in kinematic gait assessment after stroke: a systematic review protocol [version 2; peer review: 2 approved with reservations]. F1000Research 2024, 12:677 (https://doi.org/10.5256/f1000research.147192.r211235)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
  • Author Response 13 Apr 2024
    Raiff Silva, Departamento de Fisioterapia, Universidade Federal do Rio Grande do Norte, Natal, 59072-970, Brazil
    13 Apr 2024
    Author Response
    Dear reviewer, 
    We are grateful for all remarks concerning the manuscript and we have considered all points raised in our response letter. All changes were made to the new version of ... Continue reading
COMMENTS ON THIS REPORT
  • Author Response 13 Apr 2024
    Raiff Silva, Departamento de Fisioterapia, Universidade Federal do Rio Grande do Norte, Natal, 59072-970, Brazil
    13 Apr 2024
    Author Response
    Dear reviewer, 
    We are grateful for all remarks concerning the manuscript and we have considered all points raised in our response letter. All changes were made to the new version of ... Continue reading

Comments on this article Comments (0)

Version 2
VERSION 2 PUBLISHED 15 Jun 2023
Comment
Alongside their report, reviewers assign a status to the article:
Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions
Sign In
If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password.

The email address should be the one you originally registered with F1000.

Email address not valid, please try again

You registered with F1000 via Google, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Google account password, please click here.

You registered with F1000 via Facebook, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Facebook account password, please click here.

Code not correct, please try again
Email us for further assistance.
Server error, please try again.