Evaluating Wilson sims tool: Quality improvement on fall risk in UAE psychiatric patients

Crisostomo Valondo; Ahmad Muneer; Lynn Itani; Tarik Qassem; Hessa AbdulQader

doi:10.12688/f1000research.144159.1

Home Browse Evaluating Wilson sims tool: Quality improvement on fall risk in UAE...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Research Article

Evaluating Wilson sims tool: Quality improvement on fall risk in UAE psychiatric patients

[version 1; peer review: 1 approved with reservations]

Crisostomo Valondo¹, Ahmad Muneer², Lynn Itani³, Tarik Qassem⁴, Hessa AbdulQader⁵

Crisostomo Valondo¹, Ahmad Muneer², [...] Lynn Itani³, Tarik Qassem⁴, Hessa AbdulQader⁵

PUBLISHED 15 Feb 2024

Author details Author details

¹ MSc, RN, Nurse Manager Psychiatric Hospital, Al Amal Psychiatric Hospital, Al Aweer, Dubai, United Arab Emirates
² MSc, RN, Services Manager at the Addiction Clinical Academic Group, Maudsley Health, Al Amal Psychiatric Hospital, Emirates Health Services, Dubai, United Arab Emirates
³ Master’s in Public Health (MPH), Research Assistant, Maudsley Health, Al Amal Psychiatric Hospital, Emirates Health Services, Dubai, United Arab Emirates
⁴ Professor, MBBCh MSc MD FRCPsych, Consultant in Old Age Psychiatry, Lead for Mental Health for Older Adult Clinical, Academic Group, Maudsley Health, Al-Amal Hospital, Adjunct Clinical Professor of Psychiatry, Mohammed Bin Rashid University of Medicine and Health Sciences, Dubai – UAE, Honorary Consultant in Old Age Psychiatry South London and Maudsley NHS Foundation Trust London, UK, Dubai, United Arab Emirates
⁵ MSc, RN, Assistant Nursing Director, Al Amal Psychiatric Hospital, Emirates Health Services, Dubai, United Arab Emirates, Dubai, United Arab Emirates

Crisostomo Valondo
Roles: Investigation, Methodology, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing

Ahmad Muneer
Roles: Conceptualization, Writing – Original Draft Preparation, Writing – Review & Editing

Lynn Itani
Roles: Formal Analysis, Methodology

Tarik Qassem
Roles: Conceptualization, Data Curation, Investigation, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing

Hessa AbdulQader
Roles: Conceptualization, Investigation, Methodology, Project Administration, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

BACKGROUND

Falls among patients in psychiatric inpatient healthcare settings present a significant global challenge, despite ongoing efforts to reduce the risks. Falls negatively impact patient safety, post-treatment recovery, and medical costs.

AIM

This study aimed to evaluate the predictive validity of the Wilson Sims Fall Risk Assessment Tool (WSFRAT), investigate predictors of falls, and determine the optimal cut-off point for the WSFRAT. Additionally, we aimed to assess the predictive validity of clinical judgement in identifying individuals at high risk of falls.

METHODS

We conducted a psychometric study at a specialized psychiatry hospital in Dubai, United Arab Emirates, using data collected from hospital-wide quality projects between April 16, 2019, and March 31, 2021. Our sample comprised 492 patients.

RESULTS

Contrary to the recommended cut-off point of 7 in the literature, this study results indicate that the optimal cut-off point for the WSFRAT was 5+. This yielded an accuracy of 87%, a diagnostic odds ratio (DOR) of 0.728, a kappa value of 0.208, a sensitivity of 83%, and a specificity of 87%. Furthermore, the regression analyses identified significant predictors of fall risk, including age, gender, assistive device, WSFRAT, and evaluation duration. Notably, clinical judgement did not significantly predict fall risk (p=0.331).

CONCLUSION

In conclusion, the present research demonstrates that the WSFRAT is a reliable tool with high sensitivity and specificity for predicting falls in psychiatric inpatient settings. The findings emphasize the importance of employing evidence-based tools and a comprehensive assessment approach to prevent falls. Furthermore, our findings challenge the recommended cut-off point of 7 and highlight the need for further research to confirm the optimal cut-off point. Finally, this investigation revealed that clinical judgement alone is not an effective method for predicting falls in this population.

Keywords

Patient falls - risk assessment tools in Mental health settings- predictive validity - predictors of fall risk -Wilson Sims Fall Risk Assessment Tool

Corresponding author: Ahmad Muneer

Competing interests: No competing interests were disclosed.

Grant information: The author(s) declared that no grants were involved in supporting this work.

Copyright: © 2024 Valondo C et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Valondo C, Muneer A, Itani L et al. Evaluating Wilson sims tool: Quality improvement on fall risk in UAE psychiatric patients [version 1; peer review: 1 approved with reservations]. F1000Research 2024, 13:97 (https://doi.org/10.12688/f1000research.144159.1) First published: 15 Feb 2024, 13:97 (https://doi.org/10.12688/f1000research.144159.1) Latest published: 15 Feb 2024, 13:97 (https://doi.org/10.12688/f1000research.144159.1)

1. Introduction

Patient falls during hospitalization are a significant concern in healthcare due to the serious consequences they can have. These consequences range from physical injury to costly medical care and prolonged hospital stays (Watson et al., 2011). Hospital fall prevention programs are typically implemented to minimize the risk of patient falls, which include various tools for evaluating and predicting fall risk in admitted patients. However, psychiatric units pose a particular challenge, as falls are more frequent due to the combination of risk factors that patients face (Carpels et al., 2022). These risk factors include behavioural and cognitive manifestations of illnesses, medication side effects, and increased mobility during therapeutic activities (Lu et al., 2018).

Although fall prevention strategies are important in all healthcare settings, research about fall risk assessment tools specific to psychiatric patients remains limited. In 2016, Abraham looked at the applicability of several fall risk assessment tools in hospitals, and found that the majority of these tools were better suited for use in surgical or medical departments and with elderly patients, such as the Hendrich II Fall Risk Model (Abraham, 2016). While two tools have been developed for the assessment of fall risk in psychiatric patients, these studies were based on relatively small sample sizes (Edmonson et al., 2011; Healey, 2010). Specifically, the Edmonson Psychiatric Fall Risk Assessment Tool (EPFRAT) and the Wilson Sims Fall Risk Assessment Tool (WSFRAT) were identified, with the former showing a sensitivity of 63% and a specificity of 85%, and the latter having a sensitivity of 100% and a specificity of 63.1% (Edmonson et al., 2011; Healey, 2010). One notable difference between these tools is that the WSFRAT includes a field for nurses to provide their clinical judgment regarding whether a patient is at risk of falling or not. This evaluation takes priority over the final WSFRAT score.

Further research is necessary to evaluate the psychometric properties of these tools. Studies from Hong Kong and Taiwan have evaluated the psychometric properties of the WSFRAT, but these studies had limitations. In the study from Taiwan, the details of the research were not accessible, and the study from Hong Kong was limited to a geriatric patient population and calculated psychometric indicators based on the number of patients rather than the number of observations. The study recommended the WSFRAT due to its high sensitivity (Chou & Hung, 2019; Wong et al., 2021). Falls in psychiatric wards remain a persistent problem, so it is critical to obtain data from large datasets for the development of effective fall prevention strategies. Therefore, the aim of the current study is to evaluate the psychometric properties of the WSFRAT based on a large number of observations from different psychiatric wards in a specialized mental health hospital. Such research will provide valuable insights into the fall risk assessment tools for psychiatric patients and aid in the development of effective fall prevention strategies.

2. Methods

2.1 Type and design

Secondary data analyses from hospital-wide quality projects between April 16, 2019, and March 31, 2021.This study aim to Assess the predictive validity of the Wilson Sims Fall Risk Assessment Tool (WSFRAT), Another objective of our study was to determine how well nurse clinical judgment can identify those who are at a high risk of falling. The sensitivity, specificity, accuracy, positive predictive value (PPV), negative predictive value (NPV), and optimum cut-off value of the WSFRAT are all to be determined.

2.2 Place and date

The data was collected from Al Amal Psychiatric Hospital, a specialized psychiatry hospital based in Dubai, United Arab Emirates. Data collection took place between 16^th of April 2019 and 31^st of March 2021.

2.3 Ethical Approval Statement for Publication

Under the auspices of the nursing and Quality Department at Al-Amal Psychiatric Hospital, first authorized for the Quality Improvement Project (QIP) was granted, denoted by the reference number NQD-QIP-1903-WSFRAT. Subsequent approval was obtained from the Research and Development (R&D) committee at the same institution in October 2020, leading to the formal registration of the project under the reference number R&D-QIP-2010-WSFRAT.

The primary focus of this inquiry centered on assessing the effectiveness of the Wilson Sims Fall Risk Assessment Tool among psychiatric patients. The study strictly adhered to the standard treatment protocols for patients, refraining from introducing any additional procedures or interventions. As a result, the investigation was deemed exempt from an exhaustive review by the Research Ethics Committee (REC) under reference number R&D-QIP-2010-WSFRAT. This exemption was granted based on the understanding that the project’s methodology did not warrant an extensive REC evaluation due to its alignment with established ethical standards and its limited impact on routine patient care.

2.4 Population and sample

The sample comprised of 492 patients admitted to male and female acute mental health inpatient treatment units.

2.5 Data collection tools

A secondary analysis for Data was derived from hospital-wide nursing quality projects, a database of patients’ profiles was developed with the support of two senior nurse managers and information from hospital electronic medical records. The database included the following characteristics of patients: gender, age, nationality, diagnosis, number of observations, history of falls, use of assistive devices, and if the patients had a fall while hospitalized. Data on fall risk assessment was collected using the Wilson Sims Fall Risk Assessment Tool (WSFRAT).

Initially created in Michigan United States of America, the WSFRAT is completed by nurse staff during the admission of adult inpatients to the hospital. The scale consists of ratings for nine risk factors: “(a) age; (b) mental status; (c) physical status; (d) elimination; (e) sensory impairments; (f) gait or balance; (g) history of falls in the past 6 months; (h) medications (e.g., mood stabilizers, benzodiazepine agents, narcotic agents, sedative/hypnotic drugs, antipsychotic agents); and (i) a detoxification protocol” (Van Dyke et al., 2014). A low risk corresponds to a total score of 0-6 while a score ≥7 is high risk. As mentioned previously, the WSFRAT also takes into consideration the nurse’s clinical judgement and this takes precedence over the risk score for the classification of fall risk (Wilson et al., 2014).

2.6 Falls assessment in the hospital

All Hospital inpatients must be screened for risk of falls on admission and every 24 hours to determine the ongoing need for fall prevention precautions. Inpatients considered at risk of falls should be re-assessed every 12 hours, additionally, the assigned nurse/health care provider must apply the appropriate fall prevention precaution Interventions to protect patients at risk for falling and harming themselves using the least restrictive alternative available.

2.7 Data collection process

The falls risk evaluation using WSFRAT was completed upon admission, once per day and until discharge. If the patient had been admitted before the start of data collection, then the evaluation was made on 16th May 2019. Named patient’s nurse was requested to give an opinion about the patient’s overall falls risk. The opinion took binary format, whether high risk or low risk. This was done independent to patient’s score on WSFRAT.

In accordance with established criteria, a fall was precisely delineated for the purposes of this study. As per the definition provided by The National Database of Nursing Quality Indicators (NDNQI), a fall is characterized as an unplanned descent to the floor or extension of the floor, which may involve contact with any other equipment, and can occur with or without resultant injury (Montalvo, 2007). This definition aligns with the understanding that a fall constitutes an event during which an individual inadvertently comes to rest on the ground or another lower level, with or without a concurrent loss of consciousness, as outlined by Nice (2014). Furthermore, recent global health estimates as reported by the World Health Organization (WHO, 2021) specify a fall as an incident leading to an individual unintentionally resting on the ground, floor, or any other lower level. It is noteworthy that fall-related injuries encompass outcomes that range from non-fatal to fatal, underscoring the severity and diverse consequences associated with such incidents.

2.8 Data analysis

The present study employed the R programming language (Team, 2021) along with several libraries, namely psych (Revelle, 2020), cutpointr (Thiele & Hirschfeld, 2020), ggplot2 (Wickham et al., 2016), pROC (Robin et al., 2011), tidyverse (Wickham et al., 2019), and caret (Kuhn, 2015), to conduct data analysis. The analysis encompassed the computation of descriptive statistics for age, WSFRAT, and evaluation duration. T-tests were applied to discern significant distinctions between groups, and the cutpointr library was utilized to establish the optimal cut-off point through receiver operating characteristic (ROC) curve analysis, alongside measures such as sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy.

Both simple logistic regression and multiple logistic regression were employed to investigate the relationship between the binary outcome variable (Actual Fall) and individual or multiple predictor variables. Various metrics including accuracy, diagnostic odds ratio (DOR), Kappa, sensitivity, specificity, PPV, NPV, and area under the curve (AUC) were calculated and subjected to appropriate statistical tests.

The diagnostic test accuracy was computed as the ratio of correctly identified cases to all tested cases, using the formula (true positive + true negative) / (true positive + false positive + true negative + false negative). T-tests were conducted to compare continuous variables, including age, WSFRAT, and evaluation duration, across different groups.

The diagnostic odds ratio (DOR) was employed as a measure of diagnostic accuracy, combining sensitivity and specificity, with the formula (TP/FN) / (FP/TN). Kappa assessed the agreement between predicted and actual outcomes, accounting for chance agreement, using the formula (p_o - p_e) / (1 - p_e), where p_o represents observed agreement and p_e represents agreement expected by chance.

Sensitivity, as the proportion of actual positives correctly identified as positive by the test, was computed using the formula true positive / (true positive + false negative). Specificity, reflecting the proportion of actual negatives correctly identified as negative by the test, was calculated as true negative / (true negative + false positive).

Positive predictive value (PPV), denoting the proportion of positive test results that are true positives, was determined as true positive / (true positive + false positive). Negative predictive value (NPV), indicating the proportion of negative test results that are true negatives, was derived as true negative / (true negative + false negative).

Area under the curve (AUC) was employed to assess the overall performance of a diagnostic test, considering both sensitivity and specificity. It was calculated by plotting the true positive rate (sensitivity) against the false positive rate (1-specificity) at various threshold values and computing the area under the resulting curve.

For the analysis of ROC curves, including AUC calculation, confidence intervals, and comparison of multiple ROC curves, the pROC library was utilized. The ggplot2 library facilitated the creation of high-quality visualizations, including bar plots, scatter plots, and line plots, for effective presentation of findings. The tidyverse library streamlined data cleaning, manipulation, and visualization tasks through its consistent grammar and principles. Finally, the caret library was instrumental in building and assessing predictive models, encompassing data pre-processing, model training, model selection, and performance evaluation.

2.8.1R programming language

Is a free software programming language for statistical computing and graphics, which sponsored by the R Core Team and the R Foundation for Statistical computation. It is used for statistical computation and graphics. Ross Ihaka and Robert Gentleman, two statisticians, developed R, which is used by data miners, bioinformaticians, and statisticians for data analysis and creating statistical software.

3. Results

The characteristics of the selected participants in the current study are described in Table 1. A total of 492 patients with various psychiatric disorders were included in the study, with a majority of male patients (62.4%) and a median age of 32 years. The most common diagnosis was a psychotic disorder (40.0%). The fall risk in these patients was evaluated using the Wilson Sims Fall Risk Assessment Tool (WSFRAT), with a mean score of 2.2.

Table 1. Association between fall risk and various patient characteristics by Fisher exact test.

variable	n	%
Gender
M	307	37.6
F	185	62.4
Assistive Device
Yes	2	0.4
No	490	99.6
Previous History of Fall
Yes	2	0.4
No	490	99.6
Clinical Judgement
Yes	16	3.3
No	476	96.7
Fall
Yes	12	2.4
No	480	97.6
Diagnosis
Anxiety Disorder	5	1.0
Dementia	6	1.2
Personality Disorder	15	3.0
Developmental Disorder	27	5.5
Other Mental Disorders	38	7.7
Substance Use Disorder	96	19.5
Mood Disorder	108	22.0
Psychotic Disorder	197	40.0

variable	% No Recent Fall Reported	% Recent Fall Reported	Fisher exact test p-value
Gender			0.00143*
M	35.6	2.0
F	62	0.4
Assistive Device			0.04828*
No Assistive Device Used	97.4	2.2
Assistive Device Used	0.2	0.2
Previous Fall			0.000546*
No Previous Fall Reported	97.6	2.0
Previous Fall Reported	0.0	0.4
Clinical Judgement			0.331
Low risk	94.5	2.2
High risk	3.0	0.2
Diagnosis			0.0003*
Anxiety Disorder	1.0	0.0
Dementia	0.8	0.4
Personality Disorder	2.6	0.4
Developmental Disorder	5.1	0.4
Other Mental Disorders	7.7	0.0
Substance Use Disorder	19.5	0.0
Mood Disorder	22.0	0.0
Psychotic Disorder	38.8	1.2

* Statistically significant P ≤ 0.05.

We compared fall and non-fall cases finding significant differences in age, WSFRAT score, and evaluation duration between the two groups presented in Table 2. The mean age of fall cases was 46.4, while the mean age of non-fall cases was 34.3. The mean WSFRAT score of fall cases was 7.9, compared to 2.0 for non-fall cases. The mean evaluation duration of fall cases was 287.6, compared to 26.3 for non-fall cases. The Welch Two Sample t-test showed a significant difference in age, WSFRAT score, and evaluation duration between fall and non-fall cases.

Table 2. Compared fall and non-fall cases by Welch Two Sample t-test.

Variable	Mean±SD	Welch Two Sample t-test
Variable	Mean±SD	t-test	p-value
*Age*	34.6±11.9
Age in those who had fall	46.4±16.2	2.5694	0.02562*
Age in those who non-fall cases	34.3±11.7	2.5694	0.02562*
*WSFRAT score*	2.2±2.5
WSFRAT in those who had fall	7.9±4.5	4.484	0.0008966*
WSFRAT in those who had non-fall cases	2.0±2.3	4.484	0.0008966*
*Evaluation duration*	32.7±82.1
Evaluation duration in those who had falls	287.6±232.7	3.8857	0.002521*
Evaluation duration in those who non-fall cases	26.3±63.2	3.8857	0.002521*

* Statistically significant P ≤ 0.05.

We also investigated the association between fall risk and various patient characteristics which are illustrated in Table 1. A Fisher exact test showed a significant association between gender and fall risk (p=0.00143). There was also a significant association between the use of an assistive device and fall risk (p=0.0482). A previous history of falls was significantly associated with fall risk (p=0.000546). However, there was no significant association between clinical judgement and fall risk (p=0.331). The most common diagnosis was a psychotic disorder, which was significantly associated with fall risk (p=0.0003).

Receiver operating characteristic (ROC) curve analysis was conducted to determine the optimal cut points for the WSFRAT (Table 3 and Figure 1). The analysis showed that the optimal cut point at 5+ had an accuracy of 87%, a diagnostic odds ratio (DOR) of 0.728, a kappa value of 0.208, a sensitivity of 83%, a specificity of 87%, a positive predictive value (PPV) of 0.14, and a negative predictive value (NPV) of 0.99. The area under the curve (AUC) for the WSFRAT method was 0.85, indicating a good level of discrimination (Figure 1). Additionally, the cut point at 7+ had an accuracy of 94%, a DOR of 0.114, a kappa value of 0.323, a sensitivity of 67%, a specificity of 95%, a PPV of 0.24, and an NPV of 0.99. We also evaluated the accuracy of clinical judgement in predicting fall risk, with an accuracy of 95%, a DOR of 0.002, a kappa value of 0.045, a sensitivity of 8%, a specificity of 97%, a PPV of 0.06, and an NPV of 0.98.

Table 3. Receiver operating characteristic (ROC) curve analysis.

Criterion	Accuracy	DOR	Kappa	Sensitivity	Specificity	PPV	NPV	AUC
Cutoff at 5+	0.87	0.728	0.208	0.83	0.87	0.14	0.99	0.85
Cutoff at 7+	0.94	0.114	0.323	0.67	0.95	0.24	0.99	0.81
Clinical Judgement	0.95	0.002	0.045	0.08	0.97	0.06	0.98	0.53

Figure 1. Receiver Operating Curve (ROC) and Changes in the values of Area Under the Curve (AUC).

A. Receiver Operating Curve (ROC).

B. Changes in the values of Area Under the Curve (AUC) with different cut-off points.

Furthermore, we performed several regression analyses to identify predictors of fall risk. Simple logistic regression with individual predictors studied separately described in Table 4, presented that age, gender, assistive device, WSFRAT, and evaluation duration were significant predictors of fall risk. Finally, a stepwise multiple logistic regression analysis was conducted to identify predictors of future falls. The analysis included variables that showed a significant relationship in simple regression models. Although not statistically significant, the analysis showed a relationship between gender (male) (coefficient = -3.94, SE = 2.05, p = 0.05465) and future falls. The use of assistive device (coefficient = 6.32, SE = 2.63, p = 0.01641), WSFRAT score (coefficient = 0.42, SE = 0.13, p = 0.00088), and evaluation duration (coefficient = 0.01, SE = 0.002, p = 0.00014) were significantly associated with future falls. The current study revealed that the WSFRAT score was a significant predictor of fall risk in both simple and multiple logistic regression analyses.

Table 4. Simple logistic regression.

Simple logistic regression with individual predictors studied separately.

Predictive variable	Coefficient	Std. Error	z value	p value	AIC
Age	0.064	0.01965	3.237	0.00121	107.15
Gender: Male	-2.165	0.7803	-2.775	0.00553	105.93
History of previous Fall	20.437	1696.7344	0.012	0.99	101.63
Assistive device	3.774	1.4467	2.609	0.00909	112.05
WSFRAT	0.457	0.08535	5.351	<0.0001	82.449
Evaluation duration	0.012	0.002265	5.146	<0.0001	77.439
Clinical Judgement	1.036	1.0769	0.962	0.336	116.11
Diagnosis
Dementia	20.870	13070.	0.002	0.999	103.39
Developmental Disorder	19.040	13070.	0.001	0.999
Mood Disorder	0.000	13370.	0.000	1.000
Other Mental Disorders	0.000	13910.	0.000	1.000
Personality Disorder	19.690	13070.	0.002	0.999
Psychotic Disorder	18.110	13070.	0.001	0.999
Substance Use Disorder	0.000	13410.	0.000	1.000

4. Discussion

There is a scarcity of research on fall risk assessment tools for psychiatric patients. This study included 492 patients in a busy acute mental health hospital.

This prospective study aimed to evaluate the predictive validity of the WSFRAT, a tool designed to identify patients at risk of falls, in psychiatric inpatient settings. In addition, the study aimed to determine the optimal cut-off point for the WSFRAT while comparing its effectiveness to “clinical judgement,” a common method used by clinicians to assess fall risk.

The results of the study showed that the WSFRAT is a reliable tool with an accuracy of 85%. The sensitivity and specificity of the WSFRAT were also high at 84.6% and 85.5%, respectively. The optimal cut-off point was identified as 5, which has practical implications for clinical practice. The findings highlight the importance of using evidence-based tools to identify patients at risk for falls, as they can significantly reduce the incidence of falls and associated injuries.

It is worth noting that the choice of cut-off value can have a significant impact on the performance of a screening tool. The higher sensitivity and specificity values for the cut-off of 7+ suggested in the literature may be more conservative in identifying individuals at high risk of falls. In contrast, the cut-off of 5+ may be more sensitive in identifying a larger proportion of individuals at risk, but may also result in more false positives.

The unexpected inefficiency of clinical judgement in predicting falls within psychiatric inpatient settings, as revealed by this study, underscores a critical area of concern in contemporary healthcare practices. The strikingly low sensitivity of 0.08, denoting a mere 8% accuracy in identifying actual falls, challenges the conventional reliance on clinical expertise for fall risk evaluation in psychiatric care contexts.

Several complex factors may underpin this inadequacy. The intricate nature of psychiatric patients’ conditions, coupled with a myriad of environmental variables, potentially exceeds the capabilities of clinical judgement alone. While nursing expertise remains invaluable, its augmentation with standardized assessment tools tailored specifically for the psychiatric patient population might be imperative. Furthermore, nuanced environmental aspects, such as distinct patient behaviours or specific ward layouts, could significantly influence the accuracy of clinical judgement. Understanding the intricate interplay between these elements and fall risk is crucial for refining risk assessment methodologies.

The study’s revelation of clinical judgement’s high specificity (0.97) suggests its proficiency in accurately identifying non-fall cases, indicating a robust ability to rule out individuals not at risk. However, the potential pitfall lies in the overemphasis on correctly identifying non-fall instances, possibly leading to a false sense of security and overlooking individuals genuinely susceptible to falls. This underscores the need for a balanced approach, where sensitivity, the capability to identify true positive cases (actual falls), is prioritized alongside specificity.

Given these findings, it is evident that comprehensive research is imperative to delve into the intricacies of why clinical judgement falls short in psychiatric inpatient settings. Such investigations are pivotal to bridging the existing knowledge gap and may pave the way for the development of more effective and nuanced fall risk assessment strategies tailored specifically to the unique challenges posed by psychiatric patients and their environments. This study serves as a clarion call for a revaluation of current practices and advocates for an integrated approach that harmonizes clinical expertise with evidence-based tools, ultimately enhancing patient safety and well-being in psychiatric healthcare settings.

In fact, the study found that a coin toss would have very close performance (AUC=0.5) in identifying falls than clinical judgement (AUC=0.53). The Kappa statistic also confirmed that clinical judgement is only better than chance in 0.4% of cases. Therefore, in this case, clinical judgement cannot be relied upon to predict falls accurately. The possibility exists that the poor predictive power of clinical judgement in identifying fall risks in the study population is due to clinicians’ awareness of labelling patients as high-risk, prompting them to take robust preventive measures. However, this explanation is unlikely since the actual prevalence of falls in the study population was 2.4%, which is much lower than the expected rate of falls in psychiatric inpatient settings.

The study analysed predictors of fall risk in psychiatric patients using logistic regression. Age, gender, assistive device, WSFRAT, and evaluation duration were found to be significant predictors. Previous falls and clinical judgement were not significant. A comprehensive assessment should consider all factors to identify high-risk patients and prevent falls. Stepwise analysis showed that assistive device, WSFRAT, and evaluation duration remained significant predictors, but not age or gender. WSFRAT is a significant predictor of fall risk in psychiatric patients, even when controlling for other factors.

5. Limitations

The first limitation of this study is the lack of mention of any medications when assessing predictors of falls. Medications have been shown to be a significant risk factor for falls in patients with psychiatric disorders. However, due to the nature of acute mental health units, medications tend to change in acute phases of admission, which was difficult to track and analyse systematically. The lack of consideration of medications as a predictor of falls could limit the conclusion of the results, and future studies should take this into account.

The second limitation of the study is the variable duration of observation. Some patients left early against medical advice, and others left shortly after stabilization. This is a common issue in acute mental health units, where patients often have shorter lengths of stay than in other settings. The variable duration of observation could affect the accuracy of the results, and future studies should try to account for this by implementing standardized observation periods.

Despite these limitations, the study has important implications for fall prevention in psychiatric inpatient settings. The findings suggest that the WSFRAT is a reliable tool for identifying patients at risk of falls, and preventive measures should be implemented based on a more objective assessment tool such as the WSFRAT rather than relying on clinical judgement. Additionally, a comprehensive fall risk assessment that considers age, gender, assistive device use, WSFRAT score as well as other unstudied variables, such as medications, could help to identify patients at the highest risk for falls and reduce the incidence of falls in this vulnerable population.

6. Conclusion

The study highlights the importance of using evidence-based tools to identify patients at risk for falls in psychiatric inpatient settings. The WSFRAT was found to be a reliable tool with high sensitivity and specificity, and an optimal cut-off point of 5. The study also revealed that clinical judgement is not an effective method for predicting falls in this population. The study’s limitations, including the lack of consideration of medications and the variable duration of observation, should be addressed in future studies. Overall, a comprehensive fall risk assessment that considers multiple factors can help to identify high-risk patients and prevent falls in this vulnerable population. The findings of this study have important implications for fall prevention and patient safety in psychiatric inpatient settings.

7. Implications for nursing practice

Based on the study findings, there are several implications for nursing practice in psychiatric inpatient settings:

1. Use of evidence-based fall risk assessment tools: Nurses should use evidence-based tools such as the WSFRAT to identify patients at risk of falls. The tool has high sensitivity and specificity, making it a reliable way to assess fall risk. Nurses should be familiar with the updated optimal cut-off point of 5 and use this score to determine appropriate interventions.
2. Avoid relying solely on clinical judgement: The study findings suggest that relying on clinical judgement alone is not an effective method for predicting falls in psychiatric inpatient settings. Nurses should use objective assessment tools such as the WSFRAT and consider multiple factors to identify patients at high risk for falls.
3. Consider multiple factors in fall risk assessment: The study found that age, gender, assistive device use, and evaluation duration were significant predictors of fall risk in psychiatric patients. Nurses should consider these factors in addition to the WSFRAT score to identify patients at high risk for falls.
4. Address medication use: The study limitations highlight the need for future studies to consider medication use as a predictor of falls. In the meantime, nurses should be aware that medications can be a significant risk factor for falls in patients with psychiatric disorders and should monitor patients for adverse effects.
5. Implication of a Holistic Assessment: Nurses must conduct holistic evaluations that consider the patient’s emotional, psychological, and social requirements in addition to their physical health. As Patients in specialized settings may face increased stress, anxiety, and uncertainty. A holistic evaluation assists nurses in identifying and addressing these non-physical elements that can have a substantial influence on a patient’s overall health and rehabilitation. By addressing patients’ holistic needs, nurses contribute to increased mental and emotional well-being, which can affect physical recovery.
6. Implementing a centralized reporting system, nursing administration may look for patterns in everyday operations that highlight safety issues, such as inadequate documentation or possibly risky shift patterns. Using this knowledge, nurse leaders may find ways to streamline processes and reduce those risks. Furthermore, frontline Nurse management are accountable for conveying risk management goals to their employees and developing a safety culture by acting as positive role models. Nurse leaders are ultimately responsible for designing and implementing risk management strategies for their organizations, in collaboration with other health care professionals. When challenges develop, they may examine what went wrong and offer methods to enhance operations and minimize future errors.
7. Personalized Care Plans: Nurses should create personalised care plans for each patient based on their specific requirements, medical history, and treatment objectives. This entails conducting extensive evaluations and reassessing the patient’s status on a continuous basis. Patients with complicated medical issues are frequently housed in specialized settings. Personalized care plans guarantee that each patient receives the most effective treatments and interventions for their individual circumstances. This method reduces the risk of adverse events, improves outcomes, and promotes patient-centered treatment, all of which are necessary for holistic health.
8. Interdisciplinary Collaboration: To guarantee complete and coordinated treatment, nurses should work closely with other healthcare professionals such as doctors, therapists, and pharmacists. as Patients in highly specialized settings may require multifaceted treatment. Collaborative cooperation ensures that all areas of a patient’s health are addressed, including physical, psychological, and social well-being. This integrated strategy not only improves patient outcomes but also lowers the likelihood of medical mistakes and enhances the entire patient experience.
9. Implication of Patient and Family Education: Nurses should educate patients and their families about their disease, treatment plan, and self-care behaviors on a continuous basis. As Patients and their families may encounter novel and difficult medical information in specialized care settings. Education enables people to actively engage in their treatment, make educated decisions, and maintain their health outside of the hospital. This can result in better adherence to treatment programs, lower readmission rates, and a higher quality of life for patients.
Overall, the study findings emphasize the importance of using evidence-based tools and a comprehensive assessment approach to prevent falls in psychiatric inpatient settings. Nurses play a vital role in fall prevention and should be familiar with these implications to provide safe and effective care for their patients. In conclusion, these considerations are crucial for nursing practice in settings that need highly specialized patient care for a number of reasons. They highlight the comprehensive aspect of health while also improving the standard of treatment and patient outcomes. Nurses help to make healthcare more all-encompassing and patient-cantered by addressing the patient’s physical, emotional, psychological, and social well-being. In turn, this promotes confidence, raises patient satisfaction, and ultimately adds value to the conversation about nursing practice in specialized settings.

Data availability

No data are associated with this article.

References

Abraham S: Managing patient falls in psychiatric inpatient units. Health Care Manag. 2016; 35(2): 121–133. PubMed Abstract | Publisher Full Text
Carpels A, de Smet L , Desplenter S, et al.: Falls among psychiatric inpatients: a systematic review of literature. Alpha Psychiatry. 2022; 23(5): 217–222. PubMed Abstract | Publisher Full Text | Free Full Text
Chou S-P, Hung T-M: Comparison of clinical diagnostic effectiveness between psychiatric inpatient and Wilson-Sims fall risk assessment tools. Hu Li Za Zhi. 2019; 66(3): 35–45. PubMed Abstract | Publisher Full Text
Edmonson D, Robinson S, Hughes L: Development of the Edmonson psychiatric fall risk assessment tool. J. Psychosoc. Nurs. Ment. Health Serv. 2011; 49(2): 29–36. Publisher Full Text
Healey F: A GUIDE ON HOW TO PREVENT FALLS AND INJURY IN HOSPITALS. Nurs. Older People. 2010; 22(9): 16–22. Publisher Full Text
Kuhn M: Caret: classification and regression training. Astrophysics Source Code Library. 2015; ascl-1505.
Lu S-H, Chen K-H, Pan Y-C, et al.: Influence of medications and psychotic symptoms on fall risk in acute psychiatric inpatients. J. Med. Sci. 2018; 38(3): 117–121. Publisher Full Text
Montalvo I: The National Database of Nursing Quality Indicators^® (NDNQI^®). OJIN: The Online J. Issues Nurs. 2007; 12(3). Publisher Full Text
Nice.: Assessment and prevention of falls in older people. NICE guidance june 213. Nurs. Older People. 2014; 26(6).
Revelle W: psych: Procedures for Personality and Psychological Research.2020. Reference Source
Robin X, Turck N, Hainard A, et al.: pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics. 2011; 12: 77. PubMed Abstract | Publisher Full Text | Free Full Text
Team, R. C: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. 2021.
Thiele C, Hirschfeld G: Cutpointr: Improved estimation and validation of optimal cutpoints in R. ArXiv Preprint ArXiv:2002.09209. 2020.
Van Dyke D, Singley B, Speroni KG, et al.: Evaluation of fall risk assessment tools for psychiatric patient fall prevention: A comparative study. J. Psychosoc. Nurs. Ment. Health Serv. 2014; 52(12): 30–35. PubMed Abstract | Publisher Full Text
Watson WL, Clapperton AJ, Mitchell RJ: The cost of fall-related injuries among older people in NSW, 2006–07. N. S. W. Public Health Bull. 2011; 22(4): 55–59. PubMed Abstract | Publisher Full Text
Wickham H, Averick M, Bryan J, et al.: Welcome to the Tidyverse. J. Open Source Softw. 2019; 4(43). Publisher Full Text
Wickham H, Chang W, Wickham MH: Package ‘ggplot2.’. Create Elegant Data Visualisations Using the Grammar of Graphics. Version. 2016; 2(1): 1–189.
Wilson SC, Fettes S, Sims K: Gravity: It’s Not a Suggestion-It’s the Law! The Development of the WilsonSims Psychiatric Fall Risk Assessment Tool. J. Am. Psychiatr. Nurses Assoc. 2014; 20(1): 83.
Wong MMC, Pang PF, Chan CF, et al.: Wilson Sims Fall Risk Assessment Tool Versus Morse Fall Scale in Psychogeriatric Inpatients: a Multicentre Study. East Asian Arch. Psychiatr. 2021; 31(3): 67–70. PubMed Abstract | Publisher Full Text
World Health Organization: Falls, World Health Organization.2021. (Accessed: 20 September 2023). Reference Source

Comments on this article Comments (0)

Version 1

VERSION 1 PUBLISHED 15 Feb 2024

Author details Author details

Crisostomo Valondo
Roles: Investigation, Methodology, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing

Ahmad Muneer
Roles: Conceptualization, Writing – Original Draft Preparation, Writing – Review & Editing

Lynn Itani
Roles: Formal Analysis, Methodology

Tarik Qassem
Roles: Conceptualization, Data Curation, Investigation, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing

Hessa AbdulQader
Roles: Conceptualization, Investigation, Methodology, Project Administration, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

The author(s) declared that no grants were involved in supporting this work.

Article Versions (1)

version 1

Published: 15 Feb 2024, 13:97

https://doi.org/10.12688/f1000research.144159.1

© 2024 Valondo C et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Valondo C, Muneer A, Itani L et al. Evaluating Wilson sims tool: Quality improvement on fall risk in UAE psychiatric patients [version 1; peer review: 1 approved with reservations]. F1000Research 2024, 13:97 (https://doi.org/10.12688/f1000research.144159.1)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 1

VERSION 1

PUBLISHED 15 Feb 2024

Views

Reviewer Report 09 Aug 2024

Roger Hilfiker, Physiotherapy Tschopp & Hilfiker, Physiotherapy Tschopp & Hilfiker, 3902 Glis, Valais, Switzerland

Approved with Reservations

https://doi.org/10.5256/f1000research.157913.r269898

The present manuscript reports on an evaluation of the predictive validity of the Wislon SIMS Fall Risk Assessment Tool with a prospective cohort study including 492 patients. The manuscript is well written. I have some comments on the reporting:

Page 3, Methods. Please be specific whether the inclusion of participants was consecutive or whether not all eligible patients were included.
Page 4, Methods. Does the following sentence "If the patient had been admitted before the start of data collection, then the evaluation was made on 16th May 2019." mean that not all patients were included prospectively, i.e. that for some patients the data were assessed retrospectively from medical records? Please clarify.

Method section: I assume that the hospitalisation duration was different for all patients and the length of the stay might increase the risk for falls. As some of the predictor variables might be associated with the length of stay and with the falls risk, it might be helpful to discuss this aspect. This would not be a problem for prediction, but might be worth being discussed. And maybe the length of stay should be reported in the descriptive tables.

Method section: Please provide precise information on how you established the "optimal" threshold. Viewing the Figure 1B, I think that the authors dichotomised at different threshold, calculated the area under the curve for each dichotomisation and chose the threshold that provided the highest AUC. I do not know this approach, which does of course not mean that it is wrong. But please provide some arguments on why this approach is good (best with references).

The approaches I know are different: They all create one receiver operating characteristic curve, which already integrates all the different thresholds. Then one of the many methods to decide on the so called optimal threshold is applied (which of the methods from cutpointr did you chose?). The authors probably already did this with the cutpointr package and a command similar to this: opt_cut <- cutpointr(suicide, dsi, suicide, direction = ">=", pos_class = "yes", neg_class = "no", method = maximize_metric, metric = youden) [(here with the method "youden index")].

Recommended threshold: In several places of the manuscript, the authors write that the recommended threshold is 7. Please provide references to these statements.
Please discuss how the threshold was established in the primary study and how this difference in your study.

Page 7, Discussion: Please consider to remove the first sentence "There is a scarcity of research on fall risk assessment tools for psychiatric patients." to the introduction or to the discussion section where you discuss your results with other published results.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Partly
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Clinical Epidemiology

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response 26 Oct 2024

ahmad muneer, Maudsley Health- at Al Amal Hospital in Dubai., United Arab Emirates

26 Oct 2024

Author Response

Response to Reviewer Comments
We sincerely thank the reviewer for their thoughtful and constructive feedback on our manuscript. The reviewer’s insights have greatly contributed to enhancing the clarity and rigor ... Continue reading Response to Reviewer Comments
We sincerely thank the reviewer for their thoughtful and constructive feedback on our manuscript. The reviewer’s insights have greatly contributed to enhancing the clarity and rigor of our study, particularly in areas such as methodological transparency and the justification of analytical approaches. We appreciate the opportunity to address these comments, which have helped us refine our manuscript and improve its contribution to the field of fall risk assessment in psychiatric settings.
Comment 1 (Page 3, Methods): Specify whether participant inclusion was consecutive.
Response: We appreciate this insight. To clarify, participants were included consecutively, with eligibility determined by admission criteria to the acute mental health unit between April 16, 2019, and March 31, 2021. This approach minimizes selection bias and strengthens the external validity of our findings by ensuring a representative sample of psychiatric inpatients.

Comment 2 (Page 4, Methods): Clarify the timing of patient evaluation for those admitted before data collection.
Response: We recognize the need for clarification. For patients admitted prior to the official start of the study on April 16, 2019, assessments were conducted retroactively using hospital records from May 16, 2019, ensuring that evaluations aligned with the methodology of prospectively collected data. This approach enabled consistent and comprehensive data collection across the sample.

Comment 3 (Method Section): Discuss length of stay and its potential association with fall risk and predictor variables.
Response: We concur that the duration of hospital stay may indeed influence fall risk, with extended stays potentially increasing exposure to fall risks. In our revised manuscript, we will include this discussion, emphasizing that while our model was predictive, future research should consider hospitalization length as an additional factor in fall risk models. We will also add length of stay data to the descriptive tables to enhance the clarity and scope of our findings.

Comment 4 (Method Section): Provide details on establishing the "optimal" threshold for the WSFRAT.
Response: We thank the reviewer for highlighting this methodological point. In Figure 1B, we indeed tested multiple thresholds for the WSFRAT, determining the optimal point based on maximizing the area under the receiver operating characteristic curve (AUC). While our approach aligned with established practices in predictive diagnostics, we will expand the methods section to detail the rationale and cite relevant literature supporting AUC-based threshold selection (e.g., Thiele & Hirschfeld, 2020). Additionally, we utilized the cutpointr package, following commands similar to those the reviewer mentioned, such as maximize_metric with the Youden index.

Comment 5: Clarify the basis for the recommended threshold of 7 and provide supporting references.
Response: In our manuscript, the threshold of 7 was referenced based on prior studies of WSFRAT’s application in psychiatric settings (Wilson et al., 2014; Wong et al., 2021). We will include these references and expand on how our study's threshold of 5 differs from this recommendation, discussing the implications of a lower threshold for heightened sensitivity and clinical applicability.

Comment 6 (Page 7, Discussion): Move the introductory sentence on scarcity of research to a more appropriate section.
Response: We agree that this point is better placed in the introduction. The introductory sentence on the scarcity of fall risk research in psychiatric populations will be relocated to the introduction, with further context in the discussion on comparing our findings to existing literature.

By addressing these points, we aim to enhance the clarity and scientific rigor of our manuscript, ensuring it meets high standards of methodological transparency and relevance. Thank you for your valuable insights
Response to Reviewer Comments
We sincerely thank the reviewer for their thoughtful and constructive feedback on our manuscript. The reviewer’s insights have greatly contributed to enhancing the clarity and rigor of our study, particularly in areas such as methodological transparency and the justification of analytical approaches. We appreciate the opportunity to address these comments, which have helped us refine our manuscript and improve its contribution to the field of fall risk assessment in psychiatric settings.
Comment 1 (Page 3, Methods): Specify whether participant inclusion was consecutive.
Response: We appreciate this insight. To clarify, participants were included consecutively, with eligibility determined by admission criteria to the acute mental health unit between April 16, 2019, and March 31, 2021. This approach minimizes selection bias and strengthens the external validity of our findings by ensuring a representative sample of psychiatric inpatients.

Comment 2 (Page 4, Methods): Clarify the timing of patient evaluation for those admitted before data collection.
Response: We recognize the need for clarification. For patients admitted prior to the official start of the study on April 16, 2019, assessments were conducted retroactively using hospital records from May 16, 2019, ensuring that evaluations aligned with the methodology of prospectively collected data. This approach enabled consistent and comprehensive data collection across the sample.

Comment 3 (Method Section): Discuss length of stay and its potential association with fall risk and predictor variables.
Response: We concur that the duration of hospital stay may indeed influence fall risk, with extended stays potentially increasing exposure to fall risks. In our revised manuscript, we will include this discussion, emphasizing that while our model was predictive, future research should consider hospitalization length as an additional factor in fall risk models. We will also add length of stay data to the descriptive tables to enhance the clarity and scope of our findings.

Comment 4 (Method Section): Provide details on establishing the "optimal" threshold for the WSFRAT.
Response: We thank the reviewer for highlighting this methodological point. In Figure 1B, we indeed tested multiple thresholds for the WSFRAT, determining the optimal point based on maximizing the area under the receiver operating characteristic curve (AUC). While our approach aligned with established practices in predictive diagnostics, we will expand the methods section to detail the rationale and cite relevant literature supporting AUC-based threshold selection (e.g., Thiele & Hirschfeld, 2020). Additionally, we utilized the cutpointr package, following commands similar to those the reviewer mentioned, such as maximize_metric with the Youden index.

Comment 5: Clarify the basis for the recommended threshold of 7 and provide supporting references.
Response: In our manuscript, the threshold of 7 was referenced based on prior studies of WSFRAT’s application in psychiatric settings (Wilson et al., 2014; Wong et al., 2021). We will include these references and expand on how our study's threshold of 5 differs from this recommendation, discussing the implications of a lower threshold for heightened sensitivity and clinical applicability.

Comment 6 (Page 7, Discussion): Move the introductory sentence on scarcity of research to a more appropriate section.
Response: We agree that this point is better placed in the introduction. The introductory sentence on the scarcity of fall risk research in psychiatric populations will be relocated to the introduction, with further context in the discussion on comparing our findings to existing literature.

By addressing these points, we aim to enhance the clarity and scientific rigor of our manuscript, ensuring it meets high standards of methodological transparency and relevance. Thank you for your valuable insights
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 26 Oct 2024

ahmad muneer, Maudsley Health- at Al Amal Hospital in Dubai., United Arab Emirates

26 Oct 2024

Author Response

Response to Reviewer Comments
We sincerely thank the reviewer for their thoughtful and constructive feedback on our manuscript. The reviewer’s insights have greatly contributed to enhancing the clarity and rigor ... Continue reading Response to Reviewer Comments
We sincerely thank the reviewer for their thoughtful and constructive feedback on our manuscript. The reviewer’s insights have greatly contributed to enhancing the clarity and rigor of our study, particularly in areas such as methodological transparency and the justification of analytical approaches. We appreciate the opportunity to address these comments, which have helped us refine our manuscript and improve its contribution to the field of fall risk assessment in psychiatric settings.
Comment 1 (Page 3, Methods): Specify whether participant inclusion was consecutive.
Response: We appreciate this insight. To clarify, participants were included consecutively, with eligibility determined by admission criteria to the acute mental health unit between April 16, 2019, and March 31, 2021. This approach minimizes selection bias and strengthens the external validity of our findings by ensuring a representative sample of psychiatric inpatients.

Comment 2 (Page 4, Methods): Clarify the timing of patient evaluation for those admitted before data collection.
Response: We recognize the need for clarification. For patients admitted prior to the official start of the study on April 16, 2019, assessments were conducted retroactively using hospital records from May 16, 2019, ensuring that evaluations aligned with the methodology of prospectively collected data. This approach enabled consistent and comprehensive data collection across the sample.

Comment 3 (Method Section): Discuss length of stay and its potential association with fall risk and predictor variables.
Response: We concur that the duration of hospital stay may indeed influence fall risk, with extended stays potentially increasing exposure to fall risks. In our revised manuscript, we will include this discussion, emphasizing that while our model was predictive, future research should consider hospitalization length as an additional factor in fall risk models. We will also add length of stay data to the descriptive tables to enhance the clarity and scope of our findings.

Comment 4 (Method Section): Provide details on establishing the "optimal" threshold for the WSFRAT.
Response: We thank the reviewer for highlighting this methodological point. In Figure 1B, we indeed tested multiple thresholds for the WSFRAT, determining the optimal point based on maximizing the area under the receiver operating characteristic curve (AUC). While our approach aligned with established practices in predictive diagnostics, we will expand the methods section to detail the rationale and cite relevant literature supporting AUC-based threshold selection (e.g., Thiele & Hirschfeld, 2020). Additionally, we utilized the cutpointr package, following commands similar to those the reviewer mentioned, such as maximize_metric with the Youden index.

Comment 5: Clarify the basis for the recommended threshold of 7 and provide supporting references.
Response: In our manuscript, the threshold of 7 was referenced based on prior studies of WSFRAT’s application in psychiatric settings (Wilson et al., 2014; Wong et al., 2021). We will include these references and expand on how our study's threshold of 5 differs from this recommendation, discussing the implications of a lower threshold for heightened sensitivity and clinical applicability.

Comment 6 (Page 7, Discussion): Move the introductory sentence on scarcity of research to a more appropriate section.
Response: We agree that this point is better placed in the introduction. The introductory sentence on the scarcity of fall risk research in psychiatric populations will be relocated to the introduction, with further context in the discussion on comparing our findings to existing literature.

By addressing these points, we aim to enhance the clarity and scientific rigor of our manuscript, ensuring it meets high standards of methodological transparency and relevance. Thank you for your valuable insights
Response to Reviewer Comments
We sincerely thank the reviewer for their thoughtful and constructive feedback on our manuscript. The reviewer’s insights have greatly contributed to enhancing the clarity and rigor of our study, particularly in areas such as methodological transparency and the justification of analytical approaches. We appreciate the opportunity to address these comments, which have helped us refine our manuscript and improve its contribution to the field of fall risk assessment in psychiatric settings.
Comment 1 (Page 3, Methods): Specify whether participant inclusion was consecutive.
Response: We appreciate this insight. To clarify, participants were included consecutively, with eligibility determined by admission criteria to the acute mental health unit between April 16, 2019, and March 31, 2021. This approach minimizes selection bias and strengthens the external validity of our findings by ensuring a representative sample of psychiatric inpatients.

Comment 2 (Page 4, Methods): Clarify the timing of patient evaluation for those admitted before data collection.
Response: We recognize the need for clarification. For patients admitted prior to the official start of the study on April 16, 2019, assessments were conducted retroactively using hospital records from May 16, 2019, ensuring that evaluations aligned with the methodology of prospectively collected data. This approach enabled consistent and comprehensive data collection across the sample.

Comment 3 (Method Section): Discuss length of stay and its potential association with fall risk and predictor variables.
Response: We concur that the duration of hospital stay may indeed influence fall risk, with extended stays potentially increasing exposure to fall risks. In our revised manuscript, we will include this discussion, emphasizing that while our model was predictive, future research should consider hospitalization length as an additional factor in fall risk models. We will also add length of stay data to the descriptive tables to enhance the clarity and scope of our findings.

Comment 4 (Method Section): Provide details on establishing the "optimal" threshold for the WSFRAT.
Response: We thank the reviewer for highlighting this methodological point. In Figure 1B, we indeed tested multiple thresholds for the WSFRAT, determining the optimal point based on maximizing the area under the receiver operating characteristic curve (AUC). While our approach aligned with established practices in predictive diagnostics, we will expand the methods section to detail the rationale and cite relevant literature supporting AUC-based threshold selection (e.g., Thiele & Hirschfeld, 2020). Additionally, we utilized the cutpointr package, following commands similar to those the reviewer mentioned, such as maximize_metric with the Youden index.

Comment 5: Clarify the basis for the recommended threshold of 7 and provide supporting references.
Response: In our manuscript, the threshold of 7 was referenced based on prior studies of WSFRAT’s application in psychiatric settings (Wilson et al., 2014; Wong et al., 2021). We will include these references and expand on how our study's threshold of 5 differs from this recommendation, discussing the implications of a lower threshold for heightened sensitivity and clinical applicability.

Comment 6 (Page 7, Discussion): Move the introductory sentence on scarcity of research to a more appropriate section.
Response: We agree that this point is better placed in the introduction. The introductory sentence on the scarcity of fall risk research in psychiatric populations will be relocated to the introduction, with further context in the discussion on comparing our findings to existing literature.

By addressing these points, we aim to enhance the clarity and scientific rigor of our manuscript, ensuring it meets high standards of methodological transparency and relevance. Thank you for your valuable insights
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 1

VERSION 1 PUBLISHED 15 Feb 2024

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1
Version 1 15 Feb 24	read

Roger Hilfiker, Physiotherapy Tschopp & Hilfiker, 3902 Glis, Switzerland

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

6 Views

09 Aug 2024 | for Version 1

Roger Hilfiker, Physiotherapy Tschopp & Hilfiker, Physiotherapy Tschopp & Hilfiker, 3902 Glis, Valais, Switzerland

6 Views Cite this report Responses(1)

Approved With Reservations

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Partly
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Clinical Epidemiology

Respond to this report

Responses (1)

Author Response

26 Oct 2024

ahmad muneer, Maudsley Health- at Al Amal Hospital in Dubai., United Arab Emirates

Response to Reviewer Comments
We sincerely thank the reviewer for their thoughtful and constructive feedback on our manuscript. The reviewer’s insights have greatly contributed to enhancing the clarity and rigor of our study, particularly in areas such as methodological transparency and the justification of analytical approaches. We appreciate the opportunity to address these comments, which have helped us refine our manuscript and improve its contribution to the field of fall risk assessment in psychiatric settings.
Comment 1 (Page 3, Methods): Specify whether participant inclusion was consecutive.
Response: We appreciate this insight. To clarify, participants were included consecutively, with eligibility determined by admission criteria to the acute mental health unit between April 16, 2019, and March 31, 2021. This approach minimizes selection bias and strengthens the external validity of our findings by ensuring a representative sample of psychiatric inpatients.

Comment 2 (Page 4, Methods): Clarify the timing of patient evaluation for those admitted before data collection.
Response: We recognize the need for clarification. For patients admitted prior to the official start of the study on April 16, 2019, assessments were conducted retroactively using hospital records from May 16, 2019, ensuring that evaluations aligned with the methodology of prospectively collected data. This approach enabled consistent and comprehensive data collection across the sample.

Comment 3 (Method Section): Discuss length of stay and its potential association with fall risk and predictor variables.
Response: We concur that the duration of hospital stay may indeed influence fall risk, with extended stays potentially increasing exposure to fall risks. In our revised manuscript, we will include this discussion, emphasizing that while our model was predictive, future research should consider hospitalization length as an additional factor in fall risk models. We will also add length of stay data to the descriptive tables to enhance the clarity and scope of our findings.

Comment 4 (Method Section): Provide details on establishing the "optimal" threshold for the WSFRAT.
Response: We thank the reviewer for highlighting this methodological point. In Figure 1B, we indeed tested multiple thresholds for the WSFRAT, determining the optimal point based on maximizing the area under the receiver operating characteristic curve (AUC). While our approach aligned with established practices in predictive diagnostics, we will expand the methods section to detail the rationale and cite relevant literature supporting AUC-based threshold selection (e.g., Thiele & Hirschfeld, 2020). Additionally, we utilized the cutpointr package, following commands similar to those the reviewer mentioned, such as maximize_metric with the Youden index.

Comment 5: Clarify the basis for the recommended threshold of 7 and provide supporting references.
Response: In our manuscript, the threshold of 7 was referenced based on prior studies of WSFRAT’s application in psychiatric settings (Wilson et al., 2014; Wong et al., 2021). We will include these references and expand on how our study's threshold of 5 differs from this recommendation, discussing the implications of a lower threshold for heightened sensitivity and clinical applicability.

Comment 6 (Page 7, Discussion): Move the introductory sentence on scarcity of research to a more appropriate section.
Response: We agree that this point is better placed in the introduction. The introductory sentence on the scarcity of fall risk research in psychiatric populations will be relocated to the introduction, with further context in the discussion on comparing our findings to existing literature.

By addressing these points, we aim to enhance the clarity and scientific rigor of our manuscript, ensuring it meets high standards of methodological transparency and relevance. Thank you for your valuable insights

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] Abraham S: Managing patient falls in psychiatric inpatient units. Health Care Manag. 2016; 35(2): 121–133. PubMed Abstract | Publisher Full Text

[2] Carpels A, de Smet L , Desplenter S, et al.: Falls among psychiatric inpatients: a systematic review of literature. Alpha Psychiatry. 2022; 23(5): 217–222. PubMed Abstract | Publisher Full Text | Free Full Text

[3] Chou S-P, Hung T-M: Comparison of clinical diagnostic effectiveness between psychiatric inpatient and Wilson-Sims fall risk assessment tools. Hu Li Za Zhi. 2019; 66(3): 35–45. PubMed Abstract | Publisher Full Text

[4] Edmonson D, Robinson S, Hughes L: Development of the Edmonson psychiatric fall risk assessment tool. J. Psychosoc. Nurs. Ment. Health Serv. 2011; 49(2): 29–36. Publisher Full Text

[5] Healey F: A GUIDE ON HOW TO PREVENT FALLS AND INJURY IN HOSPITALS. Nurs. Older People. 2010; 22(9): 16–22. Publisher Full Text

[6] Kuhn M: Caret: classification and regression training. Astrophysics Source Code Library. 2015; ascl-1505.

[7] Lu S-H, Chen K-H, Pan Y-C, et al.: Influence of medications and psychotic symptoms on fall risk in acute psychiatric inpatients. J. Med. Sci. 2018; 38(3): 117–121. Publisher Full Text

[8] Montalvo I: The National Database of Nursing Quality Indicators^® (NDNQI^®). OJIN: The Online J. Issues Nurs. 2007; 12(3). Publisher Full Text

[9] Nice.: Assessment and prevention of falls in older people. NICE guidance june 213. Nurs. Older People. 2014; 26(6).

[10] Revelle W: psych: Procedures for Personality and Psychological Research.2020. Reference Source

[11] Robin X, Turck N, Hainard A, et al.: pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics. 2011; 12: 77. PubMed Abstract | Publisher Full Text | Free Full Text

[12] Team, R. C: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. 2021.

[13] Thiele C, Hirschfeld G: Cutpointr: Improved estimation and validation of optimal cutpoints in R. ArXiv Preprint ArXiv:2002.09209. 2020.

[14] Van Dyke D, Singley B, Speroni KG, et al.: Evaluation of fall risk assessment tools for psychiatric patient fall prevention: A comparative study. J. Psychosoc. Nurs. Ment. Health Serv. 2014; 52(12): 30–35. PubMed Abstract | Publisher Full Text

[15] Watson WL, Clapperton AJ, Mitchell RJ: The cost of fall-related injuries among older people in NSW, 2006–07. N. S. W. Public Health Bull. 2011; 22(4): 55–59. PubMed Abstract | Publisher Full Text

[16] Wickham H, Averick M, Bryan J, et al.: Welcome to the Tidyverse. J. Open Source Softw. 2019; 4(43). Publisher Full Text

[17] Wickham H, Chang W, Wickham MH: Package ‘ggplot2.’. Create Elegant Data Visualisations Using the Grammar of Graphics. Version. 2016; 2(1): 1–189.

[18] Wilson SC, Fettes S, Sims K: Gravity: It’s Not a Suggestion-It’s the Law! The Development of the WilsonSims Psychiatric Fall Risk Assessment Tool. J. Am. Psychiatr. Nurses Assoc. 2014; 20(1): 83.

[19] Wong MMC, Pang PF, Chan CF, et al.: Wilson Sims Fall Risk Assessment Tool Versus Morse Fall Scale in Psychogeriatric Inpatients: a Multicentre Study. East Asian Arch. Psychiatr. 2021; 31(3): 67–70. PubMed Abstract | Publisher Full Text

[20] World Health Organization: Falls, World Health Organization.2021. (Accessed: 20 September 2023). Reference Source

Evaluating Wilson sims tool: Quality improvement on fall risk in UAE psychiatric patients

Abstract

BACKGROUND

AIM

METHODS

RESULTS

CONCLUSION

Keywords

1. Introduction

2. Methods

2.1 Type and design

2.2 Place and date

2.3 Ethical Approval Statement for Publication

2.4 Population and sample

2.5 Data collection tools

2.6 Falls assessment in the hospital

2.7 Data collection process

2.8 Data analysis

3. Results

Table 1. Association between fall risk and various patient characteristics by Fisher exact test.

Table 2. Compared fall and non-fall cases by Welch Two Sample t-test.

Table 3. Receiver operating characteristic (ROC) curve analysis.

Figure 1. Receiver Operating Curve (ROC) and Changes in the values of Area Under the Curve (AUC).

Table 4. Simple logistic regression.

4. Discussion

5. Limitations

6. Conclusion

7. Implications for nursing practice

Data availability

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated