Keywords
Simulation-Based Training, Anesthesia Crisis Resource Management, Non-Technical Skills, Assessment of Non-Technical Skills (ANTS), Postgraduate Anesthesia Residents, Patient Safety.
-
Views
-
Downloads
How to cite this article
Taksande K, Singam DA and Thawkar V. A prospective randomized comparative study of effectiveness of simulation-based training in improving the performance of non-technical skill among postgraduate students during anaesthesia crisis resource management [version 1; peer review: 3 approved with reservations]. F1000Research 2024, 13:354 (https://doi.org/10.12688/f1000research.143437.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
Export Citation
Sciwheel
EndNote
Ref. Manager
Bibtex
ProCite
Sente
▬
✚
Study Protocol
A prospective randomized comparative study of effectiveness of simulation-based training in improving the performance of non-technical skill among postgraduate students during anaesthesia crisis resource management
[version 1; peer review: 3 approved with reservations]
PUBLISHED 23 Apr 2024
OPEN PEER REVIEW
REVIEWER STATUS
This article is included in the Datta Meghe Institute of Higher Education and Research collection.
Abstract
Corresponding author:
Karuna Taksande
Competing interests:
No competing interests were disclosed.
Grant information:
The author(s) declared that no grants were involved in supporting this work.
Copyright:
© 2024 Taksande K et al.
This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
How to cite: Taksande K, Singam DA and Thawkar V. A prospective randomized comparative study of effectiveness of simulation-based training in improving the performance of non-technical skill among postgraduate students during anaesthesia crisis resource management [version 1; peer review: 3 approved with reservations]. F1000Research 2024, 13:354 (https://doi.org/10.12688/f1000research.143437.1)
First published: 23 Apr 2024, 13:354 (https://doi.org/10.12688/f1000research.143437.1)
Latest published: 23 Apr 2024, 13:354 (https://doi.org/10.12688/f1000research.143437.1)
Introduction
Patient safety and effective crisis management are paramount in anaesthesia, where even minor errors can have significant consequences. Anaesthesia providers, particularly residents in their early postgraduate years, face the demanding challenge of responding swiftly and adeptly to unexpected perioperative crises.1 The ability to navigate such crises hinges on clinical proficiency and non-technical skills, encompassing task management, teamwork, situational awareness, and decision-making. These non-technical skills are increasingly crucial to safe and high-quality healthcare delivery.2
Simulation-based training has emerged as a transformative approach in healthcare education, offering a safe and immersive environment for learners to hone their clinical and non-technical skills. In recent years, simulation-based training has gained prominence in anaesthesia, providing a dynamic platform for anaesthesia residents to acquire and refine crisis resource management skills without exposing patients to unnecessary risk.3
This study protocol delineates a rigorous examination of the effectiveness of simulation-based training in enhancing the non-technical skills of postgraduate anaesthesia residents during anaesthesia crisis resource management. The Assessment of Non-Technical Skills (ANTS) scoring system will serve as the primary evaluative tool, allowing for the comprehensive assessment of task management, teamwork, situational awareness, and decision-making abilities in a controlled and standardized setting.4
The objective of this study is to bridge a critical knowledge gap by elucidating whether simulation-based training can significantly enhance anesthesia residents’ non-technical skills, ultimately contributing to improved patient safety during perioperative crises. As healthcare systems continually seek innovative approaches to optimize education and training, the findings of this research hold the potential to inform curricular advancements and healthcare practice standards.
Furthermore, this study seeks to explore the relationship between participant characteristics, such as years of experience and prior training exposure, and the effectiveness of simulation-based training. Such insights may lead to tailored educational interventions that cater to the unique needs of anesthesia residents at various stages of their training.5
The study’s multifaceted approach, encompassing quantitative assessments, will provide a holistic understanding of the impact of simulation-based training on anesthesia residents’ non-technical skills. The ultimate aspiration is to empower anesthesia providers with the tools and knowledge to excel in crisis resource management, enhancing patient safety and the overall quality of anesthesia care.
Aim
The aim of this study is to assess the effectiveness of simulation-based training in improving the performance of anesthesia residents during anesthesia crisis resource management using the Assessment of Non-Technical Skills (ANTS) scoring system.
Objectives
Primary objective
To evaluate and compare the performance of anesthesia residents in the following key non-technical skills domains during anesthesia crisis resource management:
1. Task management: assess the ability to organize tasks efficiently during a crisis.
2. Teamwork: evaluate teamwork skills, including communication, collaboration, and leadership within the anesthesia team.
3. Situational awareness: measure the capacity to perceive and understand critical aspects of the situation, including early detection of potential issues.
4. Decision-making: assess the quality and timeliness of decisions made by anesthesia residents in crisis scenarios.
Secondary objectives
1. To enhance anesthesia residents’ cognitive abilities in recognizing complex and challenging clinical scenarios.
2. To improve anesthesia residents’ social abilities in effective communication and interaction within the anesthesia team during crisis situations.
3. To assess the retention of improved non-technical skills among anesthesia residents over a defined followup period.
4. To gather participant feedback regarding their experiences with simulation-based training and its perceived impact on their clinical practice.
5. To explore potential correlations between demographic factors (e.g., years of experience before training) and the effectiveness of simulation-based training in improving non-technical skills.
Inclusion criteria
Participants eligible for inclusion in this study will be anaesthesia residents in their first and second years of postgraduate study (PG).
Study area
The research will be carried out within the School of Virtual Learning premises at Acharya Vinoba Bhave Rural Hospital, Jawaharlal Nehru Medical College, Sawangi Meghe, Wardha.
Setting
The study will be conducted following the approval of the ethics and screening committee of Jawaharlal Nehru Medical College, DMIHER (DU), within the School of Virtual Learning at AVBRH, JNMC, Sawangi Meghe, Wardha.
Study outcome
1. Improved non-technical skills: It is expected that participants who undergo simulation-based training will demonstrate a significant improvement in their non-technical skills, including task management, teamwork, situational awareness, and decision-making abilities. This improvement will likely be reflected in higher ANTS scores compared to baseline assessments.
2. Enhanced crisis resource management: Simulation-based training is anticipated to enhance the ability of anaesthesia residents to manage crises effectively. This includes improved coordination within the anaesthesia team, quicker and more accurate decision-making, and a heightened awareness of critical situations.
3. Positive participant feedback: Post-training feedback from participants is expected to reflect a positive experience and increased confidence in managing anaesthesia crisis scenarios. Participants are likely to find value in the hands-on, immersive nature of the training.
4. Correlation with faculty interest: The study may reveal a correlation between faculty interest in teaching using Human Patient Simulator (HPS) mannequins and the selected perioperative emergency topics. This could influence the integration of simulation-based training into anaesthesia education curricula.
5. Long-term retention of skills: The study may provide insights into the long-term retention of improved non-technical skills among anaesthesia residents. Follow-up assessments and feedback sessions after a month may indicate whether the skills acquired during training persist over time.
6. Variability in performance: The study may identify variations in performance among anaesthesia residents, which could be influenced by factors such as years of experience and prior training. This information could inform tailored educational interventions.
7. Validity of simulation-based training: The outcomes will contribute to the ongoing discussion on the validity and effectiveness of simulation-based training in healthcare education. Positive results would support the value of simulation in improving non-technical skills.
8. Contribution to patient safety: Ultimately, the study’s outcomes have the potential to contribute to improved patient safety during anaesthesia procedures. Enhanced non-technical skills among anaesthesia residents may lead to better crisis management, reduced errors, and improved patient outcomes.
Sample size
After the approval from institutional ethics committee, the study will be conducted on 24 residents from department of anaesthesiology and will be randomly allocated into two groups: Group 1 (baseline session) n=12: performance of anaesthesia resident during first baseline session of simulation scenario; and Group 2 n=12: performance of anaesthesia resident during second session of simulation scenario.
Data collection
The data collection process for the study involving anaesthesia residents in postgraduate years 1 and 2, focusing on simulation-based training for non-technical skills during anaesthesia crisis resource management, will be conducted with a high standard of American English. Below is a detailed description of the data collection process:
Subject recruitment and informed consent: Anesthesia residents in postgraduate years 1 and 2 from the Department of Anesthesia will be included as study subjects following approval from the Institutional Research Board. There are no exclusion criteria, and residents can decline participation.
Orientation session: Before the simulation sessions, all subjects will attend an orientation session. This session will cover the fundamentals of crisis evolution, patient simulation, and anaesthesia crisis resource management (ACRM). The initial hour of didactic instruction will include these key topics.
Simulation setup and pretest: Subsequently, subjects will receive a hands-on introduction to the Human Patient Simulator (HPS) mannequin and monitors, the anaesthesia machine, and the simulated operating room setting. A pre-test will also be administered to assess the baseline knowledge and skills of the participants.
Assessment for faculty: The study will assess faculty interest in teaching using HPS mannequins and select the topics of perioperative emergencies to be included in the simulation scenarios. A committee of faculty members will choose six specific perioperative emergencies:
1. Laryngospasm during induction of general anaesthesia.
2. Intraoperative ventricular tachycardia (VT).
3. Anaphylaxis.
4. Local anaesthetic toxicity (LA toxicity).
5. Intraoperative myocardial infarction (MI).
6. High spinal anesthesia.
Simulation sessions: Participants will be divided into two groups for the simulator sessions. Each session will present three distinct scenarios. In each scenario, one participant will act as the primary anesthesiologist, while the other will assume the role of a secondary anesthesiologist, helping when requested.
Scenario execution: Each scenario will involve a verbal handoff from the chief investigator to the primary anesthesiologist, including crucial patient history, conducted studies, and the current state of anaesthesia and the surgical procedure. The entire simulation will be recorded, displaying the patient’s vital signs alongside the recorded procedure.
Secondary anesthesiologist’s role: The secondary anesthesiologist will provide support as instructed but not offer crisis management advice or differential diagnoses during the scenarios.
Debriefing and feedback: Following each scenario, all subjects will participate in a videotape-assisted debriefing session based on ACRM training principles. A post-test and feedback from participants will also be collected.
Rotational participation: Throughout the study, subjects will alternate between primary and secondary roles in the three scenarios. After the first three scenarios, a debriefing will occur. Subjects will remain in the same group for the study. They return to the simulation centre a month later for their second simulation session, including three new scenarios with debriefings.
Performance evaluation: Performance evaluation will primarily focus on non-technical skills and involve primary and secondary anesthesiologists. Subjects will participate in all six scenarios, with recurrent performance evaluations based on the initial three scenarios.
Assessor training and ANTS scoring: Experienced staff anesthesiologists trained in simulation and ACRM concepts will serve as assessors, using the Assessment of Non-Technical Skills (ANTS) scoring system.4 Assessors will undergo training, which includes providing them with the user manual and background ANTS literature. After 4 hours of training, they can independently evaluate residents’ handling of simulated anaesthetic crises using videotapes. The ANTS scoring system will assess task management, teamwork, situational awareness, and decision-making skills.
Data comparison and analysis: Assessors will compare ANTS scores after evaluating videotapes of participants’ performance. ANTS uses a four-point scale to evaluate observable non-technical skills, with research videotapes sent to assessors after the study intervention phase is complete Table 1. This comprehensive data collection process will ensure a rigorous evaluation of the impact of simulation-based training on the non-technical skills of anaesthesia residents during anaesthesia crisis resource management.
Table 1. Anesthetists’ non-technical Skills system rating scale.4
Allocation
Randomisation
The allocation of participants to the simulation-based training and control groups will be achieved through a randomisation process to ensure unbiased and equitable group assignment. Randomisation will be performed using a computer-generated randomisation sequence generated by an independent statistician or through a randomisation tool or software.
1. Sequence generation: A randomisation sequence will be generated, assigning each participant a unique identification number. The sequence will be concealed from the study investigators until group assignment is required.
2. Group assignment: Participants will be assigned to the simulation-based training or control group based on the generated sequence. Group assignments will be concealed until the time of allocation to minimise selection bias.
3. Blinding: Due to the nature of the intervention (simulation-based training), it may not be feasible to blind participants or instructors. However, efforts will be made to ensure that the allocation sequence remains concealed from those involved in the allocation process.
Group characteristics
Baseline characteristics of participants, including demographic information, years of experience, and prior training exposure, will be collected to assess and report any potential group differences. Any significant differences in baseline characteristics between the groups will be considered during the data analysis phase.
Group size and allocation ratio
The study will aim to allocate an equal number of participants to both the simulation-based training group and the control group to maintain balance. The allocation ratio will be 1:1. However, if there are practical constraints, such as an uneven number of participants, this ratio may be adjusted accordingly.
Allocation concealment
The allocation sequence and group assignments will be concealed from the study investigators and participants until the allocation point. This will be achieved using sealed envelopes or an independent party responsible for allocation. Allocation concealment is essential to prevent selection bias and maintain the integrity of the randomisation process.
Monitoring and adherence
A designated study coordinator or research team member will ensure adherence to the randomisation process and proper group allocation. Monitoring will include periodic checks to confirm that the randomisation sequence is followed accurately.
Contingency plans
In the event of protocol deviations or instances where participants cannot complete the assigned intervention (e.g., due to unforeseen circumstances), a protocol for addressing such deviations and maintaining data integrity will be established and documented.
Statistical analysis
Statistical methods, including descriptive statistics, t-tests, and correlation analyses, will be employed to analyse the data using SPSS version 23. Data analysis in this study will be conducted meticulously to extract meaningful insights into the impact of simulation-based training on the non-technical skills of anaesthesia residents during anaesthesia crisis resource management. Our analytical approach encompasses several key facets:
We will perform a descriptive analysis to provide a comprehensive overview of the data collected. These entail calculating means and standard deviations for continuous variables, such as baseline participant characteristics, and computing frequencies for categorical variables. Descriptive statistics will help us understand the initial demographics and characteristics of the anaesthesia residents who participated in the study.
To assess the effectiveness of simulation-based training, we will employ comparative analysis. Specifically, we will conduct paired t-tests or Wilcoxon signed-rank tests to compare pre-training and post-training Assessment of Non-Technical Skills (ANTS) scores within groups. These tests will reveal whether the simulation-trained group exhibited statistically significant improvements in non-technical skills following the training sessions. Additionally, independent t-tests or Mann-Whitney U tests will be utilised to compare ANTS scores between the simulation-trained group and the control group, shedding light on the differential impact of the training program.
Correlation analyses will explore potential relationships between participant characteristics and training outcomes. For instance, we will investigate whether years of experience or prior training significantly affect the extent of skill improvement achieved through simulation-based training. These correlation analyses will enable us to identify any factors that may influence the effectiveness of the training program.
Data reporting
The findings of this study will be reported in a comprehensive research report, which may also be submitted for publication in peer-reviewed scientific journals. Our aim in data reporting is to present the results in a clear, organised, and transparent manner to facilitate understanding and interpretation by both the scientific community and stakeholders in healthcare education.
Data management
Ensuring secure, organised, and ethical data management is paramount to this study’s integrity and success. The following procedures will be implemented to safeguard data at every stage of the research process:
Data collection
1. Secure data collection: All data collected during the study, including Assessment of Non-Technical Skills (ANTS) scores, participant feedback, demographic information, and video recordings, will be collected using secure electronic or paper forms.
2. Anonymization: Participants’ identifiers will be replaced with unique identification codes to ensure confidentiality and privacy. Any paper documents with personal information will be securely stored in locked cabinets.
Bias
Selection bias
Bias: Participants in the study may not represent the entire anaesthesia resident population, potentially leading to skewed results.
Overcome: Implement a randomisation process to allocate participants to the simulation-based training group and the control group. Randomisation helps ensure that each participant has an equal chance of being assigned to either group, reducing the risk of selection bias.
Observer bias
Bias: The assessors evaluating participants’ performance during simulation sessions may be aware of group assignments, which could lead to biased scoring.
Overcome: Implement blinding or masking of assessors to group assignments. Ensure that assessors know whether a participant belongs to the simulation-based training or control group to minimise observer bias.
Confirmation bias
Bias: Researchers or assessors may unintentionally interpret data in a way that confirms their expectations or hypotheses.
Overcome: Maintain objectivity throughout the study. Use validated assessment tools like the Assessment of Non-Technical Skills (ANTS) scoring system to minimise subjective judgment. Pre-specify analysis plans to reduce the likelihood of post-hoc data interpretation that aligns with preconceived notions.
Ethical considerations
The Institutional Ethics Committee of Datta Meghe Institute of Higher Education and Research (DU) has granted its approval to the study protocol (Reference number: DMIHER (DU)/IEC/2023/977) Date: 10/05/2023. Before commencing the study, we will obtain written informed consent from all participants, providing them with a comprehensive explanation of the study’s objectives.
Discussion
The field of anaesthesia is marked by a critical need for anaesthesia providers, particularly residents, to possess clinical proficiency and robust non-technical skills, encompassing task management, teamwork, situational awareness, and decision-making.6 These non-technical skills are pivotal in ensuring patient safety during perioperative crises. Simulation-based training has emerged as a promising educational approach, offering a controlled environment for learners to develop clinical and non-technical competencies.7
Our study protocol outlines a rigorous examination of the potential benefits of simulation-based training for anaesthesia residents. The primary aim of this study is to evaluate whether such training significantly enhances non-technical skills, as assessed by the Assessment of Non-Technical Skills (ANTS) scoring system. The ANTS scoring system is a validated tool that comprehensively evaluates anaesthesia providers’ performance in high-stress scenarios.8 Through this study, we aim to contribute to the growing body of evidence supporting the integrating of simulation-based training into anaesthesia education curricula.
The anticipated outcomes of this study align with prior research that has demonstrated the efficacy of simulation-based training in improving non-technical skills among healthcare providers. For example, studies in surgical and medical specialities have shown that simulation-based training can significantly improve teamwork and communication skills, situational awareness, and decision-making.9,10 Given the parallels between the demands of perioperative care and other healthcare settings, we anticipate that anaesthesia residents will likewise benefit from simulation-based training.
Our study also seeks to explore potential correlations between participant characteristics, such as years of experience and prior training exposure, and the effectiveness of simulation-based training. Previous research has suggested that learners at different stages of their education and training may respond differently to simulation interventions.11 By considering these factors, we may gain insights into how to tailor educational interventions to the unique needs of anaesthesia residents at different points in their training journey.
This study’s findings can inform educational strategies and best practices in anaesthesia crisis resource management. Improved non-technical skills among anaesthesia residents can lead to more effective crisis management, reduced errors, and enhanced patient safety.4 As the healthcare landscape continues to evolve, optimising the education and training of anaesthesia providers is essential to meet the demands of complex and rapidly changing clinical environments.
Download
Export To
metrics
| Views | Downloads | |
|---|---|---|
| F1000Research | - | - |
|
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
Citations
CITE
how to cite this article
Taksande K, Singam DA and Thawkar V. A prospective randomized comparative study of effectiveness of simulation-based training in improving the performance of non-technical skill among postgraduate students during anaesthesia crisis resource management [version 1; peer review: 3 approved with reservations]. F1000Research 2024, 13:354 (https://doi.org/10.12688/f1000research.143437.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.
Open Peer Review
Current Reviewer Status:
Key to Reviewer Statuses
VIEW
HIDE
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations
A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions
Version 1
VERSION 1
PUBLISHED 23 Apr 2024
Views
4
How to cite this report:
Ahmed A and Rashid S. Reviewer Report For: A prospective randomized comparative study of effectiveness of simulation-based training in improving the performance of non-technical skill among postgraduate students during anaesthesia crisis resource management [version 1; peer review: 3 approved with reservations]. F1000Research 2024, 13:354 (https://doi.org/10.5256/f1000research.157097.r335695)
The direct URL for this report is:
https://f1000research.com/articles/13-354/v1#referee-response-335695
https://f1000research.com/articles/13-354/v1#referee-response-335695
NOTE: it is important to ensure the information in square brackets after the title is included in this citation.
Reviewer Report 22 Nov 2024
Saima Rashid,
Anaesthesiology, The Aga Khan University, Karachi, Sindh, Pakistan
Approved with Reservations
VIEWS 4
Thank you for the request to review this protocol. We have reviewed it closely and our comments are as follows:
Objectives:
How will it be assessed that secondary objectives number 1 and 2 have ... Continue reading
Objectives:
How will it be assessed that secondary objectives number 1 and 2 have ... Continue reading
CITE
HOW TO CITE THIS REPORT
Ahmed A and Rashid S. Reviewer Report For: A prospective randomized comparative study of effectiveness of simulation-based training in improving the performance of non-technical skill among postgraduate students during anaesthesia crisis resource management [version 1; peer review: 3 approved with reservations]. F1000Research 2024, 13:354 (https://doi.org/10.5256/f1000research.157097.r335695)
The direct URL for this report is:
https://f1000research.com/articles/13-354/v1#referee-response-335695
https://f1000research.com/articles/13-354/v1#referee-response-335695
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Views
2
How to cite this report:
Yao W. Reviewer Report For: A prospective randomized comparative study of effectiveness of simulation-based training in improving the performance of non-technical skill among postgraduate students during anaesthesia crisis resource management [version 1; peer review: 3 approved with reservations]. F1000Research 2024, 13:354 (https://doi.org/10.5256/f1000research.157097.r297393)
The direct URL for this report is:
https://f1000research.com/articles/13-354/v1#referee-response-297393
https://f1000research.com/articles/13-354/v1#referee-response-297393
NOTE: it is important to ensure the information in square brackets after the title is included in this citation.
Reviewer Report 28 Oct 2024
Wenlong Yao,
Wuhan Clinical Research Center for Geriatric Anesthesia, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
Approved with Reservations
VIEWS 2
This is a study protocol to investigate the role of simulated-based training in improving non-technical skills. Overall, the purpose is very clear.
I have several detailed questions about design.
1. The two groups were ... Continue reading
I have several detailed questions about design.
1. The two groups were ... Continue reading
CITE
HOW TO CITE THIS REPORT
Yao W. Reviewer Report For: A prospective randomized comparative study of effectiveness of simulation-based training in improving the performance of non-technical skill among postgraduate students during anaesthesia crisis resource management [version 1; peer review: 3 approved with reservations]. F1000Research 2024, 13:354 (https://doi.org/10.5256/f1000research.157097.r297393)
The direct URL for this report is:
https://f1000research.com/articles/13-354/v1#referee-response-297393
https://f1000research.com/articles/13-354/v1#referee-response-297393
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Views
2
How to cite this report:
Smith LO. Reviewer Report For: A prospective randomized comparative study of effectiveness of simulation-based training in improving the performance of non-technical skill among postgraduate students during anaesthesia crisis resource management [version 1; peer review: 3 approved with reservations]. F1000Research 2024, 13:354 (https://doi.org/10.5256/f1000research.157097.r301986)
The direct URL for this report is:
https://f1000research.com/articles/13-354/v1#referee-response-301986
https://f1000research.com/articles/13-354/v1#referee-response-301986
NOTE: it is important to ensure the information in square brackets after the title is included in this citation.
Reviewer Report 22 Aug 2024
Lisa Osborne Smith,
Oregon Health & Science University, Portland, OR, USA
Approved with Reservations
VIEWS 2
The study design is described as a randomized comparative study, but the description of the control group is not adequate, and it also doesn’t appear to be a control group per se. A pre-post-test design is a reasonable option for
... Continue reading
CITE
HOW TO CITE THIS REPORT
Smith LO. Reviewer Report For: A prospective randomized comparative study of effectiveness of simulation-based training in improving the performance of non-technical skill among postgraduate students during anaesthesia crisis resource management [version 1; peer review: 3 approved with reservations]. F1000Research 2024, 13:354 (https://doi.org/10.5256/f1000research.157097.r301986)
The direct URL for this report is:
https://f1000research.com/articles/13-354/v1#referee-response-301986
https://f1000research.com/articles/13-354/v1#referee-response-301986
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Comments on this article Comments (0)