Adaptation and Validation of the Interprofessional Socialization and Valuing Scale (ISVS-24) for Utilization in Multiplayer Virtual Reality Environments

Riyadh Firdaus; Aida Rosita Tantri; Rafidah Binti Atan; Sidharta Kusuma Manggala; Anggara Gilang Dwiputra; Andy Omega; Imelda Rosalyn Sianipar; Dina Muktiarti; Adisti Dwijayanti; Rima Maulida Hidayati; Tasya Claudia; Dewi Anggraeni Kusumoningrum; Luthfi Saiful Arif

doi:10.12688/f1000research.160939.1

Home Browse Adaptation and Validation of the Interprofessional Socialization and...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Research Article

Adaptation and Validation of the Interprofessional Socialization and Valuing Scale (ISVS-24) for Utilization in Multiplayer Virtual Reality Environments

[version 1; peer review: 2 approved with reservations]

Riyadh Firdaus¹, Aida Rosita Tantri ^1,2, Rafidah Binti Atan³, [...] Sidharta Kusuma Manggala¹, Anggara Gilang Dwiputra¹, Andy Omega¹, Imelda Rosalyn Sianipar², Dina Muktiarti², Adisti Dwijayanti², Rima Maulida Hidayati², Tasya Claudia², Dewi Anggraeni Kusumoningrum⁴, Luthfi Saiful Arif⁴

Riyadh Firdaus¹, Aida Rosita Tantri ^1,2, [...] Rafidah Binti Atan³, Sidharta Kusuma Manggala¹, Anggara Gilang Dwiputra¹, Andy Omega¹, Imelda Rosalyn Sianipar², Dina Muktiarti², Adisti Dwijayanti², Rima Maulida Hidayati², Tasya Claudia², Dewi Anggraeni Kusumoningrum⁴, Luthfi Saiful Arif⁴

PUBLISHED 06 Feb 2025

Author details Author details

¹ Department of Anesthesiology and Intensive Care, Faculty of Medicine, Universitas Indonesia, Central Jakarta, Jakarta, 10430, Indonesia
² Simulation-Based Medical Education and Research Center Cluster, Indonesia Medical Education and Research Institute (IMERI), Faculty of Medicine, Universitas Indonesia, Central Jakarta, Jakarta, 10430, Indonesia
³ Department of Anesthesiology, Faculty of Medicine, Malaya University, Kuala Lumpur, Kuala Lumpur, 50603, Malaysia
⁴ Medical Technology Cluster, Indonesia Medical Education and Research Institute (IMERI), Faculty of Medicine, Universitas Indonesia, Central Jakarta, Jakarta, 10430, Indonesia

Riyadh Firdaus
Roles: Conceptualization, Methodology, Supervision, Writing – Original Draft Preparation

Aida Rosita Tantri
Roles: Conceptualization, Formal Analysis, Project Administration, Writing – Original Draft Preparation

Rafidah Binti Atan
Roles: Conceptualization

Sidharta Kusuma Manggala
Roles: Conceptualization, Methodology, Software

Anggara Gilang Dwiputra
Roles: Conceptualization, Methodology, Software

Andy Omega
Roles: Conceptualization, Methodology, Software

Imelda Rosalyn Sianipar
Roles: Resources, Software, Writing – Original Draft Preparation

Dina Muktiarti
Roles: Resources, Software, Writing – Original Draft Preparation

Adisti Dwijayanti
Roles: Resources, Software, Writing – Original Draft Preparation

Rima Maulida Hidayati
Roles: Project Administration, Resources, Supervision, Writing – Review & Editing

Tasya Claudia
Roles: Data Curation, Formal Analysis, Writing – Original Draft Preparation

Dewi Anggraeni Kusumoningrum
Roles: Conceptualization, Methodology, Software

Luthfi Saiful Arif
Roles: Conceptualization, Methodology, Software

OPEN PEER REVIEW

REVIEWER STATUS

This article is included in the Public Health and Environmental Health collection.

Abstract

Background

Multiplayer Virtual Reality-Based Simulations (MPVR-based simulations) with immersive 3D environments have become an important tool in interprofessional education (IPE). However, instruments to measure interprofessional socialization in MPVR-based education are limited. The Interprofessional Socialization and Valuing Scale (ISVS) is a useful tool for evaluating interprofessional socialization. This pilot study aimed to adapt and validate the ISVS-24 for use in interprofessional MPVR-based simulations settings.

Methods

Seventy-two participants, including anesthesiology residents (at novice, junior, and senior levels), general physicians, and nurses, were recruited voluntarily. The ISVS-24 was cross-culturally adapted and reviewed by experts for content validity in the MPVR simulation context. Participants completed the adapted ISVS after undergoing an interprofessional MPVR-based simulation. Structural validity was assessed using factorial analysis through principal component analysis. Content validity was measured using the mean content validity index (CVI). Consistency validity was evaluated with Pearson correlation coefficients (PCC), and reliability was assessed using Cronbach’s alpha.

Results

The Kaiser-Meyer-Olkin test indicated sampling adequacy (0.885), and Bartlett’s sphericity test was significant (χ²(42) = 472.725, p < 0.05). A three-section structure was confirmed. The mean CVI was 0.815, with 21 valid items (Aiken’s V ≥ 0.5). Among the 72 respondents, 40 were female (55.6%) and 32 were male (44.4%); 26 were anesthesiology residents (36.1%), 23 were general physicians (31.9%), and 23 were nurses (31.9%). The overall Cronbach’s alpha was 0.959. PCC for all items exceeded the r table value (> 0.232) with p < 0.05, showing significant item relationships.

Conclusions

The adapted version of ISVS for MPVR simulation-based education has good validity and reliability to assess interprofessional socialization in an MPVR-based simulation setting.

Keywords

Interprofessional Socialization and Valuing Scale, Medical Education, Interprofessional, Multiplayer Virtual Reality Questionnaire, Validation

Corresponding author: Aida Rosita Tantri

Competing interests: No competing interests were disclosed.

Grant information: This work was supported by the Universitas Indonesia under the Publikasi Terindex Internasional (PUTI) Q2 Tahun 2022-2023 grant scheme [grant number: NKB-1217/UN2.RST/HKP.05.00/2022].
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2025 Firdaus R et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Firdaus R, Tantri AR, Atan RB et al. Adaptation and Validation of the Interprofessional Socialization and Valuing Scale (ISVS-24) for Utilization in Multiplayer Virtual Reality Environments [version 1; peer review: 2 approved with reservations]. F1000Research 2025, 14:169 (https://doi.org/10.12688/f1000research.160939.1) First published: 06 Feb 2025, 14:169 (https://doi.org/10.12688/f1000research.160939.1) Latest published: 17 Jul 2025, 14:169 (https://doi.org/10.12688/f1000research.160939.2)

Introduction

Interprofessional Education (IPE) has become a crucial aspect of advanced medical studies, playing a vital role in enhancing patient safety and refining healthcare services.^1–3 IPE promotes collaborative learning among individuals from different professional fields, which augments knowledge-sharing and problem-solving capabilities in delivering the highest quality of patient care and producing significant health outcomes.⁴ IPE seeks to impart clinical knowledge engagingly and dynamically. The benefits of IPE are clear to the students. When it comes to overcoming the limits of healthcare professionals, students who have been exposed to IPE in the curriculum have developed interprofessional partnerships throughout their practices, boosted team spirit, and fostered competencies. Their attitudes toward collaborative management and curiosity for patient care improved as a result of the implementation of IPE.^5,6

A teaching method for accomplishing learning objectives that is also used to give IPE is simulation-based education. It is modelled after real-world situations. In this case, multiplayer VR simulation-based education (MPVR-based education) has an important role in realizing interprofessional education (IPE) in promoting collaboration and communication competencies among professionals.⁷ VR has recently emerged as a potent tool for medical education that enables users to immerse themselves in and engage with computer-created three-dimensional environments, replicating real situations or simulating fictitious scenarios.^8,9 These interactive virtual experiences have been shown to outperform traditional or other digital educational methods regarding knowledge retention and skill development among health professionals. This superiority is attributed to VR’s ability to offer an immersive, interactive learning space, and its capacity to produce realistic clinical scenarios in a risk-free setting for learners.¹⁰ MPVR-based education offers group-oriented learning, teamwork simulations, and collective problem-solving activities by seamlessly incorporating interpersonal interactions within the virtual world.¹¹ The platform facilitates real-time communication, collaboration, and joint activities or simulations as a team in a shared virtual space.⁹ Although MPVR-based simulation is a rapidly developing simulation in medical education, there is not yet a suitable measuring instrument that fully measures interprofessional socialization outcomes, which captures changes in attitudes, actions, and beliefs of participants. Effective interprofessional socialization is essential as it will enhance the utilization of each individual’s skills during collaborative practice, promote efficient patient care, and enhance patient safety.

Interprofessional Socialization and Valuing Scale (ISVS) is a 24-item self-assessment questionnaire, that was developed based on interprofessional literature and focuses on changes in attitudes, actions, and beliefs that are essential to interprofessional socialization.¹² ISVS’s adaptability is evident in its successful adaptation and validation across various countries and interprofessional learning conditions.^12–14 The ability of ISVS-24 to be adapted and validated across such a wide spectrum of environments indicates its applicability among diverse socio-cultural contexts across countries, healthcare systems, education frameworks, cultural perceptions of professions, and societal expectations.

However, the existing ISVS-24 lacks validation for use in MPVR-based simulation settings, where collaborative learning and problem-solving occur in immersive 3D environments. Consequently, there is a need to develop and validate an adapted version of the ISVS-24 specifically tailored for assessing interprofessional collaboration in MPVR-based simulation settings. The objective of this research is to adapt and validate ISVS as an assessment tool for IPE in an MPVR-based education setting.

Methods

Study design

This study was a pilot study using a cross-sectional design aimed at determining the psychometric properties of the adapted ISVS-24 for a virtual environment setting and involved three phases. The first phase involved the translation of the original ISVS-24, which was in English, into Indonesian. The second phase was content validity assessment of the ISVS-24 by a panel of experts. The third phase was a pilot study conducted with participants who completed the questionnaire, which had undergone content validity processes and was adapted for use in a virtual environment setting ( Figure 1).

Figure 1. Translation and validation process.

Figure 2. Viewpoints of different users.

Participants

A total of 72 anesthesiology residents, general physicians, and nurses from the Faculty of Medicine, Universitas Indonesia – Dr. Cipto Mangunkusumo Hospital took part in this study and were voluntarily recruited. As inclusion criteria, the following factors were considered: older than 18 years, free from visual, auditory, and other sensory impairments, and agree to participate.

Development of VR-Local Anesthesia of Systemic Toxicity

This study developed Local Anesthesia of Systemic Toxicity scenes using Virtual Reality (VR-LAST) to provide an engaging and immersive learning experience that reflects students’ real-clinical scenarios. VR-LAST is a platform that provides an immersive experience of a virtual world featuring 3D-based user-generated content that supports interactive networking. This platform is used in the simulation of systemic toxicity events due to the administration of local anesthesia in patients undergoing surgery with local anesthesia.

In this platform, we created a scenario where an individual who will undergo operative action must be given anesthesia in the form of local anesthesia, which will be managed collaboratively by anesthesia residents, general practitioners, and nurses. As illustrated in Figure 2: viewpoints of different users, VR-LAST incorporates multiple user viewpoints: (A) the anesthesia resident’s view of the Non-Playable Character (NPC), (B) the general practitioner’s view, (C) the nursing student’s view, and (D) the facilitator’s view. The scenario guides participants through the preparation for anesthesia, administration of anesthesia, systemic toxicity events, and subsequent interventions, including assessment and management. The specific tasks performed by participants in the virtual environment are outlined in Supplementary Table. The operating room concept was built to help participants understand the environment and provide a realistic experience. The developed VR is ready to be tested for feasibility. It can enable participants from different times and locations to come together in real-time interactive simulation training to manage LAST emergency events. IPE using simulation-based LAST is a practical approach to prepare students for a collaborative environment, develop technical skills, and enhance soft skills, such as communication and teamwork, to prepare them as work-ready graduates.

Instruments

The Interprofessional Socialization and Valuing Scale (ISVS-24) was a 24-item self-assessment questionnaire created by King et al. The self-assessment questionnaire focuses on changes in attitudes, actions, and beliefs of participants after interprofessional training. Respondents are asked to rate their agreement or disagreement with each statement based on their own experiences and perceptions after previous training. The scale aims to capture attitudes, beliefs, and behaviours that contribute to effective teamwork and collaboration across three dimensions: self-perceived ability to work for others, value in working with others, and comfort in working with others.¹²

This study utilized virtual reality simulations that were played by three individuals. VR is conceived with specific learning objectives; comprehension of cognitive, motor, communicative, and interpersonal skills in managing local anaesthetic systemic toxicity cases in an immersive surgical room environment. A patient was standardized as NPCs (non-playable characters) with specific scenarios.

Procedure

Phase 1: Translation of the ISVS-24

Two competent native translators performed two separate translations of the source language into Indonesian, resulting in two versions of the scale. Subsequently, the two versions were compared and synthesized into one version. To accomplish this, the research team (native Indonesian speakers) discussed the wording of the items, adapted it with MPVR-based simulation, and generated a refined agreement version.

Phase 2: Adaptation and validation

To validate the content, the adapted and unanimous version of the ISVS underwent evaluation by a panel of experts made up of 4 senior anesthesiology consultants who were experts in MPVR simulation-based education. Each item was rated on a Likert scale of 1-5 based on its relevance to MPVR simulation-based setting. The scale was as follows: 1 - highly irrelevant, 2 - irrelevant, 3 - moderate, 4 - relevant, 5 - highly relevant. In addition, the experts were also requested to provide suggestions for revisions on certain questions they deemed less appropriate for multiplayer VR. After that, the assessments from the 4 experts were analyzed using Aiken’s formula. Any questionnaire items with an Aiken’s V score that did not meet validity criteria were removed.

Phase 3: Pilot study

The pilot study aimed to assess the content and comprehensibility of the scale. For the pilot study of the questionnaire, the anesthesiology residents, general physicians, and nurses (n=72) were voluntarily recruited. The participants were provided with information regarding the purpose of the study. They were given consent forms after being reminded of the voluntary nature of their participation and the confidential and anonymous handling of their data. Once they had signed the informed consent, the participants were assembled by the principal investigator in the pilot trial, who convened them in a single classroom. All participants in the pilot study were introduced and trained using VR equipment, specifically the Oculus Quest 2 headset. They were divided into small teams consisting of three persons, with one individual assigned as the team leader. All groups underwent MPVR simulation-based training. Upon finishing the simulation, the participants were asked to check the content and the understandability of the scale.

Prior to this, items deemed invalid based on the content validity analysis using Aiken’s formula were removed. Subsequently, the questionnaire was administered to 72 respondents. After using the MPVR LAST, respondents were asked to complete the ISVS questionnaire that had undergone content validity testing. The collected data were then analysed for validity and reliability of questionnaire items.

Data analysis

We used digital scoring and rating sheets to record the collected data manually. SPSS (version 22.0; IBM Corp) was applied to analyse the data quantitatively. First, a descriptive analysis of the results was conducted. The frequency and percentage were computed for the categorical variables, whereas the measures of central tendency and dispersion were determined for the quantitative variables. To determine the questionnaire’s structural validity, a factorial analysis was conducted through principal component analysis. The content validity index (CVI) of the instrument in this study was evaluated using Aiken’s formula. A measurement tool is considered adequately valid if its Aiken coefficient is greater than 0.5.^15–17 The assessment of internal consistency and reliability of the variables in this study was conducted through Cronbach’s alpha reliability analysis. Items with Cronbach’s alpha value above 0.70 were considered reliable. To evaluate the validity of the responses, the Pearson Correlation Coefficient (PCC) was calculated to assess the relationship between individual items and the overall scale.^18–21

Ethical considerations and consent to participate

This study was conducted following the Declaration of Helsinki and was approved by the Ethics Committee of the Faculty of Medicine, Universitas Indonesia - Dr. Cipto Mangunkusumo Hospital with regards of the protection of human rights and welfare in medical research, with approval number 1146/UN2.F1/ETIK/PPM.00.02/2022, on October 31, 2022. All participants provided written informed consent before participation. No participant-identifying information was included in the study data. Data were managed by researchers and assistants who were not involved in any teaching roles, and only the ID codes were noted in the database. No personal data was accessible to the instructional team in the initial phase of the research; adjustments were made to the items in the original ISVS to suit the multiplayer VR environment.

Results

Content validity

Aiken’s method was employed to assess the content validity of the questionnaire in this study. Four senior anesthesiology consultants who were experts in MPVR simulation-based education were involved in testing the content validity, utilizing the V coefficient derived from Aiken’s formula, which takes into account the number of items, judges, and the rating system. Consequently, in this investigation, each questionnaire item underwent scrutiny by the experts who evaluated its appropriateness and relevance. If a consensus (two or more experts) suggested that a specific item was unnecessary, it was subsequently removed from the questionnaire. The analysis of the content validity of the instrument used the Aiken coefficient V can be viewed in Table 1.

Table 1. Analysis table of Aiken's coefficient calculation results.

No.	Item	V
Self-perceived ability to work for others
1	Item 1	0.875
2	Item 2	0.938
3	Item 3	0.938
4	Item 4	0.813
5	Item 5	0.125^a
6	Item 6	0.875
7	Item 7	0.063^a
8	Item 8	0.688
9	Item 9	0.813
Value in working with others
10	Item 10	0.750
11	Item 11	0.875
12	Item 12	0.875
13	Item 13	0.938
14	Item 14	0.875
15	Item 15	0.750
16	Item 16	0.188^a
17	Item 17	0.750
18	Item 18	0.875
Comfort in working with others
19	Item 19	0.688
20	Item 20	0.875
21	Item 21	0.875
22	Item 22	0.750
23	Item 23	0.625
24	Item 24	0.750
Mean V		0.815

a Not valid, items were removed.

Three items that received low V Aiken’s values from the experts (V Aikens’s ≤ 0.5), were excluded from the questionnaire because they were deemed incompatible with the virtual room environment. The three items that were removed are:

1. “I am comfortable engaging in shared decision-making with clients” (item 5)
2. “I have gained a better understanding of the client’s involvement in decision-making around their care” (item 7)
3. “I have gained an appreciation for the importance of having the client and family as members of a team” (item 16)

Following this, twenty-one items with Aiken’s V values ≥ 0.5 were deemed valid and incorporated into the Multiplayer Virtual Reality Adapted Version of the Interprofessional Socialization and Valuing Scale. In addition, all experts agreed to make a slight change to item number 21 to ensure it is framed as a positive statement, changing it from “I believe that interprofessional practice is difficult to implement” to “I believe that interprofessional practice is easy to implement”.

Participant characteristics

Seventy-two participants total participated in the study, of which 40 were female (55.6%) and 32 were male (44.4%); 26 were anesthesiology residents (36.1%), 23 were general physicians (31.9%), and 23 were nurses (31.9%). The average age was 29.32 ± 2.54 years old ( Table 2).

Table 2. Demographic characteristics of subjects.

Characteristics	Mean ± SD	n (%)
Age in years	29.32 ± 2.54
Sex
Female		40 (55.6)
Male		32 (44.4)
Profession
Anesthesiology resident		26 (36.1)
General physician		23 (31.9)
Nurse		23 (31.9)

Consistency validity

The questionnaire’s validity was verified through the computation of the Pearson correlation coefficient (PCC). The critical value was 0.232, according to the table of critical values. As shown in Table 3, for all our variables, the r table was > 0.232 for p values of < 0.05. Table 3 displays the validity (Pearson’s correlation coefficient) for every item, indicating significant relationships between each item and section.

Table 3. Pearson’s correlation coefficients for measuring the validity of the questionnaire’s internal consistency.

Item No.	Statement items	PCC	Mean	SD
Self-perceived ability to work for others
1	I feel comfortable in accepting responsibility delegated to me within a team	0.66	4.51	0.56
2	I feel able to act as a fully collaborative member of the team	0.75	4.64	0.48
3	I’ve gained a better understanding of my own approach to care within an interprofessional team	0.66	4.67	0.48
4	I feel comfortable in being accountable for responsibilities I have taken on	0.76	4.61	0.52
5	I am able to listen to other members of the team	0.69	4.58	0.52
6	I feel comfortable clarifying misconceptions with other members of the team about the role of someone in my profession	0.58	4.44	0.55
7	I more highly value open and honest communication with team members	0.72	4.67	0.51
Value in working with others
8	I have gained more realistic expectations of other professionals on a team	0.70	4.61	4.91
9	I have gained an enhanced awareness of the roles of other professionals on a team	0.77	4.60	0.62
10	I see myself as preferring to work on an interprofessional team	0.81	4.60	0.52
11	I have gained an appreciation for the benefits in interprofessional teamwork	0.74	4.65	0.51
12	I have gained greater appreciation of the importance of a team approach	0.81	4.58	0.52
13	I feel comfortable initiating discussions	0.78	4.43	0.67
14	I believe that interprofessional practice will give me the desire to remain in my profession	0.80	4.57	0.53
15	I believe that interprofessional practice is not a waste of time	0.66	4.68	0.47
Comfort in working with others
16	I feel comfortable debating issues in a team	0.79	4.56	0.55
17	I am comfortable being the leader in a team situation	0.79	4.42	0.62
18	I feel confident in taking on different roles in a team (i.e., leader, participant)	0.79	4.44	0.58
19	I am able to share and exchange ideas in a team discussion	0.87	4.50	0.53
20	I feel comfortable speaking out within the team when others are not keeping the best interest of the patient in mind	0.74	4.26	0.69
21	I believe that interprofessional practice is easy to implement	0.72	4.51	0.58

The Kaiser-Meyer-Olkin test

The result of the Kaiser-Meyer-Olkin test that measures the appropriateness of the sample had a result of 0.857, which is higher than the minimum acceptable value 0.50. Bartlett’s sphericity test was significant (χ²(66)=1193.762 p < 0.05). The results of these tests demonstrated that the analysis factor was appropriate ( Table 4).

Table 4. Total variance explained.

Initial Eigen Values				Extraction sums of squared loadings
Component	Total	% of variance	Cumulative %	Total	% of variance	Cumulative %
1	11,912	56,726	56,726	11,912	56,726	56,726
2	1,106	5,266	61,992	1,106	5,266	61,992
3	1,027	4,891	66,883	1,027	4,891	66,883
4	0,991	4,717	71,600
5	0,914	4,351	75,951
6	0,781	3,721	79,671
7	0,735	3,502	83,173
8	0,580	2,763	85,936
9	0,505	2,406	88,342
10	0,440	2,098	90,440
11	0,358	1,706	92,145
12	0,334	1,590	93,735
13	0,264	1,255	94,990
14	0,236	1,125	96,115
15	0,192	0,913	97,028
16	0,170	0,810	97,838
17	0,145	0,691	98,529
18	0,121	0,577	99,106
19	0,087	0,412	99,518
20	0,055	0,261	99,780
21	0,046	0,220	100,000

Item reliability test

In this study, Cronbach’s alpha was used to measure the reliability of each item. Table 5 shows the Cronbach’s alpha result. The result showed a high internal consistency with an overall Cronbach’s alpha result of 0.959, which indicates that the adapted version of ISVS items’ were reliable.

Table 5. Cronbach’s alpha.

Sections	Number of items	Cronbach’s alpha
Self perceived ability to work for others	7 (No. 1-7)	0.861
Value in working with others	8 (No. 8-15)	0.909
Comfort in working with others	6 (No. 16-21)	0.907
Overall		0.959

Discussion

VR presents enormous potential for the future of medical education. Its applications range from surgical training and 3D visualization skills to teaching soft skills like empathy and communication. VR’s capacity for training technical competencies is well-recognized.²² However, its role in enhancing interpersonal collaboration skills still requires further study. In line with this, there’s an increasing demand for instruments that evaluate interprofessional collaboration performance in a multiplayer VR setting.

In this study, we adapted ISVS for multiplayer VR-based simulation education. Due to the distinct backdrop of the virtual world, certain items were removed during the initial validation stages. For instance, the item “I am comfortable engaging in shared decision-making with clients” was eliminated because, in a VR setting, patient are standardized as NPC with specific scenarios. Consequently, shared decision-making is unfeasible, as are two other statements concerning patient and patient’s family interactions.

In most VR-based simulation programs for medical education, patients are represented by NPC. Complex human behaviours are challenging to replicate authentically in VR. As a result, interactions with NPC might feel limited and less realistic than with real humans. The inherent limitations of NPC constrain the possible interactions between learners and patients. However, with technological advancements, particularly integrating artificial intelligence with NPC, NPC might become more sophisticated, allowing more realistic interactions.²³ Despite these challenges, VR’s capability in facilitating interprofessional collaboration lies in its ability to offer both communication and information support, enhancing users’ information processing abilities. VR might help students in achieving collaborative skills crucial for problem-solving both in and outside the classroom.^23–25

The content validity analysis, using Aiken’s V method, resulted in a final adapted questionnaire consisting of 21 valid items, after excluding three items with Aiken’s V values below 0.5. Furthermore, slight modifications were made to ensure clarity and appropriateness of the remaining items, such as rephrasing item 21 to convey a positive perspective. The high agreement among experts highlights the rigor of the adaptation process and supports the relevance of the adapted items for a VR-based environment.

The internal consistency validity, assessed using the Pearson correlation coefficient (PCC), demonstrated that all 21 items were significantly correlated with the overall scale, confirming their validity. Additionally, the reliability analysis using Cronbach’s alpha yielded a high internal consistency score (α = 0.959), indicating that the adapted ISVS is a robust tool for evaluating interprofessional collaboration outcomes in a VR setting.

The Kaiser-Meyer-Olkin (KMO) test (KMO = 0.857) and Bartlett’s sphericity test further confirmed the adequacy of the sample and the appropriateness of the factor analysis. These findings strengthen the evidence that the adapted ISVS can effectively measure interprofessional collaboration skills in multiplayer VR-based simulation environments.

However, this study has limitations. The relatively small sample size and its restriction to a single institution may limit the generalizability of the findings. Future studies should include larger and more diverse samples to enhance the reliability of the instrument. Additionally, further validation studies should explore the applicability of the adapted ISVS in different VR simulation scenarios and across various healthcare disciplines.

Despite these limitations, our findings support the use of the Multiplayer Virtual Reality Adapted Version of ISVS as a valid and reliable instrument for assessing interprofessional collaboration skills in VR-based education. By addressing the unique challenges of VR environments and adapting existing tools to fit this innovative learning modality, this study contributes to the growing body of research on VR’s role in advancing interprofessional education.

Conclusion

The adapted ISVS-24 for Multiplayer VR simulation-based education demonstrates strong validity and reliability, making it a suitable tool for assessing socialization and value aspects of interprofessional collaboration. The questionnaire provides meaningful insights into participants’ collaborative experiences within immersive virtual environments. However, further research is recommended to explore its applicability across diverse healthcare education settings and larger sample sizes. Future studies could also investigate the long-term impact of MPVR-based learning on interprofessional teamwork and clinical practice.

Data availability

Underlying data

Zenodo: Adaptation and Validation of the Interprofessional Socialization and Valuing Scale (ISVS-24) for Utilization in Multiplayer Virtual Reality Environments. https://doi.org/10.5281/zenodo.14637073.²⁶

This project contains the following underlying data:

• Aiken’s V Results.xlsx
• Validity and Reliability Test Results.xlsx
• Participants Feedback.xlsx

Extended data

This project contains the following extended data:

• Supplementary Table.docx
• Final Questionnaire (Indonesia and English Version).docx

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

Reporting guidelines

STROBE checklist for adaptation and validation of the interprofessional socialization and valuing scale (ISVS-24) for utilization in multiplayer virtual reality environments. https://doi.org/10.5281/zenodo.14637073.²⁶

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

References

1. Wong AH, Ruppel H, Gang M, et al.: “The safety dance”: a faculty development workshop partnering ipe and patient safety initiatives using simulation-based education. MedEdPORTAL. 2016; 12: 10440. Publisher Full Text
2. Hur HK, Kim KK, Lim YM, et al.: Patient safety interprofessional education program using medical error scenarios for undergraduate nursing and medical students in korea. J. Interprof. Care. 2023; 37(6): 944–953. PubMed Abstract | Publisher Full Text
3. Cleary E, Bloomfield J, Frotjold A, et al.: A realist synthesis of interprofessional patient safety activities and healthcare student attitudes towards patient safety. J. Interprof. Care. 2024 Mar-Apr; 38(2): 346–376. PubMed Abstract | Publisher Full Text
4. Mektrirat R, Sathanawongs A, Tiwananthagorn S, et al.: Achieving interprofessional education on collaborative problem-solving for covid-19 using project-based approach. Int. J. Infect. Dis. 2022 Mar; 116: S67. Publisher Full Text
5. Khalafi A, Sarvi Sarmeydani N, Akhoondzadeh R: Simulation-based interprofessional education (ipe) for enhanced attitude and teamwork of anesthesiology residents and nurse anesthesia students in iran. J. Adv. Med. Educ. Prof. 2023 Apr; 11(2): 105–112. PubMed Abstract | Publisher Full Text
6. Becker V, Jedlicska N, Scheide L, et al.: Changes in medical students´ and anesthesia technician trainees´ attitudes towards interprofessionality - experience from an interprofessional simulation-based course. BMC Med. Educ. 2022 Apr; 22(1): 273. PubMed Abstract | Publisher Full Text | Free Full Text
7. Lewis C, Enriquez D, Calabrese L, et al.: Virtual reality multiplayer interaction and medical patient handoff training and assessment. Adv. Intell. Syst. Comput. 2022; 1421: 17–23. Publisher Full Text
8. Gan W, Mok TN, Chen J, et al.: Researching the application of virtual reality in medical education: one-year follow-up of a randomized trial. BMC Med. Educ. 2023 Jan 3; 23(1): 3. PubMed Abstract | Publisher Full Text | Free Full Text
9. Jiang H, Vimalesvaran S, Wang JK, et al.: Virtual reality in medical students’ education: scoping review. JMIR Med. Educ. 2022 Feb; 8(1): e34860. PubMed Abstract | Publisher Full Text | Free Full Text
10. Haowen J, Vimalesvaran S, Myint Kyaw B, et al.: Virtual reality in medical students’ education: a scoping review protocol. BMJ Open. 2021 May 26; 11(5): e046986. PubMed Abstract | Publisher Full Text | Free Full Text
11. Du J, Shi Y, Mei C, et al.: Communication by interaction: a multiplayer vr environment for building walkthroughs. Constr. Res. Congr. 2016; 2281–2290.
12. King G, Orchard C, Khalili H, et al.: Refinement of the interprofessional socialization and valuing scale (isvs-21) and development of 9-item equivalent versions. J. Contin. Educ. Heal. Prof. 2016 Summer; 36(3): 171–177. PubMed Abstract | Publisher Full Text
13. Mahler C, Orchard C, Berger S, et al.: Translation and psychometric properties of the german version of the “interprofessional socialization and valuing scale” (isvs-21-d). J. Interprof. Care. 2023; 37(4): 655–661. PubMed Abstract | Publisher Full Text
14. Wu Q, Wang Y, Lu L, et al.: Virtual simulation in undergraduate medical education: a scoping review of recent practice. Front. Med. (Lausanne). 2022 Mar; 9: 855403. PubMed Abstract | Publisher Full Text | Free Full Text
15. Bujang MA, Khee HY, Yee LK: A step-by-step guide to questionnaire validation research. Institute for clinical Research. 2022.
16. Shrotiya VK, Dhanda U: Content validity of Assessment Instrument for Employee Engagement. SAGE Open. 2019; 9: 1–7. Publisher Full Text
17. Aiken LR: Three coefficients for analyzing the reliability and validity of ratings. Educ. Psychol. Meas. 1985; 45(1): 131–142. Publisher Full Text
18. Retnawati H: Proving content validity of self-regulated learning scale (the comparison of aiken index and expanded gregiry index). REiD (Res. Eval. Educ). 2016; 2(2): 155–164. Publisher Full Text
19. Paranhos R, Figueiredo-Filho D, Rocha E, et al.: Unveiling the mysteries of pearson’s correlation coefficient: the return. Leviathan. 2014; 8: 66–95. Publisher Full Text
20. Taber KS: The use of cronbach’s alpha when developing and reporting research instruments in science education. Res. Sci. Educ. 2018; 48: 1273–1296. Publisher Full Text
21. Tavakol M, Dennick R: Making sense of Cronbach’s alpha. Int. J. Med. Educ. 2011 Jun; 2: 53–55. PubMed Abstract | Publisher Full Text | Free Full Text
22. Schultz KS, Whitney DJ: Measurement theory in action: case studies and exercises. Thousand Oaks, CA: Sage; 2005. 9781452224749.
23. McGrath JL, Taekman JM, Dev P, et al.: Using virtual reality simulation environments to assess competence for emergency medicine learners. Acad. Emerg. Med. 2018 Feb; 25(2): 186–195. PubMed Abstract | Publisher Full Text
24. Aithal A, Aithal PS: Development and validation of survey questionnaire & experimental data – a systematical review-based statistical approach. IJMTS. 2020; 5(2): 233–251. Publisher Full Text
25. González-Pascual JL, Márquez MG, Rodríguez-Rey R, et al.: Adaptation and validation of the interprofessional socialization and valuing scale (isvs) in spanish university health sciences students. J. Interprof. Care. 2022 Nov-Dec; 36(6): 916–922. Publisher Full Text
26. Firdaus R, Tantri AR, Atan RB, et al.: Adaptation and validation of the interprofessional socialization and valuing scale (isvs-24) for utilization in multiplayer virtual reality environments. Dataset. Zenodo. 2025. Publisher Full Text

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 06 Feb 2025

Author details Author details

Riyadh Firdaus
Roles: Conceptualization, Methodology, Supervision, Writing – Original Draft Preparation

Aida Rosita Tantri
Roles: Conceptualization, Formal Analysis, Project Administration, Writing – Original Draft Preparation

Rafidah Binti Atan
Roles: Conceptualization

Sidharta Kusuma Manggala
Roles: Conceptualization, Methodology, Software

Anggara Gilang Dwiputra
Roles: Conceptualization, Methodology, Software

Andy Omega
Roles: Conceptualization, Methodology, Software

Imelda Rosalyn Sianipar
Roles: Resources, Software, Writing – Original Draft Preparation

Dina Muktiarti
Roles: Resources, Software, Writing – Original Draft Preparation

Adisti Dwijayanti
Roles: Resources, Software, Writing – Original Draft Preparation

Rima Maulida Hidayati
Roles: Project Administration, Resources, Supervision, Writing – Review & Editing

Tasya Claudia
Roles: Data Curation, Formal Analysis, Writing – Original Draft Preparation

Dewi Anggraeni Kusumoningrum
Roles: Conceptualization, Methodology, Software

Luthfi Saiful Arif
Roles: Conceptualization, Methodology, Software

Competing interests

No competing interests were disclosed.

Grant information

This work was supported by the Universitas Indonesia under the Publikasi Terindex Internasional (PUTI) Q2 Tahun 2022-2023 grant scheme [grant number: NKB-1217/UN2.RST/HKP.05.00/2022].
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (2)

version 2

Revised

Published: 17 Jul 2025, 14:169

https://doi.org/10.12688/f1000research.160939.2

version 1

Published: 06 Feb 2025, 14:169

https://doi.org/10.12688/f1000research.160939.1

© 2025 Firdaus R et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Firdaus R, Tantri AR, Atan RB et al. Adaptation and Validation of the Interprofessional Socialization and Valuing Scale (ISVS-24) for Utilization in Multiplayer Virtual Reality Environments [version 1; peer review: 2 approved with reservations]. F1000Research 2025, 14:169 (https://doi.org/10.12688/f1000research.160939.1)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 1

VERSION 1

PUBLISHED 06 Feb 2025

Views

Reviewer Report 06 Jun 2025

Danielle Rulli, The Ohio State University, Columbus, Ohio, USA

Approved with Reservations

https://doi.org/10.5256/f1000research.176903.r379678

Thank you for the opportunity to review this manuscript.

TITLE
Appropriately descriptive of the study and manuscript contents.
INTRODUCTION
There are a number of statements that need citation support. For example: "IPE seeks to impart clinical knowledge engagingly and dynamically. The benefits of IPE are clear to the students. When it comes to overcoming the limits of healthcare professionals, students who have been exposed to IPE in the curriculum have developed interprofessional partnerships throughout their practices, boosted team spirit, and fostered competencies."

This sentence needs clarification as the ISVS does indeed measure these things - "Although MPVR-based simulation is a rapidly developing simulation in medical education, there is not yet a suitable measuring instrument that fully measures interprofessional socialization outcomes, which captures changes in attitudes, actions, and beliefs of participants."

Needs citation - Effective interprofessional socialization is essential as it will enhance the utilization of each individual’s skills during collaborative practice, promote efficient patient care, and enhance patient safety.

More background in the use of MPVR-based simulation in health professions education is needed to provide the foundation for this study.

There needs to be a more robust relation of the significance of interprofessional socialization and identity to convey why this is relevant to the study. Additionally, further exploration of the ISVS (there are multiple versions) and comparator evaluations are needed to understand the selection for this study.
METHODS

ISVS-24 has already been revised,shortened, and validated by the original authors. Why was this older version chosen?

More clarity is needed on participants, as well as inclusion and exclusion criteria.

Clearer description of the intervention is needed, including how it was developed and piloted.

The authors state, "This study utilised virtual reality simulations that were played by three individuals."How were the three calibrated and how were participants oriented to using the the VR?

In the Phase 2 component, other than expertise in MPVR simulation-based education, was anyone involved that is an expert in IPE evaluation/assessment?

In Phase 3, it does not appear that all professions/students who would be involved were part of the pilot, making this component of validation questionable.

RESULTS
Regarding the content validity, why was question removal based on a handful of experts instead of being guided by a wider pilot assessment? A further explanation as to why these were removed should be in the methods in the Phase 3 section. This is assuming the final ISVS used did not include those 3 questions.

RESULTS

There is a lack of clarity on the factor analysis of individual questions to demonstrate validity.

The statements on the results of individual tests should be in the discussion. Only the results should be presented, not their interpretation.

The item reliability test data states the Cronbach alpha of 0.959 "indicates that the adapted version of ISVS items' [sic] were reliable." Please reassess this. Again, this statement should also be in the discussion, not the results.

DISCUSSION
The discussion needs to tie back to the literature in the introduction, and/or with the minimal introduction of additional literature, whether it agrees or differs with similar evidence.

Consider addressing additional limitations of the study.

CONCLUSION
Consider revising. In its current form it is somewhat reaching based on the limitations of the study design.

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Partly
Are sufficient details of methods and analysis provided to allow replication by others?

Partly
If applicable, is the statistical analysis and its interpretation appropriate?

Partly
Are all the source data underlying the results available to ensure full reproducibility?

Partly
Are the conclusions drawn adequately supported by the results?

Partly

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Interprofessional education, educational methodologies, medical/dental integration.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response 09 Aug 2025

Aida Tantri, Department of Anesthesiology and Intensive Care, Faculty of Medicine, Universitas Indonesia, Central Jakarta, 10430, Indonesia

09 Aug 2025

Author Response

We appreciate the reviewer thoughtful comments and suggestions. Please find our detailed responses below.

1. Introduction
a. We have revised the paragraph to improve clarity and accuracy, and to ensure ... Continue reading We appreciate the reviewer thoughtful comments and suggestions. Please find our detailed responses below.

1. Introduction
a. We have revised the paragraph to improve clarity and accuracy, and to ensure that each claim is appropriately supported by literature. Specifically, we rephrased the original sentences and placed the citation numbers directly after the relevant claims
b. We acknowledge that the ISVS is indeed a validated instrument that measures changes in attitudes, actions, and beliefs related to interprofessional socialization. To address this, we have revised the sentence to clarify that the gap lies not in the existence of such instruments, but in the lack of validation and adaptation of these tools for use specifically in immersive MPVR-based educational environments.
c. A reference has been added to support the role of effective interprofessional socialization in improving collaborative practice and patient safety.
d. We have expanded the background section to provide more context and examples of how MPVR has been applied in health professions education.
e. We have expanded our explanation regarding the significance of interprofessional socialization and identity formation in collaborative practice, particularly in virtual learning environments. Additionally, we elaborated on the rationale for selecting ISVS-24 over other versions and alternative instruments.

2. Methods
a. We chose to adapt the original ISVS-24, as the full and unrevised version developed by the original authors because it allows for more controlled and context-specific refinement. This ensures the instrument appropriately captures interprofessional socialisation within immersive VR settings. The explanation has been add to the method – instrument and introduction part.
b. We have added more clarity about participant
c. We have added a clearer description of the VR-LAST development process and included a brief explanation of its pilot testing by the research team. Please see the revised Methods section.
d. We have added details on how participants were oriented and briefed prior to the simulation. All participants attended a standard orientation session before each simulation that included a demonstration of the VR interface, headset usage, scenario overview, and simulation flow.
e. We acknowledge that while all panelists were highly experienced in simulation-based interprofessional education, none held formal training in IPE assessment. This is a limitation of this study and should be addressed in future validation efforts
f. We have clarified that Phase 3 involved participants from all professional groups engaged in the VR-LAST simulation, including anesthesiology residents, general physicians, and professional nurses. To avoid confusion with the earlier pilot testing of the VR intervention, we have also revised the heading of Phase 3 to better reflect the focus on instrument validation.

3. Results
a. We have clarified in the Methods – Phase 3 that three items were removed based on expert judgment in Phase 2 (content validation), and the final 21-item version was used for construct validation. No data were collected for the removed items during Phase 3
b. We have clarified the purpose and procedures of the exploratory factor analysis in the Results section to better demonstrate the construct validity of the instrument.
c. We have revised the Results section to present only the statistical outcomes and moved all interpretations to the Discussion.
d. We have corrected the grammatical error and moved the interpretation of the Cronbach’s alpha result to the Discussion section.

4. Discussion
We have revised the Discussion to clearly relate our findings back to key literature mentioned in the Introduction. Furthermore, we have expanded the Limitations section to address issues such as ecological validity and sample generalizability.

5. Conclusion
We have revised the conclusion to better reflect the study’s limitations.
We appreciate the reviewer thoughtful comments and suggestions. Please find our detailed responses below.

1. Introduction
a. We have revised the paragraph to improve clarity and accuracy, and to ensure that each claim is appropriately supported by literature. Specifically, we rephrased the original sentences and placed the citation numbers directly after the relevant claims
b. We acknowledge that the ISVS is indeed a validated instrument that measures changes in attitudes, actions, and beliefs related to interprofessional socialization. To address this, we have revised the sentence to clarify that the gap lies not in the existence of such instruments, but in the lack of validation and adaptation of these tools for use specifically in immersive MPVR-based educational environments.
c. A reference has been added to support the role of effective interprofessional socialization in improving collaborative practice and patient safety.
d. We have expanded the background section to provide more context and examples of how MPVR has been applied in health professions education.
e. We have expanded our explanation regarding the significance of interprofessional socialization and identity formation in collaborative practice, particularly in virtual learning environments. Additionally, we elaborated on the rationale for selecting ISVS-24 over other versions and alternative instruments.

2. Methods
a. We chose to adapt the original ISVS-24, as the full and unrevised version developed by the original authors because it allows for more controlled and context-specific refinement. This ensures the instrument appropriately captures interprofessional socialisation within immersive VR settings. The explanation has been add to the method – instrument and introduction part.
b. We have added more clarity about participant
c. We have added a clearer description of the VR-LAST development process and included a brief explanation of its pilot testing by the research team. Please see the revised Methods section.
d. We have added details on how participants were oriented and briefed prior to the simulation. All participants attended a standard orientation session before each simulation that included a demonstration of the VR interface, headset usage, scenario overview, and simulation flow.
e. We acknowledge that while all panelists were highly experienced in simulation-based interprofessional education, none held formal training in IPE assessment. This is a limitation of this study and should be addressed in future validation efforts
f. We have clarified that Phase 3 involved participants from all professional groups engaged in the VR-LAST simulation, including anesthesiology residents, general physicians, and professional nurses. To avoid confusion with the earlier pilot testing of the VR intervention, we have also revised the heading of Phase 3 to better reflect the focus on instrument validation.

3. Results
a. We have clarified in the Methods – Phase 3 that three items were removed based on expert judgment in Phase 2 (content validation), and the final 21-item version was used for construct validation. No data were collected for the removed items during Phase 3
b. We have clarified the purpose and procedures of the exploratory factor analysis in the Results section to better demonstrate the construct validity of the instrument.
c. We have revised the Results section to present only the statistical outcomes and moved all interpretations to the Discussion.
d. We have corrected the grammatical error and moved the interpretation of the Cronbach’s alpha result to the Discussion section.

4. Discussion
We have revised the Discussion to clearly relate our findings back to key literature mentioned in the Introduction. Furthermore, we have expanded the Limitations section to address issues such as ecological validity and sample generalizability.

5. Conclusion
We have revised the conclusion to better reflect the study’s limitations.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 09 Aug 2025

Aida Tantri, Department of Anesthesiology and Intensive Care, Faculty of Medicine, Universitas Indonesia, Central Jakarta, 10430, Indonesia

09 Aug 2025

Author Response

We appreciate the reviewer thoughtful comments and suggestions. Please find our detailed responses below.

1. Introduction
a. We have revised the paragraph to improve clarity and accuracy, and to ensure ... Continue reading We appreciate the reviewer thoughtful comments and suggestions. Please find our detailed responses below.

1. Introduction
a. We have revised the paragraph to improve clarity and accuracy, and to ensure that each claim is appropriately supported by literature. Specifically, we rephrased the original sentences and placed the citation numbers directly after the relevant claims
b. We acknowledge that the ISVS is indeed a validated instrument that measures changes in attitudes, actions, and beliefs related to interprofessional socialization. To address this, we have revised the sentence to clarify that the gap lies not in the existence of such instruments, but in the lack of validation and adaptation of these tools for use specifically in immersive MPVR-based educational environments.
c. A reference has been added to support the role of effective interprofessional socialization in improving collaborative practice and patient safety.
d. We have expanded the background section to provide more context and examples of how MPVR has been applied in health professions education.
e. We have expanded our explanation regarding the significance of interprofessional socialization and identity formation in collaborative practice, particularly in virtual learning environments. Additionally, we elaborated on the rationale for selecting ISVS-24 over other versions and alternative instruments.

2. Methods
a. We chose to adapt the original ISVS-24, as the full and unrevised version developed by the original authors because it allows for more controlled and context-specific refinement. This ensures the instrument appropriately captures interprofessional socialisation within immersive VR settings. The explanation has been add to the method – instrument and introduction part.
b. We have added more clarity about participant
c. We have added a clearer description of the VR-LAST development process and included a brief explanation of its pilot testing by the research team. Please see the revised Methods section.
d. We have added details on how participants were oriented and briefed prior to the simulation. All participants attended a standard orientation session before each simulation that included a demonstration of the VR interface, headset usage, scenario overview, and simulation flow.
e. We acknowledge that while all panelists were highly experienced in simulation-based interprofessional education, none held formal training in IPE assessment. This is a limitation of this study and should be addressed in future validation efforts
f. We have clarified that Phase 3 involved participants from all professional groups engaged in the VR-LAST simulation, including anesthesiology residents, general physicians, and professional nurses. To avoid confusion with the earlier pilot testing of the VR intervention, we have also revised the heading of Phase 3 to better reflect the focus on instrument validation.

3. Results
a. We have clarified in the Methods – Phase 3 that three items were removed based on expert judgment in Phase 2 (content validation), and the final 21-item version was used for construct validation. No data were collected for the removed items during Phase 3
b. We have clarified the purpose and procedures of the exploratory factor analysis in the Results section to better demonstrate the construct validity of the instrument.
c. We have revised the Results section to present only the statistical outcomes and moved all interpretations to the Discussion.
d. We have corrected the grammatical error and moved the interpretation of the Cronbach’s alpha result to the Discussion section.

4. Discussion
We have revised the Discussion to clearly relate our findings back to key literature mentioned in the Introduction. Furthermore, we have expanded the Limitations section to address issues such as ecological validity and sample generalizability.

5. Conclusion
We have revised the conclusion to better reflect the study’s limitations.
We appreciate the reviewer thoughtful comments and suggestions. Please find our detailed responses below.

1. Introduction
a. We have revised the paragraph to improve clarity and accuracy, and to ensure that each claim is appropriately supported by literature. Specifically, we rephrased the original sentences and placed the citation numbers directly after the relevant claims
b. We acknowledge that the ISVS is indeed a validated instrument that measures changes in attitudes, actions, and beliefs related to interprofessional socialization. To address this, we have revised the sentence to clarify that the gap lies not in the existence of such instruments, but in the lack of validation and adaptation of these tools for use specifically in immersive MPVR-based educational environments.
c. A reference has been added to support the role of effective interprofessional socialization in improving collaborative practice and patient safety.
d. We have expanded the background section to provide more context and examples of how MPVR has been applied in health professions education.
e. We have expanded our explanation regarding the significance of interprofessional socialization and identity formation in collaborative practice, particularly in virtual learning environments. Additionally, we elaborated on the rationale for selecting ISVS-24 over other versions and alternative instruments.

2. Methods
a. We chose to adapt the original ISVS-24, as the full and unrevised version developed by the original authors because it allows for more controlled and context-specific refinement. This ensures the instrument appropriately captures interprofessional socialisation within immersive VR settings. The explanation has been add to the method – instrument and introduction part.
b. We have added more clarity about participant
c. We have added a clearer description of the VR-LAST development process and included a brief explanation of its pilot testing by the research team. Please see the revised Methods section.
d. We have added details on how participants were oriented and briefed prior to the simulation. All participants attended a standard orientation session before each simulation that included a demonstration of the VR interface, headset usage, scenario overview, and simulation flow.
e. We acknowledge that while all panelists were highly experienced in simulation-based interprofessional education, none held formal training in IPE assessment. This is a limitation of this study and should be addressed in future validation efforts
f. We have clarified that Phase 3 involved participants from all professional groups engaged in the VR-LAST simulation, including anesthesiology residents, general physicians, and professional nurses. To avoid confusion with the earlier pilot testing of the VR intervention, we have also revised the heading of Phase 3 to better reflect the focus on instrument validation.

3. Results
a. We have clarified in the Methods – Phase 3 that three items were removed based on expert judgment in Phase 2 (content validation), and the final 21-item version was used for construct validation. No data were collected for the removed items during Phase 3
b. We have clarified the purpose and procedures of the exploratory factor analysis in the Results section to better demonstrate the construct validity of the instrument.
c. We have revised the Results section to present only the statistical outcomes and moved all interpretations to the Discussion.
d. We have corrected the grammatical error and moved the interpretation of the Cronbach’s alpha result to the Discussion section.

4. Discussion
We have revised the Discussion to clearly relate our findings back to key literature mentioned in the Introduction. Furthermore, we have expanded the Limitations section to address issues such as ecological validity and sample generalizability.

5. Conclusion
We have revised the conclusion to better reflect the study’s limitations.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 27 Feb 2025

Bau Dilam Ardyansyah, Medical Education, Universitas Hasanuddin, Makassar, South Sulawesi, Indonesia; Hasanuddin University Teaching Hospital, Hasanuddin University, Makassar, South Sulawesi, Indonesia; Curtin School of Allied Health, Curtin University, Perth, Western Australia, Australia

Approved with Reservations

https://doi.org/10.5256/f1000research.176903.r366123

Thank you for allowing me the opportunity to review this paper.

The study aims to adapt and validate an interprofessional outcome measure for assessing the Multiplayer Virtual Reality Environments. This manuscript has implemented several instrument development standards. ... Continue reading

Regarding the Statement:
These interactive virtual experiences have been shown to outperform traditional or other digital educational methods in terms of knowledge retention and skill development among health professionals.

To strengthen this claim, consider citing specific studies or providing empirical evidence demonstrating interactive virtual experiences' superiority over traditional or other digital learning methods. A well-supported reference would enhance the credibility of the argument.

Regarding the Instrument, ISVS-24:

For adaptation and validation studies in a new cultural and linguistic context, best practices recommend utilising existing validated instruments in the target language whenever possible. Several IPE outcome measures have already been adapted and validated in Indonesian, including RIPLS, CICS-29, CPAT, EPIS-RI, and even an Indonesian version of ISVS (ISVS-19), which has undergone thorough adaptation and validation.

This leads to an important question: Why not use one of these existing scales? What are their limitations? A thorough discussion of the shortcomings of these previously validated instruments is necessary to justify the adaptation of ISVS-24. Summarising the key findings from earlier outcome measure studies would provide a stronger contextual foundation for the research.

Additionally, ISVS-24 has been refined by its original developers into ISVS-21, which has improved psychometric properties. This raises a question: Why use ISVS-24 instead of ISVS-21? Clarifying what specific aspects of ISVS-24 make it more suitable for Multi-Participant Virtual Reality (MPVR)-based simulations compared to ISVS-21 would strengthen the rationale for this choice.
Since this study focuses on the adaptation and validation of an instrument from one language (English) to another (Indonesian), it is essential to discuss the psychometric properties of ISVS-24 to establish its reliability and validity within this new context.

Methods - Participants.

A total of 72 anesthesiology residents, general physicians, and nurses from the Faculty of Medicine, Universitas Indonesia.

Are the participants nurses or nursing students?
Inclusion criteria: no information was provided about how participants were recruited.
Since the ISVS-24 assesses interprofessional socialisation within the simulation, it is unclear whether participants were required to have prior experience working in interprofessional teams or, with or without previous exposure to VR training, were included.

Methods – Instruments.

This study utilised virtual reality simulations that were played by three individuals.

Who are these individuals? A clear layout of the participants and their roles in the simulation is needed.

Seeking permission from the original author (King et al., 2016) is highly recommended when adapting an instrument to a new context, especially if it involves translation, cultural adaptation, or modifications. Has the author obtained permission from the original author of the instrument to adapt it?

Methods – Procedure Phase 1: Translation of the ISVS-24.
The translation process of an instrument is crucial in adapting it to a new cultural context. Therefore, the translation methodology must be clearly explained.

Who conducted the translation? What were their qualifications to be designated as translators?
The translation process described in this article did not mention whether a back-translation was performed.
Briefly explain the dynamics of the translation process and how consensus was reached.

Methods – Phase 2 and Phase 3.
The methodology provides a structured approach to instrument adaptation, pilot testing, and validation. However, a clearer separation between pilot testing and validation is needed.

A panel of experts made up of 4 senior anaesthesiology consultants who were experts in MPVR simulation-based education.

This step is crucial to content validation for an interprofessional outcome measure.

Provide elaboration on the expertise of the reviewers and whether their backgrounds cover relevant fields of interprofessional education.
Why not include practitioners from diverse healthcare backgrounds, who are also the intended end users of the instrument, such as nurses?
Four reviewers are relatively small to be considered sufficient for a content validity study.
The target users of this instrument, such as anaesthesiology residents and nurses, were not involved in assessing content validity in phase 2 (should be part of a pilot study).
Phase 3 (should be part of the validation study?) used only 72 respondents, which is too small for robust psychometric analysis (factor analysis and reliability testing) of 21 items.

Results

The pilot study should primarily focus on the instrument’s items comprehensibility, feasibility, and usability. However, three items were removed solely based on the opinions of four-panel reviewers during phase 2. Why not consider removing the item based on the factor analysis? Please justify the items’ removal at this stage.
A more detailed description of the factor analysis method employed, including the type of factor analysis (e.g., exploratory or confirmatory), extraction method, rotation technique, and criteria for factor retention.
Many important indices for factor analysis results are not reported; for example, the item and factor loadings and how the factor loadings were interpreted (e.g., items retained if factor loading >0.40).
Provide the McDonald’s Omega to compliment the internal consistency reliability evaluation for each subscale (factor/domain) and the total for the scale.
A very high Cronbach alpha (> 0.95) is undesirable for a subscale or total scale.
The structural validity of the instrument, after being tested with 72 participants, was not adequately evaluated.

Discussions

It would be helpful to clarify the criteria used for removing the three items beyond just the VR setting.
Consider providing a comparison and contrast of relevant publications with the findings of this study.
The Discussion section needs improvement, as some of the narratives repeat the results presented in the Results section.

Conclusion

Consider tempering the statement by acknowledging the study's limitations, such as the small sample size, single-institution focus, and some methodology limitations.

References to instrument adaptation, standards, and validation processes need to be strengthened to ensure the incorporation of the best practices in the adaptation and validation process, including the latest statistical methods for validation.

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

No
Are sufficient details of methods and analysis provided to allow replication by others?

Partly
If applicable, is the statistical analysis and its interpretation appropriate?

Partly
Are all the source data underlying the results available to ensure full reproducibility?

Partly
Are the conclusions drawn adequately supported by the results?

Partly

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: The reviewer is a senior academic and medical doctor specializing in the development and implementation of interprofessional education and collaborative practice for health practitioners and students. Her expertise includes learning, teaching, and assessment in medical education, interprofessional education and collaborative practice, instrument development, and psychometric properties evaluation. She has led studies on adapting assessment tools for Indonesian and Australian contexts, including tuberculosis care through a Delphi study. Her research ensures culturally relevant and reliable instruments for evaluating interprofessional education and practice, enhancing collaboration among healthcare professionals and students.

CITE

Report a concern

Author Response 09 Aug 2025

Aida Tantri, Department of Anesthesiology and Intensive Care, Faculty of Medicine, Universitas Indonesia, Central Jakarta, 10430, Indonesia

09 Aug 2025

Author Response

We appreciate the reviewer thoughtful comments and suggestions. Please find our detailed responses below.

1. Introduction
In response, we have strengthened the justification for using virtual reality by adding ... Continue reading We appreciate the reviewer thoughtful comments and suggestions. Please find our detailed responses below.

1. Introduction
In response, we have strengthened the justification for using virtual reality by adding previous studies demonstrating VR’s superiority over traditional methods in improving knowledge retention and skill development.
We clarified the rationale for adapting the ISVS-24 from the original version rather than using translated instruments such as RIPLS, CPAT or ISVS-19. We also justified the selection of ISVS-24 over ISVS-21 by emphasizing the need for item-level adaptation from the source to preserve content relevance and theoretical integrity within immersive VR settings.
2. Methods - Participants
We have clarified that the participants were professional nurses, not students, and provided additional details on the inclusion criteria and recruitment process. We have clarified in the revised manuscript that participants were not required to have prior experience working in interprofessional teams or with VR training. However, their professional backgrounds ensured exposure to interprofessional collaboration in clinical settings. We also noted that previous VR experience was not a prerequisite, as the study aimed to assess interprofessional socialization within a simulated VR setting, regardless of participants’ prior VR exposure
3. Methods – Instruments
We have clarified the composition and roles of the participants in the VR simulation, specifying that each team consisted of an anesthesiology resident, a general physician, and a professional nurse assigned to interprofessional roles. We also confirm that permission to adapt and translate the ISVS-24 was obtained from the original author to use in this study.
4. Methods – Procedure Phase 1
We have clarified the qualifications of the translators, included the back-translation process, and elaborated on how consensus was reached during the translation phase to ensure cultural and contextual relevance of the adapted instrument.
5. Methods – Phase 2 and Phase 3
We have clarified the expertise of the content reviewers, acknowledged the absence of other health professions in the expert panel, and addressed this as a limitation. We also revised the manuscript to clearly distinguish between pilot testing and construct validation in Phase 3. Additionally, we acknowledged the limited sample size for factor analysis and recommended future studies to include larger and more diverse samples and expert panels.
6. Results
We have clarified that the three items were removed based on low Aiken’s V scores and expert consensus due to incompatibility with the MPVR context, and we acknowledged the limitation of not basing removal on factor analysis. We have also added a more detailed explanation of the factor analysis. While Cronbach’s alpha was used to assess internal consistency, we acknowledge in the discussion that the use of McDonald’s Omega would provide a more robust reliability estimate; however, this analysis was not conducted due to limitations in the statistical software used.
7. Discussions
We have clarified the rationale for removing three items. We have included a comparison of our findings with previous studies using ISVS or similar interprofessional education tools. We revised the Discussion section to focus more on interpretation and implications, and removed redundant restatements of the Results section to enhance clarity and purpose.
8. Conclusion
We have revised the conclusion to acknowledge the study’s limitations, including the small sample size, single-institution focus, and the need for further validation through broader future studies
We appreciate the reviewer thoughtful comments and suggestions. Please find our detailed responses below.

1. Introduction
In response, we have strengthened the justification for using virtual reality by adding previous studies demonstrating VR’s superiority over traditional methods in improving knowledge retention and skill development.
We clarified the rationale for adapting the ISVS-24 from the original version rather than using translated instruments such as RIPLS, CPAT or ISVS-19. We also justified the selection of ISVS-24 over ISVS-21 by emphasizing the need for item-level adaptation from the source to preserve content relevance and theoretical integrity within immersive VR settings.
2. Methods - Participants
We have clarified that the participants were professional nurses, not students, and provided additional details on the inclusion criteria and recruitment process. We have clarified in the revised manuscript that participants were not required to have prior experience working in interprofessional teams or with VR training. However, their professional backgrounds ensured exposure to interprofessional collaboration in clinical settings. We also noted that previous VR experience was not a prerequisite, as the study aimed to assess interprofessional socialization within a simulated VR setting, regardless of participants’ prior VR exposure
3. Methods – Instruments
We have clarified the composition and roles of the participants in the VR simulation, specifying that each team consisted of an anesthesiology resident, a general physician, and a professional nurse assigned to interprofessional roles. We also confirm that permission to adapt and translate the ISVS-24 was obtained from the original author to use in this study.
4. Methods – Procedure Phase 1
We have clarified the qualifications of the translators, included the back-translation process, and elaborated on how consensus was reached during the translation phase to ensure cultural and contextual relevance of the adapted instrument.
5. Methods – Phase 2 and Phase 3
We have clarified the expertise of the content reviewers, acknowledged the absence of other health professions in the expert panel, and addressed this as a limitation. We also revised the manuscript to clearly distinguish between pilot testing and construct validation in Phase 3. Additionally, we acknowledged the limited sample size for factor analysis and recommended future studies to include larger and more diverse samples and expert panels.
6. Results
We have clarified that the three items were removed based on low Aiken’s V scores and expert consensus due to incompatibility with the MPVR context, and we acknowledged the limitation of not basing removal on factor analysis. We have also added a more detailed explanation of the factor analysis. While Cronbach’s alpha was used to assess internal consistency, we acknowledge in the discussion that the use of McDonald’s Omega would provide a more robust reliability estimate; however, this analysis was not conducted due to limitations in the statistical software used.
7. Discussions
We have clarified the rationale for removing three items. We have included a comparison of our findings with previous studies using ISVS or similar interprofessional education tools. We revised the Discussion section to focus more on interpretation and implications, and removed redundant restatements of the Results section to enhance clarity and purpose.
8. Conclusion
We have revised the conclusion to acknowledge the study’s limitations, including the small sample size, single-institution focus, and the need for further validation through broader future studies
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 09 Aug 2025

Aida Tantri, Department of Anesthesiology and Intensive Care, Faculty of Medicine, Universitas Indonesia, Central Jakarta, 10430, Indonesia

09 Aug 2025

Author Response

We appreciate the reviewer thoughtful comments and suggestions. Please find our detailed responses below.

1. Introduction
In response, we have strengthened the justification for using virtual reality by adding ... Continue reading We appreciate the reviewer thoughtful comments and suggestions. Please find our detailed responses below.

1. Introduction
In response, we have strengthened the justification for using virtual reality by adding previous studies demonstrating VR’s superiority over traditional methods in improving knowledge retention and skill development.
We clarified the rationale for adapting the ISVS-24 from the original version rather than using translated instruments such as RIPLS, CPAT or ISVS-19. We also justified the selection of ISVS-24 over ISVS-21 by emphasizing the need for item-level adaptation from the source to preserve content relevance and theoretical integrity within immersive VR settings.
2. Methods - Participants
We have clarified that the participants were professional nurses, not students, and provided additional details on the inclusion criteria and recruitment process. We have clarified in the revised manuscript that participants were not required to have prior experience working in interprofessional teams or with VR training. However, their professional backgrounds ensured exposure to interprofessional collaboration in clinical settings. We also noted that previous VR experience was not a prerequisite, as the study aimed to assess interprofessional socialization within a simulated VR setting, regardless of participants’ prior VR exposure
3. Methods – Instruments
We have clarified the composition and roles of the participants in the VR simulation, specifying that each team consisted of an anesthesiology resident, a general physician, and a professional nurse assigned to interprofessional roles. We also confirm that permission to adapt and translate the ISVS-24 was obtained from the original author to use in this study.
4. Methods – Procedure Phase 1
We have clarified the qualifications of the translators, included the back-translation process, and elaborated on how consensus was reached during the translation phase to ensure cultural and contextual relevance of the adapted instrument.
5. Methods – Phase 2 and Phase 3
We have clarified the expertise of the content reviewers, acknowledged the absence of other health professions in the expert panel, and addressed this as a limitation. We also revised the manuscript to clearly distinguish between pilot testing and construct validation in Phase 3. Additionally, we acknowledged the limited sample size for factor analysis and recommended future studies to include larger and more diverse samples and expert panels.
6. Results
We have clarified that the three items were removed based on low Aiken’s V scores and expert consensus due to incompatibility with the MPVR context, and we acknowledged the limitation of not basing removal on factor analysis. We have also added a more detailed explanation of the factor analysis. While Cronbach’s alpha was used to assess internal consistency, we acknowledge in the discussion that the use of McDonald’s Omega would provide a more robust reliability estimate; however, this analysis was not conducted due to limitations in the statistical software used.
7. Discussions
We have clarified the rationale for removing three items. We have included a comparison of our findings with previous studies using ISVS or similar interprofessional education tools. We revised the Discussion section to focus more on interpretation and implications, and removed redundant restatements of the Results section to enhance clarity and purpose.
8. Conclusion
We have revised the conclusion to acknowledge the study’s limitations, including the small sample size, single-institution focus, and the need for further validation through broader future studies
We appreciate the reviewer thoughtful comments and suggestions. Please find our detailed responses below.

1. Introduction
In response, we have strengthened the justification for using virtual reality by adding previous studies demonstrating VR’s superiority over traditional methods in improving knowledge retention and skill development.
We clarified the rationale for adapting the ISVS-24 from the original version rather than using translated instruments such as RIPLS, CPAT or ISVS-19. We also justified the selection of ISVS-24 over ISVS-21 by emphasizing the need for item-level adaptation from the source to preserve content relevance and theoretical integrity within immersive VR settings.
2. Methods - Participants
We have clarified that the participants were professional nurses, not students, and provided additional details on the inclusion criteria and recruitment process. We have clarified in the revised manuscript that participants were not required to have prior experience working in interprofessional teams or with VR training. However, their professional backgrounds ensured exposure to interprofessional collaboration in clinical settings. We also noted that previous VR experience was not a prerequisite, as the study aimed to assess interprofessional socialization within a simulated VR setting, regardless of participants’ prior VR exposure
3. Methods – Instruments
We have clarified the composition and roles of the participants in the VR simulation, specifying that each team consisted of an anesthesiology resident, a general physician, and a professional nurse assigned to interprofessional roles. We also confirm that permission to adapt and translate the ISVS-24 was obtained from the original author to use in this study.
4. Methods – Procedure Phase 1
We have clarified the qualifications of the translators, included the back-translation process, and elaborated on how consensus was reached during the translation phase to ensure cultural and contextual relevance of the adapted instrument.
5. Methods – Phase 2 and Phase 3
We have clarified the expertise of the content reviewers, acknowledged the absence of other health professions in the expert panel, and addressed this as a limitation. We also revised the manuscript to clearly distinguish between pilot testing and construct validation in Phase 3. Additionally, we acknowledged the limited sample size for factor analysis and recommended future studies to include larger and more diverse samples and expert panels.
6. Results
We have clarified that the three items were removed based on low Aiken’s V scores and expert consensus due to incompatibility with the MPVR context, and we acknowledged the limitation of not basing removal on factor analysis. We have also added a more detailed explanation of the factor analysis. While Cronbach’s alpha was used to assess internal consistency, we acknowledge in the discussion that the use of McDonald’s Omega would provide a more robust reliability estimate; however, this analysis was not conducted due to limitations in the statistical software used.
7. Discussions
We have clarified the rationale for removing three items. We have included a comparison of our findings with previous studies using ISVS or similar interprofessional education tools. We revised the Discussion section to focus more on interpretation and implications, and removed redundant restatements of the Results section to enhance clarity and purpose.
8. Conclusion
We have revised the conclusion to acknowledge the study’s limitations, including the small sample size, single-institution focus, and the need for further validation through broader future studies
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 06 Feb 2025

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 2 (revision) 17 Jul 25	read
Version 1 06 Feb 25	read	read

Bau Dilam Ardyansyah, Universitas Hasanuddin, Makassar, Indonesia; Hasanuddin University, Makassar, Indonesia; Curtin University, Perth, Australia
Danielle Rulli, The Ohio State University, Columbus, USA

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

5 Views

02 Sep 2025 | for Version 2

5 Views Cite this report Responses(0)

Approved With Reservations

I appreciate the authors’ efforts to revise the manuscript in response to the previous comments. The overall clarity of the paper has improved, and several earlier concerns have been addressed. However, several important issues remain that should be considered further:

The current version of the Methods includes several statements describing the limitations of the study. The Methods should focus exclusively on what was done, in sufficient detail to allow replication.
A clearer justification is needed for key methodological decisions. The study involved factor analysis but included only 72 participants. Because sample size is a critical consideration in factor analysis, the rationale for this choice should be explicitly explained and supported by references or methodological guidelines.
The description of the translation procedure, particularly backward translation, remains insufficient. It is not clear whether backward translation was conducted, and Figure 1 does not show this step. A more detailed summary of the forward and backward translation processes would strengthen transparency. In particular, were there any items that required significant discussion to reach consensus, or did all 24 items proceed smoothly? How many items were reworded? Which items were they? Identification of problematic words, sentences, or phrases encountered during the translation would enrich the discussion of content validity
The description of factor analysis in the Methods and Results remains limited. It is not clear whether exploratory factor analysis (EFA) or confirmatory factor analysis (CFA) was conducted. The manuscript currently refers only to the use of principal component analysis. This distinction should be clarified and described in line with standard reporting practices.
There is no statement regarding the assessment of data normality.
Table 4 currently contains a large amount of information, much of which does not appear essential. It could be streamlined to present only the data directly relevant to determining the dimensional structure of the instrument. In addition, a table showing the factor loadings for each item, along with other results typically reported in factor analysis, would be more informative.
Although the eigenvalue criterion (>1) suggests three domains, the results in Table 4 more strongly support a unidimensional structure rather than a three-dimensional one. A more convincing and methodologically robust justification of the factorial structure could be achieved by triangulating with alternative approaches rather than relying solely on the Kaiser criterion.
The manuscript presents statements of limitation throughout the text, including in the Methods, Results, and Discussion sections. This is not conventional. Limitations should generally be reported only in the Discussion section, where they can be appropriately contextualised.
The Discussion could be further strengthened by comparing and contrasting the findings with previous ISVS-24 validation studies conducted in other languages and contexts. Highlighting similarities (e.g., factor structure, reliability, problematic items) and differences (e.g., cultural or contextual variations) would clarify how this study contributes to the broader evidence base. Thank you.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

The reviewer is a senior academic and medical doctor specializing in the development and implementation of interprofessional education and collaborative practice for health practitioners and students. Her expertise includes learning, teaching, and assessment in medical education, interprofessional education and collaborative practice, instrument development, and psychometric properties evaluation. She has led studies on adapting assessment tools for Indonesian and Australian contexts, including tuberculosis care through a Delphi study. Her research ensures culturally relevant and reliable instruments for evaluating interprofessional education and practice, enhancing collaboration among healthcare professionals and students.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

12 Views

06 Jun 2025 | for Version 1

Danielle Rulli, The Ohio State University, Columbus, Ohio, USA

12 Views Cite this report Responses(1)

Approved With Reservations

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

Partly
Are sufficient details of methods and analysis provided to allow replication by others?

Partly
If applicable, is the statistical analysis and its interpretation appropriate?

Partly
Are all the source data underlying the results available to ensure full reproducibility?

Partly
Are the conclusions drawn adequately supported by the results?

Partly

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Interprofessional education, educational methodologies, medical/dental integration.

Respond to this report

Responses (1)

Author Response

09 Aug 2025

Aida Tantri, Department of Anesthesiology and Intensive Care, Faculty of Medicine, Universitas Indonesia, Central Jakarta, 10430, Indonesia

We appreciate the reviewer thoughtful comments and suggestions. Please find our detailed responses below.

1. Introduction
a. We have revised the paragraph to improve clarity and accuracy, and to ensure that each claim is appropriately supported by literature. Specifically, we rephrased the original sentences and placed the citation numbers directly after the relevant claims
b. We acknowledge that the ISVS is indeed a validated instrument that measures changes in attitudes, actions, and beliefs related to interprofessional socialization. To address this, we have revised the sentence to clarify that the gap lies not in the existence of such instruments, but in the lack of validation and adaptation of these tools for use specifically in immersive MPVR-based educational environments.
c. A reference has been added to support the role of effective interprofessional socialization in improving collaborative practice and patient safety.
d. We have expanded the background section to provide more context and examples of how MPVR has been applied in health professions education.
e. We have expanded our explanation regarding the significance of interprofessional socialization and identity formation in collaborative practice, particularly in virtual learning environments. Additionally, we elaborated on the rationale for selecting ISVS-24 over other versions and alternative instruments.

2. Methods
a. We chose to adapt the original ISVS-24, as the full and unrevised version developed by the original authors because it allows for more controlled and context-specific refinement. This ensures the instrument appropriately captures interprofessional socialisation within immersive VR settings. The explanation has been add to the method – instrument and introduction part.
b. We have added more clarity about participant
c. We have added a clearer description of the VR-LAST development process and included a brief explanation of its pilot testing by the research team. Please see the revised Methods section.
d. We have added details on how participants were oriented and briefed prior to the simulation. All participants attended a standard orientation session before each simulation that included a demonstration of the VR interface, headset usage, scenario overview, and simulation flow.
e. We acknowledge that while all panelists were highly experienced in simulation-based interprofessional education, none held formal training in IPE assessment. This is a limitation of this study and should be addressed in future validation efforts
f. We have clarified that Phase 3 involved participants from all professional groups engaged in the VR-LAST simulation, including anesthesiology residents, general physicians, and professional nurses. To avoid confusion with the earlier pilot testing of the VR intervention, we have also revised the heading of Phase 3 to better reflect the focus on instrument validation.

3. Results
a. We have clarified in the Methods – Phase 3 that three items were removed based on expert judgment in Phase 2 (content validation), and the final 21-item version was used for construct validation. No data were collected for the removed items during Phase 3
b. We have clarified the purpose and procedures of the exploratory factor analysis in the Results section to better demonstrate the construct validity of the instrument.
c. We have revised the Results section to present only the statistical outcomes and moved all interpretations to the Discussion.
d. We have corrected the grammatical error and moved the interpretation of the Cronbach’s alpha result to the Discussion section.

4. Discussion
We have revised the Discussion to clearly relate our findings back to key literature mentioned in the Introduction. Furthermore, we have expanded the Limitations section to address issues such as ecological validity and sample generalizability.

5. Conclusion
We have revised the conclusion to better reflect the study’s limitations.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

27 Views

27 Feb 2025 | for Version 1

27 Views Cite this report Responses(1)

Approved With Reservations

Regarding the Statement:
These interactive virtual experiences have been shown to outperform traditional or other digital educational methods in terms of knowledge retention and skill development among health professionals.

Regarding the Instrument, ISVS-24:

For adaptation and validation studies in a new cultural and linguistic context, best practices recommend utilising existing validated instruments in the target language whenever possible. Several IPE outcome measures have already been adapted and validated in Indonesian, including RIPLS, CICS-29, CPAT, EPIS-RI, and even an Indonesian version of ISVS (ISVS-19), which has undergone thorough adaptation and validation.

Additionally, ISVS-24 has been refined by its original developers into ISVS-21, which has improved psychometric properties. This raises a question: Why use ISVS-24 instead of ISVS-21? Clarifying what specific aspects of ISVS-24 make it more suitable for Multi-Participant Virtual Reality (MPVR)-based simulations compared to ISVS-21 would strengthen the rationale for this choice.
Since this study focuses on the adaptation and validation of an instrument from one language (English) to another (Indonesian), it is essential to discuss the psychometric properties of ISVS-24 to establish its reliability and validity within this new context.

Methods - Participants.

A total of 72 anesthesiology residents, general physicians, and nurses from the Faculty of Medicine, Universitas Indonesia.

This study utilised virtual reality simulations that were played by three individuals.

Who are these individuals? A clear layout of the participants and their roles in the simulation is needed.

Seeking permission from the original author (King et al., 2016) is highly recommended when adapting an instrument to a new context, especially if it involves translation, cultural adaptation, or modifications. Has the author obtained permission from the original author of the instrument to adapt it?

Who conducted the translation? What were their qualifications to be designated as translators?
The translation process described in this article did not mention whether a back-translation was performed.
Briefly explain the dynamics of the translation process and how consensus was reached.

A panel of experts made up of 4 senior anaesthesiology consultants who were experts in MPVR simulation-based education.

This step is crucial to content validation for an interprofessional outcome measure.

Provide elaboration on the expertise of the reviewers and whether their backgrounds cover relevant fields of interprofessional education.
Why not include practitioners from diverse healthcare backgrounds, who are also the intended end users of the instrument, such as nurses?
Four reviewers are relatively small to be considered sufficient for a content validity study.
The target users of this instrument, such as anaesthesiology residents and nurses, were not involved in assessing content validity in phase 2 (should be part of a pilot study).
Phase 3 (should be part of the validation study?) used only 72 respondents, which is too small for robust psychometric analysis (factor analysis and reliability testing) of 21 items.

Results

The pilot study should primarily focus on the instrument’s items comprehensibility, feasibility, and usability. However, three items were removed solely based on the opinions of four-panel reviewers during phase 2. Why not consider removing the item based on the factor analysis? Please justify the items’ removal at this stage.
A more detailed description of the factor analysis method employed, including the type of factor analysis (e.g., exploratory or confirmatory), extraction method, rotation technique, and criteria for factor retention.
Many important indices for factor analysis results are not reported; for example, the item and factor loadings and how the factor loadings were interpreted (e.g., items retained if factor loading >0.40).
Provide the McDonald’s Omega to compliment the internal consistency reliability evaluation for each subscale (factor/domain) and the total for the scale.
A very high Cronbach alpha (> 0.95) is undesirable for a subscale or total scale.
The structural validity of the instrument, after being tested with 72 participants, was not adequately evaluated.

Discussions

It would be helpful to clarify the criteria used for removing the three items beyond just the VR setting.
Consider providing a comparison and contrast of relevant publications with the findings of this study.
The Discussion section needs improvement, as some of the narratives repeat the results presented in the Results section.

Conclusion

Consider tempering the statement by acknowledging the study's limitations, such as the small sample size, single-institution focus, and some methodology limitations.

Is the work clearly and accurately presented and does it cite the current literature?

Partly
Is the study design appropriate and is the work technically sound?

No
Are sufficient details of methods and analysis provided to allow replication by others?

Partly
If applicable, is the statistical analysis and its interpretation appropriate?

Partly
Are all the source data underlying the results available to ensure full reproducibility?

Partly
Are the conclusions drawn adequately supported by the results?

Partly

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Respond to this report

Responses (1)

Author Response

09 Aug 2025

Aida Tantri, Department of Anesthesiology and Intensive Care, Faculty of Medicine, Universitas Indonesia, Central Jakarta, 10430, Indonesia

We appreciate the reviewer thoughtful comments and suggestions. Please find our detailed responses below.

1. Introduction
In response, we have strengthened the justification for using virtual reality by adding previous studies demonstrating VR’s superiority over traditional methods in improving knowledge retention and skill development.
We clarified the rationale for adapting the ISVS-24 from the original version rather than using translated instruments such as RIPLS, CPAT or ISVS-19. We also justified the selection of ISVS-24 over ISVS-21 by emphasizing the need for item-level adaptation from the source to preserve content relevance and theoretical integrity within immersive VR settings.
2. Methods - Participants
We have clarified that the participants were professional nurses, not students, and provided additional details on the inclusion criteria and recruitment process. We have clarified in the revised manuscript that participants were not required to have prior experience working in interprofessional teams or with VR training. However, their professional backgrounds ensured exposure to interprofessional collaboration in clinical settings. We also noted that previous VR experience was not a prerequisite, as the study aimed to assess interprofessional socialization within a simulated VR setting, regardless of participants’ prior VR exposure
3. Methods – Instruments
We have clarified the composition and roles of the participants in the VR simulation, specifying that each team consisted of an anesthesiology resident, a general physician, and a professional nurse assigned to interprofessional roles. We also confirm that permission to adapt and translate the ISVS-24 was obtained from the original author to use in this study.
4. Methods – Procedure Phase 1
We have clarified the qualifications of the translators, included the back-translation process, and elaborated on how consensus was reached during the translation phase to ensure cultural and contextual relevance of the adapted instrument.
5. Methods – Phase 2 and Phase 3
We have clarified the expertise of the content reviewers, acknowledged the absence of other health professions in the expert panel, and addressed this as a limitation. We also revised the manuscript to clearly distinguish between pilot testing and construct validation in Phase 3. Additionally, we acknowledged the limited sample size for factor analysis and recommended future studies to include larger and more diverse samples and expert panels.
6. Results
We have clarified that the three items were removed based on low Aiken’s V scores and expert consensus due to incompatibility with the MPVR context, and we acknowledged the limitation of not basing removal on factor analysis. We have also added a more detailed explanation of the factor analysis. While Cronbach’s alpha was used to assess internal consistency, we acknowledge in the discussion that the use of McDonald’s Omega would provide a more robust reliability estimate; however, this analysis was not conducted due to limitations in the statistical software used.
7. Discussions
We have clarified the rationale for removing three items. We have included a comparison of our findings with previous studies using ISVS or similar interprofessional education tools. We revised the Discussion section to focus more on interpretation and implications, and removed redundant restatements of the Results section to enhance clarity and purpose.
8. Conclusion
We have revised the conclusion to acknowledge the study’s limitations, including the small sample size, single-institution focus, and the need for further validation through broader future studies

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. Wong AH, Ruppel H, Gang M, et al.: “The safety dance”: a faculty development workshop partnering ipe and patient safety initiatives using simulation-based education. MedEdPORTAL. 2016; 12: 10440. Publisher Full Text

[2] 2. Hur HK, Kim KK, Lim YM, et al.: Patient safety interprofessional education program using medical error scenarios for undergraduate nursing and medical students in korea. J. Interprof. Care. 2023; 37(6): 944–953. PubMed Abstract | Publisher Full Text

[3] 3. Cleary E, Bloomfield J, Frotjold A, et al.: A realist synthesis of interprofessional patient safety activities and healthcare student attitudes towards patient safety. J. Interprof. Care. 2024 Mar-Apr; 38(2): 346–376. PubMed Abstract | Publisher Full Text

[4] 4. Mektrirat R, Sathanawongs A, Tiwananthagorn S, et al.: Achieving interprofessional education on collaborative problem-solving for covid-19 using project-based approach. Int. J. Infect. Dis. 2022 Mar; 116: S67. Publisher Full Text

[5] 5. Khalafi A, Sarvi Sarmeydani N, Akhoondzadeh R: Simulation-based interprofessional education (ipe) for enhanced attitude and teamwork of anesthesiology residents and nurse anesthesia students in iran. J. Adv. Med. Educ. Prof. 2023 Apr; 11(2): 105–112. PubMed Abstract | Publisher Full Text

[6] 6. Becker V, Jedlicska N, Scheide L, et al.: Changes in medical students´ and anesthesia technician trainees´ attitudes towards interprofessionality - experience from an interprofessional simulation-based course. BMC Med. Educ. 2022 Apr; 22(1): 273. PubMed Abstract | Publisher Full Text | Free Full Text

[7] 7. Lewis C, Enriquez D, Calabrese L, et al.: Virtual reality multiplayer interaction and medical patient handoff training and assessment. Adv. Intell. Syst. Comput. 2022; 1421: 17–23. Publisher Full Text

[8] 8. Gan W, Mok TN, Chen J, et al.: Researching the application of virtual reality in medical education: one-year follow-up of a randomized trial. BMC Med. Educ. 2023 Jan 3; 23(1): 3. PubMed Abstract | Publisher Full Text | Free Full Text

[9] 9. Jiang H, Vimalesvaran S, Wang JK, et al.: Virtual reality in medical students’ education: scoping review. JMIR Med. Educ. 2022 Feb; 8(1): e34860. PubMed Abstract | Publisher Full Text | Free Full Text

[10] 10. Haowen J, Vimalesvaran S, Myint Kyaw B, et al.: Virtual reality in medical students’ education: a scoping review protocol. BMJ Open. 2021 May 26; 11(5): e046986. PubMed Abstract | Publisher Full Text | Free Full Text

[11] 11. Du J, Shi Y, Mei C, et al.: Communication by interaction: a multiplayer vr environment for building walkthroughs. Constr. Res. Congr. 2016; 2281–2290.

[12] 12. King G, Orchard C, Khalili H, et al.: Refinement of the interprofessional socialization and valuing scale (isvs-21) and development of 9-item equivalent versions. J. Contin. Educ. Heal. Prof. 2016 Summer; 36(3): 171–177. PubMed Abstract | Publisher Full Text

[13] 13. Mahler C, Orchard C, Berger S, et al.: Translation and psychometric properties of the german version of the “interprofessional socialization and valuing scale” (isvs-21-d). J. Interprof. Care. 2023; 37(4): 655–661. PubMed Abstract | Publisher Full Text

[14] 14. Wu Q, Wang Y, Lu L, et al.: Virtual simulation in undergraduate medical education: a scoping review of recent practice. Front. Med. (Lausanne). 2022 Mar; 9: 855403. PubMed Abstract | Publisher Full Text | Free Full Text

[15] 15. Bujang MA, Khee HY, Yee LK: A step-by-step guide to questionnaire validation research. Institute for clinical Research. 2022.

[16] 16. Shrotiya VK, Dhanda U: Content validity of Assessment Instrument for Employee Engagement. SAGE Open. 2019; 9: 1–7. Publisher Full Text

[17] 17. Aiken LR: Three coefficients for analyzing the reliability and validity of ratings. Educ. Psychol. Meas. 1985; 45(1): 131–142. Publisher Full Text

[18] 18. Retnawati H: Proving content validity of self-regulated learning scale (the comparison of aiken index and expanded gregiry index). REiD (Res. Eval. Educ). 2016; 2(2): 155–164. Publisher Full Text

[19] 19. Paranhos R, Figueiredo-Filho D, Rocha E, et al.: Unveiling the mysteries of pearson’s correlation coefficient: the return. Leviathan. 2014; 8: 66–95. Publisher Full Text

[20] 20. Taber KS: The use of cronbach’s alpha when developing and reporting research instruments in science education. Res. Sci. Educ. 2018; 48: 1273–1296. Publisher Full Text

[21] 21. Tavakol M, Dennick R: Making sense of Cronbach’s alpha. Int. J. Med. Educ. 2011 Jun; 2: 53–55. PubMed Abstract | Publisher Full Text | Free Full Text

[22] 22. Schultz KS, Whitney DJ: Measurement theory in action: case studies and exercises. Thousand Oaks, CA: Sage; 2005. 9781452224749.

[23] 23. McGrath JL, Taekman JM, Dev P, et al.: Using virtual reality simulation environments to assess competence for emergency medicine learners. Acad. Emerg. Med. 2018 Feb; 25(2): 186–195. PubMed Abstract | Publisher Full Text

[24] 24. Aithal A, Aithal PS: Development and validation of survey questionnaire & experimental data – a systematical review-based statistical approach. IJMTS. 2020; 5(2): 233–251. Publisher Full Text

[25] 25. González-Pascual JL, Márquez MG, Rodríguez-Rey R, et al.: Adaptation and validation of the interprofessional socialization and valuing scale (isvs) in spanish university health sciences students. J. Interprof. Care. 2022 Nov-Dec; 36(6): 916–922. Publisher Full Text

[26] 26. Firdaus R, Tantri AR, Atan RB, et al.: Adaptation and validation of the interprofessional socialization and valuing scale (isvs-24) for utilization in multiplayer virtual reality environments. Dataset. Zenodo. 2025. Publisher Full Text

Adaptation and Validation of the Interprofessional Socialization and Valuing Scale (ISVS-24) for Utilization in Multiplayer Virtual Reality Environments

Abstract

Background

Methods

Results

Conclusions

Keywords

Introduction

Methods

Study design

Figure 1. Translation and validation process.

Figure 2. Viewpoints of different users.

Participants

Development of VR-Local Anesthesia of Systemic Toxicity

Instruments

Procedure

Data analysis

Ethical considerations and consent to participate

Results

Content validity

Table 1. Analysis table of Aiken's coefficient calculation results.

Participant characteristics

Table 2. Demographic characteristics of subjects.

Consistency validity

Table 3. Pearson’s correlation coefficients for measuring the validity of the questionnaire’s internal consistency.

The Kaiser-Meyer-Olkin test

Table 4. Total variance explained.

Item reliability test

Table 5. Cronbach’s alpha.

Discussion

Conclusion

Data availability

Underlying data

Extended data

Reporting guidelines

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated