Amendments from Version 1

F1000Research

2046-1402

F1000 Research Limited

London, UK

10.12688/f1000research.72900.2

Method Article

Articles

Immersive virtual classroom as an education tool for color barrier-free presentations: a pilot study

[version 2; peer review: 3 not approved]

Fukuyama

Sayaka

Conceptualization Data Curation Formal Analysis Software Writing – Original Draft Preparation 1 Saito

Toki

Software Supervision Visualization 1 Ichikawa

Daisuke

Supervision Validation 1 Kohyama

Ayako

Investigation Supervision Writing – Original Draft Preparation 1 Oyama

Hiroshi

Conceptualization Funding Acquisition Methodology Project Administration Resources Validation Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0001-7457-1904 a 1 1Department of Clinical Information Engineering, School of Public Health, Graduate School of Medicine, University of Tokyo, 7-3-1, Hongo, Bunkyo-Ku, Tokyo, 113-0033, Japan

a hoyama@m.u-tokyo.ac.jp

No competing interests were disclosed.

20 1 2022

2021

985

14 1 2022

2022

This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Background: This study proposes an experiential approach for understanding colour vision variation using virtual reality technology.

Methods: The study design was adapted from phase 1 clinical trials. A virtual classroom was developed in a three-dimensional space, and 10 healthy university students were tested to help them understand colour vision variation.

Results: No participant interrupted the experience due to virtual reality motion sickness. Most participants noted that the virtual classroom was an excellent educational tool, which could help teachers understand the problems associated with [visual analog scale (VAS): mean ± standard deviation (SD), 9.55 ± 1.57] and obtain a better understanding of (VAS: mean ± SD, 9.04 ± 1.0) color vision variations.

Conclusions: We conducted a pilot study on the impact of immersive virtual classroom experiences as an educational tool for creating presentations free of colour vision barriers. This approach may help participants respond appropriately to children who suffer from this disorder.

color vision variation experiential education tool virtual reality

The author(s) declared that no grants were involved in supporting this work.

Revised Amendments from Version 1

This paper is a pilot study that proposes a methodology for experiential education on color vision variations. Following the advice of the reviewers, additional revisions were made. 1) An overview diagram of the methodology has been added. 2) We added that the research design was based on Phase 1 of the clinical trial. In the case of Phase 1 of a medical device or software application, exploratory studies (feasibility studies) are conducted on a small number of samples in the early stages of development. It is used to preliminarily establish the safety and efficacy of the device or application and to design the next phase of testing. 3) We added that the sample size was calculated based on the incidence of adverse events in existing papers. 4) Added the following reason for using VAS as the evaluation method: VAS has the advantage of measuring minute differences in respondents' impressions and measuring information that is difficult to relativize or quantify. We also considered using the Likert scale but did not use it because it has the disadvantage that different respondents have different reasons for their choices. 5) In the study's limitations, we added the need for future validation of this system using the Simulcheck method.

Introduction

Colour vision variation affects approximately 6–10% of males and 0.4–0.7% of females, although most people do not experience significant problems. ¹ ^, ² The human eye contains three types of cones: S (short wavelength absorbing cones) which help us see blue, M (medium) for green, and L (long) for red. An absence or deficiency of these accounts for different types of colour vision variation. ³ This variation is classified according to disorder or by the lack of cone cells, where there may be protan (protanopia), deutan (deuteranopia), and tritan (tritanopia) deficiencies. ² ^, ⁴

Patients with colour vision variation often have problems in daily life, including school life, admission to schools, and obtaining a job. ⁵ There is currently no effective treatment for this disorder. It is, therefore, necessary to consider how to use colours based on universal designs; this approach involves products or environments that are perceptible to patients with any colour vision variation. ⁵

In Japan, colour vision tests during primary school medical examinations were abolished in 2002. However, before the termination of such tests, some studies have shown that approximately 70% of primary or junior high schoolteachers were unaware of colour vision variation. Approximately 80% knew that colour vision variation could be detected during a medical examination using a colour vision test. Moreover, approximately 90% of teachers were unfamiliar with the Teaching Guidelines for Problems with Colour Vision. ⁶ Thus, many teachers lacked knowledge and an understanding of colour vision variation. After the termination of these tests, the Japanese Teaching Guidelines for Colour Vision ⁷ were published to help teachers better understand colour vision variation. In addition, the Color Universal Design Organization appraises and approves textbooks for the universal design of colours. ⁸

However, it can be difficult for teachers to become aware of students who have colour vision variation, and most teachers have not used the Japanese Teaching Guidelines for Colour Vision. ⁷ Such variation can cause problems for students, for example, they may be reprimanded by teachers who have limited knowledge of the disorder.

In 2014, the Ministry of Education, Culture, Sports, Science, and Technology in Japan instituted the Partial Revision of Ordinance for Enforcement of School Health and Safety Act for medical examinations, to help teachers learn more about colour vision variation and better assist students with such variation in learning and career guidance. However, this Act did not achieve the former aim.

There are some supporting tools for individuals with colour vision variation that use universal colour designs to help them recognise colours. ⁹ ^, ¹⁰ For example, tools have been developed to help affected individuals perceive the colours in a design from a two-dimensional picture or on a website. However, these are not designed to help educate teachers about colour vision variation; such tools include a three-dimensional (3D) space to be walked through that does not use virtual reality (VR). ¹¹

Virtual environments with a 3D space can assist learning. ¹² ^– ¹⁴ However, few studies have applied this process to teaching about the problems of students with colour vision variation. The present study developed and tested a virtual experiential learning approach for understanding colour vision variation via a simulator using VR technology in a primary school classroom.

Methods Study design

The study design was adapted from a phase 1 clinical trial, ¹⁵ which involves testing a drug on a small number of healthy people. This phase is used to determine the appropriate dosage, human response to the drug, and possible side effects. Similarly, in the case of medical devices and software applications, exploratory studies (also known as feasibility studies) are conducted in the early stages of development. These are used to preliminarily establish the safety and efficacy of the device or application and to design the next phase of testing ( Figure 1). ¹⁶

Figure 1. Methodological overview of the pilot study.

VR was used to simulate and communicate the problems of students who have colour vision variation. A primary school classroom and its teaching materials were constructed and projected in a VR space. Because approximately 70% of patients with colour vision variation are deutan-deficient, this system simulated both that and normal trichromatism so that these types of colour vision could be compared.

The teaching materials constructed in the classroom are common in primary school classrooms; some were designed based on the Japanese Teaching Guidelines for Colour Vision, which considers content about colours that are difficult for students with colour vision variation to recognise or distinguish.

Previous studies have used a head-mounted display (HMD) for experience-based, simulation-enhanced learning, ¹⁷ ^– ¹⁹ so we also used one in our system. In addition, an analogue stick was adapted as an operating device to enable users to operate and control their viewpoint manually and intuitively.

Experimental equipment

An iMac ME089J/A computer (Apple, Cupertino, CA, USA) was used as hardware for the development of the execution environment, with Windows 7 Professional (Microsoft, Redmond, WA, USA) as the operating system. Oculus Rift DK2 (Oculus VR, Irvine, CA, USA) was used as the HMD and an Xbox360 controller for Windows (Microsoft) as the analogue stick for controlling the viewpoint. Unity3D (Unity Technologies, San Francisco, CA, USA), an integrated software development environment, was used to construct the VR space with C# as the development language. “Japanese classroom set” (SbbUtutuya), a Unity software asset, was used as the 3D model for the virtual classroom and teaching materials.

Based on existing guidelines for colour vision variation, seven parameters were chosen, designed, and addressed in the virtual classroom as contents that are difficult for students with colour vision variation to recognise or distinguish: the colours of chalk, a calendar, flowers, paints, a red pen, figures or graphs, and the coding used in maps ( Figure 2). ⁷

Figure 2. Seven parameters were selected, designed, and addressed in the classroom.

“Colour Blindness Simulator for Unity” (Gulti, Tokyo, Japan), a Unity software asset, was used as a colour vision simulator; it was developed based on the theory of colour vision simulation by Brettel et al., ⁹ ^, ¹⁰ and was verified and approved by the Color Universal Design Organization. ²⁰ In this study, the deuteranope mode was used to simulate a deutan deficiency. We also included the dichromatism and trichromatism modes; the former was applied to the simulation intensity parameter, which was maximised, and the latter involved the state when the simulator was not simulating colour vision variation.

Usability and utility tests

Objective

We tested to evaluate the usability and utility of the system for educational purposes.

Experimental set-up and tasks

The sample size was calculated as the number of cases required for the mean value of adverse events within a certain margin of error. As the incidence of adverse events in VR systems ranges from 30% ²¹ to 80%, ²² ^, ²³ standard deviation of 15%, confidence level of 90% and error of 10% were used to calculate the sample size of eight cases.

The participants, who did not have colour vision variation, were recruited by opportunity sampling. The test took approximately 30 min and was performed in the authors’ study room with a single participant and a test navigator. The participants were seated when using the system ( Figure 3).

Figure 3. Participants follow the instructions of the navigator and experience the differences between normal and abnormal colour vision while moving through a virtual classroom.

Before the test started, the objectives of the test were explained, and the participant completed a pretest questionnaire. Then the participant received additional explanations regarding the outlines of the system and items in the virtual classroom that must be watched and given instructions on how to operate the controller. Then the participant was connected to the HMD to start the test; the HMD was set up based on the participant’s height.

First, the participants experienced the deuteranope mode. During this experience, the navigator in charge asked seven questions about how the participant saw colours. The participant answered the questions orally while operating their viewpoint. Second, the subject experienced the trichromat mode and answered the same questions. Finally, the subject completed a questionnaire about usability and utility (10 cm VAS) and finished the test.

VAS was used as the evaluation method in this study; this is a scale that allows respondents to freely indicate their answers for the evaluation results as lengths on a group of continuous lines. It is crucial that the respondents answer honestly, therefore, motivating the respondents is a major advantage of the VAS method. This method also has the advantage of being able to measure minute differences in respondents’ impressions, which makes it possible to relativise and quantify types of information for which this is difficult. We also considered using the Likert scale, but did not do so because it has the disadvantage that different respondents have different reasons for their choices. ²⁴

There were four items in the questionnaire: ease of operation with the controller, immersion with the HMD, clearness of the display, and VR sickness. There were two questions about whether the participant learned about problems with colour vision and whether the system promoted a better understanding of colour vision variation.

The test was consistent with the Ethical Guidelines for Medical and Health Research Involving Human Subjects (Ministry of Education, Culture, Sports, Science, and Technology, Ministry of Health, Labour, and Welfare, 2014) and was performed after informed consent was obtained from the participants. The questionnaire was completed anonymously and was self-administered. Personal information was treated in accordance with the Act on the Protection of Personal Information and the information security policy of the University of Tokyo, Tokyo, Japan. Ethics approval for this study protocol was obtained from the Research Ethics Committee of the University of Tokyo (1139).

Data analysis

R (R Development Core Team) was the statistical software used in this study. We calculated the average, standard deviation (SD), and 95% confidence interval of the VAS scores after the participants’ experiences.

Results

The participants were 10 university students (two males and eight females) at the Graduate School of Medicine, University of Tokyo; they were 21–47 years of age, with an average of 26.6 years and an SD of 7.3 years. ²⁵

No participant interrupted the experiment due to VR sickness.

All participants answered that they were familiar with the term “colour vision variation,” but only four knew situations when students with colour vision variation had difficulties. One participant answered that she had previously used colour vision variation simulation tool to check the colouring of her website.

Table 1 shows the results of the questionnaire regarding utility and usability.

Table 1. The results of the questionnaire on usability and utility (VAS, n = 10).

	Mean	SD	95% CI
	Mean	SD	Lower	Upper
Usefulness
1) Whether the participants could learn the locations of colour vision problems?	9.6	0.6	9.2	9.9
2) Whether the system promoted a better understanding of colour vision variation?	9.0	1.0	8.4	9.7
Usability
1) Controllability of the controller	7.3	1.7	6.3	8.4
2) Immersion by the head mount display	8.4	1.6	7.4	9.4
3) Is the virtual world clear?	5.8	2.2	4.4	7.1
4) The degree of VR sickness	6.6	2.5	5.0	8.2

Regarding utility, whether they learned examples of colour vision problems was rated as 9.6 ± 0.6 (VAS mean ± SD) and whether the system promoted a better understanding of colour vision variation was rated as 9.0 ± 1.0.

Regarding usability, the ease of operation was 7.3 ± 1.7, immersion with the HMD was 8.4 ± 1.6, clarity of the display was 5.8 ± 2.2, and VR sickness was 6.6 ± 2.5.

Some remarks were given in the free description field, including “to use things whose contents we can understand only by colour should be avoided,” “if the colours are similar, even if their tints are different, some people could not tell them apart,” and “we should be careful of how we display graphs: how to use colours, designs, or patterns.”

Discussion

We propose a virtual experiential learning approach using VR educational tools with the goal of improving schoolteachers’ understanding of colour vision variation.

Approximately 20% of the teachers in a previous study stated that they became aware of problems with the colours of chalk. ²⁰ The results of this study show that a few participants had not noticed difficulties in students with colour vision variation, although they knew this term. In addition, few participants had used colour simulation tools. Although the participants were students, there seemed to be minimal interest in or awareness of the problems associated with colour vision variation.

This approach can allow participants to simulate colour vision variation in a school classroom in a VR space. The contents that are likely to cause miscommunication associated with colour blindness are placed in the virtual classroom and voice guidance about them implemented. The HMD, which increased the sense of immersion, was adopted as the method for experiencing the application. The participants can experience the situations in which children feel troubled in the classroom and deal with them in an environment similar to reality.

We evaluated the method’s safety, usability, and utility using a phase 1 clinical trial in participants without colour vision variation.

Most participants experienced VR sickness. One did so during rotation movements with the controller. However, no participant interrupted their experience due to sickness. These experiences are consistent with many studies that have reported that visual rotational motion can induce motion sickness. ²⁶ ^– ²⁸ Operating the unfamiliar controller might have caused VR sickness. The user interface should therefore be improved, and the ability to rotate the controller should be restricted.

In the evaluation of system utility, the average score was 9.6 for the question “How well do you understand which items are difficult for children with colour vision variation to see or distinguish?” and the confidence interval was small. The other question item, whether the system promoted a better understanding of colour vision variation, also received an average score of 9.0. These high evaluation results indicate that the experiencers were presented with the world of the virtual classroom in both two- and three-colour modes, so that participants could experience the differences in colour by alternating between the two modes. Furthermore, while with the participants simulate being students with different colour variation, we only used the problematic points shown in Figure 2; the participants were operating the system while asking questions, which may have made it easier for them to focus on the problematic targets in the virtual classroom. A participant suggested that additional educational effects could be achieved by organizing and expanding the content.

The usability evaluation results show that the ease of operation with the controller was rated as 7.3 (±1.7). In this system, a video game controller was used as the operating device; therefore, whether the participants had experience of such a controller influenced the results. In addition, the movement speed of the viewpoint during rotation was set to slow to avoid VR sickness, which may have worsened the usability evaluation.

The average immersion of the HMD was rated as 8.4, suggesting that the participants received a high degree of immersion because their actual surroundings were eliminated while wearing the HMD and the display followed the motion of the participant’s head.

Regarding the clearness of the display, the average (5.8) was lower than that for the other parameters, and the confidence interval was large. Thus, we assume that the experience of wearing the HMD differed among the participants. Colour noise was sometimes seen in the display because the HMD tilted due to head movement or looseness of the headband. In addition, a participant stated that the HMD display resolution was low, which worsened immersion. The HMD resolution should therefore be improved.

From the results of the free-response question on whether one’s understanding of colour vision variation had increased, the reasons given for improved understanding were not only that the participants answered questions while comparing the three- and two-colour vision modes but also that the navigator explained to the participants the specific things to think about during the experience. The system could be used to develop better graphs for PowerPoint presentations, not just by schoolteachers and staff but also by students and other occupational workers.

There were limitations to this study. First, the participants were students although the system was developed for teachers. Second, the evaluations were subjective, and the teaching efficacy could not be measured quantitatively. The VAS method needs to be validated with existing scales to be used as a subjective evaluation method by users of VR systems. The Simulcheck method may be needed to evaluate this tool. ²⁹

Conclusions

We proposed a virtual experiential learning approach that allows participants to experience and demonstrate the characteristics of colour vision in children with colour vision variation. A pilot study was conducted on the impact of immersive virtual classroom experiences. With this system, schoolteachers will be able to increase their knowledge of colour vision variation and solve colour vision problems in the classroom. In the future, it is necessary to evaluate the effectiveness of this approach for new teachers.

Data availability Underlying data

OSF: Immersive virtual classroom as an education tool for color barrier-free presentations: A pilot study data. https://doi.org/10.17605/OSF.IO/3KJVR. ¹⁸

This project contains the following underlying data: •

manuscript72900RawData.xlsx (raw data)

Data are available under the terms of the Creative Commons Zero “No rights reserved” data waiver (CC0 1.0 Public domain dedication).

Extended data

OSF: Immersive virtual classroom as an education tool for color barrier-free presentations: A pilot study data. https://doi.org/10.17605/OSF.IO/3KJVR. ¹⁸

This project contains the following extended data: •

manuscript72900_Extended_data.pdf (blank copy of the questionnaire)

Data are available under the terms of the Creative Commons Zero “No rights reserved” data waiver (CC0 1.0 Public domain dedication).

Acknowledgments

The authors are grateful to Drs. T. Sakamoto and S. Ino for useful discussions. We also thank the students at the University of Tokyo who participated in the evaluation.

References 1

Birch

: Diagnosis of Defective Colour Vision. Oxford University Press;1993.

Snowden

Thompson

Troscianko

: Color vision. Basic vision: an introduction to visual perception. Revised ed. Oxford University Press;2012;132–171.

Stockman

Macleod

DIA

Johnson

: Spectral Sensitivities of the Human Cones. J. Opt. Soc. Am. A: Opt. Image Sci. Vis. Dec 1993;10(12):2491–2521. 10.1364/Josaa.10.002491

Keene

: A review of color blindness for microscopists: guidelines and tools for accommodating and coping with color vision deficiency. Microsc. Microanal. Apr 2015;21(2):279–289. 10.1017/S1431927615000173

Cole

: The handicap of abnormal colour vision. Clin. Exp. Optom. Jul 2004;87(4-5):258–275. 15312030

10.1111/j.1444-0938.2004.tb05056.x

Teaching Guidelines for Problems with Color Vision (in Japanese). Ministry of Education, Culture, Sports, Science and Technology;1989.

Japanese Teaching Guidelines for Color Vision (in Japanese). Ministry of Education Culture Sports Science and Technology;2003;4–12.

Okabe

Ito

: Color Universal Design (CUD) - How to make figures and presentations that are friendly to Colorblind people.Accessed 08, 11, 2021. Reference Source

Vienot

Brettel

Ott

: What do colour-blind people see?. Nature. Jul 13 1995;376(6536):127–128. 10.1038/376127a0

Brettel

Vienot

Mollon

: Computerized simulation of color appearance for dichromats. J. Opt. Soc. Am. A Opt. Image Sci. Vis. Oct 1997;14(10):2647–2655. 9316278

10.1364/JOSAA.14.002647

Kubota

Seki

Kano

: Development and Application of Color Universal Design Support System for Predestrian Space. IPSJ Journal. 2011;52(1):140–152.

Mills

Araujo

MMT

de : Learning through virtual reality: a preliminary investigation. Interact. Comput. Apr 1999;11(4):453–462. 10.1016/S0953-5438(98)00061-7

Panagiotis

Konstaninos

Stylianos

: E-Learning through Virtual Reality Applications: The Case of Career Counseling. Int. J. Technol. Learn. 2014;20:57–68.

Shen

C-w

J-t

Kuo

T-C

: Behavioral Intention of Using Virtual Reality in Learning. ACM. 2017:129–137.

Naeem

Syed

Xiang

: Development, Testing and Reporting of Mobile Apps for Psycho-social Interventions: Lessons from the Pharmaceuticals. J. Med. Diagn. Met. 2015;4(4). 10.4172/2168-9784.1000.191

Pietzsch

Shluzas

Pate-Cornell

: Stage-Gate Process for the Development of Medical Devices. J. Med. Devices Trans. ASME. Jun 2009 2009;3(2): ARTN 021004. 10.1115/1.3148836

Freina

Ott

: A Literature Review on Immersive Virtual Reality in Education: State of the Art and Perspectives. Rethinking Education by Leveraging the Elearning Pillar of the Digital Agenda for Europe!. 2015;I:133–141. 10.12753/2066-026x-15-020

Moro

Stromberga

Raikos

: The Effectiveness of Virtual and Augmented Reality in Health Sciences and Medical Anatomy. Anat. Sci. Educ. Nov-Dec 2017;10(6):549–559. 28419750

10.1002/ase.1696

Peden

Mercer

Tatham

: The use of head-mounted display eyeglasses for teaching surgical skills: A prospective randomised study. Int. J. Surg. Oct 2016;34:169–173. 27613125

10.1016/j.ijsu.2016.09.002

CUDO: What is Color Universal Design?.Accessed 08-31, 2020. Reference Source

Chen

RHY

: Visually Induced Motion Sickness: Effects of Translational Visual Motion Along Different Axes. Contemp. Ergon. Hum. Factor. 2011;2011:281–287.

Kim

: Characteristic changes in the physiological components of cybersickness. Psychophysiology. Sep 2005;42(5):616–625. 16176385

10.1111/j.1469-8986.2005.00349.x

Rebenitsch

Owen

: Review on cybersickness in applications and visual displays. Virtual Real. Jun 2016;20(2):101–125. 10.1007/s10055-016-0285-9

Voutilainen

Pitkaaho

Kvist

: How to ask about patient satisfaction? The visual analogue scale is less vulnerable to confounding factors and ceiling effect than a symmetric Likert scale. Article. J. Adv. Nurs. Apr 2016;72(4):946–957. 26689434

10.1111/jan.12875

Oyama

: Data from: Virtual Classroom as an Education Tool for Color Barrier-Free Presentations; A Pilot Study Data. September 15. 2021.

Golding

Gresty

: Motion sickness. Curr. Opin. Neurol. Feb 2005;18(1):29–34. 10.1097/00019052-200502000-00007

Golding

: Motion sickness susceptibility. Auton. Neurosci. Basic Clin. Oct 30 2006;129(1-2):67–76. 10.1016/j.autneu.2006.07.019

Palmisano

Mursic

Kim

: Vection and cybersickness generated by head-and-display motion in the Oculus Rift. Displays. Jan 2017;46:1–8. 10.1016/j.displa.2016.11.001

Lillo

Alvaro

Moreira

: An experimental method for the assessment of color simulation tools. J. Vis. 2014;14(8) Artn 15. 25052695

10.1167/14.8.15

10.5256/f1000research.80814.r120447

Reviewer response for version 2

Higuera-Trujillo

Juan Luis

1 Referee https://orcid.org/0000-0003-1870-2388 1Institute for Research and Innovation in Bioengineering (i3B), Universitat Politècnica de València, València, Spain

Competing interests: No competing interests were disclosed.

14 2 2022

2022

This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

recommendation

reject

I keep acknowledging the merit of your research. However, the review becomes difficult to follow. The proper way to respond to a review is issue by issue. Instead, what appears is a paragraph indicating the intention to make modifications (but no specific comments or quotes from the modified text). I understand that this may be an omission on the part of the authors, perhaps to make my reading process easier, because it is illogical for a 629-word review to coincide with the authors' 121-word response. However, a issue by issue review would be more helpful. Nevertheless, while I can appreciate the work that the authors may have done to complete the paper, I think that the paper still has some of its most critical previous limitations: imprecision in defining objectives, and (apparently) not employing methods aligned with the objectives. Please include the answer to each question (one by one), so that I can help you improve the paper.

Is the rationale for developing the new method (or application) clearly explained?

Partly

Is the description of the method technically sound?

Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Partly

If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Yes

Are sufficient details provided to allow replication of the method development and its use by others?

Reviewer Expertise:

Virtual reality; neuroarchitecture

I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.

Oyama

Hiroshi

University of Tokyo, Japan

Competing interests: No competing interests were disclosed.

17 2 2022

Dear Dr. Juan Luis Higuera-Trujillo,

I thought we had revised the paper based on your valuable advice, but I apologize for the lack of explanation about the content of the revision.

I will check the points you pointed out again and revise the paper.

Regards,

Oyama

Hiroshi

University of Tokyo, Japan

Competing interests: No competing interests were disclosed.

18 2 2022

Dear Dr. Juan Luis Higuera-Trujillo,

Thank you very much for your valuable comments. We believe that first-person experience using V R technology can create more empathy for the subject and deepen understanding. In this study, we are conducting research and development as a complement to existing educational methods for teachers to take appropriate measures to prevent students with colorblindness from being bullied in elementary and junior high schools. This research does not claim superiority over existing educational methods but complements them. In particular, we believe that it will be possible to educate, as needed, a large number of new teachers who have difficulty implementing existing methods. This study is positioned as a feasibility study before applying it to new teachers. The revised paper will be submitted again.　

Kind regards,

10.5256/f1000research.80814.r120446

Reviewer response for version 2

Skelton

Alice

1 Referee https://orcid.org/0000-0002-1496-9715 1The Sussex Colour Group, School of Psychology, University of Sussex, Brighton, UK

Competing interests: No competing interests were disclosed.

9 2 2022

2022

recommendation

reject

Thank you for taking the time to attempt to address comments from reviewers.

Unfortunately I think the problems with this paper likely mean it will not be accepted in any form. The method itself of showing people a simulation of CVD and seeing if they report a better understanding of living with CVD is not robust and does not appear meaningful for practitioners. If this is an area the authors are interested in I would encourage them to think more critically about how to best assess this method in a second study, and to treat this first study either as experiment 1 or as purely exploratory data (and make it available on e.g. OSF).

Is the rationale for developing the new method (or application) clearly explained?

Is the description of the method technically sound?

Partly

Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Yes

Are sufficient details provided to allow replication of the method development and its use by others?

Yes

Reviewer Expertise:

Colour vision, cognitive development

Oyama

Hiroshi

University of Tokyo, Japan

Competing interests: No competing interests were disclosed.

17 2 2022

Dear Dr. Alice Skelton,

Thank you for your comments.

I thought we had revised the paper based on your valuable advice, but I apologize for the lack of explanation about the content of the revision.

I will check the points you pointed out again and revise the paper.

Regards,

Oyama

Hiroshi

University of Tokyo, Japan

Competing interests: No competing interests were disclosed.

18 2 2022

Dear Dr. Alice Skelton,

Kind regards,

10.5256/f1000research.80814.r120445

Reviewer response for version 2

Chan

Teresa M

1 Referee https://orcid.org/0000-0001-6104-462X 1Continuing Professional Development Office and McMaster Education Research, Innovation, and Theory (MERIT) Program, McMaster University, Hamilton, ON, Canada

Competing interests: No competing interests were disclosed.

24 1 2022

2022

recommendation

reject

I am sorry, now the revisions are making this paper worse off. I would point out that applying a clinical trial design for this study doesn't make any sense and you should change that to explain a bit more the rationale for your approach instead of just simply stating that you modelled this off of a clinical trial.

Clinical trials are notoriously atheoretical and this is the source of much critique of these trials - and I do not feel that they are an appropriate model for this type of intervention.

Please return to your educational roots and justify this study from that vantage point. Be more specific and clear as to what you were trying to achieve.

Is the rationale for developing the new method (or application) clearly explained?

Yes

Is the description of the method technically sound?

Yes

Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Partly

If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Partly

Are sufficient details provided to allow replication of the method development and its use by others?

Partly

Reviewer Expertise:

Medical education; Health professions education; Digital learning

Oyama

Hiroshi

University of Tokyo, Japan

Competing interests: No competing interests were disclosed.

17 2 2022

Dear Dr.Teresa M. Chan,

I thought we had revised the paper based on your valuable advice, but I apologize for the lack of explanation about the content of the revision.

I will check the points you pointed out again and revise the paper.

Regards,

Oyama

Hiroshi

University of Tokyo, Japan

Competing interests: No competing interests were disclosed.

18 2 2022

Dear Dr. Teresa M. Chan,

Kind regards,

10.5256/f1000research.76511.r98822

Reviewer response for version 1

Higuera-Trujillo

Juan Luis

1 Referee https://orcid.org/0000-0003-1870-2388 1Institute for Research and Innovation in Bioengineering (i3B), Universitat Politècnica de València, València, Spain

Competing interests: No competing interests were disclosed.

1 12 2021

2021

recommendation

reject

The study contains the seeds of an interesting and very socially useful approach. In that sense, I should congratulate the authors. However, it has the major problem of presenting a very low sample size and, more critically, not fully aligning the objective with the methods employed. I understand that it is a pilot study. But a "pilot study" is not a concept that legitimises lack of depth in the paper. Such studies should try to express the methodology as justifiably and clearly as possible. However, the article (in its current version) fails to express this adequately. Other limitations of the article are a weak introduction, and discussion section. In the following review, I try to give my view on how the authors could improve the article.

Introduction

The introduction is easy to follow and well structured. However, it contains the following weaknesses:

The first paragraph is illustrative. But it would require further explanation, as it introduces concepts such as "dichromatism" with which readers may not be familiar.

A scholarly introduction should state the problem and put it in context with related literature. In this introduction the problem is stated, but it lacks academic rigour. It is a relatively short introduction, citing only 10 references. It reads more like a professional report than an academic introduction. I suggest the authors improve the scientific scope of this text.

Methods

The comment about "Godot" doesn't really make sense. If it was not used, it should not appear.

The choice of the seven parameters (the colour of chalk, etc.) should be justified.

The general experimental methodology is explained textually, but it would be a significant support to elaborate a figure indicating the whole process. The aim of this section is to allow the replicability of the study; and in its current version this would not be possible.

The questionnaires used by the authors should be listed in a table in this section. It is not enough to present the questions together with the results in the results section. The methods section should be self-explanatory.

I understand that the questionnaires used are partly based on other questionnaires. They should be referenced.

The description of the sample should not be included in the results section, but in the methods section. On the other hand, the following aspects of the sample are critical: (1) very low number of participants; (2) very different ages, especially considering the low number of participants; and (3) unbalanced gender.

Results

Question 4 of the usability questionnaire could be confusing: do high scores refer to a high degree of sickness? I understand that it meant the opposite, and that the VAS corrects this problem. But for future studies I would recommend to the authors that, if they develop their own questionnaires, all questions should be equally oriented. For example, in this question, it would have been better: "4) The degree to which you are free of VR sickness", or something similar.

It is confusing how it can be said "We have developed a VR system that allows children with colour vision mutations to experience their colour vision and demonstrate its properties", when the whole sequence has been aimed at testing the tool with university students without such problems.

Discussion

An academic discussion tries to put the results in context with the available literature. However, in this case it is essentially limited to explaining the results. The commentary on the results should be in the Results section, not in the Discussion one. The discussion should attempt to contextualise them. This does not occur in the current version of the study (or it occurs very briefly: only in the last and antepenultimate paragraphs of the discussion). I encourage authors to improve this section in order to follow the structure of a strong research article.

Is the rationale for developing the new method (or application) clearly explained?

Partly

Is the description of the method technically sound?

Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Partly

If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Yes

Are sufficient details provided to allow replication of the method development and its use by others?

Reviewer Expertise:

Virtual reality; neuroarchitecture

Oyama

Hiroshi

University of Tokyo, Japan

Competing interests: No competing interests were disclosed.

3 12 2021

Dear Dr. Juan Luis Higuera-Trujillo,

We thank you for carefully reading our manuscript and for giving valuable comments.

The ultimate goal of this research is to propose a first-person experiential education method using VR to help participants (especially new teachers) respond appropriately to children with color vision variations at the school. This paper is a report on the first stage of realizing this goal. It is positioned as a Phase 1 study of a clinical trial. Therefore, we mainly examined whether there would be any problems for healthy people and whether there would be any usability problems during the experience. The next step will be to examine the impact on new teachers.

We will revise the manuscript based on your feedback.

Sincerely

Hiroshi

10.5256/f1000research.76511.r99517

Reviewer response for version 1

Chan

Teresa M

Competing interests: No competing interests were disclosed.

10 11 2021

2021

recommendation

reject

Thank you for inviting me to complete this peer review. Overall I see the merit in this paper and feel that it is worthy of indexing.

My suggested edits are to help you shift this article from a surface reporting of what you did towards a more scholarly report of the techniques used.

Major concerns:

Why did you chose a visual analogue scale for this study? There is very little in the literature around the selection of this particular VAS scale for the purposes of your study. More substantiation (and/or more details about pilot testing) of your measurement tool would be imperative prior to indexing. What is the interrater or intrarater reliability of the VAS?

I am perplexed about the study design - there seems to be only 1 arm of individuals. The layout of the introduction led me to believe you might compare groups of individuals with colour vision variations to those who are not.

Please add your sample size calculation for this study. It is unclear why you are even reporting standard deviations or 95% CI since there is no comparator group. Please more thoroughly discuss this in the Data Analysis section as currently they do not appear beyond the table. I would suggest it is important to highlight these in the text and also discuss them in your discussion.

Finally, your utility and usability framework are unclear to me. What prior literature informed your survey tool? Please cite supporting references that helped you design your study.

Is the rationale for developing the new method (or application) clearly explained?

Yes

Is the description of the method technically sound?

Yes

Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

Partly

If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Partly

Are sufficient details provided to allow replication of the method development and its use by others?

Partly

Reviewer Expertise:

Medical education; Health professions education; Digital learning

Oyama

Hiroshi

University of Tokyo, Japan

Competing interests: No competing interests were disclosed.

15 11 2021

Dear Dr. Teresa M. Chan,

We thank you for your fruitful suggestions, especially for suggesting a better study design and valuable comments.

(1) Why did you choose a visual analog scale for this study?

Res) Although there have been reports of VAS being used in usability evaluations of surgical VR systems, there have been no reports of evaluation scales measuring the impact of VR systems such as this one. We also considered using the Likert scale but did not use it because it has the disadvantage that different respondents give different reasons for their choices. The VAS is a scale that allows respondents to freely answer their rating results in a continuous band, the length of which is on a line. It is vital to motivate respondents to answer honestly. Therefore, motivating respondents to answer is a significant advantage of the VAS method, so we used it this time. In addition, the VAS method has the advantage of measuring minute differences in respondents' impressions and being able to relativize and quantify information that is difficult to relativize and quantify. Therefore, we believe that the VAS method can accurately measure users' perceptions of the VR system. As you suggested, it is necessary to evaluate the validity of this scale. We will add the verification of the validity of this scale in the limitation section of this study.

Ref) Jokinen, E., Mikkola, T. S., & Härkki, P. (2020). Simulator training and residents' first laparoscopic hysterectomy: a randomized controlled trial. Surgical endoscopy, 34(11), 4874-4882. DOI: 10.1007/s00464-019-07270-3

(2) I am perplexed about the study design - there seems to be only one arm of individuals.

Res) This VR system is intended to be an educational tool to assist teachers in making appropriate responses to children with color vision variations. As such, this study corresponds to Phase I of the clinical trial. As you suggested, the efficacy study (Phase II) will be conducted on new teachers in elementary and middle schools.

(3) Please add your sample size calculation for this study. …

Res) This study is a Phase I clinical trial in advancing the clinical application of VR systems. The incidence of adverse events in VR systems has ranged from 30% (Chen et al., 2011) to over 80 % (Kim et al., 2005). The median value of the VAS scale is 5. If it was a normal distribution, the sample size was calculated using a two-tailed t-test of the mean values of one group with a significance level of α = 5% and a power of 80% with a mean difference of 1.5 and a standard deviation of 1. The calculation resulted in a sample size of 8.06.

Since this is a Phase I study, standard deviations and confidence intervals are shown to determine the characteristics of the population. We will add this information to the data analysis and discussion as you suggested.

We think that a sample size of 30 or more is appropriate. We will add the sample size to the limit.

Ref) Chen, W., Chen, J. Z., & So, R. H. Y. (2011). Visually induced motion sickness: Effects of translational visual motion along different axes. Contemporary ergonomics and human factors, 2011, 281-287.

Kim YY, Kim HJ, Kim EN, Ko HD, Kim HT (2005) Characteristic Changes in the Physiological Components of Cybersickness. Psychophysiology 42(5):616–662. DOI: 10.1111/j.1469-8986.2005.00349.x

Rebenitsch, L., & Owen, C. (2016). Review on cybersickness in applications and visual displays. Virtual Reality, 20(2), 101-125. DOI: 10.1007/s10055-016-0285-9

(4) Finally, your utility and usability framework are unclear to me. ….

Res) The evaluation framework and survey tools for this study were based on the following paper. We will add the following study in the Method section as a reference.

Pallavicini, F., Pepe, A., & Minissi, M. E. (2019). Gaming in virtual reality: What changes in terms of usability, emotional response and sense of presence compared to non-immersive video games? Simulation & Gaming, 50(2), 136-159. DOI: 10.1177/1046878119831420

10.5256/f1000research.76511.r98213

Reviewer response for version 1

Skelton

Alice

1 Referee https://orcid.org/0000-0002-1496-9715 1The Sussex Colour Group, School of Psychology, University of Sussex, Brighton, UK

Competing interests: No competing interests were disclosed.

3 11 2021

2021

recommendation

reject

The method suggested is using a VR classroom setting and applying the built in filters for modelling different types of colour vision deficiency (CVD) so that people without CVD can better understand the perceptual experience of people with CVD. 10 participants wear the VR headset and view a classroom environment under different examples of CVD before being asked to self-report if they have learnt about what people with CVD experience & if they think it’s useful. They are also asked some basic questions about the usability of the set up. The majority of participants do report learning about CVD. It appears that showing people what colour vision deficiency might be like makes the majority of participants understand a bit better about what colour vision deficiency is like.

There are a lot of inaccuracies in the introduction in defining what variations in colour vision are: e.g. a reference to 'trichromatism' as being a colour vision variation affecting 6-10% of males, trichromacy is actually typical colour vision; "It is difficult for patients with long or middle wave sensitive cones to distinguish red-green colors" is also incorrect - having L and M cones means you are able to discriminate reds and greens. Many textbooks with a section on colour vision would be a better place to start to find out about the physiology of colour vision (e.g. a nice accessible version is: Snowden et al. 2012 ¹).

The method of demonstrating the perceptual experience of people with colour vision deficiencies (CVD) in order to help teachers and educators better understand the experience of children in their classrooms is a good idea. It is a little hard to see what this particular method offers over what can already be gained from CVD simulators however, and this method is not compared to any other method of educating people on the experience of living with CVD, so it's hard to know how effective it is in comparison to these other methods which would likely be more accessible in terms of equipment for the majority of educators.

It’s important that interventions to teach people about individual differences are teaching people accurate information. The paper does not report how accurate the simulated versions of the classrooms are in matching the perceptual experience of people with CVD. There has been rigorous testing of the accuracy of simulations (e.g. Lillo et al. 2014 ²) which shows there is some variation in how accurate these simulations are.

Most CVD simulators do not use calibrated displays which can most accurately capture the perceptual experience of a student with CVD, and that technical aspect is also missing from the current method.

Is the rationale for developing the new method (or application) clearly explained?

Is the description of the method technically sound?

Partly

Are the conclusions about the method and its performance adequately supported by the findings presented in the article?

If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

Yes

Are sufficient details provided to allow replication of the method development and its use by others?

Yes

Reviewer Expertise:

Colour vision, cognitive development

References 1

: Basic vision: an introduction to visual perception. Oxford University Press, Oxford .2012;

: An experimental method for the assessment of color simulation tools. J Vis .2014;14(8) : 10.1167/14.8.15 15

25052695

10.1167/14.8.15

Oyama

Hiroshi

University of Tokyo, Japan

Competing interests: No competing interests were disclosed.

6 11 2021

Dear Dr. Alice Skelton,

We thank you for careful reading our manuscript and for giving useful comments.

This research aims to propose a methodology on CVD using a simulation tool based on VR technology rather than a comparative study with existing color simulations. It is based on Situated Learning Theory to support education about CVD using VR technology.

>There are a lot of inaccuracies in the Introduction....

-> We will check our references and revise the first paragraph of the Introduction.

>The method of demonstrating the perceptual experience of people with colour vision ....

->The reason for choosing medical students as subjects were to reduce knowledge bias about CVD in the evaluation.

>It’s important that interventions to teach people about individual differences....

->The "Color vision variation Simulator for Unity" used in this project is a simulation tool certified by CUDO (Color Universal Design Organization: https://www2.cudo.jp/ wp/). The HMD resolution is 1,920 x 1,080, which enables full high definition-like display. However, following your suggestion, we will add "The Simulcheck method will be needed to be evaluated." in the limitation paragraph.

Sincerely,