Immersive virtual classroom as an education tool for color barrier-free presentations: a pilot study

Introduction

Colour vision variation affects approximately 6–10% of males and 0.4–0.7% of females, although most people do not experience significant problems.^1,2 The human eye contains three types of cones: S (short wavelength absorbing cones) which help us see blue, M (medium) for green, and L (long) for red. An absence or deficiency of these accounts for different types of colour vision variation.³ This variation is classified according to disorder or by the lack of cone cells, where there may be protan (protanopia), deutan (deuteranopia), and tritan (tritanopia) deficiencies.^2,4

Patients with colour vision variation often have problems in daily life, including school life, admission to schools, and obtaining a job.⁵ There is currently no effective treatment for this disorder. It is, therefore, necessary to consider how to use colours based on universal designs; this approach involves products or environments that are perceptible to patients with any colour vision variation.⁵

In Japan, colour vision tests during primary school medical examinations were abolished in 2002. However, before the termination of such tests, some studies have shown that approximately 70% of primary or junior high schoolteachers were unaware of colour vision variation. Approximately 80% knew that colour vision variation could be detected during a medical examination using a colour vision test. Moreover, approximately 90% of teachers were unfamiliar with the Teaching Guidelines for Problems with Colour Vision.⁶ Thus, many teachers lacked knowledge and an understanding of colour vision variation. After the termination of these tests, the Japanese Teaching Guidelines for Colour Vision⁷ were published to help teachers better understand colour vision variation. In addition, the Color Universal Design Organization appraises and approves textbooks for the universal design of colours.⁸

However, it can be difficult for teachers to become aware of students who have colour vision variation, and most teachers have not used the Japanese Teaching Guidelines for Colour Vision.⁷ Such variation can cause problems for students, for example, they may be reprimanded by teachers who have limited knowledge of the disorder.

In 2014, the Ministry of Education, Culture, Sports, Science, and Technology in Japan instituted the Partial Revision of Ordinance for Enforcement of School Health and Safety Act for medical examinations, to help teachers learn more about colour vision variation and better assist students with such variation in learning and career guidance. However, this Act did not achieve the former aim.

There are some supporting tools for individuals with colour vision variation that use universal colour designs to help them recognise colours.^9,10 For example, tools have been developed to help affected individuals perceive the colours in a design from a two-dimensional picture or on a website. However, these are not designed to help educate teachers about colour vision variation; such tools include a three-dimensional (3D) space to be walked through that does not use virtual reality (VR).¹¹

Virtual environments with a 3D space can assist learning.^12–14 However, few studies have applied this process to teaching about the problems of students with colour vision variation. The present study developed and tested a virtual experiential learning approach for understanding colour vision variation via a simulator using VR technology in a primary school classroom.

Methods

Study design

The study design was adapted from a phase 1 clinical trial,¹⁵ which involves testing a drug on a small number of healthy people. This phase is used to determine the appropriate dosage, human response to the drug, and possible side effects. Similarly, in the case of medical devices and software applications, exploratory studies (also known as feasibility studies) are conducted in the early stages of development. These are used to preliminarily establish the safety and efficacy of the device or application and to design the next phase of testing (Figure 1).¹⁶

Figure 1. Methodological overview of the pilot study.

VR was used to simulate and communicate the problems of students who have colour vision variation. A primary school classroom and its teaching materials were constructed and projected in a VR space. Because approximately 70% of patients with colour vision variation are deutan-deficient, this system simulated both that and normal trichromatism so that these types of colour vision could be compared.

The teaching materials constructed in the classroom are common in primary school classrooms; some were designed based on the Japanese Teaching Guidelines for Colour Vision, which considers content about colours that are difficult for students with colour vision variation to recognise or distinguish.

Previous studies have used a head-mounted display (HMD) for experience-based, simulation-enhanced learning,^17–19 so we also used one in our system. In addition, an analogue stick was adapted as an operating device to enable users to operate and control their viewpoint manually and intuitively.

Experimental equipment

An iMac ME089J/A computer (Apple, Cupertino, CA, USA) was used as hardware for the development of the execution environment, with Windows 7 Professional (Microsoft, Redmond, WA, USA) as the operating system. Oculus Rift DK2 (Oculus VR, Irvine, CA, USA) was used as the HMD and an Xbox360 controller for Windows (Microsoft) as the analogue stick for controlling the viewpoint. Unity3D (Unity Technologies, San Francisco, CA, USA), an integrated software development environment, was used to construct the VR space with C# as the development language. “Japanese classroom set” (SbbUtutuya), a Unity software asset, was used as the 3D model for the virtual classroom and teaching materials.

Based on existing guidelines for colour vision variation, seven parameters were chosen, designed, and addressed in the virtual classroom as contents that are difficult for students with colour vision variation to recognise or distinguish: the colours of chalk, a calendar, flowers, paints, a red pen, figures or graphs, and the coding used in maps (Figure 2).⁷

Figure 2. Seven parameters were selected, designed, and addressed in the classroom.

“Colour Blindness Simulator for Unity” (Gulti, Tokyo, Japan), a Unity software asset, was used as a colour vision simulator; it was developed based on the theory of colour vision simulation by Brettel et al.,^9,10 and was verified and approved by the Color Universal Design Organization.²⁰ In this study, the deuteranope mode was used to simulate a deutan deficiency. We also included the dichromatism and trichromatism modes; the former was applied to the simulation intensity parameter, which was maximised, and the latter involved the state when the simulator was not simulating colour vision variation.

Usability and utility tests

Objective

We tested to evaluate the usability and utility of the system for educational purposes.

Experimental set-up and tasks

The sample size was calculated as the number of cases required for the mean value of adverse events within a certain margin of error. As the incidence of adverse events in VR systems ranges from 30%²¹ to 80%,^22,23 standard deviation of 15%, confidence level of 90% and error of 10% were used to calculate the sample size of eight cases.

The participants, who did not have colour vision variation, were recruited by opportunity sampling. The test took approximately 30 min and was performed in the authors’ study room with a single participant and a test navigator. The participants were seated when using the system (Figure 3).

Figure 3. Participants follow the instructions of the navigator and experience the differences between normal and abnormal colour vision while moving through a virtual classroom.

Before the test started, the objectives of the test were explained, and the participant completed a pretest questionnaire. Then the participant received additional explanations regarding the outlines of the system and items in the virtual classroom that must be watched and given instructions on how to operate the controller. Then the participant was connected to the HMD to start the test; the HMD was set up based on the participant’s height.

First, the participants experienced the deuteranope mode. During this experience, the navigator in charge asked seven questions about how the participant saw colours. The participant answered the questions orally while operating their viewpoint. Second, the subject experienced the trichromat mode and answered the same questions. Finally, the subject completed a questionnaire about usability and utility (10 cm VAS) and finished the test.

VAS was used as the evaluation method in this study; this is a scale that allows respondents to freely indicate their answers for the evaluation results as lengths on a group of continuous lines. It is crucial that the respondents answer honestly, therefore, motivating the respondents is a major advantage of the VAS method. This method also has the advantage of being able to measure minute differences in respondents’ impressions, which makes it possible to relativise and quantify types of information for which this is difficult. We also considered using the Likert scale, but did not do so because it has the disadvantage that different respondents have different reasons for their choices.²⁴

There were four items in the questionnaire: ease of operation with the controller, immersion with the HMD, clearness of the display, and VR sickness. There were two questions about whether the participant learned about problems with colour vision and whether the system promoted a better understanding of colour vision variation.

The test was consistent with the Ethical Guidelines for Medical and Health Research Involving Human Subjects (Ministry of Education, Culture, Sports, Science, and Technology, Ministry of Health, Labour, and Welfare, 2014) and was performed after informed consent was obtained from the participants. The questionnaire was completed anonymously and was self-administered. Personal information was treated in accordance with the Act on the Protection of Personal Information and the information security policy of the University of Tokyo, Tokyo, Japan. Ethics approval for this study protocol was obtained from the Research Ethics Committee of the University of Tokyo (1139).

Results

The participants were 10 university students (two males and eight females) at the Graduate School of Medicine, University of Tokyo; they were 21–47 years of age, with an average of 26.6 years and an SD of 7.3 years.²⁵

No participant interrupted the experiment due to VR sickness.

All participants answered that they were familiar with the term “colour vision variation,” but only four knew situations when students with colour vision variation had difficulties. One participant answered that she had previously used colour vision variation simulation tool to check the colouring of her website.

Table 1 shows the results of the questionnaire regarding utility and usability.

Regarding utility, whether they learned examples of colour vision problems was rated as 9.6 ± 0.6 (VAS mean ± SD) and whether the system promoted a better understanding of colour vision variation was rated as 9.0 ± 1.0.

Regarding usability, the ease of operation was 7.3 ± 1.7, immersion with the HMD was 8.4 ± 1.6, clarity of the display was 5.8 ± 2.2, and VR sickness was 6.6 ± 2.5.

Some remarks were given in the free description field, including “to use things whose contents we can understand only by colour should be avoided,” “if the colours are similar, even if their tints are different, some people could not tell them apart,” and “we should be careful of how we display graphs: how to use colours, designs, or patterns.”

Discussion

We propose a virtual experiential learning approach using VR educational tools with the goal of improving schoolteachers’ understanding of colour vision variation.

Approximately 20% of the teachers in a previous study stated that they became aware of problems with the colours of chalk.²⁰ The results of this study show that a few participants had not noticed difficulties in students with colour vision variation, although they knew this term. In addition, few participants had used colour simulation tools. Although the participants were students, there seemed to be minimal interest in or awareness of the problems associated with colour vision variation.

This approach can allow participants to simulate colour vision variation in a school classroom in a VR space. The contents that are likely to cause miscommunication associated with colour blindness are placed in the virtual classroom and voice guidance about them implemented. The HMD, which increased the sense of immersion, was adopted as the method for experiencing the application. The participants can experience the situations in which children feel troubled in the classroom and deal with them in an environment similar to reality.

We evaluated the method’s safety, usability, and utility using a phase 1 clinical trial in participants without colour vision variation.

Most participants experienced VR sickness. One did so during rotation movements with the controller. However, no participant interrupted their experience due to sickness. These experiences are consistent with many studies that have reported that visual rotational motion can induce motion sickness.^26–28 Operating the unfamiliar controller might have caused VR sickness. The user interface should therefore be improved, and the ability to rotate the controller should be restricted.

In the evaluation of system utility, the average score was 9.6 for the question “How well do you understand which items are difficult for children with colour vision variation to see or distinguish?” and the confidence interval was small. The other question item, whether the system promoted a better understanding of colour vision variation, also received an average score of 9.0. These high evaluation results indicate that the experiencers were presented with the world of the virtual classroom in both two- and three-colour modes, so that participants could experience the differences in colour by alternating between the two modes. Furthermore, while with the participants simulate being students with different colour variation, we only used the problematic points shown in Figure 2; the participants were operating the system while asking questions, which may have made it easier for them to focus on the problematic targets in the virtual classroom. A participant suggested that additional educational effects could be achieved by organizing and expanding the content.

The usability evaluation results show that the ease of operation with the controller was rated as 7.3 (±1.7). In this system, a video game controller was used as the operating device; therefore, whether the participants had experience of such a controller influenced the results. In addition, the movement speed of the viewpoint during rotation was set to slow to avoid VR sickness, which may have worsened the usability evaluation.

The average immersion of the HMD was rated as 8.4, suggesting that the participants received a high degree of immersion because their actual surroundings were eliminated while wearing the HMD and the display followed the motion of the participant’s head.

Regarding the clearness of the display, the average (5.8) was lower than that for the other parameters, and the confidence interval was large. Thus, we assume that the experience of wearing the HMD differed among the participants. Colour noise was sometimes seen in the display because the HMD tilted due to head movement or looseness of the headband. In addition, a participant stated that the HMD display resolution was low, which worsened immersion. The HMD resolution should therefore be improved.

From the results of the free-response question on whether one’s understanding of colour vision variation had increased, the reasons given for improved understanding were not only that the participants answered questions while comparing the three- and two-colour vision modes but also that the navigator explained to the participants the specific things to think about during the experience. The system could be used to develop better graphs for PowerPoint presentations, not just by schoolteachers and staff but also by students and other occupational workers.

There were limitations to this study. First, the participants were students although the system was developed for teachers. Second, the evaluations were subjective, and the teaching efficacy could not be measured quantitatively. The VAS method needs to be validated with existing scales to be used as a subjective evaluation method by users of VR systems. The Simulcheck method may be needed to evaluate this tool.²⁹

Conclusions

We proposed a virtual experiential learning approach that allows participants to experience and demonstrate the characteristics of colour vision in children with colour vision variation. A pilot study was conducted on the impact of immersive virtual classroom experiences. With this system, schoolteachers will be able to increase their knowledge of colour vision variation and solve colour vision problems in the classroom. In the future, it is necessary to evaluate the effectiveness of this approach for new teachers.

Data availability