INCISION e-learning program as a useful teaching tool to enhance surgeons ’ knowledge and skills : An Indonesian multi-center cross-sectional pilot study

Media aids are one of the most important components in the Background teaching and learning process. This pilot study program was conducted in order to assess the effectiveness of the INCISION e-learning program as teaching media in the surgical teaching and learning process, and its ability to improve surgical skills and knowledge achievement. : One intervention group and one control group were involved in this Methods study. The intervention group used the hysterectomy INCISION e-learning module, while the control group used conventional teaching approaches. The study was conducted with 14 resident surgeons in three universities in Indonesia: Universitas Indonesia, Universitas Airlangga, and Universitas Gajah Mada. The testing components used were a pre-test, post-test questionnaire (a modified Ritzman questionnaire) and direct observation of procedural skills in the operating room (OR). Data were analyzed descriptively using Mann-Whitney and Wilcoxon tests. : Using a Mann-Whitney test, we found the differences between the Results average scores of the intervention group and the control group to be statistically significant ( =0.046). A Wilcoxon test also revealed significant differences ( p p =0.028). The modified Ritzman questionnaire also revealed that the residents in the intervention group felt more confident in their surgical knowledge (82%), and made more efficient use of their time in the OR (81%). : These findings reveal a significant improvement in knowledge Conclusions 1 2 3 4 5 1 1 2 3 4 5 6 Referee Status: AWAITING PEER REVIEW 03 Dec 2018, :1888 ( First published: 7 ) https://doi.org/10.12688/f1000research.15799.1 03 Dec 2018, :1888 ( Latest published: 7 ) https://doi.org/10.12688/f1000research.15799.1 v1 Page 1 of 9 F1000Research 2018, 7:1888 Last updated: 03 DEC 2018


Introduction
The use of multimedia as a learning tool is one of the best educational techniques as it is able to engage more than one sense simultaneously, generally the senses of sight and hearing. Multimedia programs provide a variety of different stimuli, including elements of text, speech, sound and music, graphics, animations and still pictures [1][2][3] . In undergraduate and graduate medical training, the type of teaching media used depends on the institution as well as the individual teacher and the subject matter being taught 4,5 .
Traditionally, medical teachers explain theories and demonstrate procedures, followed by practice by the trainees ("see one, do one") 3,6 . There are four teaching approaches to surgical education including: standardized/simulated patients; procedure courses, videos, textbooks; web-based training; cadavers and live animals 7 . Nowadays, technological development has influenced the ways in which learning and information presentation takes place, with a variety of technological tools now supplementing and partly replacing paper books 2,3,6,[8][9][10][11] . The INCISION e-learning module is a new learning and teaching approach, comprised of an online learning platform designed to transfer procedure-specific knowledge to surgeons, gynaecologists and residents. INCISION also provides some information on pre-and postoperative care; however the primary focus is on procedure and the relevant surgical anatomy.
In this pilot study, we sought to evaluate the effectiveness of the INCISION approach on the transfer of relevant surgical knowledge, as well as to assess the strengths and weaknesses of e-learning via INCISION from the point of view of the surgery resident.

Study design
The INCISION pilot study program involved 14 surgeon residents specializing in obstetrics and gynecology (OBGYN). Inclusion criteria included residents in 3-6 th semester who had never been trained for hysterectomies, while exclusion criteria included participants who were not willing to participate in this study or had incomplete filling of the questionnaire.
We divided residents into an intervention group and a control group by random number generation, consisting of 7 surgeon residents in each group. Recruitment of the participants was done by asking residents in 3-6 th semester in person on July until August 2015 during the break after class at three universities: Universitas Indonesia, Universitas Airlangga, and Universitas Gajah Mada.
The intervention group used the hysterectomy INCISION e-learning module, while the control group used conventional teaching approaches. The distribution of participants is shown in Table 1.
This study was approved by the Ethics Committee of Faculty of Medicine, Univesitas Indonesia, on July 6th 2015 (reference number: 564/UN2.F1/ETIK/2015). Permission was obtained to perform this study in all three sites. Written consent for participation was obtained from all study participants.

Time and location
This study was conducted between July and August 2015, in the medical facilities and hospitals of: Universitas Indonesia, Universitas Airlangga and Universitas Gajah Mada.

Testing instruments
Four evaluation methods were used. First, we included a pre-test with the purpose of evaluating the residents' knowledge of procedures prior to training (Supplementary File 1). Second, a post-test was conducted after having received the course material (Supplementary File 1). Third, a Ritzman questionnaire was administered to evaluate the residents' perception of the INCISION approach on the training material, and how it impacted their knowledge (Supplementary File 2). The Ritzman questionnaire was not taken by the control group, only the intervention group. Every question is rated on a score of 1 to 7, which when converted into a percent is able to be grouped as >80% = good, 50-79% = average, and <50% = bad.
Finally, residents performed the procedure under the supervision of a qualified trainer, which was not assessed.

Procedure
Residents participating in the study were given a pre-test during the first week of the study period. This test served as an entry exam to gauge their incoming level of expertise with the particular procedure. Starting in the second week, the intervention group of residents commenced learning via the class with INCISION e-learning 2D module, while the control group of residents were taught via presentation methods.
On the third week, trainers and residents from the intervention group took the class with INCISION e-learning 3D module and discussed it together, while the control group watched the trainer and followed the operation of a patient. Finally, all residents conducted the post-test. The residents took the exam at least once during the week after the study period.
Supplementary File 3 contains information about the conventional course.

Data analysis
Data were analyzed descriptively using SPSS 21 version statistical software, using Mann-Whitney and Wilcoxon tests for statistical analysis. We used a non-parametric test due to the small sample size. The Mann-Whitney test was used to examine the differences in knowledge based on the pre-test examination. The Wilcoxon signed-rank test was also used to evaluate the difference between courses, using data from a paired-sample design. We used the Wilcoxon test to determine whether there was a difference between the intervention group and the control group at the end of the pilot program, based on the administered post-test. A p value <0.05 was considered statistically significant.

Results
All subjects completed the study; no participants declined to be part of the study. Figure 1a shows the score distribution of the pre-and posttests in the control group. Five residents decreased their score and two showed an increase. However, in the intervention group, the majority of residents increased their score, as shown in Figure 1b.
We evaluated differences between the two groups' average pre-test scores in order to determine whether there were any differences in knowledge and skill prior to intervention. Using a Mann-Whitney test, we demonstrated that the average value of the pre-test in the control group was not significantly different from that of the intervention group (p=0.561;).
Further evaluation of the average post-test scores between the control and intervention groups was important to determine whether there were differences in knowledge and skill following intervention. Using a Mann-Whitney test, we found that the average score of the intervention group (67), was significantly greater than that of the control group (53). This difference was statistically significant (p=0.046).
We then compared the pre-and post-test results within each group. The average pre-test score in the control group was 52.71, while the average post-test score in the control group was 52.71. There was no difference between the pre-and post-test results in the control group. The average pre-test score in the intervention group was 50.14, while the average post-test score was 66.93. A Wilcoxon test revealed that this difference was significant (p=0.028).
Residents in the intervention group were satisfied with the training outcome, with an average Ritzman questionnaire score of 81% ( Table 2). The highest score, 86%, was given for the  Average 77

Media
The 2D film were helpful for my understanding 76 The 3D films were helpful for my understanding 90 The online Academy was helpful for my understanding 88 The 3D film was suitable for presenting the contents 94 The 2D film was suitable for presenting the contents 80 The online Academy was suitable for presenting the contents 80 Average 83

Product Feedback
There needs to be more images / stills in the Academy. 80 There need to be more videos in the Academy 78 There need to be longer videos in the Academy statement that residents believe the content of INCISION is useful for their job. On average, residents felt that they appreciated the course (82%), that the learning atmosphere was encouraging (78%), that the learning was fun (73%), and that they obtained beneficial knowledge from the course (80%). The residents also felt that the INCISION e-learning was comprehensible (82%). The content and the language (foreign words and technical terms) was also found to be easy to comprehend (86 and 83%, respectively). They felt that they kept-up thematically with the course (82%), and that the time spent was sufficient for the theme covered (80%).
Additionally, residents in the intervention group felt that INCI-SION e-learning provides an adequate gain in knowledge (77%). They had the impression that their knowledge had expanded on a long-term basis (82%), that they would be able to remember the new themes well (78%), and that they think they will be able to sufficiently report what they had learned some time after the course (61%). They also expressed that they will apply what they learned to their day-to-day work (76%), and that they would recommend the INCISION approach to their collegues (80%). Overall, the residents rated the INCISION media as good (84%). They preferred 3D film (90%) over 2D film (76%), for ease of understanding the content. With respect to content presentation, they also found 3D film (94%) to be more suitable than 2D film (80%). They felt that the online academy both aided understanding (88%), and was an acceptable medium for presenting the content (80%).
The residents also felt more confident in their surgical knowledge (82%), and made more efficient use of their time in the OR (81%) after following the INCISION approach. They thought that the understanding of the procedure was aided by the step-by-step approach (88%), and feel patient safety would be increased due to the INCISION approach (82%). Regarding product feedback, the residents thought that more images should be provided (80%), as well as more (78%) and longer (84%) video segments.
In the in depth-interview, most residents said that INCISION e-learning via its 3D videos was able to improve knowledge and skills. They said that the advantage of using INCISION e-learning was that it can be accesed anywhere and at anytime, but there were limitations to access the program, for example, when they be located in remote areas that don't have internet access the program couldn't be used. Finally, all residents in the intervention group said that they will recommend INCISION e-learning due to their positive learning experience to their colleagues.

Discussion
The results of this study correspond to our expected outcome regarding the INCISION e-learning paradigm. The initial knowledge and skills of residents in both the control and intervention groups were similar, whereas following INCISION e-learning training of residents in the intervention group, the knowledge outcome was significantly increased. We can therefore conclude that INCISION e-learning is able to increase residents' knowledge compared to conventional learning.
The limitation of this study design was the small number of participants; however, this meant that we were able to complete this report with in-depth interviews of the residents in the intervention group, as shown in Table 3. They discussed the value of INCI-SION e-learning, including weaknesses, difficulties, and obstacles encountered during the study, as well as the difference between learning with or without the INCISION module. They also discussed whether they would recommend the program to others.
Most residents reported that INCISION e-learning was able to improve knowledge and skills via its use of 3D videos. Another reported advantage was that it can be accessed anywhere and at anytime. Reported weaknesses of the program was limitations to access should they be located in remote areas. Residents also reported that they will recommend INCISION e-learning to their colleagues due to their positive learning experience.
These results are consistent with previous studies that report the efficacy, and the satisfaction amongst users, of multimedia as learning tool for medical purposes, in particular for surgical learning 8, 9,11 .

Conclusion
These findings reveal that there were significant differences in knowledge and skill achievement between students who underwent training via the INCISION e-learning module and students who were trained via conventional teaching strategies. In addition, a questionnaire revealed that resident surgeons in the intervention group appreciated the use of INCISON e-learning.

R3
Universitas Indonesia Added value of the program is that e-learning materials and 3D lecture can improve our understanding of the visual field operations and facilitate the learning process for us to recognize the anatomy and surgical techniques.
Nothing. But since we are currently following the program that also prioritizes education in services to patients, time is a significant constraint during the INCISION program.
Yes, of course, with 3D video material we can further understand the surgical technique and anatomy.
Yes. The program is very good in improve doctor understanding and learning process, especially in a surgical operator.

R5
Universitas Indonesia The videos are very clear, detail, in a rhythm that is not too fast.
Our education system that very busy, cause we did not see the online video. Internet network is not sufficiently reliable for live streaming. I only had one time to look at the demo video together with the trainer.
Yes. After viewing the video, it becomes a better understanding of anatomy.
However, the skill remains to be honed with practice.
I recommend because the given video provide basic skills that required before moving into real action.

Universitas
Gajah Mada Values over Incision Academy is a learning media that comes with 3D video allows us to more completely understand the operating procedures that anatomically closer to the original condition at the time of surgery. Moreover, it can be accessed online, make student can learn anywhere, anytime. Besides the accesses journal links that accompanies the theories given, it is easier for students to learn more about the procedures and theories.
Difficulties experienced is due to the high mobility of us out of town, with internet access that is not capable, consequently we are often difficult to access incision Academy online.
Yes it is easy to understand. With the incision Academy online, all learners have the same standard procedure if compared with learning that I get so far.
Yes, I would recommend with explain the added value as I mentioned in point number 1.

Grant information
The author(s) declared that no grant(s) were involved in supporting this work.