INCISION e-learning program as a useful teaching tool to enhance surgeons’ knowledge and skills: An Indonesian multi-center cross-sectional pilot study

Achmad Kemal Harzif; M Nurhadi Rahman; Pungky Mulawardhana; Nadia Shafira; Tricia Dewi Anggraeni; Kanadi Sumapraja; Dwiana Ocviyanti

doi:10.12688/f1000research.15799.1

Home Browse INCISION e-learning program as a useful teaching tool to enhance surgeons’...

ALL Metrics

-

Views

-

Downloads

Get PDF

Get XML

Export

▬

✚

Research Article

INCISION e-learning program as a useful teaching tool to enhance surgeons’ knowledge and skills: An Indonesian multi-center cross-sectional pilot study

[version 1; peer review: 1 approved with reservations]

Achmad Kemal Harzif ¹, M Nurhadi Rahman², Pungky Mulawardhana³, [...] Nadia Shafira⁴, Tricia Dewi Anggraeni⁵, Kanadi Sumapraja¹, Dwiana Ocviyanti⁶

Achmad Kemal Harzif ¹, M Nurhadi Rahman², [...] Pungky Mulawardhana³, Nadia Shafira⁴, Tricia Dewi Anggraeni⁵, Kanadi Sumapraja¹, Dwiana Ocviyanti⁶

PUBLISHED 03 Dec 2018

Author details Author details

¹ Division of Reproductive Endocrinology and Infertility, Department of Obstetric and Gynecology, Faculty of Medicine, Universitas Indonesia, Dr.Cipto Mangunkusumo General Hospital, Central of Jakarta, DKI Jakarta Province, Indonesia
² Department of Obstetrics and Gynecology, Faculty of Medicine, Gajah Mada University, Sarjito Hospital, Jogjakarta, Jogjakarta Special Region, Indonesia
³ Department of Obstetrics and Gynecology, Faculty of Medicine, Universitas Airlangga, Dr. Soetomo Hospital, Surabaya, East Java, Indonesia
⁴ Department of Obstetric and Gynecology, Faculty of Medicine, Universitas Indonesia, Dr.Cipto Mangunkusumo General Hospital, Central of Jakarta, DKI Jakarta Province, Indonesia
⁵ Division of Gynecologic Oncology, Department of Obstetric and Gynecology, Faculty of Medicine, Universitas Indonesia, Dr.Cipto Mangunkusumo General Hospital, Central of Jakarta, DKI Jakarta Province, Indonesia
⁶ Division of Social Obstetrics and Gynecology, Department of Obstetric and Gynecology, Faculty of Medicine, Universitas Indonesia, Dr.Cipto Mangunkusumo General Hospital, Central of Jakarta, DKI Jakarta Province, Indonesia

Achmad Kemal Harzif
Roles: Conceptualization, Funding Acquisition, Project Administration, Supervision, Writing – Review & Editing

M Nurhadi Rahman
Roles: Conceptualization, Supervision, Writing – Review & Editing

Pungky Mulawardhana
Roles: Conceptualization, Project Administration, Supervision, Writing – Review & Editing

Nadia Shafira
Roles: Investigation, Methodology, Project Administration, Resources, Writing – Original Draft Preparation

Tricia Dewi Anggraeni
Roles: Supervision, Writing – Review & Editing

Kanadi Sumapraja
Roles: Supervision, Writing – Review & Editing

Dwiana Ocviyanti
Roles: Supervision, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

Background: Media aids are one of the most important components in the teaching and learning process. This pilot study program was conducted in order to assess the effectiveness of the INCISION e-learning program as teaching media in the surgical teaching and learning process, and its ability to improve surgical skills and knowledge achievement.
Methods: One intervention group and one control group were involved in this study. The intervention group used the hysterectomy INCISION e-learning module, while the control group used conventional teaching approaches. The study was conducted with 14 resident surgeons in three universities in Indonesia: Universitas Indonesia, Universitas Airlangga, and Universitas Gajah Mada. The testing components used were a pre-test, post-test questionnaire (a modified Ritzman questionnaire) and direct observation of procedural skills in the operating room (OR). Data were analyzed descriptively using Mann-Whitney and Wilcoxon tests.
Results: Using a Mann-Whitney test, we found the differences between the average scores of the intervention group and the control group to be statistically significant (p=0.046). A Wilcoxon test also revealed significant differences (p=0.028). The modified Ritzman questionnaire also revealed that the residents in the intervention group felt more confident in their surgical knowledge (82%), and made more efficient use of their time in the OR (81%).
Conclusions: These findings reveal a significant improvement in knowledge and skill achievement in residents that underwent training via the INCISION e-learning module, compared to residents taught via conventional teaching strategies.

Keywords

INCISION, e-learning, surgeon, skills, knowledge

Corresponding author: Achmad Kemal Harzif

Competing interests: No competing interests were disclosed.

Grant information: The author(s) declared that no grants were involved in supporting this work.

Copyright: © 2018 Harzif AK et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

How to cite: Harzif AK, Rahman MN, Mulawardhana P et al. INCISION e-learning program as a useful teaching tool to enhance surgeons’ knowledge and skills: An Indonesian multi-center cross-sectional pilot study [version 1; peer review: 1 approved with reservations]. F1000Research 2018, 7:1888 (https://doi.org/10.12688/f1000research.15799.1) First published: 03 Dec 2018, 7:1888 (https://doi.org/10.12688/f1000research.15799.1) Latest published: 03 Dec 2018, 7:1888 (https://doi.org/10.12688/f1000research.15799.1)

Introduction

The use of multimedia as a learning tool is one of the best educational techniques as it is able to engage more than one sense simultaneously, generally the senses of sight and hearing. Multimedia programs provide a variety of different stimuli, including elements of text, speech, sound and music, graphics, animations and still pictures^1–3. In undergraduate and graduate medical training, the type of teaching media used depends on the institution as well as the individual teacher and the subject matter being taught^4,5.

Traditionally, medical teachers explain theories and demonstrate procedures, followed by practice by the trainees (“see one, do one”)^3,6. There are four teaching approaches to surgical education including: standardized/simulated patients; procedure courses, videos, textbooks; web-based training; cadavers and live animals⁷. Nowadays, technological development has influenced the ways in which learning and information presentation takes place, with a variety of technological tools now supplementing and partly replacing paper books^2,3,6,8–11. The INCISION e-learning module is a new learning and teaching approach, comprised of an online learning platform designed to transfer procedure-specific knowledge to surgeons, gynaecologists and residents. INCISION also provides some information on pre- and postoperative care; however the primary focus is on procedure and the relevant surgical anatomy.

In this pilot study, we sought to evaluate the effectiveness of the INCISION approach on the transfer of relevant surgical knowledge, as well as to assess the strengths and weaknesses of e-learning via INCISION from the point of view of the surgery resident.

Methodology

Study design

The INCISION pilot study program involved 14 surgeon residents specializing in obstetrics and gynecology (OBGYN). Inclusion criteria included residents in 3–6^th semester who had never been trained for hysterectomies, while exclusion criteria included participants who were not willing to participate in this study or had incomplete filling of the questionnaire.

We divided residents into an intervention group and a control group by random number generation, consisting of 7 surgeon residents in each group. Recruitment of the participants was done by asking residents in 3–6^th semester in person on July until August 2015 during the break after class at three universities: Universitas Indonesia, Universitas Airlangga, and Universitas Gajah Mada.

The intervention group used the hysterectomy INCISION e-learning module, while the control group used conventional teaching approaches. The distribution of participants is shown in Table 1.

Table 1. Resident surgeon distribution.

	Control Group (n=7)		Intervention Group (n=7)
	N	%	N	%
University
Universitas Indonesia	3	42.9	3	42.9
Universitas Airlangga	2	28.6	2	28.6
Universitas Gajah Mada	2	28.6	2	28.6

This study was approved by the Ethics Committee of Faculty of Medicine, Univesitas Indonesia, on July 6th 2015 (reference number: 564/UN2.F1/ETIK/2015). Permission was obtained to perform this study in all three sites. Written consent for participation was obtained from all study participants.

Time and location

This study was conducted between July and August 2015, in the medical facilities and hospitals of: Universitas Indonesia, Universitas Airlangga and Universitas Gajah Mada.

Testing instruments

Four evaluation methods were used. First, we included a pre-test with the purpose of evaluating the residents’ knowledge of procedures prior to training (Supplementary File 1). Second, a post-test was conducted after having received the course material (Supplementary File 1). Third, a Ritzman questionnaire was administered to evaluate the residents’ perception of the INCISION approach on the training material, and how it impacted their knowledge (Supplementary File 2). The Ritzman questionnaire was not taken by the control group, only the intervention group. Every question is rated on a score of 1 to 7, which when converted into a percent is able to be grouped as >80% = good, 50-79% = average, and <50% = bad.

Finally, residents performed the procedure under the supervision of a qualified trainer, which was not assessed.

In-depth interviews were also conducted with the intervention group for assessment value of INCISION e-learning. Residents were interviewed by the research team in the Obstetrics and Gynecology Department's Meeting Room. The research team wrote notes during the interview only. The residents were asked about added value of using INCISION e-learning; weakness, difficulties or obstacles encountered during the study with INCISION e-learning; the difference between INCISION e-learning and conventinal teaching; and whether the residents will recomend INCISION e-learning for their friends.

Procedure

Residents participating in the study were given a pre-test during the first week of the study period. This test served as an entry exam to gauge their incoming level of expertise with the particular procedure. Starting in the second week, the intervention group of residents commenced learning via the class with INCISION e-learning 2D module, while the control group of residents were taught via presentation methods.

On the third week, trainers and residents from the intervention group took the class with INCISION e-learning 3D module and discussed it together, while the control group watched the trainer and followed the operation of a patient. Finally, all residents conducted the post-test. The residents took the exam at least once during the week after the study period.

Supplementary File 3 contains information about the conventional course.

Data analysis

Data were analyzed descriptively using SPSS 21 version statistical software, using Mann-Whitney and Wilcoxon tests for statistical analysis. We used a non-parametric test due to the small sample size. The Mann-Whitney test was used to examine the differences in knowledge based on the pre-test examination. The Wilcoxon signed-rank test was also used to evaluate the difference between courses, using data from a paired-sample design. We used the Wilcoxon test to determine whether there was a difference between the intervention group and the control group at the end of the pilot program, based on the administered post-test. A p value <0.05 was considered statistically significant.

Results

All subjects completed the study; no participants declined to be part of the study.

Figure 1a shows the score distribution of the pre- and post-tests in the control group. Five residents decreased their score and two showed an increase. However, in the intervention group, the majority of residents increased their score, as shown in Figure 1b.

Figure 1.

Distribution of pre- and post-test scores from (a) the control group and (b) the intervention group.

We evaluated differences between the two groups’ average pre-test scores in order to determine whether there were any differences in knowledge and skill prior to intervention. Using a Mann-Whitney test, we demonstrated that the average value of the pre-test in the control group was not significantly different from that of the intervention group (p=0.561;).

Further evaluation of the average post-test scores between the control and intervention groups was important to determine whether there were differences in knowledge and skill following intervention. Using a Mann-Whitney test, we found that the average score of the intervention group (67), was significantly greater than that of the control group (53). This difference was statistically significant (p=0.046).

We then compared the pre- and post-test results within each group. The average pre-test score in the control group was 52.71, while the average post-test score in the control group was 52.71. There was no difference between the pre- and post-test results in the control group. The average pre-test score in the intervention group was 50.14, while the average post-test score was 66.93. A Wilcoxon test revealed that this difference was significant (p=0.028).

Residents in the intervention group were satisfied with the training outcome, with an average Ritzman questionnaire score of 81% (Table 2). The highest score, 86%, was given for the statement that residents believe the content of INCISION is useful for their job. On average, residents felt that they appreciated the course (82%), that the learning atmosphere was encouraging (78%), that the learning was fun (73%), and that they obtained beneficial knowledge from the course (80%). The residents also felt that the INCISION e-learning was comprehensible (82%). The content and the language (foreign words and technical terms) was also found to be easy to comprehend (86 and 83%, respectively). They felt that they kept-up thematically with the course (82%), and that the time spent was sufficient for the theme covered (80%).

Table 2. Ritzman questionnaire analysis.

Ritzman Questionnaire	Average Score/item
Training Outcome
Overall, I like the course	82
The learning atmosphere was agreeable	78
The learning was fun	73
I find the approach useful for my job	86
Investing time in the course was useful	82
I can apply the content of the course in my job	88
I derive personal use from the course	80
Average	81
Comprehension
The contents were comprehensible	86
The language (foreign words and technical term) was comprehensible	83
I kept up thematically in the course.	82
The time was sufficient for the theme covered.	80
Average	83
Knowledge gain
I have the impression that my knowledge has expanded on a long-term basis	82
I will be able to remember the new themes well	78
I think that I will bstill be able to report what I learned some time after the course.	71
I will appl what I learned to my day-to-day work	76
I would recommend the INCISION approach to my collegues	80
Average	77
Media
The 2D film were helpful for my understanding	76
The 3D films were helpful for my understanding	90
The online Academy was helpful for my understanding	88
The 3D film was suitable for presenting the contents	94
The 2D film was suitable for presenting the contents	80
The online Academy was suitable for presenting the contents	80
Average	84
Customized additions:
Training outcome
I feel more confident in my surgical knowledge after following the INCISION Approach	82
I feel I made more efficient use of my time in the OR after following the INCISION approach.	81
My understanding of the procedure was helped by the step-by-step approach.	88
I feel patient safety is increased due to the INCISION approach	82
Average	83
Product Feedback
There needs to be more images / stills in the Academy.	80
There need to be more videos in the Academy	78
There need to be longer videos in the Academy	84
Average	80

Additionally, residents in the intervention group felt that INCISION e-learning provides an adequate gain in knowledge (77%). They had the impression that their knowledge had expanded on a long-term basis (82%), that they would be able to remember the new themes well (78%), and that they think they will be able to sufficiently report what they had learned some time after the course (61%). They also expressed that they will apply what they learned to their day-to-day work (76%), and that they would recommend the INCISION approach to their collegues (80%). Overall, the residents rated the INCISION media as good (84%). They preferred 3D film (90%) over 2D film (76%), for ease of understanding the content. With respect to content presentation, they also found 3D film (94%) to be more suitable than 2D film (80%). They felt that the online academy both aided understanding (88%), and was an acceptable medium for presenting the content (80%).

The residents also felt more confident in their surgical knowledge (82%), and made more efficient use of their time in the OR (81%) after following the INCISION approach. They thought that the understanding of the procedure was aided by the step-by-step approach (88%), and feel patient safety would be increased due to the INCISION approach (82%). Regarding product feedback, the residents thought that more images should be provided (80%), as well as more (78%) and longer (84%) video segments.

In the in depth-interview, most residents said that INCISION e-learning via its 3D videos was able to improve knowledge and skills. They said that the advantage of using INCISION e-learning was that it can be accesed anywhere and at anytime, but there were limitations to access the program, for example, when they be located in remote areas that don’t have internet access the program couldn’t be used. Finally, all residents in the intervention group said that they will recommend INCISION e-learning due to their positive learning experience to their colleagues.

Dataset 1.Answers for Ritzman questionnaire about the perception of INCISION e-learning in the intervention group.

Dataset 2.Raw data for pre and post test scores in both control and intervention groups.

Discussion

The results of this study correspond to our expected outcome regarding the INCISION e-learning paradigm. The initial knowledge and skills of residents in both the control and intervention groups were similar, whereas following INCISION e-learning training of residents in the intervention group, the knowledge outcome was significantly increased. We can therefore conclude that INCISION e-learning is able to increase residents’ knowledge compared to conventional learning.

The limitation of this study design was the small number of participants; however, this meant that we were able to complete this report with in-depth interviews of the residents in the intervention group, as shown in Table 3. They discussed the value of INCISION e-learning, including weaknesses, difficulties, and obstacles encountered during the study, as well as the difference between learning with or without the INCISION module. They also discussed whether they would recommend the program to others.

Table 3. Interview output.

Student ID	University	Question
		What is added value of INCISION e-learning?	What the weakness, difficulties or obstacles encountered during the study with INCISION Academy online learning?	Is INCISION e-learning make the material easier to understand? According to you, how much difference when compared with conventional teaching?	Will you recommend INCISION online learning Academy to your friends? Please explain why.
R3	Universitas Indonesia	Added value of the program is that e- learning materials and 3D lecture can improve our understanding of the visual field operations and facilitate the learning process for us to recognize the anatomy and surgical techniques.	Nothing. But since we are currently following the program that also prioritizes education in services to patients, time is a significant constraint during the INCISION program.	Yes, of course, with 3D video material we can further understand the surgical technique and anatomy.	Yes. The program is very good in improve doctor understanding and learning process, especially in a surgical operator.
R5	Universitas Indonesia	The videos are very clear, detail, in a rhythm that is not too fast.	Our education system that very busy, cause we did not see the online video. Internet network is not sufficiently reliable for live streaming. I only had one time to look at the demo video together with the trainer.	Yes. After viewing the video, it becomes a better understanding of anatomy. However, the skill remains to be honed with practice.	I recommend because the given video provide basic skills that required before moving into real action.
R1	Universitas Indonesia	The value is we can learn more real because of the technology used by the Incision. And accessible because it can be learned anywhere (there is online module).	Constraints and difficulties may be time limited.	Yes. The material easier to understand because the real picture given by 3D program.	Yes. I would recommend this program to other colleagues because it simplifies the process of learning curve.
R14	Universitas Airlangga	The value is we can learn more easily, can be accessed anytime anywhere and the tutorial steps allows us to more completely understand,	This program depends to internet connection so if there is a trouble about it, we will difficult to access INCISION e-learning.	Yes, of course. INCISION e-learning makes more understood the surgical technique and anatomy.	I recommend this program because it’s very useful, especially to the surgeon.
R12	Universitas Airlangga	Explanation of the tutorial steps by using audio-visual operations making it easier to learn.	The videos using cadaver that is still lacking in a real application in the field.	Easy to understand, because it directly learn step by step in audio visual.	I recommend this tutorial, because the use of audio- visual methods.
R8	Universitas Gajah Mada	The value of Incision is we could have access to learning materials anytime and anywhere.	Constraints/difficulties during the learning is in the phase of “action with instructors from the incision”. This phase in our implementation is delayed quite long-time. Actually number of cases of hysterectomy procedures is enough, but the obstacles to bring together participants and instructors schedule quite difficult. If we are at a hospital center, the instructor may not necessarily be in place or have no time to scheduled operations, on the contrary when the instructor can be scheduled operations, our participants were in affiliates hospital.	Yes, with Incision learning materials more easily understood. By reading the material and then look at the video (especially in 3D) then it is easier to understand the content and have a clear perception of each step of a procedure.	Yes. By recommending Incision elearning, I hope my colleagues can get the material and understanding as I got when follow this lesson.
R10	Universitas Gajah Mada	Values over Incision Academy is a learning media that comes with 3D video allows us to more completely understand the operating procedures that anatomically closer to the original condition at the time of surgery. Moreover, it can be accessed online, make student can learn anywhere, anytime. Besides the accesses journal links that accompanies the theories given, it is easier for students to learn more about the procedures and theories.	Difficulties experienced is due to the high mobility of us out of town, with internet access that is not capable, consequently we are often difficult to access incision Academy online.	Yes it is easy to understand. With the incision Academy online, all learners have the same standard procedure if compared with learning that I get so far.	Yes, I would recommend with explain the added value as I mentioned in point number 1.

Most residents reported that INCISION e-learning was able to improve knowledge and skills via its use of 3D videos. Another reported advantage was that it can be accessed anywhere and at anytime. Reported weaknesses of the program was limitations to access should they be located in remote areas. Residents also reported that they will recommend INCISION e-learning to their colleagues due to their positive learning experience.

These results are consistent with previous studies that report the efficacy, and the satisfaction amongst users, of multimedia as learning tool for medical purposes, in particular for surgical learning^8,9,11.

Conclusion

These findings reveal that there were significant differences in knowledge and skill achievement between students who underwent training via the INCISION e-learning module and students who were trained via conventional teaching strategies. In addition, a questionnaire revealed that resident surgeons in the intervention group appreciated the use of INCISON e-learning.

Data availability

F1000Research: Dataset 1. Answers for Ritzman questionnaire about the perception of INCISION e-learning in the intervention group., https://doi.org/10.5256/f1000research.15799.d227255¹²

F1000Research: Dataset 2. Raw data for pre and post test scores in both control and intervention groups., https://doi.org/10.5256/f1000research.15799.d227256¹³

Grant information

The author(s) declared that no grants were involved in supporting this work.

Acknowledgements

The authors would like to thank all participants and supervisors that contributed to the work in this study.

Supplementary material

Supplementary File 1: Pre- and post-test used.

Click here to access the data

Supplementary File 2: Ritzman questionnaire about the perception of INCISION e-learning.

Click here to access the data

Supplementary File 3: Conventional Hysterectomy Course Schedule.

Click here to access the data

Faculty Opinions recommended

References

1. Aloraini S: The impact of using multimedia on students’ academic achievement in the college of education at king saud university. Journal of King Saud University-Languages and Translation. 2012; 24(2): 75–82. Publisher Full Text
2. Issa N, Mayer RE, Schuller M, et al.: Teaching for understanding in medical classrooms using multimedia design principles. Med Educ. 2013; 47(4): 388–396. PubMed Abstract | Publisher Full Text
3. Shariff U, Seretis C, Lee D, et al.: The role of multimedia in surgical skills training and assessment. Surgeon. 2016; 14(3): 150–163. PubMed Abstract | Publisher Full Text
4. Masic I: E-learning as new method of medical education. Acta Inform Med. 2008; 16(2): 102–17. PubMed Abstract | Publisher Full Text | Free Full Text
5. Issa N, Mayer RE, Schuller M, et al.: Research on the applications of multimedia auxiliary teaching system with the computer to the education of clinical medicine. Medical Education. 2015; 47(4).
6. Shariff U, Kullar N, Haray PN, et al.: Multimedia educational tools for cognitive surgical skill acquisition in open and laparoscopic colorectal surgery: a randomized controlled trial. Colorectal Dis. 2015; 17(5): 441–450. PubMed Abstract | Publisher Full Text
7. Parmar DJ, Shah C, Parmar RD: Effectiveness of four step approach for procedures (skill) teaching: Rct. The Southeast Asian Journal of Case Report and Review. 2013; 2(5): 281–288. Reference Source
8. Choules AP: The use of elearning in medical education: a review of the current situation. Postgrad Med J. 2007; 83(978): 212–216. PubMed Abstract | Publisher Full Text | Free Full Text
9. Funke K, Bonrath E, Mardin WA, et al.: Blended learning in surgery using the Inmedea Simulator. Langenbecks Arch Surg. 2013; 398(2): 335–340. PubMed Abstract | Publisher Full Text
10. Fingeret AL, Martinez RH, Hsieh C, et al.: Watch what happens: using a web-based multimedia platform to enhance intraoperative learning and development of clinical reasoning. Am J Surg. 2016; 211(2): 384–389. PubMed Abstract | Publisher Full Text
11. Norman E, Furnes B: The relationship between metacognitive experiences and learning: Is there a difference between digital and non-digital study media? Comput Hum Behav. 2016; 54: 301–309. Publisher Full Text
12. Harzif AK, Rahman MN, Mulawardhana P, et al.: Dataset 1 in: INCISION e-learning program as a useful teaching tool to enhance surgeons’ knowledge and skills: An Indonesian multi-center cross-sectional pilot study. F1000Research. 2018. http://www.doi.org/10.5256/f1000research.15799.d227255
13. Harzif AK, Rahman MN, Mulawardhana P, et al.: Dataset 2 in: INCISION e-learning program as a useful teaching tool to enhance surgeons’ knowledge and skills: An Indonesian multi-center cross-sectional pilot study. F1000Research. 2018. http://www.doi.org/10.5256/f1000research.15799.d227256

Comments on this article Comments (0)

Version 1

VERSION 1 PUBLISHED 03 Dec 2018