F1000Research F1000Research 2046-1402 F1000 Research Limited London, UK 10.12688/f1000research.109876.2 Research Article Articles Evaluating the effect of digital technology on the learning of orthodontic cephalometric measurement

[version 2; peer review: 1 approved, 1 approved with reservations]

 Yu Xin Conceptualization Visualization Writing – Original Draft Preparation 1 2 3 Tian Yu Investigation Visualization Writing – Original Draft Preparation https://orcid.org/0000-0002-5256-046X 1 2 Li Dandan Formal Analysis Methodology Writing – Review & Editing 1 2 Sun Wen Formal Analysis Project Administration Writing – Review & Editing 1 2 Wang Hua Data Curation Resources Writing – Review & Editing 1 2 Yuan Siyang Investigation Visualization Writing – Review & Editing 4 Yan Bin Data Curation Supervision Writing – Review & Editing a 1 2 Wang Lin Project Administration Supervision Writing – Review & Editing 1 2 Pan Yongchu Conceptualization Funding Acquisition Writing – Review & Editing https://orcid.org/0000-0002-6899-5994 b 1 2 Jiangsu Key Laboratory of Oral Diseases, Jiangsu Key Laboratory of Oral Diseases, Nanjing, 210029, China Department of Orthodontics, The Affiliated Stomatological Hospital of Nanjing Medical University, Nanjing, 210029, China Department of Orthodontics, Affiliated Nantong Stomatological Hospital of Nantong University, Nantong, 226006, China School of Dentistry, University of Dundee, Dundee, DD1 4HN, UK byan@njmu.edu.cn panyongchu@njmu.edu.cn

No competing interests were disclosed.

 12 12 2022 2022 11 328 5 12 2022 Copyright: © 2022 Yu X et al. 2022 This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Background: This study aimed to evaluate the effectiveness of integrating digital technology into cephalometric measurement teaching.

Methods: In total, 94 undergraduates of stomatology were recruited and randomly allocated to two groups. According to the cross-over design, both groups completed cephalometric measurements by the traditional hand-drawn method and digital technology (the Dolphin software) in different order. In the traditional hand-drawn method, students depicted the outline of the craniofacial anatomical structures on sulfuric transfer paper first, then marked the measurement points and completed the measurement of line spacings and angles; by digital technology, they marked the points in the software and adjust the automatically generated outlines of the structures to obtain the results. Two professional orthodontists were invited as instructors and their measurements were set as standards. An online questionnaire was also used to investigate students' attitudes toward digital technology being used in the cephalometric teaching process.

Results: There were significant differences of students’ measurements ( P 1-SNA<0.01, P 1-SNB=0.01 and P 1-L1-NB (mm)<0.01; SNA: sella-nasion-subspinale angle, SNB: sella-nasion-supramental angle, L1-NB (mm): the distance from the lower central incisor tip to the nasion-supramental plane) between the traditional method and digital technology. Besides, the results of most items by digital technology were closer to the standards than those by the traditional method, including five items with statistical significance ( P 2-SNB<0.05, P 2-L1-NB (mm)<0.01, P 2-FMA<0.05, P 2-FMIA<0.05, P 2-IMPA<0.01), while three items were the opposite ( P 2-SNA<0.05, P 2-ANB (mm)<0.01, P 2-NA-PA<0.01). The questionnaire showed more students preferred digital technology (33%) as a better teaching method than the traditional method (2%) and 72% of participants thought they had mastered 50-80% of cephalometric knowledge after the course.

Conclusions: This study demonstrated effectiveness and acceptance of the course applying digital technology during the cephalometric teaching process.

 orthodontic teaching; cephalometric measurement; traditional hand-drawn method; digital technology; the Dolphin software Jiangsu Provincial Key Medical Discipline ZDXKA2016026 Priority Academic Program Development of Jiangsu Higher Education Institutions PAPD-2018-87 Nanjing Medical University 2019ZC027 ZC32015JG01 This work was supported by the Educational Project of Nanjing Medical University (2019ZC027 and ZC32015JG01), the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD-2018-87), and the Jiangsu Provincial Key Medical Discipline (ZDXKA2016026). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Amendments from Version 1

This version has modified the title and abstract to better conclude this pedagogical experiment. In addition, the introduction to processes of this pedagogical experiment was divided into parts of students and instructors, to make the contents clearer. Figure 1 was revised according to the corresponding contents

 Introduction

Orthodontics, as a discipline of stomatology, aims to study various kinds of malocclusion, including deformities of teeth, jaws and the craniofacial region. 1 Diagnosis is the most important part of orthodontic clinical work, among which cephalometric measurement is an essential procedure. 2 Nevertheless, cephalometric measurement indicators are numerous and complicated. In the past, the traditional hand-drawn method was applied, which required reading lamps, sulfuric acid transfer paper, dividers and so on. In addition, previous studies suggested that undergraduates tended to show less confidence in reading and measuring lateral cranial radiographs. 3

As continuous developments are seen in digital technology, such technology has been applied in the education for undergraduates in recent decades. Digital technology possesses vivid images and operable processes, making it more intuitive, interactive and understandable than simply imparting theoretical knowledge to students, which may help improve the effect of teaching practice. 4 – 6 Buchanan JA et al. found that before starting clinical work, students could better master theoretical knowledge through simulated operation or computer-aided learning method. 7 , 8 It was also reported that students' attitudes towards computer-aided learning and digital technique were positive. 9 – 13 Digital technology has offered great potential for dental education as well. 14 For instance, Nagy ZA et al. reported that the Dental Teacher software could help students more efficiently learn the preparation technique of onlay restorations and facilitate their individual performances. 15 Liu L et al. also found the digital training system might be a good alternative to the traditional training method in the preclinical practice of tooth preparation. 16

The digital cephalometric analysis system, widely incorporated in intelligent software, was developed to computerize the manual tasks and output the specific results automatically. It was reported to be more time-saving than traditional measurement method and helpful for reducing unnecessary errors during the measurement process. 17 Farooq et al. also found that the accuracy of cephalometric measurement by digital tracing with FACAD ® was similar with the manual method. Furthermore, its advantages of digital imaging, such as quality improvement, file transmission and archiving made digitalized cephalometric analysis preferrable in daily use. 18

The Dolphin software ® (Dolphin Imaging & management solution, America) is widely applied in the field of orthodontics, possessing functions like storage and management of patients’ information and images. After users upload computerized tomography photographs, it can also achieve three-dimensional imaging, cephalometric measurement and treatment effect prediction. It incorporates more than 400 cephalometric analytical methods. This software has been reported to have the potential as an animation textbook for medical college students. 19 Although it was assumed to exert some positive effects in the teaching process, there is still lack of research investigating the effectiveness of applying it in orthodontic cephalometric teaching process. Thus, the aim of this study was to examine the effectiveness of applying digital technology (the Dolphin software ®) in teaching cephalometric measurement.

 Methods Ethical considerations

This study followed the guidelines of the Nanjing Medical University ethics review committee and received the approval of the committee (approval number: PJ2019-053-001). All participants gave written informed consent.

 Study size

G*Power software version 3.1.9.7 (RRID:SCR_013726) was used to estimate the required sample size for this study. This study used two independent t tests to calculate the number of students needed. The study power was set at 90% and alpha value set at 0.05. Based on these, a minimum sample of 86 subjects was required.

 Participants

This study approached fourth-year undergraduate students of Stomatology in Nanjing Medical University by inviting them to attend this course, followed by their voluntary registration. All the students, consisting of 63 female and 31 male students (around 21-23 years old), agreed to participate in this pedagogical experiment and signed the informed consent form. All the students did not learn about the method of measuring the lateral cranial radiographs and use the Dolphin software before. We also invited two orthodontist faculty members with over 5 years of clinical experience as instructors.

 Study design

This study was conducted from May 12, 2020 to June 16, 2020. The flow diagram of this pedagogical study was shown in Figure 1. For students, it was divided into pre-test learning, group and cephalometric measurement, and questionnaire survey. For instructors, it was divided into monitoring activity, cephalometric measurement and inter- and intra-class correlation coefficient of their measurements.

Flow diagram of the teaching experiment.

Part of students

Pre-test learning

The total teaching period consisted of 7 credit hours, including 4 credit hours of theoretical class and 3 credit hours for practical instruction. During the theoretical class, one instructor imparted relevant knowledge to the students in detail, including the positions of anatomical markers and the meanings of commonly used measurement items.

In the practical instruction class, another instructor guided 94 students to review the basic knowledge and showed them how to complete cephalometric measurement by the traditional method and digital technology (the Dolphin software ®). For the traditional method, the sulfuric acid transfer paper was fixed to the radiograph with a clip. Then, the patient’s soft tissue profile and hard tissue anatomical structures were depicted on the reading lamp. Finally, the commonly used anatomical points were identified on the sulfuric acid transfer paper and the measurement was completed with the ruler and protractor.

While using digital technology, the instructor adjusted results of line spacing on the lateral cranial radiograph to their actual size at first. Then, the instructor accomplished the measurement by adjusting the gray contrast value and other auxiliary methods. After learning the relevant knowledge of commonly used cephalometric measurement points and items, students were encouraged to review relevant contents after class.

Group and cephalometric measurement

One week after the end of the pre-test instruction, 94 students were randomly allocated into two groups through the RAND function in Excel software (Microsoft ® Excel 2019MSO (2201 Build 16.0.14827.20198 version for 64 bit) and were required to complete the same cephalometric measurement by both traditional method and digital technology (the Dolphin Imaging ® 11.8).

According to the cross-over design, one group took the traditional method first to complete the measurement and then used the Dolphin software ®, while the other group completed in the opposite order. A total of 15 cephalometric items ( Figure 2) were measured, such as the angle between Sella-Nasion plane and the Nasion-Subspinale plane (SNA), the angle between Sella-Nasion plane and the Nasion-Supramental plane (SNB), the angle between the Nasion-Subspinale plane and the Nasion-Supramental plane (ANB). The measurements of the traditional method and Dolphin software ® were recorded, respectively. The collection and input of these data were completed by three postgraduates, with two postgraduates responsible for the inputting and the third one in charge of checking.

Illustration of 15 commonly used measurement items.

1. SNA: the angle between the sella-nasion plane and the nasion-subspinale plane; 2. SNB: the angle between sella-nasion plane and the nasion-supramental plane; 3. ANB: the angle between the nasion-subspinale plane and the nasion-supramental plane; 4. NP-FH: the posterior-inferior angle between the facial plane and the Frankfort horizontal plane; 5. NA-PA: the angle between the nasion-subspinale plane and the pogonion-subspinale plane; 6. U1-NA (mm): the distance from the upper central incisor tip to the nasion-subspinale plane; 7. U1-NA: the angle between the upper central incisor and the nasion-subspinale plane; 8. L1-NB (mm): the distance from the lower central incisor tip to the nasion-supramental plane; 9. L1-NB: the angle between the lower central incisor and the nasion-supramental plane; 10. U1-L1: the angle between the upper central incisor and the lower central incisor; 11.Y axis angle: the anterior-inferior angle between the Frankfort horizontal plane and the sella-pogonion plane; 12. Po-NB (mm): the distance from the pogonion point to the nasion-supramental plane; 13. FMA: the angle between the Frankfort horizontal plane and the mandibular plane; 14. FMIA: the angle between the Frankfort horizontal plane and the long axis of the lower central incisor; 15. IMPA: the angle between the long axis of the lower central incisor and the mandibular plane. The profile tracing was completed with the Uceph software (Uceph, Chengdu, China).

 Questionnaire survey

In order to survey the effectiveness of applying the digital software in cephalometric teaching and students’ attitudes toward it (the Dolphin software ®), we designed an online questionnaire and collected results by Wenjuanxing (Ranxing Co. Ltd., Changsha, China). For example, to investigate how difficult students considered cephalometry is, we set three options ranging from “very difficult”, “kind of tough” to “easy”. As to the mastery degree of students after the course, the options were “50-80%”, “20-50%” to “0-20%”. Gender of the participants was recorded from the university records. The detailed questionnaire list and corresponding options are shown in Table 1.

Students' attitudes towards the application of digital technology.
Questionnaire list Option list
Are you interested in studying cephalometric measurement? A: very interested (55%) B: a little interested (43%) C: not interested at all (2%)
How helpful do you think cephalometric measurement is to the diagnosis of malocclusion? A: very helpful (77%) B: a little helpful (21%) C: not helpful at all (2%)
How tough do you think cephalometric measurement is? A: very difficult (21%) B: kind of tough (66%) C: easy (13%)
Which do you prefer as the better teaching method? A: digital technology (33%) B: traditional method (2%) C: both are acceptable (65%)
How well do you think you master the cephalometric measurement analysis? A: 50%-80% (72%) B: 20%-50% (28%) C: 0-20% (0%)

Part of instructors

Monitoring activity

During this pedagogical experiments, two instructors took responsibility for guiding students with cephalometric measurements, assuring the measurement order of two student groups, monitoring the process of giving out and recollecting the questionnaire, as well as assistance in the data analysis.

Cephalometric measurement & Inter- and intra-class correlation coefficient test

Two instructors also measured the same radiograph by two methods. One week later, they measured the same radiograph by two methods again. The measurements by one instructor at one-week’s interval were used for calculating intra-class correlation coefficient (ICC intra), and measurements by two instructors were used for calculating inter-class correlation coefficient (ICC inter).

 Statistical analysis

The quantitative data of cephalometric measurements were analyzed by the statistical software SPSS 18.0 (IBM Corporation, Armonk, NY, RRID:SCR_016479). The measurements of students by two methods were compared using the independent sample t test, as well as comparing them with corresponding standards, respectively, with the level of significance set as P<0.05.

As for questionnaire data, we obtained the statistical data through the built-in function on the online questionnaire platform, as it provided the constituent ratio of each option and participants list. Then we performed a descriptive analysis of these results.

Anatomical points calibration

Before the course, the standard of anatomical markers on lateral cranial radiographs were calibrated. Firstly, two orthodontists reviewed the basic definitions and meanings of common anatomical points and items. Then, they measured a lateral cranial radiograph on paper and by software, followed by discussing and unifying the anatomical position standard. Subsequently, they respectively completed cephalometric measurement of another lateral cranial radiograph. The intraclass correlation coefficient (ICC) was greater than 90%, showing consistency of cephalometric measurement between them.

 Results Standards of cephalometric measurement

As shown in Supplementary Table 1 and 2, the inter- and intra-ICC were all surpassed 90%, and there were no significant differences between two instructors’ measurements, and the measurements by two methods. As the result, the means of two instructors’ measurements were set as the standards.

 Accuracy of students’ cephalometric measurement

Statistically significant differences were observed in measurements of SNA, SNB and the distance from the lower central incisor tip to the nasion-supramental plane (L1-NB (mm)) between the traditional method and digital technology ( P 1-SNA<0.01, P 1-SNB=0.01, P 1-L1-NB (mm)<0.01) ( Table 2), while other items showed no significant differences. Besides, the measurements by digital technology were closer to the standard values than those by the traditional method. The accuracy of five items measurements using digital technology was significantly higher, including SNB, L1-NB (mm), the angle between the Frankfort horizontal plane and the mandibular plane (FMA), the angle between the Frankfort horizontal plane and the long axis of the lower central incisor (FMIA), and the angle between the long axis of the lower central incisor and the mandibular plane (IMPA) ( P 2-SNB<0.05, P 2-L1-NB (mm)<0.01, P 2-FMA<0.05, P 2-FMIA<0.05, P 2-IMPA<0.01). However, five items presented the opposite result, among which three items were statistically significant (SNA, ANB and the angle between the Nasion-Subspinale plane and the pogonion-subspinale plane (NA-PA) ( P 2-SNA<0.05, P 2-ANB<0.01, P 2-NA-PA<0.01).

Comparison of students' measurements and standard values by the digital technology and traditional method.
Measurement items Digital technology Traditional method P 1 P 2
Test value ( X ¯ ±SD) Standard value d ¯ 1 Test value ( X ¯ ±SD) Standard value d ¯ 2
SNA (°) 92.46±1.88 92.00 1.53 91.41±1.47 92.10 1.19 <0.01 <0.05
SNB (°) 88.47±0.86 88.30 0.69 88.03±1.35 88.30 0.94 0.01 <0.05
ANB (°) 3.76±1.83 3.70 1.43 3.50±0.86 3.80 0.75 0.25 <0.01
NP-FH (°) 89.32±1.62 90.70 1.83 88.95±1.61 90.90 1.89 0.13 0.76
NA-PA (°) 5.99±3.60 6.40 2.79 5.98±1.86 6.50 1.56 0.98 <0.01
U1-NA (mm) 5.07±1.86 6.20 1.71 5.20±1.41 6.30 1.35 0.61 0.06
U1-NA (°) 21.26±3.77 19.70 3.41 21.69±4.75 19.90 3.88 0.50 0.26
L1-NB (mm) 6.09±0.48 6.70 0.73 5.76±0.75 6.80 1.00 <0.01 <0.01
L1-NB (°) 25.10±2.24 26.45 2.19 25.69±3.40 26.60 2.68 0.16 0.09
U1-L1 (°) 129.68±4.67 130.25 4.04 130.16±5.97 130.50 4.82 0.54 0.08
Y axis (°) 64.45±0.99 62.80 1.79 64.22±1.93 62.70 1.91 0.32 0.52
Po-NB (mm) 1.13±0.46 1.05 0.37 1.21±0.44 1.02 0.35 0.21 0.54
FMA (°) 28.89±2.05 28.25 1.71 29.31±2.51 28.50 2.16 0.22 <0.05
FMIA (°) 63.46±2.78 63.75 2.27 63.36±3.52 63.80 2.90 0.83 <0.05
IMPA (°) 87.94±2.78 88.00 2.31 87.80±4.03 88.20 3.30 0.79 <0.01

X: Mean of students’ measurements; SD: Standard deviation of students’ measurements; Standard value: Mean of two orthodontists’ measurements by traditional method or digital technology; d ¯ 1 : Mean of absolute values of differences between students’ results and standard values by digital software; d ¯ 2 : Mean of absolute values of differences between students’ results and standard values by traditional method; P 1: The P value of the independent sample t test on students’ measurements by two methods; P 2: The P value of the independent sample t test on absolute values of differences between students’ results and the standards by two methods.

SNA: the angle between the sella-nasion plane and the nasion-subspinale plane; SNB: the angle between sella-nasion plane and the nasion-supramental plane; ANB: the angle between the nasion-subspinale plane and the nasion-supramental plane; NP-FH: the posterior-inferior angle between the facial plane and the Frankfort horizontal plane; NA-PA: the angle between the nasion-subspinale plane and the pogonion-subspinale plane; U1-NA (mm): the distance from the upper central incisor tip to the nasion-subspinale plane; U1-NA: the angle between the upper central incisor and the nasion-subspinale plane; L1-NB (mm): the distance from the lower central incisor tip to the nasion-supramental plane; L1-NB: the angle between the lower central incisor and the nasion-supramental plane; U1-L1: the angle between the upper central incisor and the lower central incisor; Y axis angle: the anterior-inferior angle between the Frankfort horizontal plane and the sella-pogonion plane; Po-NB (mm): the distance from the pogonion point to the nasion-supramental plane; FMA: the angle between the Frankfort horizontal plane and the mandibular plane; FMIA: the angle between the Frankfort horizontal plane and the long axis of the lower central incisor; IMPA: the angle between the long axis of the lower central incisor and the mandibular plane.

 Attitudes of students towards the digital technology

We assigned the questionnaires to all the participants with 82 of 94 students filling out the questionnaire and the response rate was 87%. The statistical results of the questionnaire are shown in Table 1. Among the respondents, 66% thought studying cephalometry was very difficult and 21% thought it was kind of tough. After instruction, review and practice, 72% of them considered they had mastered 50-80% of relevant knowledge and a few students thought they had mastered 20-50%. About 33% of students preferred the digital technology than traditional method (2%) as a better teaching method and 65% held that both were acceptable, which indicated good acceptance by students of digital technology applied in the teaching process. In addition, 98% of participants expressed their interest in studying cephalometry and considered cephalometric analysis helpful to diagnosis of malocclusion.

 Discussion

Cephalometric measurement is essential for diagnosis and treatment plan design of patients with malocclusions, which are the most significant procedures in orthodontic clinical work. Orthodontic educators put forward that orthodontic teaching for undergraduates should focus on diagnosis and recognition of problems. 20 However, there is not a generally accepted teaching method, and a wide variation of course durations and contents exist in different dental colleges and faculties. 21 How to arouse students’ interest and achieve better teaching effects is a major problem faced by orthodontic educators. Since only a few reports explored this aspect, we designed this didactical experiment.

In our study, three cephalometric items measured by two methods were statistically different, while other items were basically similar. These results implied that digital technology could achieve similar results to the traditional method during cephalometric measurement. Additionally, the majority of items measured by digital technology were closer to the standards, including five statistically significant items, which suggested students could achieve more accurate results by digital technology. This may be attributed to the function of digital software to adjust the gray contrast value of X-ray films (Supplementary figure 1), making it easier to identify the unclear points on the printed paper. In addition, the automatic generation of results also helped to avoid evitable errors during manual measurement. These results were in accordance with previous studies, which suggested that the accuracy of digital measurement on 3-dimensional cone beam computed tomography images was basically similar to or even higher than that of manual measurement. 22 , 23

However, in spite of the convenience it provided, the digital technology may lead to a lack of deep understanding of corresponding contents, such as definitions and meanings of these measurement items. The traditional method could better cultivate practical abilities of students and enhance their memory of relevant knowledge. Besides, although the accuracy of some items obtained by digital technology were significantly higher, a few items showed the opposite result (SNA, ANB and NA-PA). The subspinale point (the A point) was associated with these three items, which suggested that the traditional method was more accurate than digital technology in positioning the subspinale point on the lateral cranial radiograph. The subspinale point is the most concave point of the arc from the anterior nasal spine point (the ANS point) to the superior prosthion point (the Spr point). Compared with digital measurement, the advantage of manual measurement is that the arc can be traced on paper, and some auxiliary instruments like ruler and protractor can help to locate the subspinale point (Supplementary figure 2). These results indicated abundant experience was required to identify the subspinale point, reminding both orthodontic educators and students to devote more time and energy to deep learning it.

Before the course, the majority of students showed fear towards abstract and complex concepts, assuming cephalometric measurement was difficult to master. However, with instruction and practice, most students could master 50-80% of relevant knowledge, which could result from digital technology realizing visualization of numerous and complicated anatomical markers. Previous studies found that visualization was extremely attractive to young students, and significantly aroused their interest and sense of participation. 24 , 25 Our survey confirmed that more students preferred digital technology (33%) as a better teaching method than the traditional method. As a result, the application of digital technology in teaching cephalometric measurement was widely accepted by students and contributed to favorable teaching results.

This study still had some limitations. For example, the sample size could be further enlarged. Secondly, the measurement time for students were not strictly required, which may have resulted in the underperformances or supernormal performances of students. Thirdly, exploration of student’s attitudes towards digital technology being applied in this course was not sufficient. The mentioned issues needed to be improved may interrupt us from to accurately assessing the real advantages and disadvantages of the appliance of this digital technology. These findings could guide and encourage university orthodontic teachers to apply this technology in cephalometric teaching and pay more attention to considering the position identification of the subspinale point.

 Conclusion

This study investigated the effect of applying the digital technology (the Dolphin software ®) in cephalometric learning, demonstrating better effect and acceptance of this technology.

 Data availability Extended data

Figshare: Evaluating the effect of digital technology on the learning of orthodontic cephalometric measurement, https://doi.org/10.6084/m9.figshare.21550599.v1. 26

This project contains the following extended data:

Supplementary File.docx (Supplementary figure 1, 2; Supplementary table 1, 2)

The results of students’ measurements.xlsx

Statistical results of the questionnaire.xlsx

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

 References Ditmarov A : Orthodontics: Orthodontics vs orthodontiya. Br. Dent. J. Jul 13 2018;225(1):2. 30002557 10.1038/sj.bdj.2018.549 Jheon AH Oberoi S Solem RC : Moving towards precision orthodontics: An evolving paradigm shift in the planning and delivery of customized orthodontic therapy. Orthod. Craniofac. Res. Jun 2017;20(Suppl 1):106–113. 28643930 10.1111/ocr.12171 Drage NA Atkin PA Farnell DJJ : Dental and maxillofacial radiology: confidence, knowledge and skills in the newly graduated dentist. Br. Dent. J. Apr 2020;228(7):546–550. 10.1038/s41415-020-1425-x Zitzmann NU Matthisson L Ohla H : Digital Undergraduate Education in Dentistry: A Systematic Review. Int. J. Environ. Res. Public Health. May 7 2020;17(9). 32392877 10.3390/ijerph17093269 Inquimbert C Tramini P Romieu O : Pedagogical Evaluation of Digital Technology to Enhance Dental Student Learning. Eur. J. Dent. Feb 2019;13(1):053–057. 31242508 10.1055/s-0039-1688526 Silveira MS Cogo ALP : The contributions of digital technologies in the teaching of nursing skills: an integrative review. Rev. Gaucha Enferm. Jul 13 2017;38(2):e66204.Contribuições das tecnologias educacionais digitais no ensino de habilidades de enfermagem: revisão integrativa. 28723986 10.1590/1983-1447.2017.02.66204 Buchanan JA : Use of simulation technology in dental education. J. Dent. Educ. Nov 2001;65(11):1225–1231. 10.1002/j.0022-0337.2001.65.11.tb03481.x Urbankova A : Impact of computerized dental simulation training on preclinical operative dentistry examination scores. J. Dent. Educ. Apr 2010;74(4):402–409. 20388813 10.1002/j.0022-0337.2010.74.4.tb04885.x Al-Jewair TS Qutub AF Malkhassian G : A systematic review of computer-assisted learning in endodontics education. J. Dent. Educ. Jun 2010;74(6):601–611. 10.1002/j.0022-0337.2010.74.6.tb04905.x Abbey LM Arnold P Halunko L : CASE STUDIES for Dentistry: development of a tool to author interactive, multimedia, computer-based patient simulations. J. Dent. Educ. Dec 2003;67(12):1345–54. Al-Jewair TS Azarpazhooh A Suri S : Computer-assisted learning in orthodontic education: a systematic review and meta-analysis. J. Dent. Educ. Jun 2009;73(6):730–739. 19491350 10.1002/j.0022-0337.2009.73.6.tb04752.x Rosenberg H Grad HA Matear DW : The effectiveness of computer-aided, self-instructional programs in dental education: a systematic review of the literature. J. Dent. Educ. May 2003;67(5):524–532. 12809187 10.1002/j.0022-0337.2003.67.5.tb03654.x Schott TC Arsalan R Weimer K : Students' perspectives on the use of digital versus conventional dental impression techniques in orthodontics. BMC Med. Educ. Mar 12 2019;19(1):81. 30866910 10.1186/s12909-019-1512-3 Ren Q Wang Y Zheng Q : Survey of student attitudes towards digital simulation technologies at a dental school in China. Eur. J. Dent. Educ. Aug 2017;21(3):180–186. 10.1111/eje.12198 Nagy ZA Simon B Tóth Z : Evaluating the efficiency of the Dental Teacher system as a digital preclinical teaching tool. Eur. J. Dent. Educ. Aug 2018;22(3):e619–e623. 29797383 10.1111/eje.12365 Liu L Li J Yuan S : Evaluating the effectiveness of a preclinical practice of tooth preparation using digital training system: A randomised controlled trial. Eur. J. Dent. Educ. Nov 2018;22(4):e679–e686. 29952122 10.1111/eje.12378 Chen SK Chen YJ Yao CC : Enhanced speed and precision of measurement in a computer-assisted digital cephalometric analysis system. Angle Orthod. Aug 2004;74(4):501–507. 10.1043/0003-3219(2004)074<0501:Esapom>2.0.Co;2 Farooq MU Khan MA Imran S : Assessing the Reliability of Digitalized Cephalometric Analysis in Comparison with Manual Cephalometric Analysis. J. Clin. Diagn. Res. Oct 2016;10(10):Zc20–zc23. 27891451 10.7860/jcdr/2016/17735.8636 Hongyu Chen CW : Wenli Lai The introduction of Dolphin software and its clinical application in orthodontics. Journal of Chinese Physician. 2015;17(04):611–613. Rock WP O'Brien KD Stephens CD : Orthodontic teaching practice and undergraduate knowledge in British dental schools. Br. Dent. J. Mar 23 2002;192(6):347–351. 15552073 10.1038/sj.bdj.4801371 Derringer KA : Undergraduate orthodontic teaching in UK dental schools. Br. Dent. J. Aug 27 2005;199(4):224–232. 16127406 10.1038/sj.bdj.4812615 Shahidi S Oshagh M Gozin F : Accuracy of computerized automatic identification of cephalometric landmarks by a designed software. Dentomaxillofac. Radiol. 2013;42(1):20110187. 23236215 10.1259/dmfr.20110187 Gupta A Kharbanda OP Sardana V : Accuracy of 3D cephalometric measurements based on an automatic knowledge-based landmark detection algorithm. Int. J. Comput. Assist. Radiol. Surg. Jul 2016;11(7):1297–1309. 26704370 10.1007/s11548-015-1334-7 Triepels CPR Smeets CFA Notten KJB : Does three-dimensional anatomy improve student understanding?. Clin. Anat. Jan 2020;33(1):25–33. 31087400 10.1002/ca.23405 Amer RS Denehy GE Cobb DS : Development and evaluation of an interactive dental video game to teach dentin bonding. J. Dent. Educ. Jun 2011;75(6):823–831. 21642529 10.1002/j.0022-0337.2011.75.6.tb05111.x Tian Pan Y : Evaluating the effect of digital technology on the learning of orthodontic cephalometric measurement. figshare.Dataset.2022. 10.6084/m9.figshare.21550599.v1 10.5256/f1000research.140872.r157996 Reviewer response for version 2 Corte-Real Ana 1 2 Referee https://orcid.org/0000-0003-2477-1857 Forensic Dentistry Laboratory, Faculty of Medicine, University of Coimbra, Coimbra, Portugal Faculty of Medicine, University of Coimbra, Coimbra, Portugal

Competing interests: No competing interests were disclosed.

 11 1 2023 Copyright: © 2023 Corte-Real A 2023 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

The authors improved the manuscript. However, some suggestions should be pointed out in the Methods section.

"All the students use (/used?) the Dolphin software before"… is it really what the authors intended to write?

The original information of the manuscript is its impact on orthodontic learning and students' new skills and satisfaction (instead of "student's acceptance towards it"). It is difficult to understand the sentence, "The collection and input of these data were completed by three postgraduates, with two postgraduates responsible for the inputting and the third one in charge of checking." It should be clarified. The student's contribution to the cephalometric analysis and their performance and satisfaction evaluations should be clear. 

Intra and inter-class correlations were based on the instructors' data. What is its impact on the present study?

Is the work clearly and accurately presented and does it cite the current literature?

No

If applicable, is the statistical analysis and its interpretation appropriate?

No

Are all the source data underlying the results available to ensure full reproducibility?

No

Is the study design appropriate and is the work technically sound?

No

Are the conclusions drawn adequately supported by the results?

No

Are sufficient details of methods and analysis provided to allow replication by others?

No

Reviewer Expertise:

Medical and Dentistry Higher Education and Health Quality

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Tian Yu Jiangsu Key Laboratory of Oral Diseases, China

Competing interests: No competing interests existed.

 20 2 2023

The authors improved the manuscript. However, some suggestions should be pointed out in the Methods section.

Response: We are grateful for the reviewer’s helpful suggestions. As shown below, we have addressed the reviewer’s concerns, and provided point-by-point responses. We hope that the changes meet your approval.

1. "All the students use (/used?) the Dolphin software before"… is it really what the authors intended to write?

Response: Thank you for pointing out this mistake. We have revised the sentence to “All the students have neither learned about the method of measuring the lateral cranial radiographs nor used the Dolphin software before.”

2. The original information of the manuscript is its impact on orthodontic learning and students' new skills and satisfaction (instead of "student's acceptance towards it").

Response: Thank you for your suggestion. According to your advice, we have revised the phrase to “orthodontic learning, students' new skills and satisfaction towards it”.

3. It is difficult to understand the sentence, "The collection and input of these data were completed by three postgraduates, with two postgraduates responsible for the inputting and the third one in charge of checking." It should be clarified. The student's contribution to the cephalometric analysis and their performance and satisfaction evaluations should be clear.

Response: Thank you for your suggestion. To make it clear, we have revised the sentence to “The collection, input and check of these data were completed by three authors.”

4. Intra and inter-class correlations were based on the instructors' data. What is its impact on the present study?

Response: Thank you for your question. In order to evaluate students’ performances, we needed standards. Thus, we invited the experienced instructors and set their measurement results as standards after checking the stability and consistency of their measurements.

 10.5256/f1000research.140872.r157997 Reviewer response for version 2 Maspero Cinzia 1 2 Referee https://orcid.org/0000-0002-5930-1698 Fondazione IRCCS Cà Granda, Ospedale Maggiore Policlinico, Milan, Italy Department of Orthodontics, Faculty of Medicine, University of Milan, Milan, Italy

Competing interests: No competing interests were disclosed.

 4 1 2023 Copyright: © 2023 Maspero C 2023 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve

All my requirements have been addressed. The manuscript has been improved and, in my opinion, it is suitable for indexing.

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Yes

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Partly

Are the conclusions drawn adequately supported by the results?

Partly

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

Orthodontic, Orthognati surgery, cephalometic, maxillary expansion, interceptive orthodontics, fixed appliances, clear aligners, pediatric dentistry

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

 10.5256/f1000research.121428.r135780 Reviewer response for version 1 Corte-Real Ana 1 2 Referee https://orcid.org/0000-0003-2477-1857 Forensic Dentistry Laboratory, Faculty of Medicine, University of Coimbra, Coimbra, Portugal Faculty of Medicine, University of Coimbra, Coimbra, Portugal

Competing interests: No competing interests were disclosed.

 7 9 2022 Copyright: © 2022 Corte-Real A 2022 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation reject

The topic is interesting for assessing learning from the student's perspective, but the title does not correspond to the content. The methodology section is the weakest part of the work. The authors should adjust the study design, including the analysis of intra and inter-observer accuracy and monitoring activities during the study. The selection of participants should include the inclusion and exclusion criteria as the previous use of digital technology for cephalometric measurements impacts the students' skills.

Is the work clearly and accurately presented and does it cite the current literature?

No

If applicable, is the statistical analysis and its interpretation appropriate?

No

Are all the source data underlying the results available to ensure full reproducibility?

No

Is the study design appropriate and is the work technically sound?

No

Are the conclusions drawn adequately supported by the results?

No

Are sufficient details of methods and analysis provided to allow replication by others?

No

Reviewer Expertise:

Medical and Dentistry Higher Education and Health Quality

I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.

Tian Yu Jiangsu Key Laboratory of Oral Diseases, China

Competing interests: No competing interests were disclosed.

 7 11 2022

Response: We are grateful for the reviewer’s helpful suggestions. As shown below, we have addressed the reviewer’s concerns, and provided point-by-point responses. We hope that the changes meet your approval.

1. The topic is interesting for assessing learning from the student's perspective, but the title does not correspond to the content.

Response: Thank you for your valuable suggestions. We fully understand and agree with your suggestion about the relationship between the title and content of this article. We’ve revised the title to make it more suitable for this study “Evaluating the effect of digital technology on the learning of orthodontic cephalometric measurement”.

2. The methodology section is the weakest part of the work. The authors should adjust the study design, including the analysis of intra and inter-observer accuracy and monitoring activities during the study.

Response: Thank you for your valuable suggestion. We added in Methods section that “The measurements by one instructor at one-week’s interval were used for calculating intra-class correlation coefficient (ICC intra), and measurements by two instructors were used for calculating inter-class correlation coefficient (ICC inter).”, and in Standards of cephalometric measurement section that “The inter- and intra-ICC were all surpassed 90%, and there were no significant differences between two instructors’ measurements, and the measurements by two methods”.

As regard to the monitoring activity, we’ve added that “During this pedagogical experiments, two instructors took responsibility for guiding students with cephalometric measurements, assuring the measurement order of two student groups, monitoring the process of giving out and recollecting the questionnaire, as well as assistance in the data analysis.” in the Monitoring activity section in page 8.

3. The selection of participants should include the inclusion and exclusion criteria as the previous use of digital technology for cephalometric measurements impacts the students' skills.

Response: Thank you for your suggestions. We agree with your advice on the selection of participant students. We’ve added in “Participants” section in line 5, page 6 that “…All the students did not learn about the method of measuring the lateral cranial radiographs and used the Dolphin software before”.

 10.5256/f1000research.121428.r148740 Reviewer response for version 1 Maspero Cinzia 1 2 Referee https://orcid.org/0000-0002-5930-1698 Fondazione IRCCS Cà Granda, Ospedale Maggiore Policlinico, Milan, Italy Department of Orthodontics, Faculty of Medicine, University of Milan, Milan, Italy

Competing interests: No competing interests were disclosed.

 7 9 2022 Copyright: © 2022 Maspero C 2022 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

Thank you for submitting your research. The manuscript is interesting but some changes are necessary before taking it into consideration for publication. Here are my concerns:

ABSTRACT: "Besides, the results of most items by digital technology were closer to the standards than those by the traditional method" What do you mean with standards?

INTRODUCTION: replace the word subdiscipline with discipline.

"These factors may not only affect accuracy of measurements, but also contribute to destruction of students’ enthusiasm for further learning." Are you sure with this sentence?

STUDY DESIGN: pre-test instruction. Please better explain what do you mean and what instruction have been given.

FLOW CHART: anatomical point calibration: calibrated their standard. What do you mean? Specify which standard.

CEFALOMETRIC MEASUREMENTS: it is not clear if the same lateral teleradiography have been used for all the students. Better specify.

Illustration of 15 commonly used measurement items: which kind of tracing have been used?

Please better specify the aim of this study.

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Yes

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Partly

Are the conclusions drawn adequately supported by the results?

Partly

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

Orthodontic, Orthognati surgery, cephalometic, maxillary expansion, interceptive orthodontics, fixed appliances, clear aligners, pediatric dentistry

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Tian Yu Jiangsu Key Laboratory of Oral Diseases, China

Competing interests: No competing interests were disclosed.

 7 11 2022

Thank you for submitting your research. The manuscript is interesting but some changes are necessary before taking it into consideration for publication. Here are my concerns:

Response: We are grateful for the reviewer’s helpful suggestions. As shown below, we have addressed the reviewer’s concerns, and provided point-by-point responses. We hope that the changes meet your approval.

1. ABSTRACT: "Besides, the results of most items by digital technology were closer to the standards than those by the traditional method" What do you mean with standards?

Response: Thank you for your valuable suggestion. The means of two instructors’ measurements were set as the standards, and we’ve added this in the abstract (line 4 in Results, page 2).

2. INTRODUCTION: replace the word subdiscipline with discipline.

Response: Thank you for your suggestion. According to your advice, we’ve used the word “discipline” in line 1, page3.

3. "These factors may not only affect accuracy of measurements, but also contribute to destruction of students’ enthusiasm for further learning." Are you sure with this sentence?

Response: Thank you for your advice. We’ve deleted the sentence.

4. STUDY DESIGN: pre-test instruction. Please better explain what do you mean and what instruction have been given.

Response: Thank you for your suggestion. We’ve checked the manuscript, and described the pre-test instruction in detail in page 6, which included 7 credit hours and let students learn about the 15 cephalometric items and two measurement methods.

5. FLOW CHART: anatomical point calibration: calibrated their standard. What do you mean? Specify which standard.

Response: Thank you for your valuable suggestions. We revised the subtitle and described in detail that “Before the course, the inter- and intra-class correlation coefficient of two instructors’ measurements were tested. Firstly, two orthodontists reviewed and discussed the locations of anatomical points. Then, they measured a lateral cranial radiograph by traditional and software methods. One week later, they measured the same radiograph by two methods again. The measurements by one instructor at one-week’s interval were used for calculating intra-class correlation coefficient (ICC intra), and measurements by two instructors were used for calculating inter-class correlation coefficient (ICC inter)” in Inter- and intra-class correlation coefficient of two instructors’ measurements section.

6. CEFALOMETRIC MEASUREMENTS: it is not clear if the same lateral teleradiography have been used for all the students. Better specify.

Response: Thank you for your suggestion. We’ve added in line 2, page 7 that “One week after the end of the pre-test instruction, 94 students were randomly allocated into two groups through the RAND function in Excel software (Microsoft® Excel® 2019MSO (2201 Build 16.0.14827.20198 version for 64 bit) and were required to complete cephalometric measurement of the same lateral cranial radiograph by both traditional method and digital technology (the Dolphin Imaging® 11.8).”.

7. Illustration of 15 commonly used measurement items: which kind of tracing have been used?

Response: Thank you for your question. The tracing was completed with the Uceph software, and we’ve added “The profile tracing was completed with the Uceph software (Uceph, Chengdu, China).” in the illustration of Figure 2.

8. Please better specify the aim of this study.

Response: Thank you for your suggestion. We’ve revised in abstract and introduction section that “This study aimed to evaluate the effect of applying digital technology in cephalometric measurement teaching and students’ acceptance towards it”.