Evaluating the effect of digital technology on the learning of orthodontic cephalometric measurement

Background: This study aimed to evaluate the effectiveness of integrating digital technology into cephalometric measurement teaching. Methods: In total, 94 undergraduates of stomatology were recruited and randomly allocated to two groups. According to the cross-over design, both groups completed cephalometric measurements by the traditional hand-drawn method and digital technology (the Dolphin software) in different order. In the traditional hand-drawn method, students depicted the outline of the craniofacial anatomical structures on sulfuric transfer paper first, then marked the measurement points and completed the measurement of line spacings and angles; by digital technology, they marked the points in the software and adjust the automatically generated outlines of the structures to obtain the results. Two professional orthodontists were invited as instructors and their measurements were set as standards. An online questionnaire was also used to investigate students' attitudes toward digital technology being used in the cephalometric teaching process. Results: There were significant differences of students’ measurements ( P 1-SNA<0.01, P 1-SNB=0.01 and P 1-L1-NB (mm)<0.01; SNA: sella-nasion-subspinale angle, SNB: sella-nasion-supramental angle, L1-NB (mm): the distance from the lower central incisor tip to the nasion-supramental plane) between the traditional method and digital technology. Besides, the results of most items by digital technology were closer to the standards than those by the traditional method, including five items with statistical significance ( P 2-SNB<0.05, P 2-L1-NB (mm)<0.01, P 2-FMA<0.05, P 2-FMIA<0.05, P 2-IMPA<0.01), while three items were the opposite ( P 2-SNA<0.05, P 2-ANB (mm)<0.01, P 2-NA-PA<0.01). The questionnaire showed more students preferred digital technology (33%) as a better teaching method than the traditional method (2%) and 72% of participants thought they had mastered 50-80% of cephalometric knowledge after the course. Conclusions: This study demonstrated effectiveness and acceptance of the course applying digital technology during the cephalometric teaching process.


Introduction
Orthodontics, as a discipline of stomatology, aims to study various kinds of malocclusion, including deformities of teeth, jaws and the craniofacial region. 1 Diagnosis is the most important part of orthodontic clinical work, among which cephalometric measurement is an essential procedure. 2Nevertheless, cephalometric measurement indicators are numerous and complicated.In the past, the traditional hand-drawn method was applied, which required reading lamps, sulfuric acid transfer paper, dividers and so on.In addition, previous studies suggested that undergraduates tended to show less confidence in reading and measuring lateral cranial radiographs. 3 continuous developments are seen in digital technology, such technology has been applied in the education for undergraduates in recent decades.5][6] Buchanan JA et al. found that before starting clinical work, students could better master theoretical knowledge through simulated operation or computer-aided learning method. 7,80][11][12][13] Digital technology has offered great potential for dental education as well. 14For instance, Nagy ZA et al. reported that the Dental Teacher software could help students more efficiently learn the preparation technique of onlay restorations and facilitate their individual performances. 15Liu L et al. also found the digital training system might be a good alternative to the traditional training method in the preclinical practice of tooth preparation. 16e digital cephalometric analysis system, widely incorporated in intelligent software, was developed to computerize the manual tasks and output the specific results automatically.It was reported to be more time-saving than traditional measurement method and helpful for reducing unnecessary errors during the measurement process. 17Farooq et al. also found that the accuracy of cephalometric measurement by digital tracing with FACAD ® was similar with the manual method.Furthermore, its advantages of digital imaging, such as quality improvement, file transmission and archiving made digitalized cephalometric analysis preferrable in daily use. 18e Dolphin software ® (Dolphin Imaging & management solution, America) is widely applied in the field of orthodontics, possessing functions like storage and management of patients' information and images.After users upload computerized tomography photographs, it can also achieve three-dimensional imaging, cephalometric measurement and treatment effect prediction.It incorporates more than 400 cephalometric analytical methods.This software has been reported to have the potential as an animation textbook for medical college students. 19Although it was assumed to exert some positive effects in the teaching process, there is still lack of research investigating the effectiveness of applying it in orthodontic cephalometric teaching process.Thus, the aim of this study was to examine the effectiveness of applying digital technology (the Dolphin software ® ) in teaching cephalometric measurement.

Ethical considerations
This study followed the guidelines of the Nanjing Medical University ethics review committee and received the approval of the committee (approval number: PJ2019-053-001).All participants gave written informed consent.

Study size
G*Power software version 3.1.9.7 (RRID:SCR_013726) was used to estimate the required sample size for this study.This study used two independent t tests to calculate the number of students needed.The study power was set at 90% and alpha value set at 0.05.Based on these, a minimum sample of 86 subjects was required.

Participants
This study approached fourth-year undergraduate students of Stomatology in Nanjing Medical University by inviting them to attend this course, followed by their voluntary registration.All the students, consisting of 63 female and 31 male students (around 21-23 years old), agreed to participate in this pedagogical experiment and signed the informed consent form.All the students have neither learned about the method of measuring the lateral cranial radiographs nor used the Dolphin software before.We also invited two orthodontist faculty members with over 5 years of clinical experience as instructors.

REVISED Amendments from Version 2
To make the article more easily to understand, some minor mistakes and phrases have been revised.

Study design
This study was conducted from May 12, 2020 to June 16, 2020.The flow diagram of this pedagogical study was shown in Figure 1.For students, it was divided into pre-test learning, group and cephalometric measurement, and questionnaire survey.For instructors, it was divided into monitoring activity, cephalometric measurement and inter-and intra-class correlation coefficient of their measurements.

Pre-test learning
The total teaching period consisted of 7 credit hours, including 4 credit hours of theoretical class and 3 credit hours for practical instruction.During the theoretical class, one instructor imparted relevant knowledge to the students in detail, including the positions of anatomical markers and the meanings of commonly used measurement items.
In the practical instruction class, another instructor guided 94 students to review the basic knowledge and showed them how to complete cephalometric measurement by the traditional method and digital technology (the Dolphin software ® ).For the traditional method, the sulfuric acid transfer paper was fixed to the radiograph with a clip.Then, the patient's soft tissue profile and hard tissue anatomical structures were depicted on the reading lamp.Finally, the commonly used anatomical points were identified on the sulfuric acid transfer paper and the measurement was completed with the ruler and protractor.
While using digital technology, the instructor adjusted results of line spacing on the lateral cranial radiograph to their actual size at first.Then, the instructor accomplished the measurement by adjusting the gray contrast value and other auxiliary methods.After learning the relevant knowledge of commonly used cephalometric measurement points and items, students were encouraged to review relevant contents after class.

Group and cephalometric measurement
One week after the end of the pre-test instruction, 94 students were randomly allocated into two groups through the RAND function in Excel software (Microsoft ® Excel 2019MSO (2201 Build 16.0.14827.20198version for 64 bit) and were required to complete the same cephalometric measurement by both traditional method and digital technology (the Dolphin Imaging ® 11.8).
According to the cross-over design, one group took the traditional method first to complete the measurement and then used the Dolphin software ® , while the other group completed in the opposite order.A total of 15 cephalometric items (Figure 2) were measured, such as the angle between Sella-Nasion plane and the Nasion-Subspinale plane (SNA), the angle between Sella-Nasion plane and the Nasion-Supramental plane (SNB), the angle between the Nasion-Subspinale plane and the Nasion-Supramental plane (ANB).The measurements of the traditional method and Dolphin software ® were recorded, respectively.The collection, input and check of these data were completed by three authors.

Questionnaire survey
In order to survey the effectiveness of applying the digital software in cephalometric teaching and students' attitudes toward it (the Dolphin software ® ), we designed an online questionnaire and collected results by Wenjuanxing (Ranxing Co. Ltd., Changsha, China).For example, to investigate how difficult students considered cephalometry is, we set three options ranging from "very difficult", "kind of tough" to "easy".As to the mastery degree of students after the course, the options were "50-80%", "20-50%" to "0-20%".Gender of the participants was recorded from the university records.The detailed questionnaire list and corresponding options are shown in Table 1.

Monitoring activity
During this pedagogical experiments, two instructors took responsibility for guiding students with cephalometric measurements, assuring the measurement order of two student groups, monitoring the process of giving out and recollecting the questionnaire, as well as assistance in the data analysis.

Cephalometric measurement & Inter-and intra-class correlation coefficient test
Two instructors also measured the same radiograph by two methods.One week later, they measured the same radiograph by two methods again.The measurements by one instructor at one-week's interval were used for calculating intra-class correlation coefficient (ICC intra ), and measurements by two instructors were used for calculating inter-class correlation coefficient (ICC inter ).

Statistical analysis
The quantitative data of cephalometric measurements were analyzed by the statistical software SPSS 18.0 (IBM Corporation, Armonk, NY, RRID:SCR_016479).The measurements of students by two methods were compared using the independent sample t test, as well as comparing them with corresponding standards, respectively, with the level of significance set as P<0.05.
As for questionnaire data, we obtained the statistical data through the built-in function on the online questionnaire platform, as it provided the constituent ratio of each option and participants list.Then we performed a descriptive analysis of these results.

Anatomical points calibration
Before the course, the standard of anatomical markers on lateral cranial radiographs were calibrated.Firstly, two orthodontists reviewed the basic definitions and meanings of common anatomical points and items.Then, they measured a lateral cranial radiograph on paper and by software, followed by discussing and unifying the anatomical position standard.Subsequently, they respectively completed cephalometric measurement of another lateral cranial radiograph.The intraclass correlation coefficient (ICC) was greater than 90%, showing consistency of cephalometric measurement between them.

Standards of cephalometric measurement
As shown in Supplementary Table 1 and 2, the inter-and intra-ICC were all surpassed 90%, and there were no significant differences between two instructors' measurements, and the measurements by two methods.As the result, the means of two instructors' measurements were set as the standards.SNA: the angle between the sella-nasion plane and the nasion-subspinale plane; SNB: the angle between sella-nasion plane and the nasion-supramental plane; ANB: the angle between the nasion-subspinale plane and the nasion-supramental plane; NP-FH: the posteriorinferior angle between the facial plane and the Frankfort horizontal plane; NA-PA: the angle between the nasion-subspinale plane and the pogonion-subspinale plane; U1-NA (mm): the distance from the upper central incisor tip to the nasion-subspinale plane; U1-NA: the angle between the upper central incisor and the nasion-subspinale plane; L1-NB (mm): the distance from the lower central incisor tip to the nasion-supramental plane; L1-NB: the angle between the lower central incisor and the nasion-supramental plane; U1-L1: the angle between the upper central incisor and the lower central incisor; Y axis angle: the anterior-inferior angle between the Frankfort horizontal plane and the sella-pogonion plane; Po-NB (mm): the distance from the pogonion point to the nasion-supramental plane; FMA: the angle between the Frankfort horizontal plane and the mandibular plane; FMIA: the angle between the Frankfort horizontal plane and the long axis of the lower central incisor; IMPA: the angle between the long axis of the lower central incisor and the mandibular plane.
Accuracy of students' cephalometric measurement Statistically significant differences were observed in measurements of SNA, SNB and the distance from the lower central incisor tip to the nasion-supramental plane (L1-NB (mm)) between the traditional method and digital technology (P 1-SNA <0.01, P 1-SNB =0.01, P 1-L1-NB (mm) <0.01) (Table 2), while other items showed no significant differences.Besides, the measurements by digital technology were closer to the standard values than those by the traditional method.The accuracy of five items measurements using digital technology was significantly higher, including SNB, L1-NB (mm), the angle between the Frankfort horizontal plane and the mandibular plane (FMA), the angle between the Frankfort horizontal plane and the long axis of the lower central incisor (FMIA), and the angle between the long axis of the lower central incisor and the mandibular plane (IMPA) (P 2-SNB <0.05, P 2-L1-NB (mm) <0.01, P 2-FMA <0.05, P 2-FMIA <0.05, P 2-IMPA <0.01).However, five items presented the opposite result, among which three items were statistically significant (SNA, ANB and the angle between the Nasion-Subspinale plane and the pogonion-subspinale plane (NA-PA) (P 2-SNA <0.05, P 2-ANB <0.01, P 2-NA-PA <0.01).

Attitudes of students towards the digital technology
We assigned the questionnaires to all the participants with 82 of 94 students filling out the questionnaire and the response rate was 87%.The statistical results of the questionnaire are shown in Table 1.Among the respondents, 66% thought studying cephalometry was very difficult and 21% thought it was kind of tough.After instruction, review and practice, 72% of them considered they had mastered 50-80% of relevant knowledge and a few students thought they had mastered 20-50%.About 33% of students preferred the digital technology than traditional method (2%) as a better teaching method and 65% held that both were acceptable, which indicated good acceptance by students of digital technology applied in the teaching process.In addition, 98% of participants expressed their interest in studying cephalometry and considered cephalometric analysis helpful to diagnosis of malocclusion.

Discussion
Cephalometric measurement is essential for diagnosis and treatment plan design of patients with malocclusions, which are the most significant procedures in orthodontic clinical work.Orthodontic educators put forward that orthodontic teaching for undergraduates should focus on diagnosis and recognition of problems. 20However, there is not a generally accepted teaching method, and a wide variation of course durations and contents exist in different dental colleges and faculties. 21How to arouse students' interest and achieve better teaching effects is a major problem faced by orthodontic educators.Since only a few reports explored this aspect, we designed this didactical experiment.
In our study, three cephalometric items measured by two methods were statistically different, while other items were basically similar.These results implied that digital technology could achieve similar results to the traditional method during cephalometric measurement.Additionally, the majority of items measured by digital technology were closer to the standards, including five statistically significant items, which suggested students could achieve more accurate results by digital technology.This may be attributed to the function of digital software to adjust the gray contrast value of X-ray films (Supplementary figure 1), making it easier to identify the unclear points on the printed paper.In addition, the automatic generation of results also helped to avoid evitable errors during manual measurement.These results were in accordance with previous studies, which suggested that the accuracy of digital measurement on 3-dimensional cone beam computed tomography images was basically similar to or even higher than that of manual measurement. 22,23wever, in spite of the convenience it provided, the digital technology may lead to a lack of deep understanding of corresponding contents, such as definitions and meanings of these measurement items.The traditional method could better cultivate practical abilities of students and enhance their memory of relevant knowledge.Besides, although the accuracy of some items obtained by digital technology were significantly higher, a few items showed the opposite result (SNA, ANB and NA-PA).The subspinale point (the A point) was associated with these three items, which suggested that the traditional method was more accurate than digital technology in positioning the subspinale point on the lateral cranial radiograph.The subspinale point is the most concave point of the arc from the anterior nasal spine point (the ANS point) to the superior prosthion point (the Spr point).Compared with digital measurement, the advantage of manual measurement is that the arc can be traced on paper, and some auxiliary instruments like ruler and protractor can help to locate the subspinale point (Supplementary figure 2).These results indicated abundant experience was required to identify the subspinale point, reminding both orthodontic educators and students to devote more time and energy to deep learning it.
Before the course, the majority of students showed fear towards abstract and complex concepts, assuming cephalometric measurement was difficult to master.However, with instruction and practice, most students could master 50-80% of relevant knowledge, which could result from digital technology realizing visualization of numerous and complicated anatomical markers.Previous studies found that visualization was extremely attractive to young students, and significantly aroused their interest and sense of participation. 24,25Our survey confirmed that more students preferred digital technology (33%) as a better teaching method than the traditional method.As a result, the application of digital technology in teaching cephalometric measurement was widely accepted by students and contributed to favorable teaching results.
This study still had some limitations.For example, the sample size could be further enlarged.Secondly, the measurement time for students were not strictly required, which may have resulted in the underperformances or supernormal performances of students.Thirdly, exploration of student's attitudes towards digital technology being applied in this course was not sufficient.The mentioned issues needed to be improved may interrupt us from to accurately assessing the real advantages and disadvantages of the appliance of this digital technology.These findings could guide and encourage university orthodontic teachers to apply this technology in cephalometric teaching and pay more attention to considering the position identification of the subspinale point.

Conclusion
This study investigated the effect of applying the digital technology (the Dolphin software ® ) in cephalometric learning, demonstrating better effect and acceptance of this technology.

Data availability
Extended data Figshare: Evaluating the effect of digital technology on the learning of orthodontic cephalometric measurement, https:// doi.org/10.6084/m9.figshare.21550599. 26is project contains the following extended data: -Supplementary File.docx (Supplementary figure 1, 2; Supplementary table 1,  The original information of the manuscript is its impact on orthodontic learning and students' new skills and satisfaction (instead of "student's acceptance towards it").It is difficult to understand the sentence, "The collection and input of these data were completed by three postgraduates, with two postgraduates responsible for the inputting and the third one in charge of checking."It should be clarified.The student's contribution to the cephalometric analysis and their performance and satisfaction evaluations should be clear.Response: Thank you for your suggestion.We've revised in abstract and introduction section that "This study aimed to evaluate the effect of applying digital technology in cephalometric measurement teaching and students' acceptance towards it".
Competing Interests: No competing interests were disclosed.
The benefits of publishing with F1000Research: Your article is published within days, with no editorial bias • You can publish traditional articles, null/negative results, case reports, data notes and more • The peer review process is transparent and collaborative • Your article is indexed in PubMed after passing peer review • Dedicated customer support at every stage • For pre-submission enquiries, contact research@f1000.com

Figure 1 .
Figure 1.Flow diagram of the teaching experiment.

Figure 2 .
Figure 2. Illustration of 15 commonly used measurement items.1. SNA: the angle between the sella-nasion plane and the nasion-subspinale plane; 2. SNB: the angle between sella-nasion plane and the nasion-supramental plane; 3. ANB: the angle between the nasion-subspinale plane and the nasion-supramental plane; 4. NP-FH: the posteriorinferior angle between the facial plane and the Frankfort horizontal plane; 5. NA-PA: the angle between the nasionsubspinale plane and the pogonion-subspinale plane; 6. U1-NA (mm): the distance from the upper central incisor tip to the nasion-subspinale plane; 7. U1-NA: the angle between the upper central incisor and the nasion-subspinale plane; 8. L1-NB (mm): the distance from the lower central incisor tip to the nasion-supramental plane; 9. L1-NB: the angle between the lower central incisor and the nasion-supramental plane; 10.U1-L1: the angle between the upper central incisor and the lower central incisor; 11.Y axis angle: the anterior-inferior angle between the Frankfort horizontal plane and the sella-pogonion plane; 12. Po-NB (mm): the distance from the pogonion point to the nasionsupramental plane; 13.FMA: the angle between the Frankfort horizontal plane and the mandibular plane; 14.FMIA: the angle between the Frankfort horizontal plane and the long axis of the lower central incisor; 15.IMPA: the angle between the long axis of the lower central incisor and the mandibular plane.The profile tracing was completed with the Uceph software (Uceph, Chengdu, China).

2 )-
The results of students' measurements.xlsx-Statistical results of the questionnaire.xlsxData are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

Table 1 .
Students' attitudes towards the application of digital technology.

Table 2 .
Comparison of students' measurements and standard values by the digital technology and traditional method.P 1 : The P value of the independent sample t test on students' measurements by two methods; P 2 : The P value of the independent sample t test on absolute values of differences between students' results and the standards by two methods.

have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
No competing interests were disclosed.Department of Orthodontics, Faculty of Medicine, University of Milan, Milan, Italy 2 Fondazione IRCCS Cà Granda, Ospedale Maggiore Policlinico, Milan, Italy Thank you for submitting your research.The manuscript is interesting but some changes are necessary before taking it into consideration for publication.Here are my concerns:ABSTRACT: "Besides, the results of most items by digital technology were closer to the standards than those by the traditional method" What do you mean with standards?These factors may not only affect accuracy of measurements, but also contribute to destruction of students' enthusiasm for further learning."Are you sure with this sentence?
○Competing Interests: 1 ○ INTRODUCTION: replace the word subdiscipline with discipline.○ "○ FLOW CHART: anatomical point calibration: calibrated their standard.What do you mean?Specify which standard.○ CEFALOMETRIC MEASUREMENTS: it is not clear if the same lateral teleradiography have been used for all the students.Better specify.○ Illustration of 15 commonly used measurement items: which kind of tracing have been used?○ Please better specify the aim of this study.○ Is

the work clearly and accurately presented and does it cite the current literature? Yes Is the study design appropriate and is the work technically sound? Partly Are sufficient details of methods and analysis provided to allow replication by others? Yes If applicable, is the statistical analysis and its interpretation appropriate? Yes Are all the source data underlying the results available to ensure full reproducibility? Yes Are the conclusions drawn adequately supported by the results? Partly
Orthodontic, Orthognati surgery, cephalometic, maxillary expansion, interceptive orthodontics, fixed appliances, clear aligners, pediatric dentistry I Competing Interests: No competing interests were disclosed.Reviewer Expertise:

confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. for
calculating intra-class correlation coefficient (ICC intra ), and measurements by two instructors were used for calculating inter-class correlation coefficient (ICC inter )" in Inter-

and intra-class correlation coefficient of two instructors' measurements section. 6. CEFALOMETRIC MEASUREMENTS: it is not clear if the same lateral teleradiography have been used for all the students. Better specify. Response:
Thank you for your suggestion.We've added in line 2, page 7 that "One week after the end of the pre-test instruction, 94 students were randomly allocated into two groups through the RAND function in Excel software (Microsoft® Excel® 2019MSO (2201 Build 16.0.14827.20198version for 64 bit) and were required to complete cephalometric measurement of the same lateral cranial radiograph by both traditional method and digital technology (the Dolphin Imaging® 11.8).". 7.

Illustration of 15 commonly used measurement items: which kind of tracing have been used? Response:
Thank you for your question.The tracing was completed with the Uceph software, and we've added "The profile tracing was completed with the Uceph software (Uceph, Chengdu, China)." in the illustration of Figure 2.
8. Please better specify the aim of this study.