Introduction of structured physical examination skills to second year undergraduate medical students

Introduction: Effective learning of physical examination skills (PES) requires suitable teaching and learning techniques and assessment methods. The Tribhuvan University (Nepal) curriculum recommends involving the departments of Medicine and Surgery in PES training (PEST) for second year students as a part of early clinical exposure. The project was developed to make teaching/learning of PES structured, involving eight clinical sciences departments and using appropriate methods for teaching and assessment in KIST Medical College, Nepal. Methods: Irby’s three stages of clinical teaching model (Preparation, Teaching, Reflection), was applied for teaching. Skill acquisition was based on Millers’ learning pyramid at “show how level” and Dreyfus’ competency model at “competent level”. Teaching/learning was conducted in small groups. A tutorial, demonstration and practice (TDS) model was developed for teaching/learning techniques based on a simple five-step method for teaching clinical skills. Assessment of effectiveness of training was done at “reaction level” as per Kirkpatrick’s model based on students’ feedback, “shows how level” as per Miller’s pyramid of learning by OSCE and “competent level” as per Dreyfus’ model using retro-pre questionnaire. Results: The analysis of retro-pre questionnaire based on the Dreyfus model found the average skill score (max score 184), before the introduction of the project module as 15.9 (median = 13.5) and after as 116.5 (median = 116). A paired t-test showed the difference to be statistically significant (100.5±23 and 95% CI 95.45 – 105.59). The average overall feedback score for the students on PES training based on seven items on a five point Likert scale was found to be 4.30. The mean total objective structured clinical examination (OSCE) score was 3.77 (SD+/- 0.33) out of 5; 80% of students scored more than 70%. Conclusion: Students learned most of the skills with the implementation of the structured PES module and did well in the OSCE. Students and faculty were satisfied with the training and assessment.

The project was developed to make teaching-learning of PES structured, and involved eight clinical departments using appropriate teaching, learning and assessment methods. Students are provided with an opportunity to learn basic physical examination skills in gynecology and obstetrics, orthopedics, ear, nose and throat (ENT), ophthalmology and pediatrics, as well as general medicine and surgery in their second year Bachelor of Medicine and Bachelor of Surgery degrees (MBSS) so that they get sufficient time to learn reasoning, diagnostic, procedural, therapeutic and counseling skills during their clinical years (third, fourth and final year). The objective of the project was that at the end of the second year, students should be able to initiate and perform a basic physical examination of an adult suffering from medical, surgical, gynecology and obstetric, orthopedic, ENT and eye diseases, as well as a basic physical examination of a child.

I. Development of the module
Faculty members of departments involved in the project identified basic PES to be learnt by students. A checklist for each selected PES was prepared based on Hutichson's Clinical Methods (22nd edition) 4 and was peer reviewed and finalized by a core project committee.

II. Orientation of faculty members
All faculty members involved in teaching received teacher training before commencement of the module. They were oriented with regards to the implementation of the project in a mini-workshop. The details regarding grouping of students, the posting schedule of various groups in different departments in rotation, the approach to teaching and learning, the teaching-learning strategy and the assessment modalities were also shared with them.

III. Approach to teaching
Physical examination skills involve psychomotor skills. For teaching physical examination skills, Irby's three stages of clinical teaching were applied 5 . These are: preparation (stage I), teaching (stage II) and reflection (stage III).

IV. Approach to learning
Skills acquisition was based on Millers' Learning Pyramid 6 at the "show how level" and Dreyfus' competency model 6 at the "competent level". Miller's four levels of learning are: 1) Whether the learner has knowledge of the skill; 2) Whether the learner knows how the skill is performed; 3) Whether the learner shows how to perform the skill in a controlled or simulated setting; and 4) Finally, whether the learner actually does the skill in clinical practice.
The basic principle of the Dreyfus model is that the student progresses through five stages of proficiency in this specific order: novice, advanced beginner, competent, proficient, and expert 6 .
V. Teaching-learning strategies Teaching-learning was conducted in small groups. The one hundred students were divided into seven groups of 14-15 students; each group was further divided into two subgroups of seven or eight students. Each group was posted for four weeks each in Medicine Units I and II, surgery, pediatrics, and gynecology and obstetrics, and each subgroup for two weeks in family medicine, ENT, ophthalmology and orthopedics in rotation. Students learned PES related to the cardiovascular system (CVS) and respiratory system (RSS) in Medicine Unit I, the peripheral nervous system (PNS) and central nervous system (CNS) in Medicine Unit II, the examination of the abdomen in Surgery, the musculoskeletal system (MS) in orthopedics, general PES in family medicine, obstetrics and gynecology examination in the obstetrics and gynecology department, ear, nose and throat examination in ENT and eye examination in ophthalmology. Structured PEST (S-PEST) sessions were held for four hours every Monday for 28 weeks between February and August 2011.
Based on the method used for teaching clinical skills in the American College of Surgeon's advanced trauma life support course, a tutorial, demonstration and practice (TDP) model was developed. Each S-PEST session had three sub-sessions: Tutorial (T), Demonstration (D) and Practice (P). The 'Tutorial' element covered the overview by the faculty preceptor on skills to be taught; 'Demonstration' involved actually demonstrating each of the skills taught with a stepwise description, while 'Practice' involved performance/practice of each demonstrated skill by the students using a sequential description to be observed by the preceptor. This model follows five (conceptualization, visualization, verbalization, practice and correction and reinforcement) of the seven psychomotor teaching principles based on the taxonomy of psychomotor domain (the other two being skill mastery and skill autonomy) 7 .
In most sessions, demonstration and performance/practice were conducted on real patients either in the ward or outpatient department (OPD). Some sessions were conducted on simulated patients, while in a few sessions the students themselves consented to be simulated patients. The following instruments were used for assessment:

VI. Assessment
1. The retro-pre questionnaire for assessing learners' self-reported changes. The retrospective post-then-pre design is a popular way to assess learners' self-reported changes in knowledge, awareness, skills, confidence, attitudes or behaviors. It takes less time, is less intrusive and for self-reported change, avoids pre-test sensitivity and response shift bias that result from pre-test overestimation or underestimation 11 . 2. A feedback questionnaire to assess the perception of teaching and learning sessions of S-PEST from students and faculty members. 3. The OSCE was used for the end of the posting assessment. Standardized patients (SP) were used in the OSCE. They were trained to follow students' commands for various aspects of the physical examination. SP were healthy individuals from our house keeping department who consented to be SP. They were given prior briefing regarding appropriate mannerisms and how to respond to students' commands during OSCEs. A faculty observer at each station used a checklist to rate each student's performance.
The following components were developed by the station authors for each OSCE station:  An instruction sheet for the examinee.  A checklist for the assessment of the skill being examined at that station.
 A detailed patient profile for the standardized patient.  A list of the equipment, instruments etc required at the station.

VII. Data management and analysis
Data was analysed using SPSS version 18.0.

VIII. Ethical considerations
The institutional Research Committee of KIST Medical College approved the project.

Results
A. Students' self-reported changes in perceived skill levels using the retro-pre questionnaire Forty-six skills, representing various systems in different departments during the training were included in the retro-pre questionnaire. Two to four skills from the 'must know' category from each subject/chapter were included in this questionnaire. Each skill was scored out of 4. Individual skill scores were added to get overall scores. A paired t-test was used to evaluate the difference in overall scores before and after the module. Scores were found to follow a normal distribution as confirmed by a Shapiro-Wilk test.
The average perceived skills score (the maximum score being 184) before the module was 15.9, which increased to 116.5 after the module. Students perceived that their level of skill improved after the module. The result from the paired t-test showed that the difference is highly statistically significant (mean 100.5 with SD+/-23 and 95% confidence interval 95.45 -105.59), which means that students did learn most of the skills after the Structured Physical Examination Skills Training (S-PEST) module and it did influence them.

B. Feedback of students regarding S-PEST
Around 75% of students filled in the feedback questionnaire which used a five point Likert scale. The questions were on the objectives of the session, facilitator/preceptor role, satisfaction with learning activities in each sub-session, overall rating of session and there were two open ended questions ("Suggestion/s for improvement" and "Any other comment/s"). Respondents gave scores out of 5, the higher the score, the higher the satisfaction.
The average scores of the different questions and global scores (overall session ratings) were calculated. The average feedback score was found to be 4.30 (maximum score 5) and the overall global score was 8.34 (maximum score 10) (see Figure 1 for scores and average scores). The agreement between global (subjective) and overall items average (objective) scores were found to be 78% (Pearson's Correlation).
Almost all respondents recognized the importance of each subsession (Tutorial, Demonstration and Practice). The most frequent remark obtained from the open-response category question was to "provide more time for practice".  I & II, MS, General PE, Eye, and ENT). The mean total OSCE score obtained by students in each station was 3.77 with a standard deviation (SD+/-) of 0.33 (the maximum score was 5). Eighty percent of the students scored more than 70% (26 students scored more than 80%, 55 students between 70 and 80%, and 15 students between 60% and 70%). A graph of the OSCE scores is shown in Figure 2.

Discussion
Clinical skills acquisition is a major focus of education for health professionals extending from undergraduate to postgraduate and continuing to professional education 12 .
The current trend in medical education is to introduce clinical teaching early, within the first two years of the curriculum, to help  students understand the relevance of the basic sciences to clinical practice and to provide instruction in basic clinical skills in a standardized fashion 13 .
Following these trends, Tribhuvan University (TU), Kathmandu, included early clinical exposure (ECE) in the curriculum, revised in 2008. The clinical examination, a required course for second year students, concentrates on the teaching of physical examination skills in two departments (Medicine and Surgery) with no defined method of teaching/learning and assessment 3 .
The importance of structured clinical education has long been recognized. It provides equal learning opportunities and a suitable environment for everyone to acquire clinical skills and competencies. Modules are especially suitable for outcome-based adult learning programs and maximizing adult learning 14,15 . With this purpose in mind, a teaching/learning module was developed in this project at the KIST Medical College Nepal KISTMC (affiliated to TU) for teaching basic physical examination skills to second year students as part of early clinical exposure. KISTMC involved 8 clinical science departments and made teaching, learning and assessment structured.
Teaching-learning was conducted in small groups as small group teaching and learning is considered effective in clinical settings for tutorials and demonstrations 16 .
Physical examination skills are largely psychomotor skills. For teaching physical examination skills, Irby's three stages of clinical teaching were applied (Preparation, Teaching and Reflection) 5,17 . Though faculty members and students reflected on experiences at the end of each session, these reflections could not be recorded 5 .
All the faculty members involved in teaching received teachertraining before commencement of the course and were oriented about the implementation of the project 18 .
Students were well informed about the project implementation but a limitation was that the students' stage of competency could not be assessed at the beginning of the project (this is why the retro-pre questionnaire was used). Faculty members too were generally satisfied with the S-PEST. They commented that with the implementation of this module, students' clinical reasoning and diagnostic skills seemed to develop early on in the students' clinical years. Faculty members too felt that the students required more time for practice. They suggested that models, manikins and videos may be used for demonstration in addition to real and simulated patients.

Skill acquisition was based on Millers' Learning
Patsy Stark and F. Fortune had previously suggested that models may not be appropriate for teaching/learning skills but manikins and videos could be used instead 20 . They suggested that dedicated and structured clinical skills training is the most important factor, whether it takes place in a skills centre, in the ward or in the community 20  Analysis of our retro-pre model in line with the Dreyfus model of skill acquisition suggests that students did learn most of the skills following the implementation of the S-PEST Module. One limitation is that although the retro-pre model may reveal valuable information, it is not a substitute tool for an objective measure or a gold standard, but can be used where a large number of skills are to be assessed at one point in time. Don W. Scott et al. from the University of Chicago Pritzker School of Medicine used retro-pre modelling for assessing teaching skills and they recommended this method for assessment 23 .
The OSCE is a proven valid and reliable, formative and summative tool for assessing the clinical skills learned by health sciences students 24 . Standardized patients (healthy individuals trained to portray all the characteristics of an actual patient and to provide constructive feedback) were used in the OSCE. Students did well in the OSCE but one limitation was that only one skill from each system was assessed out of several skills taught in each system. Students and Faculty members also seemed to be satisfied with the OSCE process.

Conclusion
Students that were introduced to S-PEST acquired basic PES used to examine medical, surgical, gynecological and obstetric, orthopedic, ENT, and eye adult patients, as well as pediatric patients. It is expected that S-PEST will enhance medical students' performance during their clinical years. In this study, the students did well in OSCE and both students and Faculty members were satisfied with the training and assessment.

Current Referee Status:
Referee The authors are to be applauded for their thoroughness and attention to accepted Article content: pedagogical models when designing and setting up their curriculum. However, there is little description of the actual content and emphasis of the curricular components, and the methods by which key knowledge and skills were actually assessed in the OSCEs. It is not clear if the OSCEs had been validated, or whether there was any comparison group for analysis of effect. There was no information to determine if the OSCE was used in any novel way.
The methods used to introduce the curriculum were presented in some detail, but as stated above Data: there were significant areas not described. The figures were unhelpful and the data could be better portrayed in table form.
The conclusions were substantiated by the data presented, but without information about Conclusions: any comparison or control group, it is difficult to gauge the impact of the curricular intervention (over that of secular trend etc). It is also not clear what these conclusions add to the existing literature.
I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
No competing interests were disclosed. Due to word limitation criteria, every detail of content cannot be included in the Article content: Due to word limitation criteria, every detail of content cannot be included in the Article content: article; however, adequate information is provided in the introduction, methods and result section. Curricular components are mentioned briefly in the introduction. Contents are described briefly in method section under development of module, orientation of faculty members, approach to teaching, approach to learning, teaching/learning strategies and assessment section. OSCE was validated, the details of which are mentioned in the comments in response to Dr. Deborah Korenstein (See comments in response to referee 2).
OSCE was conducted in a novel way. The summary of practical arrangements for OSCE at KIST Medical College is as follows: 29 May 2013 Referee Report: Title and Abstract: The title is OK, though I would describe it as a structured physical exam skills (italicized words added by me). The abstract focuses largely on the theoretical educational program models underpinning the approaches to evaluation. I think it would be clearer to focus concretely on concisely describing the evaluation performed. The theoretical rationale for that approach can be described in the paper itself.
Overall, the methods and design are adequately explained, though there are a few Article content: important exceptions. The authors go into great detail about the theoretical models underpinning their approach to evaluation. This is fine but is not really critical. The critical components of both the survey and the OSCE are whether these are validated instruments. I assume the authors created both for this project, but it is important for them to state that these tools have not been previously validated (unless of course they have). In addition, I did not find the Figures particularly helpful. They might be more clearly displayed in table format than as graphs.
The conclusions are reasonable, though I would add the caveat that because the Conclusions: measurement instruments were not validated it is difficult to be certain of their reliability. Also, the authors should emphasize that the meaning of the OSCE scores in particular is not clear since there is no comparison group and the OSCE has not been validated.
The data is limited, but the curriculum design is presented in adequate detail. Data: I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
No competing interests were disclosed. Faculty members involved in teaching physical examination skills were instructed to draft two OSCE stations for each system within 8 weeks time.
Second workshop of OSCE academic team for two half days was organized in July 2010 Step II: to assess and finalize all three components of OSCE stations, develop instructions for examiner/assessor and scoring (marking) criteria. Editing of all three components of OSCE stations was done by experienced senior faculty members.
All the documents developed in second workshop printed in first week of August for Step III: conducting OSCE session in the end of August 2010 for first batch of undergraduate students.
OSCE session conducted in the last week of August 2010 and the written feedback Step IV: obtained from the examiners/assessors, faculty involved in teaching physical examination skills and students. This session was considered as piloting for OSCE session to be conducted for batch II in 2011 for proposed project.
Feedback taken from the examiners/assessors, faculty involved in teaching physical Step V: examination skills and students compiled, analyzed and shared with OSCE academic team in a meeting arranged in September 2010. Also score obtained on each station and total score by students shared in the same meeting. The team fully satisfied with the entire process and made some recommendations for conducting session for batch II in 2011 for proposed project.
OSCE academic team developed in 2010, reactivated in May 2011. Faculty members Step VI involved in teaching were assigned to review all three components of OSCE stations i.e. Tasks (stem), checklists and instruction/direction (training information) to simulated patients used in 2010 for assessment of first batch technically and as well as in the light of recommendation of OSCE academic team for conducting next session. OSCE Academic Team in depth reviewed all three components of OSCE stations in July 2011 in meeting cum workshop for two half days, edited & finalized. The documents printed in August 2011 and OSCE session held on September 27, 2011 for second batch (for project).
Validated tools were used for assessment Mean of score obtained by students (98 Conclusions: . students) on each OSCE station was calculated. Then overall mean was calculated. This data is included in article.
All the available data is included in data files of the article. Data: