Teaching-learning behavior in medicine according to students’ perspective: what most influences academic achievement?

Participants and context

A total of 443 medical students of the preclinical phase were recruited to participate in this study. The study was divided into three stages: item construction, validation and reliability, and assessing students’ perspectives on teaching-learning behaviour. Item construction was conducted by exploratory factor analysis to 105 medical students from different cohorts during November-December 2018. A total of 182 medical students were involved in conducting validation and reliability stages using confirmatory factor analysis. This took three months from January to March 2019. Taking the survey involved 156 medical students to assess students’ perspectives regarding teaching-learning behavior during four months in 2020.

Data collection

Item construction phase

This section described the psychometric properties that have been previously identified to develop an assessment instrument of the teaching-learning behaviour from the student’s perspective. Exploratory factor analysis (EFA) was conducted on 105 medical students from two batches, namely year three and year four, with ages ranging between 19–23 years old used random sampling strategies. There were 46 items of questionnaire from the qualitative study that preceded item development. The themes for blueprint development came from a review of literature on the topic of the measure so that scale and subscale can be identified (Lynn, 1986). This phase used 105 medical students as participants, representing a subject-to-item ratio of 2-3:1 (Worthington & Whittaker, 2006) with a purposive sampling strategy.

Before conducting factor analysis, Kaiser-Myer-Olkin (KMO) and Bartlett’s test of sphericity were performed to obtain sampling adequacy (Worthington & Whittaker, 2006). Sampling adequacy was achieved if it has Kaiser-Myer-Olkin (KMO) value of great than 0.5. After the sample was adequate, the extraction method and rotation in the oblique or orthogonal manner using SPSS version 21 were carried out. Confirmation of the data extraction that can be loaded in the same factor or scale was done using the maximum likelihood.

Inclusion criteria of the item construction phase were as follows: fresh year medical student, active status and registered in the university database.

Validation and reliability

The instrument validation phase was carried out through confirmatory factor analysis (CFA). Validity determines whether an instrument truly measures what it is intended to measure. At the same time, reliability assesses the consistency of the result or outcome so that the research can be replicated (Lynch et al., 2004). This study used construct validity to assess the instrument. In terms of confirmatory factor analysis, conducting factor analysis can examine construct behaviour (Agung et al., 2017; Anthoine et al., 2014).

According to Standards of Educational and Psychological Measurement, a source of validity has five components, namely content, response process, internal structure, relationship to other variables, and consequences (Cook & Beckman, 2006; Michels et al., 2016). In this study, validity was assessed by content evidence, internal structure, and correlation to other variables. Content evidence refers to two stage process in terms of blueprint item development and domain identification for affective/personality measure (Michels et al., 2016). This is due to measure of teaching-learning behavior in accord with student’s perspective. Content evidence also drawn relevance of content and representativeness of the test construct Whilst internal structure of this study refers to factor analysis data from the results of confirmatory factor analysis and reliability (Cook & Beckman, 2006). The last source of validity of this study-correlation to other variables, we have correlated the instrument with outcome of student achievement (GPA).

Reliability is the test used to assess the compatibility of the result of measurement (Stalmeijer et al., 2008), and it is identified by a coefficient ranging from very low coherence < 0.2 to very high coherence > 0.9 (Shirali et al., 2018). Reliability (r₁₁) is the proportion of observed variance that is due to errors of measurement (Aiken, 1996). In other words, reliability refers to the degree to which a questionnaire measures the concepts of interest consistently (e.g., are the items enough? are the scores reproducible?) (Van Der Vleuten & Schuwirth, 2005).

This part describes the stage of the validation and reliability of an instrument that exists previously in initial testing based on the EFA result. Using structural equation modelling, we applied confirmatory factor analysis (CFA) toward 182 medical students. A total of 182 students were selected by simple random.

The inclusion criteria to participate in this phase are as follows: the 2^nd level medical students and above level who have conducted learning activity of generic phase, this is due to assess the teaching-learning behavior, students must have a comprehension regarding the foundational concept of good learning.

True survey to assess students’ perspective

The 156 students from batches 2017, 2018, and 2019 were selected and invited to participate in this survey with a random sampling strategy. The sample size was determined based on an estimated proportion population as formulated:

Inclusion criteria of medical students are as follows: the active students on the academic year run, have followed academic activity > 1 year. The inclusion criteria were established to anticipate the occurrence of dropout research and ensure the consistency of the data. Students who are still at 1^st-year votes in the-adaptation of teaching-learning and attendance rates are still fluctuating. Therefore, only the students in 2^nd-year and above are involved in this research.

The 33-items questionnaire was distributed to 156 medical students of UNISBA for four months from September to December 2020. Upon receiving written consent, the participants were asked to fill out the instrument for 15–25 minutes. The participants were asked to score their agreement for each item in the questionnaire on a 5-point Likert scale ranging from 1 (disagree) to 5 (strongly agree).

This study has some potential of bias as follows: filling out questionnaires may cause dishonesty for fear of affecting performance assessments by lecturers. Medical students possibly feel uncomfortable expressing their thoughts. In anticipation of potential bias, the researcher ensured that involvement in the study was voluntary and there was no implication toward their assessment.

In summary, the phases passed in this study, the sample size, and the method approach taken can be seen in Figure 1.

Figure 1. Steps summary of participants involving in the study.

Data analysis

Items construction

Using SPSS version 21, the extraction method of maximum likelihood (ML) was implemented to extract the data. The Varimax rotation method was applied with Kaiser Normalization to simplify and describe the data structure. A factor analysis conducted on the 105 completed questionnaires indicated an adequate sample size had been achieved, as evidenced by a KMO value of 0.705 (> 0.5) and Bartlett’s test of sphericity of p < 0.001 (X² = 2584.12).

The detailed analysis identified seven factors based on the initial Eigenvalue of more than one (Figure 2). These seven factors were retained in the study and accounted for 53.77% of the total variance after nine iterations (Table 1). The instrument items were grouped into the same component based on the factor loading value of more than 0.4 (Cabrera-Nguyen, 2010).

Figure 2. Factors in the seven components that explained the variable of student’s instrument based on Eigenvalue > 1.

Thirty-three items were retained as an indicator variable (loading value of > 0.4). Thirteen subjects were removed from 46 items because they had a loading value of less than 0.4. The items excluded were q2, q3, q5, q6, q7, q8, q22, q30, q31, q34, q35, q39, and q40. The Cronbach’s alpha values for all 33 items were 0.913, indicating very high consistency (Guilford, 1978; Van Der Vleuten & Schuwirth, 2005).

Validation through confirmatory factor analysis

After the sample achieves fitness, conducting confirmatory factor analysis to determine the t loading factor as an indicator validity and error measurement as indicator reliability of each item. It was performed using the LISREL 8.8 program. Counting t loading factor is deemed valid if it has a value of more than 1.96 (95% degree of confidence) and shows a significance representing its latent variable. Then, the lambda and delta value to structural equation modelling was determined. The distribution frequency from each sample group was represented with a table and graph.

True survey

A pre-tested performance captures the demographic data, admission test, the value of GPA (grade point average), and perception of teaching-learning behaviour among the undergraduate medical students of UNISBA. The data collected were tabulated and analyzed using the Statistical Package for Social Sciences (SPSS) version 24.0. The presenting of results in terms of numbers and proportions. The linear regression test was conducted to identify the influence of perception variable on academic achievement with its 95% Confidence interval (CI) and then proceeded with Dunnett’s posthoc multiple comparison test for comparison purposes. In this study, p-value <0.05 was considered as statistically significant, it does deem variable perception influence the academic achievement in terms of grade point average (GPA).

The variables of this study were identified based on the definition variable following. Gender was defined as generally conceived as a set of characteristics or traits that are associated with specific biological sex (male or female). The age is when someone has been alive or something has existed (<20 years old and >20 years old). Student's perception was recognition and interpretation of students regarding sensory information and how they respond to the data. Academic achievement resulted from students’ academic performance in terms of GPA and was categorized as low-moderate if GPA < 2.75, called high if GPA 2.75-3.50, and excellent if GPA > 3.50.

Ethical considerations

The study protocol complied with the Helsinki Declaration, and the Joint Committee approved it of Research and Ethics of the medical faculty of Universitas Islam Bandung, Indonesia. This research has received ethical approval from the medical faculty of Bandung Islamic university research ethics committee no. 005/Ethic committee FK/VI/2017.

Participant consent

All the participants involved in this study have gave written consent and agreed to participate, also permitted the data to be analyzed and published in a scientific journal.

Item construction

Table 1 indicated that the items were grouped into the same component based on the factor loading value of more than 0.4. For example, learning material and body of knowledge (Factor 1) consist of items q13, q14, q15, q16, q17, q18, q19, q20, q23, and q24 as the loading values were above 0.40. The other items were also classified in the same manner. As seen in Table 1, the matrix pattern of the Varimax rotation method with Kaiser Normalization showed the grouping of the items based on a loading value of more than 0.4 (Kusmiati, Dharmmika & Dewi, 2021).

Thirty-three items were retained as an indicator variable (loading value of > 0.4). Thirteen subjects were removed because they had a loading value of less than 0.4. The items excluded were q2, q3, q5, q6, q7, q8, q22, q30, q31, q34, q35, q39, and q40. The Cronbach’s alpha values for all 33 items were 0.913, indicating very high consistency (Guilford, 1978; Van Der Vleuten & Schuwirth, 2005). The Cronbach’s alpha values of each domain as follows: value of 0.921 for lecturer ability, 0.813 for learning material, 0.817 for exam effectiveness, 0.639 for learning experience, 0.758 for difficulty level of the exam, 0.750 for learning material suitability, and 0.645 for integrated material.

Items retained of the instrument were 33 items and represented all indicators to assess the teaching-learning behaviour starting from the body of knowledge to the assessment method. The factor of the lecturer’s capacity had the most items among the factors (10 things). In contrast, elements such as the difficulty level of the exam and linking material within the learning module were factors that had the fewest questions (two items).

Validation and reliability

The results of the t-loading values and β-reliability of 183 medical students can be seen in Figures 3 and 4.

Figure 3. The result of confirmatory factor analysis of student instruments (3 components).

Figure 4. The result of confirmatory factor analysis of student instrument (4 components).

Figures 3 and 4 showed that the loading factor of each item against a factor is 1 to 7. The value of the t loading element of all things was higher than the critical point 1.96, so the observed variable (indicator variable) describes the latent variable (7 factors) and shows a significant influence of the latent variables on its indicator variables. Both Figure 3 and Figure 4 show that the instrument model is acceptable by looking at degrees of freedom all model have p-value <0.001, while RMSEA values indicate good fit (<0.08) and reasonable fit (<0.14) (Boerebach et al., 2014).

Thirty-three questions represent seven analysis groups as latent variables (Figures 3 and 4). The connection between latent variables with 33 items as indicator variables is analyzed using confirmatory factor analysis (CFA). All 33 questions described a significant relationship to their respective factor groups, with a range of t-loading values from 7.55 to 16.47. Each factor group (factors 1–7) is represented by the questions in its respective factor group.

Table 2 indicates the amount of item/subscales for each factor based on EFA result.

Figure 4 showed that the latent variable of exam effectiveness (f4) has four indicator variables, with the strongest indicator being question no. 25 (t value = 14.91). Questions no. 26–29 represented the latent variable for the suitability of learning (f5). Question no. 27 was the strongest indicator for the latent variable of learning suitability with a test (t value = 13.97). Meanwhile, the latent variables of the difficulty level of the exam (f6) and learning module variables (f7) are represented most strongly by a question no. 31 and no. 33 (t value = 3.76; 10.93). These latent variables each have two items as their indicator variable.

Students’ perspective regarding teaching-learning behavior

After data cleaning, a total of 152 samples were remaining because four data were incomplete in terms of there is no data of gender. Table 3 shows the socio-demographic information of the students (n = 152). Many of the students are females (69.08%), with the age group was dominated by the age of greater than 20 years old. The Year 2 students (19.74%) were those who had sat for their end of the second-semester summative examination, while the Year 3 students (44.08%) were those who had taken their fourth summative examination. The students of Year 4 (36.18%) are those who had taken a review for a whole core module of medicine. The number of students who had taken regular admission tests was more than those who took specific interest and ability test with the ratio of 6:1 (or 131:21 student) due to allocation for specific interest test was limited of 20% from all new students accepted.

This study also depicted that academic achievement in terms of GPA value was dominated by the group of high achievement (77.33 %). This result showed that the academic performance of medical students in our faculty had obtained good achievement.

The Most factor that influences the academic achievement

The output of the perception of teaching-learning behaviour to the academic achievement (GPA) was stated significant if p-value less than 0.05 with confidence interval amount of 95%.

Based on Table 4, it was found that perception of learning experience and exam effectiveness were two factors influencing academic achievement from the students’ perspective. It was understood that learning experience is the most important aspect in influencing academic achievement. This is due to how a student thinks about learning and studying determines how he/she accomplish assignments and evaluation tasks. Conversely, the learner’s experience with evaluation and assessment processes determines how the student approaches learning (Struyven et al., 2005). Besides, a positive learning experience furnishes students with chunks of time that intentionally prompt them in achieving their learning goals (Schmidt et al., 2019). Undeniably, the learning experience of students changes over time and is dependent on the interventions, peer interactions, and changing circumstances (Fredricks et al., 2016). Its impact on faculty can help them to improve the contemporary design of teaching and learning. This is due to having the ability to assess students' experience which indicates utilizing good self-monitoring, self-assessment, and metacognitive reflection (Giannakos et al., 2020).

The academic achievement depicted success in curriculum delivery. This finding rhymes with the statement of Dent and Harden that curriculum has six aspects: content, educational strategies; learning experiences; assessment process; learning outcomes; and the slight contrast point, learning environment (Dent et al., 2018). Two things under this study are in terms of learning experience and assessment method (exam significance). In this context, students perceived that the assessment addressed more on conformity with learning materials and the effectiveness of grade score achievement that includes the level of difficulty. In this regard, students are learning more about practice aspects and their goals in the undertaken examination.

Experience is an episode, a chunk of time to remember; it is feelings and thoughts, motives and activities, all closely knitted together. Learning is an experience (Schmidt et al., 2019) that portrays students’ feelings and thoughts, motives, and actions when they interact with the teaching and learning environment. Other variables of perception that slightly influence though not significant, namely perception regarding lecturer capacity and integrated module (79.46% and 77.80%). An interesting finding from this study is about the ability of lecturers, students assume there are 10 items that must be owned by a lecturer. The ten items are grouped into 4 abilities including: teaching kills, expertise in teaching materials, personality styles, and interaction with students. According to the students, the lecturer’s competency or capacity was an essential factor in encouraging success in achieving academic performance. This is in line with the Indonesian directorate general of higher education that a lecturer ought to have four competencies, as follows pedagogical competency, knowledge or subject competency, personal competency, and social competency. Pedagogical competency relates to the teacher’s ability in managing the teaching-learning process. They comprise five aspects, namely learning climate, modelling, coaching, exploration, and articulation (Boerboom et al., 2011).

The result of confirmatory factor analysis (Figure 3) found that the essential aspect of lecturer capacity was about lecturer ability in mastering the material taught (C18) and in managing the teaching process (C17) because both items have the highest of t- loading values (16.47 and 15.85). It was understood that pedagogical competence consists of the ability to deal with the understanding of students, designing, implementing, and evaluating learning outcomes as a representing the academic achievement, as well as developing students to embody their various potential (Mulyasa, 2012). To summarize, if the teacher can deliver the lessons in line with the student’s expectations, it makes the teaching-learning process run effectively and efficiently. Student’s perceptions regarding the pedagogical competence of lecturers influence positive student motivation; therefore it could affect the performance of academics indirectly (Lumbantobing, 2020).

Meanwhile, statement item about learning experience was the most represented by experiencing when learn of clinical skill practice (C10). Variable indicator in exam effectiveness (Figure 3) was most indicated by the results of an oral exam and objective structured clinical examination (OSCE) (C25). Academic success in medical education is more emphasized to the clinical skill learning, ability in diagnostic reasoning, and clinical reasoning. In contrast, related aspects to them reflect learning experiences about clinical practice and clinical diagnosis.

The integrated module is a crucial factor in learning medicine. This is due to the teaching method in problem based-learning was incorporating multidisciplinary-system-based courses rather than discipline-specific ones (Atta & AlQahtani, 2015). One of the approaches in problem based-learning is the integrated module. This is related to greater knowledge acquisition and more excellent improvement in clinical skills than the lecture-based approach. The finding in this study showed that integrated module/material also affects academic achievement. Similar to McParland's study that the performance of the students holding PBL was better in both multiple-choice questions and the final exam (Atta & AlQahtani, 2015).

This study can be generalized to medical students of other medical faculty that have similar characteristics to our faculty. This is due to each institution have different characteristics.

Limitations and future research

The original 46-item and 33-item questionnaires were distributed to the medical student in the Indonesian language. There could also be a misinterpretation of meaning when the factors extracted from the literature were translated from English to Indonesian language for the questionnaire and when it was translated again into English for publication. To generalize the results of research at other medical faculty in Indonesia given the limited number of samples, more research is needed with a larger sample which involves other samples of medical faculty.

Underlying data

Figshare: Raw data of the research-Teaching learning behavior in medicine according to students’ perspective, https://doi.org/10.6084/m9.figshare.16456983 (Kusmiati et al., 2021b).

This project contains the following underlying data:

Figshare: Table 3 characteristic of demographic data of students, https://doi.org/10.6084/m9.figshare.16934977 (Kusmiati et al., 2021a).

Extended data

Figshare: List of questions and description of the questionnaire-Teaching learning behavior in medicine according to students' perspective, https://doi.org/10.6084/m9.figshare.16457001.

This project contains the following extended data:

Figshare: Figure 1 steps summary of participant involving in the study, https://doi.org/10.6084/m9.figshare.16935064.

Data are available under the terms of the Creative Commons Zero “No rights reserved” data waiver (CC0 1.0 Public domain dedication).