Teaching-learning behavior in medicine according to students’ perspective: what most influences academic achievement? [version 1; peer review: 1 approved with reservations]

Background: The medical student is a part of education that has a pivotal role in contributing to the teaching-learning process. Assessing the learning process by student perception can give valuable input to predict academic achievement. This study aims to identify the student’s perspective regarding teaching-learning behaviour and which factor most influences academic achievement. Methods: A total of 443 medical students of the preclinical phase were selected to participate in this study. Design of the study comprised three single-groups time sequences with the observational approach. This study is divided into three phases: item construction by conducting exploratory factor analysis (EFA), validation through confirmatory factor analysis (CFA), and true surveys. Sample size calculation employed formulation of subject to item ratio for EFA and CFA; true survey utilized the estimate proportion population. Using regression linear, we determine the most influential factor to academic performance. Results: Results were summarized following two factors that influencing academic performance, namely learning experience (p value=0.013, r=0.041) and exam effectiveness (p value=0.041, r=0.028). Our work highlights the lecturer capacity and integrated module have contributed the academic success (79.46% and 77.80%). Conclusions: Our finding has novelty in which contribute the knowledge regarding exam significance on medicine. Meanwhile, related to the learning experience domain is increasingly proving an essential factor to achieve academic performance. The major strength of this study is the systematic manner in which it was conducted. Open Peer Review


Introduction
Identifying the academic achievements of students and learning behavior is one of the essential factors in evaluating the success of curriculum delivery (Klug et al., 2013). The medical student is a part of education that has a pivotal role in contributing to the teaching-learning process. This is due to medical students are internal stakeholders who are directly involved with the teaching provided by lecturers. They feel for themselves how the learning process is performed. Therefore, assessing the learning process by student perception can give valuable input to diagnose academic achievement.
Focusing on students' learning behaviour seems a promising approach for identifying the academic performance in addition to teacher's competence (Klug et al., 2013)‚ precise curriculum, and learning environment (Struyven et al., 2005). Thus, we wanted to look at a students' perspective regarding teaching-learning behaviour which will be further explained in the methods section. A theoretical basis was needed to develop the new measurement method.
A study conducted by Yeo and Chang in Daegu, Korea, showed that the achievement level of the students was approximately 60% to 70% (Yeo & Chang, 2019). This is related to the content curriculum, while the students' selfevaluation reliability can predict academic achievement. Besides, by observing teaching-learning behavior, we can assess their perception of the teaching process. Perception is a person's response to what is seen or felt to affect thought patterns. If a person's perception of an object is good, there will be a sense of pleasure and interest in that object. Likewise, in the learning process, if it is following their abilities and expectations, students will feel content and give a good perception of it. Learning behaviour was defined as observable behavioural patterns that students demonstrate as they approach and undertake learning tasks. Such learning behaviours include noticeable problem-solving strategies, flexibility, attention, and responses to learning situations (competition, novelty, error, etc.) in classroom settings and facing the exam. Several studies showed the relationship between students' learning behaviour and their academic performance observed by the teacher (Klug et al., 2013;Lin et al., 2019). The new focus set a need for measuring teaching-learning behaviour from students' perspective was not done yet. Learning behaviour are learned actions that enable students to access learning and interact with others productively in the community. Teaching behaviour was defined as action, interaction and communication of teacher with the students in the instructional process. Teaching behavior that is instructionally supportive in terms of providing opportunities for students to respond, to choose, or to receive positive feedback, promotes academic achievement (Trickey et al., 2015).
Thus, in this study, we first identified item construction regarding teaching-learning behaviour from students' perspective by conducting psychometric properties; validating a tool consisting of seven domains using confirmatory factor analysis. Second and most importantly, this study aims to investigate the student's perception regarding teaching-learning behaviour and which factor that most influences academic achievement.

Participants and context
A total of 443 medical students of the preclinical phase were recruited to participate in this study. The study was divided into three stages: item construction, validation and reliability, and assessing students' perspectives on teaching-learning behaviour. Item construction was conducted by exploratory factor analysis to 105 medical students from different cohorts during November-December 2018. A total of 182 medical students were involved in conducting validation and reliability stages using confirmatory factor analysis. This took three months from January to March 2019. Taking the survey involved 156 medical students to assess students' perspectives regarding teaching-learning behavior during four months in 2020.

Data collection Item construction phase
This section described the psychometric properties that have been previously identified to develop an assessment instrument of the teaching-learning behaviour from the student's perspective. Exploratory factor analysis (EFA) was conducted on 105 medical students from two batches, namely year three and year four, with ages ranging between 19-23 years old used random sampling strategies. There were 46 items of questionnaire from the qualitative study that preceded item development. This phase used 105 medical students as participants, representing a subject-to-item ratio of 2-3:1 (Worthington & Whittaker, 2006) with a purposive sampling strategy.
Before conducting factor analysis, Kaiser-Myer-Olkin (KMO) and Bartlett's test of sphericity were performed to obtain sampling adequacy (Worthington & Whittaker, 2006). Sampling adequacy was achieved if it has Kaiser-Myer-Olkin (KMO) value of great than 0.5. After the sample was adequate, the extraction method and rotation in the oblique or orthogonal manner using SPSS version 21 were carried out. Confirmation of the data extraction that can be loaded in the same factor or scale was done using the maximum likelihood.
Inclusion criteria of the item construction phase were as follows: fresh year medical student, active status and registered in the university database.

Validation and reliability
The instrument validation phase was carried out through confirmatory factor analysis (CFA). Validity determines whether an instrument truly measures what it is intended to measure. At the same time, reliability assesses the consistency of the result or outcome so that the research can be replicated (Lynch et al., 2004). This study used construct validity to assess the instrument. In terms of confirmatory factor analysis, conducting factor analysis can examine construct behaviour (Agung et al., 2017;Anthoine et al., 2014).
Reliability is the test used to assess the compatibility of the result of measurement (Stalmeijer et al., 2008), and it is identified by a coefficient ranging from very low coherence < 0.2 to very high coherence > 0.9 (Shirali et al., 2018). Reliability (r 11 ) is the proportion of observed variance that is due to errors of measurement (Aiken, 1996). In other words, reliability refers to the degree to which a questionnaire measures the concepts of interest consistently (e.g., are the items enough? are the scores reproducible?) (Van Der Vleuten & Schuwirth, 2005).
This part describes the stage of the validation and reliability of an instrument that exists previously in initial testing based on the EFA result. Using structural equation modelling, we applied confirmatory factor analysis (CFA) toward 182 medical students. A total of 182 students were selected by simple random.
The inclusion criteria to participate in this phase are as follows: the 2 nd level medical students and above level who have conducted learning activity of generic phase, this is due to assess the teaching-learning behavior, students must have a comprehension regarding the foundational concept of good learning.

True survey to assess students' perspective
The 156 students from batches 2017, 2018, and 2019 were selected and invited to participate in this survey with a random sampling strategy. The sample size was determined based on an estimated proportion population as formulated: Inclusion criteria of medical students are as follows: the active students on the academic year run, have followed academic activity > 1 year. The inclusion criteria were established to anticipate the occurrence of dropout research and ensure the consistency of the data. Students who are still at 1 st -year votes in the-adaptation of teaching-learning and attendance rates are still fluctuating. Therefore, only the students in 2 nd -year and above are involved in this research.
The 33-items questionnaire was distributed to 156 medical students of UNISBA for four months from September to December 2020. Upon receiving written consent, the participants were asked to fill out the instrument for 15-25 minutes. The participants were asked to score their agreement for each item in the questionnaire on a 5-point Likert scale ranging from 1 (disagree) to 5 (strongly agree).
This study has some potential of bias as follows: filling out questionnaires may cause dishonesty for fear of affecting performance assessments by lecturers. Medical students possibly feel uncomfortable expressing their thoughts. In anticipation of potential bias, the researcher ensured that involvement in the study was voluntary and there was no implication toward their assessment.
In summary, the phases passed in this study, the sample size, and the method approach taken can be seen in Figure 1.

Data analysis Items construction
Using SPSS version 21, the extraction method of maximum likelihood (ML) was implemented to extract the data. The Varimax rotation method was applied with Kaiser Normalization to simplify and describe the data structure. A factor analysis conducted on the 105 completed questionnaires indicated an adequate sample size had been achieved, as evidenced by a KMO value of 0.705 (> 0.5) and Bartlett's test of sphericity of p < 0.001 (X 2 = 2584.12).
The detailed analysis identified seven factors based on the initial Eigenvalue of more than one ( Figure 2). These seven factors were retained in the study and accounted for 73.42% of the total variance after nine iterations ( Table 1). The instrument items were grouped into the same component based on the factor loading value of more than 0.4.

Validation through confirmatory factor analysis
After the sample achieves fitness, conducting confirmatory factor analysis to determine the t loading factor as an indicator validity and error measurement as indicator reliability of each item. It was performed using the LISREL 8.8 program.   Counting t loading factor is deemed valid if it has a value of more than 1.96 (95% degree of confidence) and shows a significance representing its latent variable. Then, the lambda and delta value to structural equation modelling was determined. The distribution frequency from each sample group was represented with a table and graph.

True survey
A pre-tested performance captures the demographic data, admission test, the value of GPA (grade point average), and perception of teaching-learning behaviour among the undergraduate medical students of UNISBA. The data collected were tabulated and analyzed using the Statistical Package for Social Sciences (SPSS) version 24.0. The presenting of results in terms of numbers and proportions. The linear regression test was conducted to identify the influence of perception variable on academic achievement with its 95% Confidence interval (CI) and then proceeded with Dunnett's posthoc multiple comparison test for comparison purposes. In this study, p-value <0.05 was considered as statistically significant, it does deem variable perception influence the academic achievement in terms of grade point average (GPA).
The variables of this study were identified based on the definition variable following. Gender was defined as generally conceived as a set of characteristics or traits that are associated with specific biological sex (male or female). The age is when someone has been alive or something has existed (<20 years old and >20 years old). Student's perception was recognition and interpretation of students regarding sensory information and how they respond to the data. Academic achievement resulted from students' academic performance in terms of GPA and was categorized as low-moderate if GPA < 2.75, called high if GPA 2.75-3.50, and excellent if GPA > 3.50.

Ethical considerations
The study protocol complied with the Helsinki Declaration, and the Joint Committee approved it of Research and Ethics of the medical faculty of Universitas Islam Bandung, Indonesia. This research has received ethical approval from the medical faculty of Bandung Islamic university research ethics committee no. 005/Ethic committee FK/VI/2017.

Participant consent
All the participants involved in this study have gave written consent and agreed to participate, also permitted the data to be analyzed and published in a scientific journal.

Results and discussion
Item construction Table 1 indicated that the items were grouped into the same component based on the factor loading value of more than 0.4. For example, learning material and body of knowledge (Factor 1) consist of items q13, q14, q15, q16, q17, q18, q19, q20, q23, and q24 as the loading values were above 0.40. The other items were also classified in the same manner. As seen in Table 1, the matrix pattern of the Varimax rotation method with Kaiser Normalization showed the grouping of the items based on a loading value of more than 0.4 (Kusmiati, Dharmmika & Dewi, 2021).
Thirty-three items were retained as an indicator variable (loading value of > 0.4). Thirteen subjects were removed because they had a loading value of less than 0.4. The items excluded were q2, q3, q5, q6, q7, q8, q22, q30, q31, q34, q35, q39, and Items retained of the instrument were 33 items and represented all indicators to assess the teaching-learning behaviour starting from the body of knowledge to the assessment method. The factor of the lecturer's capacity had the most items among the factors (10 things). In contrast, elements such as the difficulty level of the exam and linking material within the learning module were factors that had the fewest questions (two items).

Validation and reliability
The results of the t-loading values and β-reliability of 183 medical students can be seen in Figures 3 and 4. Figures 3 and 4 showed that the loading factor of each item against a factor is 1 to 7. The value of the t loading element of all things was higher than the critical point 1.96, so the observed variable (indicator variable) describes the latent variable (7 factors) and shows a significant influence of the latent variables on its indicator variables.
Thirty-three questions represent seven analysis groups as latent variables (Figures 3 and 4). The connection between latent variables with 33 items as indicator variables is analyzed using confirmatory factor analysis (CFA). All 33 questions described a significant relationship to their respective factor groups, with a range of t-loading values from 7.55 to 16.47. Each factor group (factors 1-7) is represented by the questions in its respective factor group.     Students' perspective regarding teaching-learning behavior After data cleaning, a total of 152 samples were remaining because four data were incomplete in terms of there is no data of gender. Table 3 shows the socio-demographic information of the students (n = 152). Many of the students are females (69.08%), with the age group was dominated by the age of greater than 20 years old. The Year 2 students (19.74%) were those who had sat for their end of the second-semester summative examination, while the Year 3 students (44.08%) were those who had taken their fourth summative examination. The students of Year 4 (36.18%) are those who had taken a review for a whole core module of medicine. The number of students who had taken regular admission tests was more than those who took specific interest and ability test with the ratio of 6:1 (or 131:21 student) due to allocation for specific interest test was limited of 20% from all new students accepted.
This study also depicted that academic achievement in terms of GPA value was dominated by the group of high achievement (77.33 %). This result showed that the academic performance of medical students in our faculty had obtained good achievement.
The Most factor that influences the academic achievement The output of the perception of teaching-learning behaviour to the academic achievement (GPA) was stated significant if p-value less than 0.05 with confidence interval amount of 95%.
Based on Table 4, it was found that perception of learning experience and exam effectiveness were two factors influencing academic achievement from the students' perspective. It was understood that learning experience is the most important aspect in influencing academic achievement. This is due to how a student thinks about learning and studying determines how he/she accomplish assignments and evaluation tasks. Conversely, the learner's experience with evaluation and assessment processes determines how the student approaches learning (Struyven et al., 2005). Besides, a positive learning experience furnishes students with chunks of time that intentionally prompt them in achieving their learning goals (Schmidt et al., 2019). Undeniably, the learning experience of students changes over time and is dependent on the interventions, peer interactions, and changing circumstances (Fredricks et al., 2016). Its impact on faculty can help them to improve the contemporary design of teaching and learning. This is due to having the ability to assess students' experience which indicates utilizing good self-monitoring, self-assessment, and metacognitive reflection (Giannakos et al., 2020). The academic achievement depicted success in curriculum delivery. This finding rhymes with the statement of Dent and Harden that curriculum has six aspects: content, educational strategies; learning experiences; assessment process; learning outcomes; and the slight contrast point, learning environment (Dent et al., 2018). Two things under this study are in terms of learning experience and assessment method (exam significance). In this context, students perceived that the assessment addressed more on conformity with learning materials and the effectiveness of grade score achievement that includes the level of difficulty. In this regard, students are learning more about practice aspects and their goals in the undertaken examination.
Experience is an episode, a chunk of time to remember; it is feelings and thoughts, motives and activities, all closely knitted together. Learning is an experience (Schmidt et al., 2019) that portrays students' feelings and thoughts, motives, and actions when they interact with the teaching and learning environment. Other variables of perception that slightly influence though not significant, namely perception regarding lecturer capacity and integrated module (79.46% and 77.80%). An interesting finding from this study is about the ability of lecturers, students assume there are 10 items that must be owned by a lecturer. The ten items are grouped into 4 abilities including: teaching kills, expertise in teaching materials, personality styles, and interaction with students. According to the students, the lecturer's competency or capacity was an essential factor in encouraging success in achieving academic performance. This is in line with the Indonesian directorate general of higher education that a lecturer ought to have four competencies, as follows pedagogical competency, knowledge or subject competency, personal competency, and social competency. Pedagogical competency relates to the teacher's ability in managing the teaching-learning process. They comprise five aspects, namely learning climate, modelling, coaching, exploration, and articulation (Boerboom et al., 2011).
The result of confirmatory factor analysis ( Figure 3) found that the essential aspect of lecturer capacity was about lecturer ability in mastering the material taught (C18) and in managing the teaching process (C17) because both items have the highest of t-loading values (16.47 and 15.85). It was understood that pedagogical competence consists of the ability to deal with the understanding of students, designing, implementing, and evaluating learning outcomes as a representing the academic achievement, as well as developing students to embody their various potential (Mulyasa, 2012). To summarize, if the teacher can deliver the lessons in line with the student's expectations, it makes the teaching-learning process run effectively and efficiently. Student's perceptions regarding the pedagogical competence of lecturers influence positive student motivation; therefore it could affect the performance of academics indirectly (Lumbantobing, 2020).
Meanwhile, statement item about learning experience was the most represented by experiencing when learn of clinical skill practice (C10). Variable indicator in exam effectiveness (Figure 3) was most indicated by the results of an oral exam and objective structured clinical examination (OSCE) (C25). Academic success in medical education is more emphasized to the clinical skill learning, ability in diagnostic reasoning, and clinical reasoning. In contrast, related aspects to them reflect learning experiences about clinical practice and clinical diagnosis. The integrated module is a crucial factor in learning medicine. This is due to the teaching method in problem based-learning was incorporating multidisciplinary-system-based courses rather than discipline-specific ones (Atta & AlQahtani, 2015). One of the approaches in problem based-learning is the integrated module. This is related to greater knowledge acquisition and more excellent improvement in clinical skills than the lecture-based approach. The finding in this study showed that integrated module/material also affects academic achievement. Similar to McParland's study that the performance of the students holding PBL was better in both multiple-choice questions and the final exam (Atta & AlQahtani, 2015).
This study can be generalized to medical students of other medical faculty that have similar characteristics to our faculty. This is due to each institution have different characteristics.

Conclusions
Overall, our work demonstrates that student perception of learning experience and exam effectiveness affect academic performance. Other factors that contribute to achieving academic success are lecturer capacity and the integrated modules. We provide evidence that lecturer capacity and integrated modules can enhance academic success in the educational process indirectly.

Limitations and future research
The original 46-item and 33-item questionnaires were distributed to the medical student in the Indonesian language. There could also be a misinterpretation of meaning when the factors extracted from the literature were translated from English to Indonesian language for the questionnaire and when it was translated again into English for publication. To generalize the results of research at other medical faculty in Indonesia given the limited number of samples, more research is needed with a larger sample which involves other samples of medical faculty.

Umatul Khoiriyah
Faculty of Medicine, Medical Education Unit, Universitas Islam Indonesia, Yogyakarta, Indonesia The manuscript describes the scale development of teaching-learning behaviour from the student perspective. It also explains the factors from this behaviour that influence student achievement. The authors had written the manuscript clearly; however, some remarks are identified to improve the quality of the papers.
The main topic explained in this paper is scale development, which needs a validity framework. It is recommended to use the current validity framework, viewing the validity as the extent to which the evidence and theories supporting the interpretation of assessment results align with the tool's underlying construct (AERA, APA, 1999). In this current framework, validity evidence consists of content evidence, response process, internal structure, correlation to other variables, and consequence. For detail, please refer to one of these references: Cook, D. A., & Beckman, T. J. (2006). Current concepts in validity and reliability for psychometric instruments: Theory and application. The American Journal of Medicine, 119(2), 166.e167-166.e116.

2.
It is suggested that the aim of this study is to collect validity evidence of the tool measuring student perspectives of teaching and learning behaviour. Based on the current description in this paper, I think that the validity evidence in this tool includes content, response process, and correlation to another variable.
The authors have explained the EFA and CFA in detail, which is part of internal structure evidence. Moreover, the authors have correlated the tool with student achievement. This analysis is part of the evidence of "correlation to other variables". However, the explanation about the content evidence is not clear. The step in developing the items such as blueprint development, review expert (if it is available) should be informed to the reader. The author should state the domain number underlying the construct of the tool so the reader can understand easily why the EFA was extracted in 7 domains. The response process evidence by tying the items to some students also should be stated clearly. The response process is usually conducted before collecting data for EFA analysis.
So, it is suggested to revise the manuscript (abstract and main text) based on the current validity framework. Furthermore, since the aim would be revised, the method, results, discussion, and conclusion should also be revised to answer the research aim.
In terms of data analysis, I note that the authors extracted the data into seven domains in the EFA part even though the scree plot (figure 2) shows that three domains are better for this data. Please clarify the reason in the method section.
The EFA results also show that factors 5 (the difficulty level of the exam) and 7 (integrated and material) only consist of 2 items. The CFA also show similar results. Why did they not merge this domain with another domain? The literature said that a good subscale consists of at least three items.
In this study, the analysis of CFA was divided into two parts ( figure 3 and figure 4). Please clarify why the seven factors were not analyzed together in one model since they derive from one construct. Furthermore, I also notice that the model in figure 3 is not quite fit since it does not fulfil the fit indexes (RMSEA < 0.1 and chi-square/df < 3)