Effect of problem-based learning on students’ attitude towards learning physics: a cohort study

Design of the study

This study aimed at determining the effect of PBL instruction on students’ attitude towards physics. The study followed a quantitative approach with a quasi-experimental design employing, specifically, a pre-test–post-test control group experimental design following a randomized solomon four group design (Dimitrov & Rumrill, 2003) in which PBL as an intervention was applied on the experimental group while the control group was instructed using traditional instructions as elaborated in the literature review section.

Sample and sampling methods

The study was conducted from January to April 2021 with 419 physics students of the 13^th grade (aged between 16 and 21) in Mitooma District-Southwestern Uganda. Selection of participating schools and allocation to the treatment and comparison groups were based on simple random sampling. Cluster sampling technique (Creswell, 2014) was employed in this study where intact classes were used as units of analysis. 19 classes from 19 schools were employed. The number of students in each school can be found in our data article (Kanyesigye et al., 2022b). For instance, the sample of female students in the pre- and post-test groups was 53, while the sample of male students in the experimental group was 79. In the control group, there were 39 female students and 68 male students. Similarly, the sample of public-school students was 39 in the pre-test and post-test, while the sample of private school students was 92 in the experimental group. There were also 55 students in public school and 51 students in private school included in the control group. The population and sample sizes used in this study were taken at a 95% confidence interval in accordance with Morgan (2006). All secondary schools in Mitooma district are both day and boarding, with only two single girls’ schools. Thus, all schools included are day and boarding schools. Note that there was no exclusion criteria.

Training of physics teachers in the PBL process

A two-day, six-hour professional training was organized on 10^th and 11^th January 2021 at Ruhinda Secondary School-Mitooma district and was attended by 30 physics teachers from Mitooma district in Southwestern Uganda. This training was designed as part of the study (Kanyesigye et al., 2022d). The participants were invited to the training depending on their teaching subject (physics) in Mitooma district. The main purpose of the professional training was to enhance secondary school in-service physics teachers’ knowledge of PBL and was guided by the following objectives:

Participants were split into groups of five. The trainer for the Secondary Science and Mathematics (SESEMAT) program in the western region of Uganda and the first author served as facilitators for the formed groups. The SESEMAT trainer is one of the experts in Uganda that were first trained in the implementation of competence-based curriculum and appointed at the regional level by the government to train other teachers. As such, during the proposal development and development of the training content, this expert played a big role. So, inviting him to facilitate the training was based on his expertise and the fact that he had participated in the material development and validation of this study research instrument. The first author is a national examiner of physics. During proposal development, the author consulted the physics teachers (participants of this study) on which topics in physics generally posed greater challenges to students. Among other topics, the topic of waves was pointed out based on the generally poor performance of students in wave concepts and the fact that questions on wave concepts are also mostly dodged. So, the author took up the topic of waves for the PhD project. The roles for the participants and the training leader were defined at the start of the training. In formulating PBL questions, the topic of waves was selected as agreed prior to the course upon by majority of the participants, using direct messaging (WhatsApp) with the first author, in the training based on possession of prior knowledge. The aim of the training, was to provide the participants with the ability to draft real-life-based problems on the concepts of waves using online resources and textbooks.

Implementation of the intervention

All activities were carried out throughout the study during normal class hours (four hours per week for four weeks). Before starting the experimentation, a pre-test was administered to both experimental and control groups in the first week — 10 physics teachers who handled the experimentation group first trained students about PBL strategies before starting the experiment. Experimental-1 (pre- and post-test design) accommodated 132 students, experimental-2 (post-test only design) accommodated 99 students, control-1 (pre- and post-test design) accommodated 107 students, and control-2 (post-test only design) accommodated 81 students. During the implementation, the teachers of the experimental groups in each lesson would purposively divide students into five to six-member groups, and each group would be presented a problem to research and after make a presentation to the whole class under the guidance of their respective teachers. This implementation was monitored at all levels by the researchers. Each group member, in one way or the other, had a responsibility to fulfill. The members were required to participate during the group discussions actively. Members in their groups were required to share their knowledge, ideas, and experiences about the solution of the given problem. Before group discussion, each member was expected to make individual study and be able to represent, communicate and evaluate and assess their learning either individually or at group level. In case of need for guidance, the teacher would pose open-ended general questions for students to think about critically. Students would, at the end of each activity, evaluate each other in relation to participation, preparation, interpersonal skills contribution to the progress of the group.

On the other hand, students in the control group were instructed (under supervision from the researchers) using traditional approaches where the teachers copied notes either from textbooks or from their notebooks and dictated them to students to write down in their notebooks. A few teachers would write about one or two problems on the chalkboard and again solve them as students watched. They would thereafter be referring the students to either textbooks and/past-papers for trial questions, and students’ solutions were hardly harmonized. The exercise lasted for twelve weeks, after which a post-test similar to the pre-test was administered to the participating students. Note that a researcher was present for all lessons across all groups. This was possible because different schools taught the topic of waves at different periods in the term. So, throughout the whole period of data collection, the authors were in the field. The notes from these observations can be found at Kanyesigye et al. (2022c).

Data collection methods

Data was collected using the Views About Science Survey (VASS) developed by Ibrahim Halloun (Halloun, 1996; Halloun & Hestenes, 1996a, 1996b) focused on attitude. “Attitudes and beliefs surveys that are commonly used in physics courses (such as VASS) are about how students perceive the discipline of physics or their particular physics course” (Madsen et al., 2020, p.90). This tool is a valid and reliable tool to fit our research objectives and has two versions, P20, and P05.07 (Halloun & Hestenes, 1996b). We used the current P05.07 version accessible from the PhysPort website. This version contains 33 items that measure students learning outcome before and after covering a certain content. VASS is not multiple-choice¹, items are formulated based on Halloun’s Contrasting Alternatives rating scale (Cars) and done in 40 minutes. Items are rated on a 5-point scale and each scale an ‘a’ revealing negative attitude and a ‘b’ statement revealing a positive attitude (see Figure 1). The first scale (a) >> (b) means mostly (a), rarely (b); the second scale (a) > (b) means more (a) than (b); the third scale (a) = (b) means equally (a) and (b); the fourth scale (b) > (a) means more (b) than (a); and the fifth scale (b) >> (a) means mostly (b), rarely (a).

Figure 1. Example of Views About Science Survey type of item and its rating.

To ensure content validity of the study instruments, the instrument was presented to four research experts before its adoption to ensure that it matched the problem under investigation as recommended by scores (Gay, Mills & Airasian, 2012; Samsudin et al., 2019). The experts were selected from Secondary Science and Mathematics Teachers (SESEMAT) association in Uganda, and from the department of science education in the University of Rwanda College of Education and Mbarara University of Science and Technology and were requested to indicate whether the items were relevant to the problem under investigation. After collecting their opinions on every item, the content validity ratio (C.V.R) for each item was calculated based on the formula by Lawshe (1975):

In this study (N = 4 and n_e = 3), when the C.V.R. values for all items were averaged, the items with the CVR bigger than 0.49 remained, and the Content Validity Index (C.V.I.) was obtained as 0.98. Comparing this value with the critical value of 0.99 (Lawshe, 1975), the difference was considered good hence the instrument was considered valid.

Suppose a research instrument is to be considered reliable. In that case, it must prove that if it were to be used on a similar group of participants under the same conditions, it would still result in similar results (Cohen et al., 2007). Achievement of consistency gives the researcher assurance that the results obtained represent the achievement of the individual participants (Fraenkel et al., 2012). After the survey was accepted for adoption, a pilot study was administered to 42 randomly selected students among the study participants. The internal consistency of the study instrument was determined by computing the Cronbach’s Alpha which gave a 0.73 coefficient, and this value rendered it reliable according to Gliem and Gliem (2003). We have also employed and administered the mechanical waves concept survey (Tongchai et al., 2008), a valid and standard tool available at physport.com for triangulation purpose.

Ethical considerations

After the approval of the proposal, an ethical clearance (Ref: 03/DRI-CE/067/EN/gi/2020) was obtained from research and innovation office, University of Rwanda, College of Education and thereafter an authorisation letter to conduct research in Uganda was given by the permanent secretary, Ministry of Education and Sports – Uganda. A week before the start of data collection, participants first signed written informed consent letters. Participants who were below 18 years were asked to consult their parents and in turn, their parents consented on their behalf. Each participant was given a code and was referred to only by that code. No monetary compensation was given to participants. Participants were free to withdraw from the study at any time without penalty and were also free not to answer any questions or respond to any research situations if they chose so.

Data analysis

In VASS development, Halloun (1996) classified students’ views or attitude in three profiles; expert, transitional, and folk. We have based our analysis on these distinct categories. The most common way to score these surveys is to collapse students’ responses into two categories depending on whether they are the same as an expert physicist would give (called “percent expert like response” or “percent favorable response”) (Madsen et al., 2020). We used both MS Excel 2016 (Microsoft, 2016) (RRID:SCR_016137) and SPSS 23.0 (IBM Corp, 2015) (RRID:SCR_016479) to analyze data. We computed frequencies of students in each of the VASS categories (1-5) using “countif” function in Excel 2016. We then merged these five categories into three categories. Thus, we averaged the first two categories into folks-like attitude, last two categories into physics expert-like attitude, and the third (middle) category remained as it is and named transitional attitude. We averaged these frequencies across 33 VASS items to make displayable percentages along with the Solomon four groups. We then followed this procedure to compute frequencies across gender and school type. The “shift” in percent favorable responses is calculated by subtracting the pre-test class average percent favorable from the post-test class average percent favorable. This metric tells you how students’ expert-like or favorable beliefs about physics changed from the start to the end of their physics course (Madsen et al., 2020). To measure the correlation between students’ attitude and performance, we computed and compared the average scores of mechanical waves conceptual survey (MWCS) correct answers and average scores of VASS for each student. To measure the statistical significance between test groups and gender or school factors, we employed nonparametric tests in SPSS and computed Mann-Whitney U or Kruskal-Wallis tests where appropriate.

Classification of students’ attitudes about learning physics before and after learning through PBL

Figure 2 displays the overall results of learning physics. By considering only the groups of students who sat for both pre-and post-tests, it is seen that most of the students (66%), within the experimental group, display high and similar attitude (folks-like attitude) before learning via PBL approach. However, after learning about mechanical waves, there was a great shift from attitude in both students in control and experimental groups. The difference between these two groups was 19% more students (79% of students in experimental and 60% of students in control group) changed their attitude towards physics expert-like due to PBL. It is also noted that in the second stream of students who did not sit for pre-test, only 3% of students taught using PBL, alongside 13% of those who were taught in traditional methods, had a folk-like attitude after completing the lessons.

Figure 2. Classification of students’ attitude about learning physics before and after learning through problem-based learning.

Table 1 presents the statistical significances of the groups discussed in Figure 3 above. The students’ attitude from either control or experimental groups was similar before learning mechanical waves. However, it diverged after learning, and a physics expert-like attitude was developed more in the experimental group. Therefore, we retained the hypothesis that assumed that the distribution of mean scores of all students was the same at the pre-test stage and rejected the hypothesis that assumed the distribution of mean scores of all students was the same at the post-test stage. Nevertheless, there was no statistically significant difference (p > 0.05) between female and male students and between public and private schools both at pre- and post-test stages.

Figure 3. Independent-Samples Mann-Whitney U test across control and experimental groups.

Note: y-axis shows average scores and x-axis shows number of students in %.

Figure 2 displays the VASS mean scores of students in control and experimental groups at pre- and post-test stages. It can be visualized that students’ attitudes shifted to higher scores in the experimental group than the control group after learning mechanical waves (at the pre-test stage). Note that higher scores refer to a more “expert” like attitude.

Figure 4 displays box and whisker plots across Solomon’s four-group design. It is seen that the experimental group (those who learned with PBL) either performed both pre- and post-test or those who performed only post-test had similar high VASS average scores compared to their counterparts in the control group (those who learned with chalk and board).

Figure 4. Independent-Samples Kruskal-Wallis test across Solomon four-group design.

We have investigated two factors (students’ gender and school type/ownership) that could influence students’ attitude toward learning physics when learned with or without PBL.

Case 1.Gender difference

Since gender showed no statistical (p > 0.05) effect (see Table 1), Figure 5 displays the classification of students’ attitude about learning physics before and after learning through PBL according to gender. Thus, both male and female students were able to shift from folks-like to physics expert-like attitude after learning mechanical waves. The sample for female students at both pre- and post-test was 53 while that of male was 79 in experimental group. Likewise, there wre 39 female and 68 male students in control group.

Figure 5. Classification of students’ attitude about learning physics before and after learning through PBL according to gender.

It can be seen that 65% female students had folk-like views in pre-test that shifted to 83% expert-like in post-test due to the PBL intervention. A similar shift in male students was observed from 66% to 77%, from folk to expert-like views. However, such big shift was not observed in a control group. For instance, the shift in expert-like views was from 8% to 60% and from 8% to 61% among females and males respectively.

Case 2. Public versus private school

Since public and private schools did not show any statistical (p > 0.05) effect (see Table 1), Figure 6 displays the classification of students’ attitude about learning physics before and after learning through PBL across the type of schools. Thus, students in public or private were able to shift from folks-like to physics expert-like attitude after learning mechanical waves. The sample for students from government school at both pre- and post-test was 39 while that of those from private was 92 in experimental group. Likewise, there were 55 students in government school and 51 students in private school.

Figure 6. Classification of students’ attitude about learning physics before and after learning through PBL according to public or private school.

It can be seen that 66% government students had folk-like views in pre-test, that shifted to 80% expert-like in post-test due to the PBL intervention. A similar shift in private students was observed from 66% to 78%, from folk to expert-like views. However, such big shift was not observed in a control group. For instance, there was only 60% students from government school and 60% students from private school shifted to expert-like views.

Correlation between students’ attitude and performance

After measuring the effect of PBL on students’ attitude, we went further and correlated the scores from the MWCS and VASS scores. Figure 7 presents these results. Surprisingly, a positive attitude, such as physics expert-like, was found to not significantly correlate with students’ performance scores. A small correlation coefficient of 0.35 was found in overall students at the post-test stage. This was large compared with a very small correlation coefficient of 0.02 found in overall students at the pre- test stage. Note that a correlation between 0.00 and 0.49 is considered low, 0.50 and 0.69 is considered medium, while 0.70 and 1.00 is considered as high.

Figure 7. Correlation scores between MWCS and VASS across Solomon four-group design.

Specifically, a negative correlation (but very small) coefficient (-0.03) was found in a control group (in a post-test only design), while a positive correlation (but small) coefficient (0.10) was found in a control group (in a pre- and post-test design) at pre-test stage (Table 2).

Likewise, a series of negative correlation coefficients (such as -0.19 found in Experim group of pre- and post-test design) were found at the post-test stage with only a positive correlation (but a small) coefficient (.08) in a control group that performed only post-test.

Underlying data

Mendeley Data: Data for measuring impact of problem-based learning during learning mechanical waves: MWCS, VASS, RTOP. https://doi.org/10.17632/rdtcgstmps.3 (Kanyesigye et al., 2022c)

This project contains the following underlying data:

Extended data

Mendeley Data: Teacher Training in Implementing Problem-Based Learning. https://doi.org/10.17632/b28d3p7kf8.1 (Kanyesigye, 2022)

This project contains the following extended data.

Mendeley Data: Data for measuring impact of problem-based learning during learning mechanical waves: MWCS, VASS, RTOP. https://doi.org/10.17632/rdtcgstmps.3 (Kanyesigye et al., 2022c)

This project contains the following extended data.

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).