Widening gap between college students’ perceptions and abilities: The Dunning-Kruger phenomenon in critical thinking appraisal in higher education

Introduction

Critical thinking is the logical evaluation of problems or situations based on evidence to create plausible actions to be taken to solve or correct a real or hypothetical dilemma (Alsaleh, 2020; Liu & Pásztor, 2022; Sutiani, 2021). Sternberg argued that individuals are able to achieve their goals if they practice analytical, creative, and practical thinking, the elements of successful intelligence. Similarly, Carter and Kravits (2014) suggested that critical thinking allows students to succeed in college. In Keys to College Success, Carter and Kravits (2014) advocated for the active teaching of critical thinking to develop students’ autonomous analytical ability, nourish creative skills, and provide spaces for the exercise of decision-making. Thus, the educators constantly urge higher learning institutions to measure and improve critical thinking levels among their students (Erikson & Erikson, 2019; Liu & Pásztor, 2022).

One of the driving motivations underlying this research is the lack of thorough assessments of critical thinking among Middle Eastern college students. More specifically, much of the available research utilized Western translated instruments like the California Critical Thinking Appraisal or The Watson-Glasser Critical Thinking Appraisal. Such instruments are heavily focused on the analytical thinking ability element minimizing the creative or practical elements. Furthermore, most researchers have focused on actual critical thinking skills assessment ignoring the important role of critical thinking dispositions. The tendency to be inquisitive, open-minded, motivated and persevering are all essential ideation states required for the optimal execution of critical thinking. Self-reported measurement of students’ critical thinking ability on analytical, creative, and practical elements is missing from the literature, especially among Middle Eastern and North African college students (Abduljaber et al., 2025; Al-Mahrooqi & Denman, 2020; Elyas & Al-Zahrani, 2019).

The present research asks the following set of research questions. What is the prevalence of analytical, creative, and practical thinking among college students in the Middle East? To what extent do language of instruction or gender influence critical thinking self-reported ratings on analytical, creative, and practical thinking? To what degree are analytical, creative, and practical thinking independent from each other? By the same token, the purpose of the study is to estimate the levels of critical thinking among a large sample of students attending Qatar University in Doha. Similarly, the study estimates the associations between demographic factors like language and gender and critical thinking levels. Finally, the analysis examines whether the three elements of critical thinking are correlated or not.

Findings conclude that Qatar University students self-reported a very high overall rating of critical thinking. Similarly, the 803 participants indicated a very high ability in all three areas of critical thinking. The independent samples t tests reported no discernible practical differences in scores between male and female students. By the same token, being instructed in Arabic or English has minimal practical effect on critical thinking self-reporting. The correlations between analytical, creative, and practical thinking were too low to conclude that all three elements measure the same underlying construct.

The ongoing research contributes to the study of critical thinking in the Middle East in several respects. First, this investigation raises serious concerns about the potential emergence of a Dunning-Kruger Effect in this area of research. There is a clear contradiction between the very high self-reported ratings in this research and the poor performance on other assessments in prior studies. Future researchers are recommended to utilize other standardized assessments for measuring critical thinking dispositions like the California Critical Thinking Dispositions Appraisal. Second, this study emphasizes the importance of measuring all elements of critical thinking rather than simply focusing on analytical ability. The evidence shows that creative and practical thinking are equally important in developing students’ college and career readiness.

Literature review

Methods

This research follows a quantitative descriptive cross-sectional survey design. The quantitative methodology is selected because it offers the researcher the ability to estimate the prevalence of critical thinking among various samples (Abduljaber, 2018a; Abduljaber, 2018b; Onder, 2019a; Onder, 2019b), one of the goals driving this research. Additionally, descriptive designs are appropriate for majoring relationships among variables like analytical, creative, and practical thinking, another primary goal for this investigation. Furthermore, a cost effective and timely measurement approach to critical thinking is to collect respondents’ self-reported ratings of their abilities. Thus, a survey of a large sample of students attending different courses across various semesters constitutes a suitable way of obtaining data to measure the level of critical thinking among college students. Prior to data collection activities, Qatar University Research Ethics committee approved the investigation. The Qatar University Institutional Review Board (QU-IRB) approval number is QU-IRB 1487-E/21, and it was approved on February 22, 2021. Participants completed written informed consent forms before taking part in the study (see the consent form) (Yousef, W. 2025). Responses were anonymous to ensure confidentiality. No identifying information were collected throughout any phase during the research.

Data analysis

Descriptive statistics were used to estimate the prevalence of each of the critical thinking dimensions among students. First, the means of each item, as well as the dimensions, were calculated. Second, the average score for all items were calculated and considered as the total score measuring prevalence. Since there are 30 items measured on a 1-5 ordinal scale, the prevalence score ranged between 30 and 150. Higher scores corresponded to better levels of critical thinking.

Second, a series of independent t tests were performed to investigate whether gender or language of instruction carried any significant effects on observed results. Also, simple means’ comparisons were made based on the calculated averages per semester for each dimension of critical thinking. If the means’ difference between any two groups was found to be statistically significant, it was interpreted further to shed light on the potential effect of the attribute on the variable. Note that simple means’ comparisons were demonstrated using histograms and averages tables.

Furthermore, a correlational analysis between the scores of each dimension across every semester was performed. The purpose of this analysis was to investigate whether the three dimensions are related. If the correlations among all of them are high, then all of the three dimensions measure the same underlying construct. On the other hand, if the correlations among the three were low, then they do not measure the same variable. In the latter scenario, each dimension is deemed to be autonomous from other dimensions measuring an independent skill.

Figure 1 shows the prevalence of critical thinking among Qatar University 803 students who completed the Sternberg’s questionnaire. Overall, students have a very good self-reported rating of their critical thinking 120.37 on a 30-150 scale. Across all three dimensions, analytical, creative, and practical thinking, students self-reported their resemblance to the practice characteristics listed in the statements as Somewhat Like Me or Definitely Like Me. More specifically, the analytical thinking scores across the three semesters for all students who completed that portion of the questionnaire is 122. Similarly, the creative thinking score for all students was 117.87. By the same token, the practical thinking score for the entire sample was 121.25. Examining all such prevalence scores, one finds that Qatar University students rated themselves as very good as seen in the designated categories in Figure 1.

Figure 1. Prevalence of critical thinking among Qatar University students (Source: Authors).

Results

Qatar University students rated themselves high on analytical thinking. Table 2 demonstrates the means of ten items measuring students analytical thinking abilities. Note that all items possessed high means ranging between 3.9 and 4.3 (last column). Students’ self-reported perceptions about their abilities to define problems, formulate questions, use evidence, and reason with logical arguments were high regardless of gender or semester. Similarly, students self-rated themselves as creative thinkers by providing high scores on the ten items measuring various facets of innovative thinking as shown in Table 2. All items had meanings between 3.1 and 4.2 (last column). Interestingly, Qatar University students appeared to be cautious by being less willing to rate their abilities to take risks by making mistakes as high as other creative thinking items. Overall, students believe that they generate a diversity of unique ideas in group settings and find innovative solutions by defying rules and routines. In the same vein, students rated their practical thinking abilities to be high. All means of the ten items measuring practical thinking ranged between 3.5 and 4.2 (last column). Table 3 shows summation of each critical thinking dimension (i.e., analytical thinking, creative thinking, and practical thinking). Students perceived themselves as doers, problem-solvers, disciplined and organized. In sum, Qatar University students perceive themselves as analytical, creative, and practical thinkers.

Figure 2 shows students self-reported critical thinking scores on the three dimensions in each semester with available data. Note that each score is out of 50 because each dimension is measured with ten items each ranging between 1 and 5. The score is the summation of the ten items. Within each semester, analytical, creative and practical thinking averages were high and not very different. Simultaneously, the differences among scores over time were minimal. The changes in scores could be caused by a wide array of factors. Regardless of the factors generating the difference, the scores have not changed much ranging from 38 to 42.

Figure 2. Comparison of the dimensions of critical thinking in terms of semesters (Source: Authors).

Figure 3 displays students’ scores for analytical, creative and practical thinking by language of instruction in the Spring of 2020. Note that Arabic instructed students scored slightly higher than English instructed students. The differences in scores, however, were minimal. Qatar is an Arabic speaking country and Qatar University is home to a majority of Arabic speaking students. Therefore, it would be expected to experience more difficulty in courses taught in the English language causing a miniscule drop in critical thinking scores.

Figure 3. Comparison of the dimensions of critical thinking in terms of instruction language (Source: Authors).

Table 4 demonstrates the results of an independent samples t-test investigating whether sex influences total scores on critical thinking. The dependent variable is the total score resulting from the summation of all 30 items per student generating a score ranging between 30 and 150. The independent variable is the self-reported student sex. Results show that females score higher than males. Also, the means’ difference is statistically significant at the 0.01 level with a relatively large t statistic value. Therefore, females tend to be more critical thinkers compared to males at Qatar University according to the self-reported data available.

Table 5 shows the results of an independent samples t-test examining whether language of instruction affects scores on critical thinking. Note that the dependent variable is the sum of all 30 items on the test. Further, the independent variable is language measured by Arabic or English. Results show that Arabic taught students scored higher than English taught students. Nevertheless, the means’ difference is only statistically significant at the 0.10 level. Such a result alludes to the conclusion that language did not cause the differences observed in the test scores.

Table 6 shows the results of bivariate correlations between analytical, creative and practical thinking for the Spring 2020. Note that practical thinking is negatively associated with both analytical and creative thinking. Nevertheless, the magnitude of the correlations is minimal. By the same token, analytical thinking had an inverse relationship with creative thinking. All associations, however, are small or non-significant.

Table 7 shows the bivariate correlations between analytical, creative, and practical thinking for Fall 2020. Note that practical thinking is conversely related to analytical and creative thinking. More importantly, the inverse association between practical thinking and analytical thinking is stronger compared to the association between practical thinking and creative thinking. Note that the correlation between analytical thinking and creative thinking is negligible and close to zero.

Table 8 demonstrates the bivariate correlations between analytical, creative and practical thinking in the Spring of 2021. Note that analytical thinking is conversely related to creative thinking. By the same token, practical thinking is negatively associated with both analytical and creative thinking. Note, however, that the strength of such correlations is weak. It appears that if a student scores high on practical thinking, the student will likely score lower on both analytical and creative thinking.

One of the patterns worthies of mention is that Qatar University students rated their practical thinking abilities in a more favorable way compared to analytical or creative thinking. Figure 4 demonstrates the highest six means among all the items considered in this research. Note that four of them are within the purview of practical thinking. Students rated their abilities to perform or do tasks, apply learned knowledge, and completion of tasks higher than other items on the questionnaire. With respect to analytical thinking, students rated their desire to observe convincing evidence as the highest item in that dimension. Concerning creative thinking, students rated being energized when exposed to new learning experiences as the highest item within that set of indicators.

Figure 4. Comparison of the dimensions of critical thinking in terms of instruction language (Source: Authors).

Simultaneously, students rated themselves low in few items. The least six means, as seen in Figure 5, were associated with three practical thinking items, two creative thinking items, and one analytical thinking item. Shifting between tasks or problems, adjustment to new teaching styles, and time management were the least skills students rated themselves in. By the same token, allowing oneself to make mistakes was one of the least creative thinking items students rated themselves on. With respect to analytical thinking, students indicated a dislike to analyze causes and effects.

Figure 5. Comparison of the dimensions of critical thinking in terms of instruction language (Source: Authors).

Discussion

The findings in this research contradict the literature on critical thinking abilities among college students in the Middle East (Abrami et al., 2015; Chen et al., 2018; Lee et al., 2016; Miterianifa et al., 2019). Unlike past research reporting low scores on critical thinking assessments, this study documented high self-reported analytical, creative, and practical thinking scores among Qatar University students. In a related study, Yousef (2021) concluded that Qatar University students regardless of gender, language of instruction, years’ standing or major scored low on a mathematics literacy test designed to measure students’ abilities in identifying, implementing, and evaluating strategy. This result confirms the existence of the mismatch between self-reported measures and observed assessments of students’ abilities in critical thinking.

This research uncovers a Dunning-Kruger’s effect reality in the assessment of critical thinking among Arab college students like Qatar University attendees. The Dunning-Kruger effect holds that individuals rate their abilities much higher compared to their actual skills (Kruger & Dunning, 1999). One of the reasons driving this phenomenon is the fear of losing face reputation among peers and instructors. Students desire to be perceived as smart and intelligent, and therefore would likely rate their skills at unrealistically high values. By the same token, confidence is a reversed quality in modern society, and no college student aspires to be perceived incontinent in his or her circles. To appear confident, students are more likely to rate themselves as knowledgeable in areas where they truly need serious training and development. Many students believe that their skills and intelligence in one domain transfers to others. Such a thought is absolutely untenable because knowing how to solve complex mathematics problems does not translate into becoming a great English language writer.

One of the most important findings in this research is the validation of Sternberg’s three-pronged conception of critical thinking (Sternberg, 1999). Unlike other researchers who believe that critical thinking is a unidimensional construct, this study documented low bivariate correlations between analytical, creative, and practical thinking. Such a conclusion leads to the belief that each domain of the three critical thinking possesses unique skills independent of others (Yousef, 2021). This result challenges existing critical thinking instructional models that overwhelmingly focus on analytical ability more than creativity or decision making and implementation.

Conclusions

This investigation examined critical thinking dispositions among college students from their perspectives. Results indicated that students rate their analytical, creative, and practical thinking as high. Additionally, gender and language of instruction seem to have little to no effect on students’ critical thinking ratings. The correlational analysis reported low associations between the three types of critical thinking pointing to the validity of Sternberg’s measurement model. The three distinct dimensions are independent from each other, noting a need for teaching all of them at the college level rather than simply focusing on analytical thinking.

The evidence presented throughout this paper is important for several considerations. First, past analyses noted the limited ability of college students in critical thinking skills’ assessments. Such a trend suggests the emergence of the Dunning-Kruger phenomenon. Students may rate their skills as high. However, when tested, their actual ability is much lower. Furthermore, the rigorous testing of students ‘critical thinking dispositions is important given the limited study of the topic. The implementation of validated instruments like the California Critical Thinking Dispositions Appraisal is crucial in determining prevalence among college students’ populations especially in neglected settings like The Middle East.

Ethics and consent

Prior to data collection activities, Qatar University Research Ethics committee approved the investigation. The Qatar University Institutional Review Board (QU-IRB) approval number is QU-IRB 1487-E/21, and it was approved on February 22, 2021. Participants completed written informed consent forms before taking part in the study (see the consent form) (Yousef, W. 2025). Responses were anonymous to ensure confidentiality. No identifying information were collected throughout any phase during the research.