Faculty expectations and academic achievement: a multilevel analysis of the Pygmalion Effect in university students

2.1 Instruments

For the development of this study, two levels of analysis were established: Pygmalion from the faculty level and the student level. For the faculty level, the following instruments were applied: the Teacher Expectation Questionnaire (TEQ), aimed at measuring faculty expectations; the Implicit Theories of Intelligence Scale (ITIS), focused on the fixed or malleable view of intelligence; and the Student Potential Perception Scale (SPPS), which evaluates faculty perception of student potential.

For the student level, the Student Perception of Expectations (SPED), which collects the student’s perception of the teacher’s expectations, and the Classroom Climate Scale (CCS), which measures classroom climate from the student perspective, were utilized.

All instruments were adapted and validated specifically for this research, based on previously established theories and constructs, which allowed for structuring key dimensions for the analysis of the Pygmalion effect and identifying its main components in the higher education context.

Classroom climate was assessed using an adaptation of the Classroom Climate Scale (CCS), developed and validated for the Ecuadorian context based on both Latin American and international theoretical and empirical references. The version used was grounded in three key studies: the three-dimensional model proposed by Sriklaub et al. (2015), oriented towards student-centered learning; the TCCS scale by Sørlie and Hukkelberg (2023), which evaluates climate from the faculty perception; and the work of López et al. (2018), who validated a classroom climate instrument in Chilean students using a mixed approach and mass application.

For the construction of the instrument evaluating the perception of faculty expectations (SPED) regarding the Pygmalion effect, the definition of the constructs was primarily based on the study by Witty and Debaryshe (1994), which allowed for structuring parallel items for faculty and students, grouped into dimensions clearly defined through factor analysis. Complementarily, empirical and theoretical references were incorporated from Ardiles and Escobar (1997), who addressed student expectations regarding the teaching role in personal, pedagogical, and technical dimensions; from Gürşimşek (1999), whose work evidenced the differential perception of teacher treatment towards students of different performance levels; and from López et al. (2018), who validated a classroom climate scale from a Latin American perspective.

The design of the Student Potential Perception Scale (SPPS) instrument was based on the study by Ceroni et al. (2016), who identified three key dimensions in faculty perception of the student body: academic potential, learning difficulties, and inappropriate behavior. Based on this theoretical and psychometric structure, the original items were adapted and expanded for application in the Ecuadorian higher education context. The questionnaire allowed for obtaining a reliable measure of the faculty’s anticipatory judgment regarding the student’s future performance, an essential component of the Pygmalion effect.

The present research relies on the work of Abd-El-Fattah and Yates (2006), who present a scale with 14 items on entity and incremental intelligence beliefs, grounded in the underlying theory. The ITIS is supported by an exploratory and confirmatory factor analysis, affirming the existence of two differentiated and psychometrically solid factors. These reliability coefficients indicate that the subscales of the obtained factors exceed.75, which is considered acceptable. Structural equivalence was also verified in student samples completing questionnaires at universities in Australia and Egypt, as well as between males and females, supporting the cross-cultural validity of the scale’s use. Additionally, the results presented by Liu (2021) are included, arguing the influence that these types of beliefs exert on achievement and intrinsic motivation. Through this scale, this work seeks to answer how the beliefs faculty hold regarding the nature of intelligence affect the expectations they set for their students, which constitutes the core of the Pygmalion effect.

The design of the Teacher Expectation Questionnaire (TEQ) instrument is fundamentally based on the theoretical model developed by Brophy (1983), which systematized research on educational expectations and the role they play in generating self-fulfilling prophecies within the school context. This author stated that the expectations education professionals have regarding their students’ capabilities translate into observable interaction patterns, particularly in affective climate, feedback, level of demand, and opportunities for participation, which configured the design of the TEQ. The contributions of Cooper (1979), who formulates a model on the diffusion of expectations and the impact they have on performance, were also added, as well as those of Rubie-Davies (2006), whose empirical research showed that teacher expectations cumulatively affect the academic self-evaluation students grant themselves.

It should be noted that each instrument was composed of various dimensions measured through Likert-type scale items. The organizational structure is detailed in Figure 1.

Figure 1. Structure of instruments, dimensions, and items applied by variable level (ND and NA).

Note: ND = Faculty Level (from Spanish Nivel Docente). TEQ = Teacher Expectation Questionnaire. ND1-D1 = Emotional climate. ND1-D2 = Feedback. ND1-D3 = Level of demand and instruction. ND1-D4 = Response opportunities. ITIS = Implicit Theories of Intelligence Scale. ND2-D1 = Fixed view. ND2-D2 = Malleable view of intelligence. SPPS = Student Potential Perception Scale. ND3-D1 = Potential in the classroom. ND3-D2 = Learning difficulties. ND3-D3 = Inappropriate behavior. ND3-D4 = Socioemotional skills. NA = Student Level (from Spanish Nivel Alumno). SPED = Student Perception of Expectations. NA1-D1 = Promotion of student participation and involvement. NA1-D2 = Personal consideration toward the student. NA1-D3 = Avoidance of negative interactions. NA1-D4 = Lack of recognition or praise. CCS = Classroom Climate Scale. NA2-D1 = Physical environment. NA2-D2 = Faculty-student interactions. NA2-D3 = Peer relationships. NA2-D4 = Faculty orientation toward learning. P = Number of questions per dimension.

2.2 Analysis of instrument reliability and validity

The validation and relevance of the instruments’ content were evaluated through expert judgment (n = 3), ensuring that the items adequately reflect and cover the construct measured in each category. Aiken’s V coefficient was utilized to quantify the consensus among the judges regarding the pertinence of each item. Subsequently, the Delphi method was applied to consolidate consensus regarding the final structure of the instruments.

Aiken’s V test showed high content validity: the TEQ obtained a V coefficient of .85; ITIS, .88; and SPPS, .90. Regarding the student level, the SPED and CCS obtained V coefficients of .87 and .89, respectively. This demonstrates that, in all cases, the items present a significant degree of agreement among the experts, supporting their content validity.

Internal consistency was evaluated using Cronbach’s α coefficient, yielding the following results: for the TEQ, a coefficient of .81; ITIS, .79; and SPPS, .85, regarding the student-level instruments, the SPED and CCS reached coefficients of .83 and .88, respectively. This reflects satisfactory and consistent reliability, confirming the psychometric adequacy of the instruments for application in the study.

2.3 Exploratory Factor Analysis (EFA)

Data processing and statistical analysis were performed using Jamovi 2.3 software, based on the R language, utilizing robust extraction methods.

Spearman correlation matrices were constructed to explore bivariate relationships between the dimensions of the instruments administered to faculty and students. This preliminary analysis allowed for evaluating the strength and direction of associations between variables, as well as verifying the statistical suitability of applying Exploratory Factor Analysis (EFA) by identifying significant covariation patterns among the items.

An EFA was conducted with the purpose of reducing the dimensionality of faculty and student perceptions and revealing the underlying structure within each dimension. In the case of students, eight theoretical dimensions related to classroom climate, faculty expectations, and negative interactions were grouped. For this purpose, the Minimum Residuals (MinRes) method was employed—suitable when multivariate normality is not assumed—along with an orthogonal Varimax rotation to obtain interpretable factors.

The number of factors was determined using Horn’s Parallel Analysis, which compares actual eigenvalues with those generated from random matrices, supported by Kaiser’s criterion (eigenvalues >1). This procedure indicated the retention of three factors which, after Varimax rotation, collectively explained 69.23% of the total variance:

In terms of proportional explained variance, FE1 accounted for 42.25%, FE2 for 29.16%, and FE3 for 28.60% of the factorial model. Factor loadings greater than 0.40 were considered significant, ensuring that each item was solidly associated with a single factor, thus guaranteeing a clear and coherent conceptual structure for subsequent use in regression models.

The EFA for the faculty scale was conducted using the same methodological strategy, applying the MinRes method and Varimax rotation. In an initial stage, all proposed dimensions were included; however, the “Opportunities” dimension (ND1_D4) did not show a clear dominant loading, and was therefore excluded from the model to ensure parsimony.

Subsequently, three factors emerged with loadings greater than.40, labeled according to the predominant dimensions:

These three factors explained 59.20% of the total cumulative variance. Of the variance explained by the model, FD1 contributed 34.83%, FD2 contributed 35.20%, and FD3 contributed 30.08%, reflecting a balanced distribution among the identified constructs.

2.4 Multilevel regression analysis

To evaluate the predictive capacity of the factors identified in the EFA regarding academic achievement, a hierarchical modeling strategy was followed. In a first phase, a robust regression model was fitted, including exclusively the student-level factors (Level 1). In this initial stage, all student dimensions showed statistically significant effects on academic performance.

Subsequently, to incorporate the hierarchical structure of the data and comply with the multilevel approach, faculty-level factors (Level 2) were introduced, specifically those linked to expectations (FD2). The inclusion of second-level predictors generated a variation in the student-level coefficients, showing that faculty expectations act as a determinant predictor of performance.

Thus, it was possible to demonstrate that the inclusion of the second level of analysis offers a more complete perspective of educational dynamics, suggesting that the effect of student perceptions can be moderated or enhanced through faculty expectations.

The proposed hierarchical multilevel model was:

Where:

$Y_{\mathit{ij}}$ : academic achievement of student i in classroom j.

${\beta }_0$ : global intercept (general average of achievement).

$X_{1\mathit{ij}}$ : student factor 1 (FE1 – Classroom Climate).

$X_{2\mathit{ij}}$ : student factor 2 (FE2 – Negative Interactions).

$X_{3\mathit{ij}}$ : student Factor 3 (FE3 – Faculty Expectations).

$Z_j$ : faculty factor (FD2 – Faculty Expectations).

${\beta }_1\dots {\beta }_4$ : estimated regression coefficients (fixed effects).

$u_{0j}$ : random effect of the faculty level (variance between professors).

${\epsilon }_{\mathit{ij}}$ : random error of the student level (residual variance).

This mathematical model validates the multilevel analysis by decomposing the error term into two components: the variability attributed to the faculty member ( $u_{0j}$ ) and the individual variability of the student ( ${\epsilon }_{\mathit{ij}}$ ).

3.1 Descriptive analysis and factor structure

Initially, an Exploratory Factor Analysis (EFA) was conducted with the purpose of revealing the latent structure underlying the evaluated dimensions. This procedure allowed for the identification of several distinct conceptual factors for both faculty and students based on the original dimensions.

Based on this factorial structure, composite variables were generated which were subsequently employed as predictors in the regression models. These models allow for analyzing how the perceptions collected through the Faculty Level (ND) and Student Level (NA) instruments are associated with academic performance, empirically evidencing the factors that configure the Pygmalion effect in higher education.

In Figure 2, significant positive associations are evidenced among the Emotional Climate (ND1-D1), Feedback (ND1-D2), and Level of Demand and Instruction (ND1-D3) dimensions of the TEQ instrument. Spearman correlation coefficients of $r=.55$ between Emotional Climate and Feedback, $r=.62$ between Emotional Climate and Level of Demand, and $r=.64$ between Feedback and Level of Demand are identified, all with high statistical significance (p < .001). This covariation suggests the presence of a strong shared latent component linked to explicit expectations.

Figure 2. Spearman correlation matrix, distributions, and scatterplots among faculty-level dimensions.

The main diagonal displays the distribution of each variable. The lower triangle presents the scatterplots, and the upper triangle indicates the Spearman correlation coefficients (r). Asterisks indicate statistical significance: * p < .05, ** p < .01, *** p < .001. ND = Faculty Level (from Spanish Nivel Docente).

Regarding the Malleable View of Intelligence (ND2-D2) dimension, results show a significant correlation with the Potentialities (ND3-D1) dimension ( $r=.64$ , p = .001). In contrast, the Learning Difficulties (ND3-D2) and Inappropriate Behavior (ND3-D3) dimensions present weaker correlations ( $r$ between .40 and .43) with the remaining variables, and in several cases do not reach statistical significance. This reinforces the notion that these variables represent differentiated constructs with a lower degree of structural dependence within the faculty model.

In Figure 3, significant positive correlations are observed among the student-level dimensions. Of note is a strong association between Faculty-student interactions (NA2-D2) and Peer relationships (NA2-D3), with a coefficient of $r=.72$ (p < .001), as well as between Physical environment (NA2-D1) and Faculty-student interactions ( $r=.66$ , p < .001). These results reflect a structured covariation pattern among the Classroom Climate Scale (CCS) dimensions.

Figure 3. Spearman correlation matrix, distributions, and scatterplots among student-level dimensions.

The main diagonal displays the density of the variables. The upper triangle contains the correlation coefficients (r) and the lower triangle the corresponding scatterplots. * p < .05, ** p < .01, *** p < .001. NA = Student Level (from Spanish Nivel Alumno).

Likewise, relevant correlations are identified among the SPED instrument dimensions, particularly between Promotion of student participation and involvement (NA1-D1) and Personal consideration toward the student (NA1-D2) ( $r=.68,p<0.001$ ), as well as between Avoidance of negative interactions (NA1-D3) and Lack of recognition or praise (NA1-D4) ( $r=.71,p<.001$ ). These findings suggest a coherent student perception regarding faculty treatment and its emotional and motivational implications.

Figure 4 presents the faculty-level factor loadings, organized into three latent factors.

Figure 4. Exploratory factor model and standardized loadings of the faculty level (ND).

The diagram shows the factor loadings (λ) obtained in the EFA. FD1 = Theories of intelligence; FD2 = Faculty expectations; FD3 = Perception of potential. Curved arrows indicate correlations between latent factors. ND = Faculty Level (from Spanish Nivel Docente).

The first factor, Theories of Intelligence (FD1), integrates Fixed View (ND2-D1) (λ = 1.00) and Malleable View (ND2-D2) (.68). Both dimensions, evaluated with the Implicit Theories of Intelligence Scale, reflect faculty beliefs regarding the student’s cognitive plasticity. The high loading of the Fixed View indicates that this dimension centrally defines the construct.

The second factor, Faculty Expectations (FD2), groups the dimensions of the Teacher Expectation Questionnaire: Emotional Climate (ND1-D1) (.59), Feedback (ND1-D2) (.90), and Level of Demand and Instruction (ND1-D3) (.64). These dimensions evidence that expectations are operationalized in the creation of an affective environment, the quality of feedback, and academic rigor.

The third factor, Perception of Student Potential (FD3), derived from the Student Potential Perception Scale, includes Potential in the Classroom (ND3-D1) (.79), Learning Difficulties (ND3-D2) (.63), and Inappropriate Behavior (ND3-D3) (.76). The lowest loading was observed in Socioemotional Skills (ND3-D4) (.42). These associations indicate that faculty judgment is a multidimensional construction encompassing positive attributes and perceived deficits.

Finally, the Response Opportunities (ND1-D4) dimension was excluded from the model due to presenting an insufficient loading.

Figure 5 details the factorial structure of the student level, identifying three conceptual factors that explain student perception regarding the Pygmalion effect in the university context.

Figure 5. Exploratory factor model and standardized loadings of the student level (NA).

Factor loadings (λ) are presented on the unidirectional arrows. FE1 = Classroom Climate; FE2 = Negative Interactions; FE3 = Faculty Expectations. NA = Student Level (from Spanish Nivel Alumno).

The first factor, Classroom Climate (FE1), groups the dimensions of the Classroom Climate Scale. The Faculty-student interactions dimension (.96) acts as the principal component, followed by Peer relationships (.68), Physical environment (.63), and Faculty orientation toward learning (.61). This reflects high structural cohesion, suggesting that the pedagogical relationship is the core that defines the student’s global perception of the environment.

The second factor, Negative Interactions (FE2), integrates Avoidance of negative interactions (NA1-D3) (.99) and Lack of recognition or praise (NA1-D4) (.71). These dimensions describe a pattern of emotional distancing and absence of faculty encouragement.

The third factor, Faculty Expectations (FE3), is composed of Promotion of student participation and involvement (NA1-D1) (.86) and Personal consideration toward the student (.72). This construct reflects the support, trust, and motivation that the faculty member projects.

Based on the analysis of the maximum loadings (.96 for interaction and .99 for avoidance), it is evidenced that behavioral and affective components are the central markers of the student experience.

3.2 Determinants of academic achievement: Regression models

The relationship between faculty- and student-level factors and academic achievement is fundamental to understanding how perceptions and beliefs influence student performance. Regression analysis allows for the estimation of the specific weight of each identified factor.

To analyze the joint effects, a model with six variables was initially fitted ( Table 1). However, preliminary multicollinearity analysis revealed significant statistical redundancy among the faculty-level factors. Furthermore, substantial shared variance was evidenced between Theories of Intelligence (FD1) and Perception of Potential (FD3) with Faculty Expectations (FD2). This suggests that explicit expectations (FD2) act as the integrating construct that crystallizes underlying beliefs. Consequently, considering model parsimony and estimation stability, FD2 was retained as the main predictor.

Table 2 presents the fit of the hierarchical multilevel model. At Level 1 (Student), it is observed that FE1 Classroom Climate has a positive and significant effect (B = 0.10; p < .05), implying that as the perception of the environment improves, average performance increases. FE2 Negative Interactions shows a positive coefficient (B = 0.09), although its significance level (p = .092) suggests a marginal effect. Regarding FE3 Student-Perceived Faculty Expectations (p = .055) results indicate a favorable positive trend.

At the Faculty Level, FD2 Faculty Expectations presents a strong coefficient (B = 1.18; p < .001). This indicates that faculty expectations constitute the most relevant predictor of academic achievement, surpassing individual variables. This finding provides solid evidence of the Pygmalion effect in university environments.

This allows for generating the equation:

Where:

$Y_{\mathit{ij}}$ : estimated academic average of student i with faculty member j.

$0.08$ : global model intercept.

$\mathit{FE}{1}_{\mathit{ij}}$ : perceived classroom climate (NA).

$\mathit{FE}{2}_{\mathit{ij}}$ : negative interactions (NA).

$\mathit{FE}{3}_{\mathit{ij}}$ : perceived expectations (NA).

$\mathit{FD}{2}_j$ : faculty expectations (ND).

${\epsilon }_{\mathit{ij}}$ : residual random error.

The coefficient associated with FE1 (0.10) indicates that for each unit increase in positive climate perception, a 0.10-point increase in the average is expected. The coefficient corresponding to FD2 (1.18) suggests that a one-unit increase in faculty expectations is associated with a 1.18-point increase in the group’s average academic achievement. This effect was statistically significant (p < .001), conferring high inferential robustness.