Validating instructional practice scale for university instructors in Ethiopia

2.1 Study design, setting, and the instrument

The purpose of this study was to validate the instructional practice tool in the context of Ethiopian public university instructors. Hence, we employed a cross-sectional descriptive survey method to gather data from the target population at certain points in time. We randomly select the southern part of the country. All eight universities identified in it, are categorized based on generation (year of establishment). Four public universities—Arbaminch, Dilla, Wachamo, and Jinka—representing the first, second, third, and fourth generations, respectively, were utilized to select participants.

The items were first developed by Abundo (2019), Benosa (2018), and Sergio (2018) to account for the instructional practice of teachers through classroom observations to compile their thesis at Bicol University. By 2022, Bibon extracted those items and compiled them in the form of a scale responded in five alternative responses - never, rare, sometimes, frequently, and always. Grounding on the constructivism of teaching and learning and the suggestion of educational institutions, Bibon (2022) classified into three categories of instructional practice. These include—planning (8 indicators), delivering (9 indicators), and assessing (8 indicators)—the scale is meant to measure the instructions used by scientific teachers. In his study, the scale’s Cronbach alpha of .86 indicated better internal consistency when assessing the construct.

2.2 Population (or participants and sampling)

The study’s target population consisted of instructors at Arbaminch, Dilla, Wachamo, and Jinka universities. Various sample sizes have been suggested to perform factor analysis. For instance, the following criteria are deemed excellent: at least ten times as many subjects as variables (Everitt, 1975; Nunnally, 1978); at least 100 subjects (Gorsuch, 1983; Kline, 1994); sample size to the number of variables (e.g., three to six subjects per variable) (Cattell, 1978); sample size-to-parameter ratio of 20:1 (Jackson, 2003); and 50 - Very poor, 100 - poor, 200 - fair, 300 - good, 500 - very good, and 1,000 or more scale of sample adequacy are excellent (Comrey & Lee, 1992). According to Comrey and Lee, a total of 1300 individuals were chosen to detect structures, representing an excellent sample size (1000) accounting for a maximum response error of 30%.

Kothari’s (2004) stratified proportional sample size formula, nh = (Nh/N)*n, was employed to draw participants proportionally from the four universities. Where N represents the entire population size, Nh represents the sample size for the h^th stratum, nh represents the sample size, and n is the sample size. Therefore, nh calculated as follows: 432 for Arbaminch University out of 1,720 instructors, 318 for Dilla University out of 1,263 instructors, 281 for Wachamo out of 1,119 instructors, and 269 for Jinka University out of 1,069 instructors, assuming N = 5,171 and n = 1300.

Since they were either inadequately completed, incomplete, or not returned, 46 response papers were removed. The 1,254-participant data were randomly divided into two groups, 627 participants in each group. The data was then utilized to find patterns of structure and confirm them using confirmatory factor analysis (627).

2.3 Procedures

2.3.1 Content validity evaluation

According to Almohanna, Win, Meedya, and Vlahu-Gjorgievska (2022), reliable instruments yield reliable data. Lawshe’s (1975) content validity quantitative evaluation method was used to evaluate each item by nine experienced subject matter experts (SMEs) from the following fields: social psychology, educational planning and management, curriculum and instructional provision, educational measurement and evaluation, and so on. The formula for computation is displayed as follows:

CVR = content validity ratio

ne = number of panellists pointing to the item as ‘essential’

N = total number of panellists

A three-point rating system was used to rank each item on the draft data-gathering tool (1 not essential, 2 useful but not essential, and 3 essential). CVR has a value between -1 and +1. The item is deemed acceptable and clear if the value is positive; it should be reworded, modified, or rejected if the value is negative; and it is deemed necessary and legitimate if 50% of the panellists in the N size assess the item as essential. In general, every item satisfies the acceptable standard of ≥.75 (Lawshe, 1975), suggesting that the items are extremely important. The overall mean of all items in the scale using the Content Validity Index (CVI) statistical technique was .88, exceeding ≥.70 standard given by Tilden, Nelson, and May (1990), and ≥.8 suggested by Davis (1992).

2.3.2 Data collection

This study was conducted in 2022 to 2023/24 academic year. The participants were recruited between May 1st and May 30th, 2023, and data was gathered from June 1st to June 30th, 2023. The questionnaire was administered in person. We gathered an endorsement letter and reached out to department heads and deans of colleges and institutes. They were given a brief overview of the study’s objectives, the possible participants, the type of data collection tool, and the typical amount of time needed to complete the questionnaire. Through establishing a communication channel with these high-ranking officials at various stages, a survey was dispersed around departmental offices. After that, it was distributed at random among instructors who provided consent until the target number of participants was attained.

2.3.3 Data analysis

Cleansing of data was done before data analysis. As a result, eight response sheets that were improperly filled out and 3 that were not returned were excluded. SPSS-23 (https://www.ibm.com/support/pages/downloading-ibm-spss-statistics-23) and a free trial version smartPLS-4 (https://www.smartpls.com/downloads/) were utilized for the data analysis. SmartPLS was used to investigate confirmation factor analysis, and descriptive statistics and exploratory factor analysis were carried out using SPSS.

2.4 Ethics and consent

The Center for Educational Research along with the Office of Research and Dissemination and the Office of Vice President for Research and Technology Transfer at Dilla University have ensured that the issue under investigation complies with academic research criteria and ethical standards on 13/01/2023.

Potential participants received brief instructions regarding the study’s overall goal and the characteristics of the data collection instrument. The representatives from the above-mentioned offices on the same date confirmed that collecting oral consent from participants is sufficient for the present study. Accordingly, we obtained verbal informed consent from each participant. The confidentiality of the participants was greatly protected by avoiding mentioning their names and other relevant identifiers during the data collecting and reporting procedure. Representatives from Center for Educational Research, Office of Research and Dissemination, and the Office of Vice President for Research and Technology Transfer at Dilla University were approved the unharmfull nature of the data collection tool and assumed the number of participants and confirmed that collecting verbal consent from participants is sufficient for the present study.

3.1 Socio-demographic characteristics

Table 2 indicates that 1,254 instructors took part. Approximately 956 (76.2%) participants were male, and 298 (23.8 %) participants were female, making up around three-fourths and one-fourth of the total, respectively. The age distribution has a mean of 34.16 years and a standard deviation of 4.37, falling between the minimum age of 28 and the maximum age of 45. This number appears to be in line with the distributions of work experiences and academic ranks. There were 1186 (94.6%) master’s degree holders, 50 (4%) PhD holders and 18 (1.4%) assistant lecturers as the final minimum size. This means that the instructors in the minimum, maximum, and average age groups will be covered, accordingly. One year of work experience at a university is the minimum, while sixteen years is the maximum. Ultimately, 852 (67.9 %) participants, or the higher two-thirds, underwent training for higher education teaching under the Higher Diploma Program (HDP). In contrast, approximately one-fourth of instructors were not.

3.2 Exploratory Factor Analysis (EFA)

We employed the Direct Oblimin with Kaiser Normalization Rotation Method, a Maximum Likelihood (ML) Extraction Method, an Eigenvalue exceeding one, and a factor loading cut-off value of .5 (greater than the default criteria, i.e., .3). By assuming that the observed variables are normally distributed, ML produces factor structures with high correlations of indicators. With big sample sizes, ML yields estimates that are effective, less skewed, and less variable.

3.2.1 Assumption test result

To move forward with EFA, multiple assumptions were examined. According to George and Mallery (2019) and Hair, Hult, Ringle, and Sarstedt (2022), the data distribution resulted in a relatively normal distribution, falling within the range of ±1, -.394 for skewness, and 0.014 for kurtosis. Other tests, such as the Kolmogorov-Smirnova, Shapiro-Wilk, and Z values or critical ratios for normalcy, have shown minor violations (skewed to negative).

Variance Inflation Factor (VIF) results for instructional planning, delivery, and assessment were 1.052, 1.049, and 1.004, respectively. Tolerance values were.95 for instructional planning, .954 for instructional delivery, and .996 for instructional assessment sub-scales. As indicated in the literature, both tests verified that there was no issue with multicollinearity with this set of data. for example, a VIF higher than 5 to 10 (Kim, 2019), VIF greater than 10 and a tolerance value < 0.10 (Hair, Black, Babin, & Anderson, 2010) indicates a potential problem of multicollinearity, a VIF < 5 (Ringle, Da Silva, & Bido, 2014; Rogerson, 2001) and even 4 (Pan & Jackson, 2008) are considered acceptable.

The internal consistency of the overall and subscale items was checked using Cronbach alpha, resulting in .874, .928, .886, .786 and instructional planning, delivery, assessment subscales and overall scale, respectively ( Table 4). As stated by Sarstedt (2019), this guarantees the measurement’s unidimensionality and sub-dimensionality nature and satisfies the need for EFA analysis (.7 minimum criteria). The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy is .846, which is around the desired (≥.70) category, according to Kaiser (1974), Hoelzle & Meyer (2013), and Lloret, Ferreres, Hernandez, & Tomas (2017). Bartlett’s Test of Sphericity, 7921.264 (p=.00), further confirms that the data are suitable for EFA analysis ( Table 3).

3.2.2 Exploratory factor analysis result

Six factors were obtained by rotating the matrix. But there wasn’t a single item loaded in the sixth factor. In the fourth factor, only two items (items 5 and 6) and in the fifth factor, only one item (item 16) were loaded. Therefore, the last three factors were eliminated since they did not meet the criteria for having three to five items in each component and were not appropriate for conducting confirmatory factor analysis, as stated by (MacCallum, Widaman, Zhang, & Hong, 1999; Raubenheimer, 2004). Additionally, there was no loading in any factor for items 8, 13, 14, 15, and 17. This means that items were loaded below the given threshold (.50). Eight items were eliminated overall.

Table 4 illustrates the rotation of eight items to the instructional assessment (IA) subscale (loading .573 to .821), four items to the instructional delivery (ID) subscale (loading .854 to .892), and five items to the instructional planning (IP) subscale (loading .569 to .874). For IA, ID, IP, and overall scale, the internal consistency or reliability values of .886, .928, .874, and .806 exhibit robust (Taber, 2018) and good dependability (Salkind, 2015; Tavakol and Dennick, 2011; Lavrakas, 2008).

According to the total variance explained analysis, the three components collectively account for 60.95% of the variance in instructional practice. This demonstrates that irrespective of rotation methods and disciplines, 50% explained variance is sufficient (Sürücü, Şeşen, & Maslakçı, 2021; Beavers, Lounsbury, Richards, Huck, Skolits, & Esquivel, 2013; Hair, Sarstedt, Pieper, & Ringle, 2012; Pett, Lackey, & Sullivan, 2003). Furthermore, instructional delivery and instructional planning variables accounted for roughly comparable variance (18.51% and 18.47%), respectively. Whereas the instructional evaluation factor explains the relatively highest share of variance (23.97%) ( Table 5).

When the sample size is 200 or higher, Cattell’s Scree Plot test is still another trustworthy method to ascertain the number of components (Sürücü, Yikilmaz, & Maslakci, 2022). Starting with the fourth component indicates a notable linear trend in the eigenvalue pattern ( Figure 1). We reasonably retained the three factors at 60.95%.

Figure 1. Scree plot factor.

Furthermore, we conducted parallel analysis to confirm wether the number of components in EFA loading and scree plot are actual factors or due to chance. Six hundred twenty seven cases, 25 items, 95% specifications, and principal components analysis method were specified. As a result, only the first three rawdata egenvalues are greater than the respective prcntyle of the random data egenvalues ( Figure 2). The parallel analysis confitrmed that three components are extracted in EFA as suggested by (O’Connor, 2000).

Figure 2. Parallel analysis plot.

3.3 Confirmatory Factor Analysis (CFA) Result

3.3.1 Common Method Bias (CMB)

CMB analysis is advised as the data samples were obtained by a questionnaire and/or all variables were obtained from the same individuals (Podsakoff, MacKenzie, Lee, & Podsakoff, 2003). As a result, we used Harman’s single-factor test to assess CMB, and the results showed that it explains 24.284% of the variation, which is lower than the 50% acceptable limit.

3.3.2 Measurement Model

Convergent validity, internal consistency reliability, and discriminant validity were examined using a reflective measurement approach. The extent to which a component is positively correlated with another factor that assesses the same construct is known as convergent validity. Factor loadings and average variance extraction were used for testing it. As a result, Items 22, 20, 23, and 25 in the instructional assessment factor and Item 1 in the instructional planning factor were loaded .398, .582, .606, .694, and .698 respectively ( Figure 3).

Figure 3. Outer loading and AVE values before modification.

According to Hair, Hult, Ringle, and Sarstedt (2016), Henseler, Ringle, & Sarstedt (2014), and Hair, Black, and Babin (2010), this indicates below the threshold (≥.7). Following a sequential removal and reanalysis of the first three relatively low-loaded items (items 22, 20, and 23), item 25 improved from .694 to .723, satisfying the threshold. As a result, according to Sarstedt, Ringle, and Hair et al. (2017), Henseler et al. (2014), and Hair et al. (2010), the Average Variance Explained (AVE) in instructional assessment also improved from .476 ( Figure 3) to .607 ( Figure 4), met the minimum acceptable criterion (>.5). To satisfy the minimally acceptable standards of item loading values and AVE values, four items—three from the instructional assessment and one from the instructional planning—were generally removed.

Figure 4. Outer loading and AVE values after modification.

Raykov’s rho coefficient, composite reliability (CR), and Cronbach alpha were employed to assess the construct validity and reliability ( Table 6). As per (Hair, Hult, Ringle, & Sarstedt, 2017; Hair, Black, Babin, & Anderson, 2010), the outcome satisfies the acceptable criterion for all (.7 to .95).

The degree of differentiation between a component and another component is known as discriminant validity. The results of the tests on the heterotrait–monotrait (HTMT) ratio of correlations between constructs and the Fornell-Larcker criterion are displayed in Table 7 and fall within an acceptable range. According to Hair, Risher, Sarstedt, & Ringle (2019); Henseler, Ringle, & Sarstedt (2014); Fornell, Larcker (1981), this indicates that each construct’s square root of the AVE (all bold crossing values) in the Fornell-Larcker criterion exceeds its intercorrelations with other constructs or greater than the absolute measure of any correlation. Henseler et al. (2014) criticized the Fornell and Larcker test for not consistently detecting the absence of discriminant validity in some study scenarios.

In order to evaluate discriminant validity, we thus looked at an alternative set of HTMT criteria. The results indicate that there is no discriminant validity issue, according to Hair et al. (2019) and Henseler et al. (2014). Because there is a positive correlation between IP and IA (r = .242, p =.00), IP and ID (r = .1, p =.00), and IA and ID (r = .137, p =.000). The ratio of correlations is also getting closer to zero. The two-tailed confidence interval at the.05 significance level does not include one.

3.3.3 Structural model

We checked the estimated model’s goodness-of-fit. There was a .08 marginal standard root means square residual (SRMR). Based on Brown’s (2015) analysis, the model is appropriate if ≤ 0.08. We also used the variance inflation factor (VIF) to assess for multicollinearity amongst the latent components. Multicollinearity problems are indicated by a VIF of greater than five (Hair, Risher, Sarstedt, & Ringle, 2019; Sarstedt, Ringle, Hair, 2017). Since all of the numbers in Table 8 are ≤ 3.529, multicollinearity is not an issue for the model.