Validity and reliability of the questionnaire of academic knowledge of teachers of basic general education

Introduction

The concern and analysis about the knowledge possessed by teachers in general, and in particular those of basic general education, persists within the academic debate and in professional practice.^1–3 The nature of teachers’ education has become an important field of study considering the role it has in society.

In Ecuador, studies on the subject are scarce. For this reason, the need arose to carry out this research, with the intention of proposing a questionnaire to determine the origin of the academic knowledge of teachers of basic general education. In the bibliographic review, the absence of instruments to understand the configuration of the knowledge that a teacher puts into practice in the classroom was determined. From the perspective of this research, this fact is decisive for the quality of education.

In recent years, several studies have addressed the impact of teachers’ professional knowledge on the quality of teaching. There is a prevalence towards the analysis of pedagogical knowledge of the content,⁴ which can be associated with a traditional conception of education.

At the same time, the academic community recognizes that a teacher’s level of knowledge is an indicator of the quality of their behaviour in the classroom.⁵ This implies understanding this fact in terms of relational categories. It is worth mentioning that Shulman⁶ was a precursor in considering that a teacher’s knowledge affects the student’s assessment of his capacity and with it of quality. This criterion becomes an important conceptual support for the analysis that occupies this work.

Additionally, it is striking that in the studies consulted,^7,8 there is no evidence that accurately determines the factors that make up the knowledge of a teacher, which function as support for the profession of teacher. It is possible that this is due to the multiplicity of ways in which knowledge is manifested and reached.

However, in mainstream literature, several authors have shown interest not only in the analysis of knowledge of the teacher,^9–11 but also in its implementation by designing tools that allow them to identify student´s learning styles and their pedagogical usefulness.^12,13

Building on this, it can be added that academic knowledge and its relationship with the teaching praxis are characterized firstly by being integrated into practice, which implies a continuous process of assimilation and organization of knowledge by the individuals involved. Secondly, they are situated within a temporal category that recognizes their emergence and evolution within a specific historical context and are related to a variety of knowledge.¹⁴ The combination of theoretical and practical knowledge that a teacher possesses is a result of their academic training and professional experience in the corresponding field.¹⁵

In other words, academic knowledge is considered an active, dynamic, and transformative relationship which is used to apprehend, understand and act on an environment. These relationships have the peculiarity of manifesting, reconfiguring and being open to debate. That is to say, these gradual development characteristics are the result of different processes, which are related to the teacher’s academic training in a specific disciplinary area, as well as their professional, affective, cultural, and social experiences that the teacher assumes.

Within this order of ideas, it should be considered that academic knowledge has been the center of attention when proposing new didactic strategies that the educator must consider according to the changes that are taking place in society. Likewise, several educational reforms that have been carried out from the point of view of teacher training take this knowledge as a starting point.

In Ecuador it is observed that in basic education, the role of the teacher is focused on the realization of micro-curricular plans and the incorporation of methodological strategies that involve the student at the center of the educational process. Additionally, it is noticeable that the teacher must develop appropriate resources and the types of evaluation to be carried out. Precisely, one of the aspects indicated to comply with the above comes from the combination of the teacher’s knowledge and professional skills.¹⁰ To achieve this goal, they must achieve an innovation of educational work with flexibility in different contexts, as well as the exercise of a systematic pedagogical praxis.

It is also linked to certain performances developed by the teacher. Important knowledges are identified: to know the area of knowledge that teaches; to know how to teach the subject; to know how to teach in general; and to know how people learn.

The work “The seven necessary knowledges for the education of the future” by Edgar Morín¹⁶ published by the United Nations Educational, Scientific and Cultural Organization (UNESCO), opened a space for reflection on education and the implications for the future of the quality of academic knowledge. In this regard, it has been possible to identify a variety of knowledge that teachers must possess (Table 1).

The aspects presented in Table 1 are based on the vision of reconfiguring the training of future teachers of basic education based on the paradigm shift of educational processes from teaching to learning. In this way, we proceed to solve those problems of praxis and those that directly affect the integral development of the student. This determines important adjustments not only of teachers, but of students and educational institutions. The former must act as true leaders and agents of change and transformation of didactic praxis.¹⁷

Another important aspect to consider is given by the notions of implicit theories,¹⁸ which configure the knowledge indicated. A review of several studies,¹⁹ reports that teachers face difficulties in practice motivated by the way they perceive the teaching of a discipline, especially in the use of strategies loaded with values for which they are not prepared.

It shifts from focusing on what teachers know about the practice to their perceptions of its possibilities. This approach changes the understanding of the social context as an important influence on the way teachers make decisions about their practice. The role of previous experiences is fundamental in teachers’ perception of specific teaching situations.²⁰

The teachers’ perception of their practice refers to how teachers interpret and view their own work in the classroom; that is, how they understand and value their own performance in teaching. This includes the understanding of their role as educators, the pedagogical strategies they possess, their relationships with students, as well as their ability to respond to unforeseen situations, among other aspects. The teachers’ perception of their practice can influence their decisions and actions in the classroom, as well as their ability to improve and adapt to new situations.

A question arises before the proposal of this systematic construction carried out by teachers: why is it necessary to understand the experience of the profession, as well as the disciplinary structure of which it was formed and its concretion in practice? The answer may be oriented to the lack or weak mastery that this can exercise on knowledge of the subject, in addition to the curriculum and vocational training dictated or oriented in a fragmented and institutional way.

A second answer can be given by the specific disposition of the pedagogical situation through the educator that implies simplifications and exemplifications that allow learning on the part of the student.^13,21

Regardless of the historical period in which they were carried out, the studies do not respond to the need to break a pre-established disposition of disciplinary knowledge as a structuring mechanism of teaching and therefore of training. It is considered vital to reconfigure the vision that the teaching profession presents in practice.

Given these dilemmas and fragmentations, it is essential to determine the nature or origin of this knowledge oriented by personal identity, the work context and the experience accumulated by teachers.

These elements constitute an argument to conduct a new study that develops the construct of academic knowledge as a central theoretical core, which contributes to a better understanding of the factors that interact with the knowledge raised and that are expressed in the development of teaching practice.

Teachers need to identify academic knowledge to plan, teach and reflect effectively on instruction²²; for this purpose, measurement instruments are limited.²³ In the literature review, no studies or research were identified that analyse the knowledge of teachers in the conditions of the Ecuadorian territory and even more so in the educational units of the Ibarra Canton; nor the nature or origin of these.

It is recognized that there is a significant interest in determining the configurations of knowledge; however, evidence of its validity is still lacking. It is precisely this aspect that this proposal seeks to resolve. The objective of the study is to establish the construct validity and reliability of the questionnaire on the nature or origin of the academic knowledge of teachers of basic general education.

This knowledge has a broad, plural, integrating connotation, linked to social aspects. The proposed conceptual delimitation limits an academic knowledge closely linked to the exercise of reflection on knowledge. In this sense, basic education teachers do not limit themselves to teaching a pre-established curriculum, but develop their didactic activity based on practical knowledge, including their values and beliefs.²⁴

This last aspect is determined by macro and micro contextual factors, in addition to national educational policies and personal experiences.²⁵ In this way, the orientation of the present research towards the daily practices that interrelate praxis, the tradition of the trade and the disciplinary structure in which the teacher has been trained is established.

Methods

Results

Of the 27 teachers of basic general education who participated in the research, 81% were female and 19% were male. The median age was 40 years (range: 24-57 years). An important factor has to do with years of teaching experience, which averages 12.37 years (range: 1-30 years). 68% of teachers have degrees in Education Sciences.

Table 2 reports the knowledge of the teachers. Table 3 reports the reliability between the different evaluators (Cohen’s Kappa) (n=8), where the result obtained was substantial (range: κ=0.61- κ=0.80) for two aspects (sufficiency and clarity) and moderate (range: κ=0.41- κ=0.60) for the remaining (coherence). The sufficiency has obtained the highest values, allowing us to consider that the different indicators belong to the same dimension considered as the academic knowledge of the teachers.

As for the descriptive analysis, it presents an average response that goes between 4 and 5, which assigns values between “Agree” and “Totally agree”, where 87% belongs to the latter value (Table 4).

Table 5 presents the analysis of global reliability of the instrument. Cronbach’s alpha is 0.901, and it is estimated as an excellent level.⁴² For the principal components analysis, a validation analysis of assumptions was developed using the Bartlett statistic and the KMO value. Bartlett’s sphericity test was used to show if there were significant correlations in the data. The values <0.05 support that the technique is adequate (165,591; gl=21; p<0.000),⁴³ the model is appropriate and does not present sphericity. The KMO sample adequacy measure was 0.778, which is considered medium.

It was considered that the recommended ratio of commonalities should be ≥ 0.40.⁴⁴ Of the 7 indicators that are part of the questionnaire, all are considered useful, and have adequate factorial weights. The interval obtained was (=6.15 and =0.96) (Table 5).

The results of the application of the principal components extraction method in the factor analysis with varix rotation indicate the retention of two factors for presenting an eigenvalue greater than 1⁴⁵; see Table 6.

When considering both, an explained variance of 82.74% is obtained. Figure 1 presents the eigenvalue scatter plot of each of the factors; 2 factors have eigenvalues higher than (>1).

Figure 1. Scatter plot of each of the factors.

Thus, the first indicator called “In the preparation of the categories I pay more attention to the knowledge of the disciplines” that retains 65.28%; and a second indicator “I evaluate the concepts of everyday life and in this way I structure the contents” with 17.46%.

Discussion and conclusions

The possibilities of academic knowledge in education have not been fully explored.⁴⁵ None of the previous studies has yet explored this knowledge, and the different configurations that support them. Although other studies have addressed the issue,^46,47 they have only done so from the perspective of teachers’ content knowledge. That is, the studies have focused particularly on teachers’ professional knowledge for effective teaching.⁴⁵

The original instrument from which this questionnaire was adapted was an important measure that can be used to determine the nature or origin of teachers’ academic knowledge. It includes several indicators that in the context of Ecuadorian basic general education have not been recognized or valued by experts previously. Previous studies have described the importance of teachers’ application of their knowledge in the classroom, which allows complex reasoning processes to be involved, selectively identifying those most relevant, from their perspective, to put them in context in a particular situation.⁴⁸

However, the relationship of the implicit theories of the teachers, in addition to the different ways that they teach their classes, have been ignored or not understood their true consequence in didactic practice. The proposal presented also shows differences with respect to the “competence model for teachers’ pedagogical content knowledge.”.⁴⁹ The reliability analysis obtained an excellent level. These values are consistent with,⁵⁰ authors who in their research obtained (α=0.981), with values greater than (α=0.80) in all indicators. This reveals a good internal consistency of the instrument, similar to Monroy.⁴⁹

In line with Şimşek,⁴⁹ the idea of considering academic knowledge as a coherent and plausible construct is supported. This makes it possible for the validated questionnaire to become an excellent candidate to identify the nature or origin of the academic knowledge of teachers of basic general education without the need to reduce or eliminate indicators from the instrument.⁵¹ Currently, there are few scales that meet these conditions: fast administration, no need to delete items, and high values of Cronbach’s alpha. It is shown that the instrument has high values of internal consistency, measured by Cronbach’s coefficient, as well as by its reliability.⁵² The scale presented constitutes a reliable tool,⁵³ to assess the nature or origin of the academic knowledge of teachers of general basic education.

According to the results of the exploratory factor analysis, the Kaiser-Meyer-Olkin value fits with other studies conducted in the area of education.^49,54 In some of these,⁵⁵ different items were removed from the scales for a factorial load value less than 0.50; however, in the current analysis the values obtained are above the standardized threshold,⁵⁶ indicating that the correlations between questions are statistically significant and sufficiently strong⁵⁷ for the analysis of questions. These considerations are strengthened by studies that support the presence of values on a scale between =0.70 and =0.95.⁵⁸ The principal components analysis using the varimax rotation method to explore the factorial structure of the teachers’ academic knowledge scale was based on the criterion (eigenvalue>1).⁵⁹ From the 7 initial indicators the procedure of reduction. That is, the questionnaire on the nature or origin of academic knowledge of teachers of basic general education represents a robust and valid framework to understand certain notions of implicit theories and the role of experiences as the basis and support of academic knowledge. In reference to the different indicators that make up the scale, they have adequate factorial weights and no item presented commonality <0.55 (ranges=0.553 and =0.795).⁶⁰ This has been supported by published and reviewed studies.^61,62 which demonstrate the usefulness and saturation of the scale through its different items in order to constitute a theoretical construct and that it allows the identification of the nature or origin of the academic knowledge of teachers quickly and reliably with an instrument composed of 7 indicators.

The present study examined the validity and reliability of a scale on the nature or origin of the academic knowledge of teachers of basic general education, which can facilitate research in this field and provides a conceptual framework to understand certain notions of implicit theories and the role of experiences as the basis and sustenance of academic knowledge. The ‘Questionnaire on the nature or origin of academic knowledge of teachers of basic general education’ presents an advantage over other instruments since it focuses on the beliefs and implicit theories that are part of the teacher’s thinking. To this end, it determines the role played by disciplinary knowledge and its didactic treatment; the definition of the sources that contribute most to the teaching exercise. In addition, it can demonstrate reflection and change during their professional life.

The levels of reliability between the different evaluators show an excellent intraobserver correlation. Evidence related to reliability analysis was collected for the entire instrument with a Cronbach’s alpha value (α=0.901), considering the 7 indicators, which is estimated as an excellent level. Bartlett’s sphericity test showed that, if there are significant correlations in the data, this supports that the technique is adequate and that the model is appropriate and does not present sphericity. Of the 7 indicators that are part of the questionnaire, all are considered useful and have adequate factorial weights. The usefulness and saturation of each of these to constitute the theoretical construct related to the academic knowledge of teachers of basic general education is demonstrated. This constitutes a relevant, valid and reliable instrument to determine the nature or origin of the academic knowledge of teachers in a fast and reliable manner.

The present study has some limitations, which are not related to the fulfilment of the proposed objective; however, it is necessary to mention them. The first is related to the absence of other questionnaires as a standard with which to compare the results obtained. We have reported on the non-existence of measures that serve as a standard. In light of this, the strategy adopted was based on the review of bibliographic material that reflected studies on the construction of similar tools from the methodological point of view and their application to education. A second limitation of the present study considers that the extrapolation of the results obtained for use in other educational units that did not participate in the sample is based on the participation of (n=27), teachers from the same number of institutions. However, this can be minimized considering the responsibility.

The Questionnaire on the Nature or Origin of Academic Knowledge of Basic General Education Teachers focuses on the implicit beliefs and theories of basic education teachers. It determines the role of disciplinary knowledge and its didactic treatment, as well as the sources that contribute the most to teaching practice and reflection during their professional life. The reliability test showed that the questionnaire is acceptable with a Cronbach’s alpha of =0.704, standard deviation values between =0.76 and =0.97, and appropriate factorial weights for each item. The questionnaire is relevant, valid, and reliable, tailored to the needs of basic general education teachers, and confirming the construct validity and reliability of the questionnaire.