CATPO as an Innovative Evaluation Framework for GASING-Based Mathematics Learning: Bridging Educational Evaluation and Instructional Innovation

2.1. CIPP model in mathematics education evaluation

The CIPP (Context, Input, Process, Product) model is one of the most widely used frameworks for evaluating educational programs, including mathematics learning. Its comprehensive structure allows evaluators to assess needs, resources, implementation quality, and outcomes simultaneously. In elementary education, the CIPP model has been shown to provide a systematic basis for evaluating mathematics instruction by examining curriculum relevance, teaching practices, and student achievement (Qadriah et al., 2022). At the secondary level, curriculum evaluations using CIPP reveal its effectiveness in identifying strengths and weaknesses in mathematics programs, particularly in relation to instructional design and learning outcomes (Keskin, 2020; Ojimba et al., 2022). In higher education contexts, CIPP has been employed to construct evaluation index systems for teaching quality, demonstrating its adaptability to different institutional settings (Li & Hu, 2022). These applications indicate that the model supports evidence-based decision-making and continuous improvement. However, most studies emphasize program management aspects rather than detailed classroom interactions, suggesting limitations when evaluating innovative pedagogies that rely heavily on teaching–learning processes.

2.2. Innovations in mathematics education

Contemporary mathematics education is undergoing significant transformation driven by the need to develop higher-order thinking skills, creativity, and real-world problem-solving abilities. Reviews of modern instructional approaches highlight a shift toward student-centered learning, inquiry-based methods, and collaborative problem-solving (Grace & Gayathripriya, 2025; Luzano, 2025). Such innovations aim to move beyond procedural fluency toward deep conceptual understanding. Various instructional strategies have been explored to support these goals. Community-based learning approaches, for instance, connect mathematical knowledge with real-life contexts, thereby enhancing problem-solving competence and learner motivation (Dankunprasert et al., 2025). Tiered instruction and discipline-based competitions have also been shown to foster students’ innovative capabilities by accommodating diverse learning needs (Wang, 2025). Evidence-based pedagogical strategies further emphasize the importance of active engagement, formative assessment, and meaningful learning experiences (Tang, 2025). In specialized fields such as engineering education, mathematics instruction increasingly integrates interdisciplinary applications to improve relevance and transferability of knowledge (Pepin et al., 2021). Additionally, evaluation practices themselves are evolving alongside instructional innovations, particularly within flipped and blended learning environments where traditional assessment models may be insufficient (Yangari & Inga, 2021).

2.3. Technology-enhanced mathematics learning and evaluation

Technological advancement has introduced new possibilities for mathematics teaching and learning, including digital platforms, simulations, augmented reality, and interactive tools. Research shows that constructivist digital learning environments can significantly enhance students’ problem-solving abilities by supporting exploration, visualization, and immediate feedback (Chaiarwut et al., 2025). Similarly, technology-based mathematics games integrating augmented reality and haptic feedback have demonstrated effectiveness in reducing anxiety and increasing engagement among primary school students (Andryananda et al., 2025). Technology also plays a crucial role in expanding educational access, particularly in underdeveloped or remote regions. Integrating local culture with technological innovation can strengthen educational independence and sustainability (Mara, 2025). However, the success of technology-enhanced learning depends on alignment with pedagogical objectives and contextual conditions. Consequently, evaluation frameworks must capture not only learning outcomes but also usability, engagement, and implementation fidelity. Furthermore, equipping educators with the necessary digital competencies is just as critical as the technology itself to ensure these frameworks are effectively applied in diverse classroom settings.

2.4. Humanistic and 21st-century approaches in mathematics teaching

Recent research emphasizes the importance of humanizing mathematics education by connecting abstract concepts to students’ real-life experiences and socio-cultural contexts. In technical and vocational education settings, humanistic pedagogy has been found to improve student engagement, confidence, and perceived relevance of mathematics (Vimbelo & Bayaga, 2024). Moreover, models grounded in 21st-century competencies highlight inquiry, collaboration, and knowledge construction as essential elements of effective learning. Studies on contemporary mathematics learning models demonstrate that investigation and construction stages support deeper understanding and critical thinking skills (Tampa et al., 2024). These perspectives reinforce the need for evaluation approaches that consider not only cognitive outcomes but also affective and contextual dimensions of learning.

2.5. Stake countenance model in educational evaluation

The Stake Countenance evaluation model provides a descriptive–judgmental framework focusing on antecedents, transactions, and outcomes. Unlike decision-oriented models, it emphasizes the comparison between intended plans and actual implementation, enabling evaluators to identify discrepancies that affect program effectiveness. This model has been applied to assess various educational programs, including competency development and readiness outcomes among students (Kenaphoom, 2025). Its strength lies in capturing qualitative aspects of instructional processes, classroom interactions, and learner experiences. Such insights are particularly valuable when evaluating innovative teaching methods where implementation quality significantly influences results. Nevertheless, when used alone, the model may lack the systemic perspective needed to analyze broader contextual and organizational factors.

2.6. Need for an integrative evaluation framework for innovative mathematics learning

The literature indicates that mathematics education today is characterized by pedagogical innovation, technological integration, and diverse learning contexts. While the CIPP model provides a comprehensive macro-level evaluation of programs, the Stake Countenance model offers detailed micro-level analysis of instructional processes. Neither model independently captures the full complexity of innovative learning approaches such as GASING-based mathematics instruction. Therefore, integrating these complementary frameworks can produce a more holistic evaluation system that examines context, resources, implementation processes, instructional interactions, and outcomes simultaneously. The proposed CATPO model seeks to address this need by bridging traditional program evaluation with instructional innovation. Such a framework is expected to generate comprehensive evidence for improving mathematics education practices, particularly in primary school settings where foundational numeracy development is critical.

3.1. Research design

This study employed a mixed methods research design to develop and apply the CATPO evaluation framework for GASING-based mathematics learning. Mixed methods research integrates quantitative and qualitative approaches to obtain a comprehensive understanding of complex educational phenomena, particularly when evaluating instructional innovations and program effectiveness (Creswell & Creswell, 2018). Quantitative data were used to assess measurable outcomes and patterns, while qualitative data provided in-depth insights into implementation processes, contextual factors, and participant experiences. The design aligns with contemporary mathematics education research, where innovative teaching methods often require multidimensional evaluation combining performance indicators, observational evidence, and stakeholder perceptions (Saparbayeva et al., 2025; Odekeye & Jita, 2025). Such integrative approaches are especially relevant for programs that aim to transform learning practices rather than merely improve test scores.

3.2. Research context and participants

The study was conducted among primary school mathematics teachers implementing GASING-based instruction. Teachers play a central role in translating pedagogical innovations into classroom practice; therefore, their competencies, instructional decisions, and perceptions are critical components of program evaluation. Research on mathematics education consistently highlights that the effectiveness of innovative approaches depends heavily on teacher readiness, contextual adaptation, and instructional strategies (Cevikbas & Kaiser, 2023; Mashingaidze, 2024).

A total of 125 teachers participated in the study. Participants were selected using purposive sampling to ensure that all respondents had direct experience implementing GASING-based mathematics instruction. This sampling approach is appropriate for evaluation studies focusing on specific program implementation contexts where expertise and involvement are required.

The sample represented teachers from diverse primary school settings, including urban and semi-urban schools with varying levels of resources, student characteristics, and institutional support. Such diversity was intended to capture a broad range of implementation conditions and enhance the ecological validity of the findings. Participation was voluntary, and all respondents provided informed consent prior to data collection. The sample size was considered adequate for descriptive evaluation and mixed-methods analysis aimed at assessing program effectiveness and implementation processes.

3.3. Development of the CATPO evaluation framework

The CATPO model was developed through the integration of the CIPP evaluation model and the Stake Countenance approach to create a holistic framework capable of assessing both systemic and instructional dimensions of the program. The CIPP model provides a decision-oriented structure for evaluating context, input, process, and product, making it suitable for curriculum and program assessment across educational levels (Dizon, 2023; Bilgin & Yıldız, 2024). It has been widely applied in evaluating teacher education, curriculum implementation, and self-learning programs (Alhajia et al., 2020; Al-Shanawani, 2019). However, CIPP primarily emphasizes program management and outcomes. To capture detailed instructional transactions and classroom dynamics, elements of the Stake Countenance model were incorporated. The resulting CATPO framework was designed to evaluate antecedent conditions, implementation processes, learning interactions, and outcomes simultaneously, thereby bridging macro-level program evaluation with micro-level instructional analysis.

As illustrated in Figure 1, the CIPP model conceptualizes evaluation as a linear system consisting of Context, Input, Process (implementation), Product, and Outcome. This approach emphasizes decision-oriented evaluation, focusing on program planning, resource allocation, implementation quality, and results. In contrast, the Stake Countenance model organizes evaluation into Antecedents, Transactions, and Outcomes, with an additional judgment component that compares intended plans with actual implementation. This model highlights the interactive processes occurring during program execution and the qualitative dimensions of program performance.

Figure 1. Integrative structure of the CATPO evaluation framework based on the CIPP and stake countenance models.

The CATPO framework synthesizes these perspectives into a unified structure tailored to the evaluation of numeracy learning programs using the GASING method in elementary schools. Specifically, CATPO adopts the contextual orientation of CIPP while incorporating the process sensitivity of the Countenance model.

3.4. Instruments and data collection

Data were collected using multiple instruments to ensure comprehensive evaluation:

The use of multiple data sources aligns with best practices in program evaluation and strengthens the credibility of findings through triangulation.

3.5. Data analysis procedures

Quantitative data were analyzed using descriptive statistics to summarize trends in implementation and outcomes. Where appropriate, inferential techniques were applied to examine relationships among variables. Qualitative data from interviews and observations were analyzed using thematic analysis, allowing patterns related to instructional innovation, challenges, and contextual influences to emerge. This combined analytic approach reflects current trends in mathematics education research, where both performance data and experiential insights are needed to evaluate innovative teaching methods (Tirado-Olivares et al., 2025; Paul et al., 2025). The integration of findings from different data types enabled a comprehensive assessment of program effectiveness.

3.6. Ethical considerations

This study adhered to established ethical standards for research involving human participants. Ethical approval was obtained prior to data collection from the Research Ethics Committee of Universitas Negeri Yogyakarta, Indonesia, with approval number 753/UN34.17/LT/2026, covering the research period from 23 February to 30 April 2026. The study was conducted in accordance with applicable national regulations and internationally recognized ethical guidelines for educational research. Participation in the study was entirely voluntary. Informed consent was obtained verbally from all participants prior to their involvement. Verbal consent was considered appropriate due to the minimal-risk nature of the study and the educational setting, where formal written consent procedures were not deemed necessary and could potentially disrupt natural participation. Before providing consent, participants were clearly informed about the study’s purpose, procedures, and their rights, including the right to withdraw at any time without penalty. To ensure confidentiality and anonymity, all identifying information was removed from the data records, and participants were assigned codes during analysis and reporting. Data were securely stored and used solely for research purposes. No personally identifiable information is disclosed in this publication.

3.7. Validity and reliability

To ensure the trustworthiness of findings, several strategies were employed:

These procedures are consistent with recommended practices for evaluating complex educational programs and innovations.

4.1. Development and validation of the CATPO model

The CATPO evaluation model was developed through a research and development process integrating elements of the CIPP model and the Stake Countenance framework. The development stages included a preliminary study, model design, expert review, main field testing, operational testing, revision, implementation, and finalization.

Figure 2 The research and development process of the CATPO evaluation model used to assess the Ende Pandai Berhitung–GASING program. The process began with a preliminary study involving data collection through interviews, observations, focus group discussions, and literature review, followed by preliminary field testing and analysis. Subsequent stages included model design, revision, main field testing, operational testing, implementation, final evaluation, and dissemination. The final output was a validated CATPO evaluation model accompanied by a guideline/manual for practical application.

Figure 2. Research and development process of the CATPO evaluation model for the ende pandai Berhitung–GASING program.

The preliminary study involved interviews, observations, focus group discussions, and literature analysis to identify evaluation needs for GASING-based mathematics learning. Findings indicated the necessity of a framework capable of simultaneously assessing contextual readiness, instructional processes, outcomes, and broader impacts.

The resulting CATPO model comprises five domains: Context, Antecedent, Transaction, Product, and Outcomes. These domains collectively represent systemic conditions, preparatory factors, instructional interactions, immediate learning results, and long-term impacts.

To guide interpretation of evaluation results, standardized performance criteria were established for each domain ( Table 1). Scores were converted into percentages of the maximum possible score and classified into three categories: Poor (0.00–49.99%), Moderate (50.00–69.99%), and Good (70.00–100%). Scores of 70% or higher indicate satisfactory program performance.

4.2. Main field testing results

Main field testing was conducted to examine the clarity, feasibility, validity, and objectivity of the CATPO model prior to large-scale implementation. Teachers who had experience with GASING-based instruction evaluated the model using structured instruments.

Table 2 presents the results of the main field testing. The model demonstrated high performance across all evaluated aspects, with scores ranging from 79.8% to 85.4%. Instrument validity obtained the highest score (85.4%), indicating that the indicators adequately captured the constructs intended for evaluation. Clarity of indicators (83.1%) and objectivity of scoring (81.9%) were also rated highly, suggesting that the model is understandable and minimizes subjective judgment. Feasibility of procedures received a slightly lower score (79.8%), though still within the “Good” category. The average score for the main field testing was 82.6%, indicating that the CATPO model is appropriate for school-based program evaluation.

4.3. Operational field testing results

Operational field testing examined the performance of the CATPO model in real evaluation settings. Participants applied the model during program implementation and assessed its usability and effectiveness.

Results are summarized in Table 3. Ease of use achieved a score of 84.7%, indicating that users found the model practical for application. Decision support value obtained the highest score (86.5%), suggesting that the model provides useful information for program improvement. Data reliability was rated at 83.8%, reflecting consistency in evaluation results across users. Time efficiency received the lowest score (77.3%), although still within the “Good” category. Overall, the operational testing results demonstrate that the CATPO model functions effectively in authentic educational contexts.

4.4. Evaluation of the GASING program using CATPO

The validated CATPO model was subsequently applied to evaluate the Ende Pandai Berhitung–GASING mathematics program. Evaluation results for each domain are presented below.

4.4.1. Context Evaluation

The Context domain assesses environmental conditions influencing program implementation, including relevance, policy support, infrastructure readiness, and student baseline readiness.

As shown in Table 4, program relevance achieved the highest score (84.2%), indicating strong alignment with local educational needs and numeracy priorities. Policy support also received a high rating (78.5%), reflecting adequate backing from educational authorities and school leadership. Infrastructure readiness scored 71.3%, suggesting generally sufficient resources with some variability across schools. Student baseline readiness obtained a score of 68.9%, categorized as Moderate, indicating variation in students’ initial numeracy competence. The overall Context score was 75.7%, classified as Good.

4.4.2. Antecedent Evaluation

The Antecedent domain examines preparatory conditions such as teacher competence, training adequacy, availability of learning materials, and administrative support.

According to Table 5, learning materials availability received the highest score (76.4%), followed by administrative support (73.2%) and teacher competence (72.6%), all within the Good category. Training adequacy scored 69.8%, categorized as Moderate, indicating that professional development support was less optimal compared to other preparatory factors. The overall Antecedent score was 73.0%, indicating satisfactory readiness for program implementation.

4.4.3. Transaction Evaluation

The Transaction domain represents instructional processes occurring during classroom implementation, including teaching strategies, interaction patterns, engagement, and classroom management.

As shown in Table 6, student engagement achieved the highest score (93.4%), followed by teacher–student interaction (91.2%). Use of GASING strategies scored 88.5%, and classroom management obtained 86.7%. All indicators were classified as Good, with exceptionally high values compared to other domains. The overall Transaction score was 90.0%, indicating very strong instructional implementation.

4.4.4. Product Evaluation

The Product domain assesses immediate learning outcomes, including cognitive and affective improvements.

According to Table 7, learning motivation received the highest score (86.9%), followed by mathematics confidence (84.5%), numeracy improvement (82.6%), and conceptual understanding (80.3%). All indicators were categorized as Good. The overall Product score was 83.6%, suggesting substantial positive effects on students’ learning outcomes.

4.4.5. Outcomes Evaluation

The Outcomes domain examines broader impacts beyond immediate classroom results, including institutional and professional effects.

As presented in Table 8, teacher professional growth achieved the highest score (81.2%), followed by school academic performance (78.4%), community perception (76.1%), and program sustainability (74.6%). All indicators were within the Good category.

The overall Outcomes score was 77.6%, indicating positive program impacts at the institutional and community levels.

4.5. Overall evaluation profile

A summary of evaluation results across all CATPO domains is shown in Table 9.

The overall program score was 80.0%, classified as Good (High Performance). Among the domains, Transaction obtained the highest score (90.0%), followed by Product (83.6%), Outcomes (77.6%), Context (75.7%), and Antecedent (73.0%). These results indicate that the program demonstrated strong performance across all evaluation dimensions, with particularly high effectiveness in instructional implementation.

4.6. Qualitative findings supporting the evaluation

Qualitative data from interviews, classroom observations, and document analysis provided additional insights into program implementation and complemented the quantitative findings. Teachers reported that GASING-based instruction increased student participation and enthusiasm during mathematics lessons. Observations indicated active involvement of students in problem-solving activities and interactive learning tasks. Teachers also noted improvements in students’ confidence when dealing with mathematical problems. However, qualitative data also revealed challenges related to training and initial implementation. Some teachers reported difficulties in adapting to new instructional strategies due to limited prior experience with the GASING approach. Variations in students’ initial abilities were also observed, influencing the pace of instruction. Overall, qualitative findings support the quantitative results, particularly the high Transaction scores and moderate Antecedent indicators.

5.1. Integrative strength of the CATPO evaluation framework

A key contribution of this study lies in demonstrating the feasibility of integrating systemic and transactional evaluation perspectives. Traditional models often prioritize either program structure (e.g., context and resources) or instructional processes (e.g., classroom interactions), but rarely both in a balanced manner. The CATPO model bridges this gap by incorporating five interrelated aspects—Context, Antecedent, Transaction, Product, and Outcomes—allowing evaluators to trace how initial conditions influence implementation quality and, ultimately, program impact. The overall results ( Table 9) show that while all domains achieved “Good” performance, Transaction obtained the highest score. This pattern suggests that effective classroom implementation can occur even when preparatory conditions are not optimal, highlighting the central role of teachers’ instructional practices in determining program success. Conversely, the relatively lower scores for Context and Antecedent indicate that systemic readiness remains an important but often underdeveloped component of educational reform initiatives.

5.2. Importance of contextual and preparatory conditions

Findings from the Context evaluation ( Table 4) reveal strong program relevance and policy support but moderate student baseline readiness. This suggests that the program addresses genuine educational needs, yet variations in initial learner competence may influence learning trajectories. Similarly, the Antecedent results ( Table 5) show adequate teacher competence and resource availability but highlight insufficient training intensity. These results underscore the importance of preparatory investments before implementing instructional innovations. Programs introduced without sufficient teacher development or diagnostic assessment of student readiness may achieve uneven outcomes across schools. Strengthening professional development and preparatory support could therefore enhance program effectiveness and reduce variability.

5.3. Effectiveness of instructional implementation

The Transaction domain ( Table 6) represents the core instructional processes occurring in classrooms and recorded the highest performance. High scores for teacher–student interaction and student engagement indicate that GASING-based learning successfully promotes active participation and collaborative problem-solving. This finding is particularly significant because engagement is widely recognized as a prerequisite for meaningful learning in mathematics. The strong classroom dynamics observed suggest that GASING’s emphasis on intuitive reasoning and enjoyable activities aligns well with primary students’ developmental characteristics. The method appears to reduce anxiety and increase confidence, which are critical affective factors influencing mathematics achievement. These findings reinforce the view that pedagogical innovation can substantially improve learning experiences even within existing structural constraints.

5.4. Impact on learning outcomes

Product evaluation results ( Table 7) demonstrate substantial improvements in numeracy skills, conceptual understanding, motivation, and confidence. The combination of cognitive and affective gains indicates that the program supports holistic learning rather than merely improving test performance. Such outcomes are particularly important at the primary level, where early experiences with mathematics shape long-term attitudes and achievement patterns. The Outcomes domain ( Table 8) further shows positive effects beyond the classroom, including teacher professional growth and improved institutional performance. These broader impacts suggest that the program functions not only as a teaching method but also as a catalyst for organizational development. However, the slightly lower sustainability score implies that continued support and resource allocation will be necessary to maintain long-term benefits.

5.5. Implications for educational evaluation practice

The results highlight several practical advantages of the CATPO model for program evaluation. First, the model provides a diagnostic profile of strengths and weaknesses across program stages, enabling targeted improvements rather than generalized recommendations. Second, the standardized criteria ( Table 1) facilitate transparent interpretation of scores, making evaluation findings more accessible to stakeholders. Third, the inclusion of outcome-level indicators allows evaluators to assess long-term impact, which is often overlooked in short-term program reviews. Importantly, the discrepancy between high Transaction scores and moderate Context/Antecedent scores suggests that improving preparatory conditions could yield even greater program effectiveness. This demonstrates the model’s capacity to identify leverage points for policy intervention.

5.6. Implications for mathematics education innovation

The success of the GASING program as evaluated through CATPO underscores the importance of learner-centered pedagogies in addressing foundational numeracy challenges. Traditional mathematics instruction frequently emphasizes procedural mastery, which can lead to disengagement and superficial understanding. In contrast, GASING promotes conceptual reasoning through enjoyable and accessible activities, making mathematics less intimidating for young learners. The observed improvements in motivation and confidence indicate that positive emotional experiences are closely linked to cognitive development. By fostering a supportive learning environment, the program helps students build both competence and self-efficacy. These findings support contemporary perspectives that effective mathematics education must address affective as well as cognitive dimensions.

5.7. Limitations and future research directions

Although the findings are promising, several limitations should be considered. The evaluation was conducted within a specific regional context, which may limit generalizability to other educational systems. Differences in resources, teacher qualifications, and policy environments could influence program outcomes elsewhere. Additionally, the study relied primarily on teacher-reported and school-level data; long-term tracking of student achievement was beyond the scope of the research.

Future studies should examine the CATPO model’s applicability across diverse contexts and subject areas. Longitudinal research could also assess whether gains in motivation and numeracy persist over time. Incorporating digital data sources or learning analytics may further enhance the model’s precision and scalability.

6.1 Policy implications

The results offer several important implications for policymakers and educational leaders:

6.2 Implications for practice

For practitioners, the CATPO model provides a practical tool for diagnosing strengths and weaknesses of educational programs and guiding targeted improvements. Teachers and school administrators can use the framework to monitor implementation fidelity, evaluate student progress, and inform instructional decision-making.