An Evidence-Based DIPPO Evaluation Model for the Implementation of Project-Based Learning (P5) to Enhance 21st-Century Skills and Work Readiness

2.1 Research design

This study employed a mixed-methods evaluative research design grounded in evidence-based evaluation principles. Mixed-methods approaches are widely recognized for their capacity to provide comprehensive insights into complex educational programs by integrating quantitative measurements with qualitative contextual understanding (Omeka et al., 2024; Garcia & Cerado, 2020). The design was structured around the DIPPO evaluation framework, enabling systematic assessment of program design, implementation processes, learning products, and outcome-level impacts related to 21st-century skills and work readiness.

The evaluative orientation of this study aligns with prior evidence-based educational research that emphasizes continuous improvement and decision-oriented evaluation (Bondie, 2015; Kearney, 2016). Furthermore, the integration of mixed data sources supports triangulation, enhancing the credibility and robustness of evaluation findings, particularly in project-based and technology-enhanced learning contexts (Spatıotı et al., 2023; Patel et al., 2025).

2.2 Participants

Participants were selected using a purposive sampling strategy to ensure relevance to the implementation of Project-Based Learning (P5). The study involved key educational stakeholders, including students, subject teachers, and school administrators, who were directly engaged in P5 implementation. This sampling approach is consistent with prior evaluative and educational design studies that prioritize contextual relevance and stakeholder engagement in program evaluation (Garcia & Cerado, 2020; Zhao et al., 2022).

Student participants were drawn from secondary schools implementing P5 projects, while teacher participants included educators responsible for designing and facilitating project-based learning activities. School administrators contributed evaluative insights related to program planning, resource allocation, and outcome utilization. Such multi-stakeholder participation enabled a holistic evaluation across DIPPO dimensions and supported evidence-based conclusions relevant to both instructional practice and policy decision-making (Kearney, 2016).

2.3 Instruments

Multiple data collection instruments were developed and validated to align with the DIPPO evaluation dimensions. Quantitative instruments included structured questionnaires and rating scales designed to measure perceptions of program design, instructional processes, 21st-century skills development, and work readiness outcomes. The development of these instruments followed evidence-based evaluation guidelines to ensure content validity, clarity, and usability (Garcia & Cerado, 2020).

Qualitative data were collected through semi-structured interviews and document analysis, enabling deeper exploration of implementation challenges, pedagogical practices, and contextual factors influencing P5 outcomes. In addition, reflective records and learning artifacts produced during project implementation were analyzed to capture authentic evidence of student engagement and competency development, consistent with evidence-based pedagogical assessment practices (Bondie, 2015).

Where applicable, digital data from learning platforms and project documentation were incorporated to strengthen evaluative accuracy and support continuous improvement (Larsari et al., 2023). This multi-instrument approach ensured comprehensive data coverage across cognitive, behavioral, and contextual dimensions of learning effectiveness (Spatıotı et al., 2023).

2.4 Data analysis

Quantitative data were analyzed using descriptive and inferential statistical techniques to examine trends, relationships, and outcome patterns across DIPPO components. Descriptive statistics were used to summarize central tendencies and dispersion, while inferential analyses supported comparisons across stakeholder groups and implementation phases. Such quantitative evaluation practices are commonly applied in comprehensive educational system assessments and quality evaluation studies (Xu et al., 2022; Burns, 2024).

Qualitative data were analyzed using thematic analysis, involving systematic coding and categorization to identify recurring patterns related to program implementation, skill development, and work readiness. This approach enabled the interpretation of contextual evidence and supported the explanation of quantitative findings. In line with recent advances in educational evaluation, qualitative data were also reviewed using structured analytic frameworks to ensure consistency and transparency (Zhao et al., 2022).

To enhance analytical rigor, triangulation was conducted across data sources and methods. This process aligns with evidence-based evaluation practices that emphasize reliability, validity, and actionable insight generation (Okoye et al., 2023; Okoye et al., 2022). The integration of quantitative and qualitative findings enabled the formulation of evidence-based conclusions and informed recommendations for improving Project-Based Learning (P5) implementation through the DIPPO evaluation model.

2.5 Ethical considerations

This study was conducted in accordance with the principles of the Declaration of Helsinki and established ethical standards for educational research. Ethical approval for this study was obtained from the Research Ethics Committee of Universitas Negeri Yogyakarta prior to data collection (Approval No. T/197.5/UN34.9/PT.01.04/2025). Prior to data collection, oral informed consent was obtained from all participants, including students, teachers, and school administrators. Participants were provided with clear information regarding the research objectives, procedures, potential risks and benefits, the voluntary nature of participation, and their right to withdraw at any stage without academic or professional consequences (Kearney, 2016). Only those who verbally agreed were included in the study. For student participants under 18 years of age, oral informed consent was first obtained from their parents or legal guardians through school-mediated communication, followed by oral assent from the students themselves in language appropriate to their age and comprehension level. Oral consent was chosen instead of written consent for three main reasons: (1) to reduce administrative burden on schools and families and avoid disrupting regular teaching–learning activities; (2) to minimize discomfort or anxiety among students who might perceive written forms as formal evaluation or institutional pressure rather than voluntary participation; and (3) to respect local cultural and institutional practices in which verbal agreement is commonly accepted in school-based educational research, particularly when no physical risk is involved.

To ensure ethical accountability, the researchers documented the consent process in field notes, including the date, setting, and confirmation that participants and guardians had received adequate information before agreeing to participate. To ensure confidentiality and anonymity, all personal identifiers were removed from the datasets, and coded identifiers were used during data analysis and reporting. All data were stored securely and were accessible only to the research team. Ethical considerations related to the use of digital learning artifacts and institutional documents followed evidence-based educational research practices to prevent data misuse and ensure transparency (Bondie, 2015; Garcia & Cerado, 2020).

2.6 Validity, reliability, and trustworthiness

To ensure the rigor of the evaluation as shown as at Table 1. Operationalization of DIPPO Evaluation Indicators, this study applied multiple strategies to establish validity, reliability, and trustworthiness across quantitative and qualitative components. Content validity of the evaluation instruments was ensured through alignment with the DIPPO framework and evidence-based evaluation principles, as recommended in prior educational evaluation research (Garcia & Cerado, 2020; Spatıotı et al., 2023). Instrument items were reviewed to ensure clarity, relevance, and coverage of key constructs related to project-based learning, 21st-century skills, and work readiness. Reliability of quantitative instruments was assessed using internal consistency measures. Consistent with best practices in educational system evaluation, reliability analysis supported the stability and consistency of measurement across participant groups and implementation phases (Xu et al., 2022).

For qualitative data, trustworthiness was enhanced through triangulation of data sources (students, teachers, administrators), methods (questionnaires, interviews, document analysis), and evaluative dimensions. This approach aligns with evidence-based and mixed-methods evaluation standards emphasizing credibility, dependability, and confirmability (Okoye et al., 2023; Okoye et al., 2022). Member checking and peer debriefing were employed where feasible to reduce interpretive bias and strengthen analytic transparency.

3.1 Results of design and input evaluation

The design evaluation examined the alignment of P5 objectives with 21st-century skills and work readiness outcomes. As shown in Table 2, the Design dimension achieved a high mean score (M = 4.21, SD = 0.54), with 83.3% of respondents indicating strong agreement (≥4). These findings suggest that the conceptual design of P5 is well aligned with competency-based learning principles, supporting transferable and future-oriented skills. This result is consistent with prior studies emphasizing the importance of coherent program design in effective educational evaluation (Rozak et al., 2022; Nuha et al., 2024).

From Table 2, Summary of DIPPO Evaluation Results shown that in contrast, the Input dimension yielded a comparatively lower mean score (M = 3.78, SD = 0.63), indicating moderate readiness in terms of teacher capacity, learning resources, and technological infrastructure. This pattern aligns with previous evaluation research highlighting input variability as a common challenge in large-scale curriculum implementation (Santiko et al., 2022; Lai et al., 2022). The identification of this gap provides a clear basis for evidence-based decision-making related to resource optimization.

Figure 1, Mean scores across DIPPO evaluation dimensions, supporting comparative evaluation and prioritization of improvement areas. presents the mean scores across the five DIPPO evaluation dimensions. The Process dimension achieved the highest mean score (M = 4.32), followed by Design (M = 4.21), indicating high implementation fidelity and strong alignment with program objectives. Product (M = 4.10) and Outcome (M = 4.05) also demonstrated favorable results, reflecting effective outputs and positive program impact. In contrast, the Input dimension recorded the lowest mean score (M = 3.78), suggesting comparatively lower readiness of resources and initial conditions. Overall, the findings confirm program effectiveness while highlighting Input as a priority area for improvement.

Figure 1. Mean scores across DIPPO evaluation dimensions, supporting comparative evaluation and prioritization of improvement areas.

3.2 Results of process and product evaluation

The process evaluation revealed high levels of student engagement and instructional fidelity during P5 implementation (M = 4.32, SD = 0.49). Observational and survey data indicated active collaboration, sustained participation, and the use of differentiated instructional strategies. These findings are consistent with prior research demonstrating that project-based and blended learning environments promote deeper engagement and active learning processes (Oise et al., 2025; Hidayah, 2024).

Regarding the product evaluation, student-generated project outputs demonstrated satisfactory to high quality across creativity, teamwork, and problem-solving indicators (M = 4.10, SD = 0.57). This aligns with previous project-based learning studies emphasizing learning artifacts as valid indicators of applied competence and instructional effectiveness (Puspitasari, 2020; Marlena et al., 2025). Variations in product complexity suggest contextual influences such as project theme and resource availability, a pattern also reported in educational service quality evaluations (Rozak et al., 2022; Cook et al., 2024).

Figure 2, Radar charts of process evaluation indicators—engagement, collaboration, initiative, and problem solving—from the perspectives of students, teachers, and administrators, illustrating variations in perceived process quality and implementation fidelity presents three radar charts comparing process evaluation indicators—engagement, collaboration, initiative, and problem solving—from the perspectives of students, teachers, and administrators. Overall, the indicators exhibit consistently high scores on a five-point Likert scale, indicating strong process quality and high implementation fidelity. The non-uniform radar shapes reflect differences in emphasis across stakeholder groups: students show stronger perceptions of engagement and problem solving, teachers emphasize collaboration and initiative, while administrators highlight initiative as a key indicator of effective implementation oversight. Despite these variations, the overall pattern suggests a stable and well-executed learning process across stakeholders.

Figure 2. Radar charts of process evaluation indicators—engagement, collaboration, initiative, and problem solving—from the perspectives of students, teachers, and administrators, illustrating variations in perceived process quality and implementation fidelity.

3.3 Results of outcome evaluation: 21st-century skills and work readiness

Outcome evaluation focused on changes in 21st-century skills and work readiness, measured through pre–post comparisons. As presented in Table 3, Pre–Post Analysis of 21st-Century Skills, all skill dimensions showed statistically significant improvements (p < .001), with medium to large effect sizes (Cohen’s d = 0.71–0.93). The largest gains were observed in creativity and collaboration, reflecting the collaborative and problem-oriented nature of P5 projects.

These results corroborate earlier evidence that project-based learning effectively enhances transferable skills and employability-related competencies (Puspitasari, 2020; Widiastuti & Mariono). From an evaluation standpoint, the outcome findings confirm the suitability of the DIPPO model for capturing both educational and workforce-oriented impacts, consistent with multidimensional evaluation approaches (Chen & Yu, 2022; Liu, 2022).

Figure 3, Pre–Post Changes in 21st-Century Skill Scores visually reinforces the statistical findings reported in Table 3 by illustrating consistent and substantial pre–post gains across all 21st-century skills. The magnitude of improvement, as indicated by the Δ mean values, corresponds with large effect sizes (Cohen’s d ranging from 0.71 to 0.93), suggesting that the observed changes are not only statistically significant but also educationally meaningful. Notably, creativity and collaboration exhibited the largest gains, aligning with prior research highlighting the effectiveness of project-based and blended learning environments in fostering higher-order and transferable skills. These results indicate that the intervention produced robust learning impacts rather than marginal improvements, supporting its pedagogical relevance and scalability.

Figure 3. Pre–post changes in 21st-century skill scores.

3.4 Stakeholder comparison and evidence-based synthesis

To examine perceptual differences across stakeholder groups, a one-way ANOVA was conducted ( Table 4, Stakeholder Comparison Across DIPPO Dimensions). The results revealed a statistically significant difference only in the Input dimension (p = .017), indicating divergent perceptions of resource readiness between teachers and administrators. No significant differences were observed in other dimensions, suggesting shared understanding of P5 effectiveness across stakeholders.

Figure 4, Evidence-based decision quadrant mapping Input readiness against Outcome effectiveness to inform strategic resource optimization rather than program redesign. visualizes the relationship between Input readiness and Outcome effectiveness across DIPPO dimensions using an evidence-based decision quadrant. The positioning of dimensions indicates that, despite a statistically significant difference in the Input dimension across stakeholder groups (F = 4.38, p = .017), overall outcome effectiveness remains consistently high. Most dimensions fall within the sustain and scale or strengthen inputs while maintaining outcomes quadrants, suggesting that the program’s core design and implementation do not require fundamental redesign. Instead, the findings support targeted resource optimization and capacity strengthening, particularly in input-related aspects, to enhance program sustainability.

Figure 4. Evidence-based decision quadrant mapping Input readiness against Outcome effectiveness to inform strategic resource optimization rather than program redesign.

3.5 Evidence-based interpretation

Overall, the pilot findings indicate that the Evidence-Based DIPPO Evaluation Model effectively captures the multidimensional performance of P5 implementation. Strong results in Design, Process, Product, and Outcome dimensions, coupled with a moderate Input score, provide actionable evidence for targeted improvement. These findings reinforce the value of structured evaluation models in informing evidence-based decisions within contemporary curriculum reforms (Santiko et al., 2022; Liu, 2022).

4.1 DIPPO evaluation model as a decision-oriented framework

The results demonstrate that the DIPPO model effectively captured variations across design, input, process, product, and outcome dimensions, supporting its suitability as a comprehensive evaluation framework. High scores in the Design and Process dimensions indicate strong conceptual alignment and implementation fidelity, reinforcing Dippo’s (1988) original proposition that evaluation should connect educational design with occupational relevance. This finding also aligns with subsequent theoretical expansions of Dippo’s work, which emphasize education as a socially situated and practice-oriented enterprise (Alsop et al., 2007; Britzman et al., 1997).

Compared with widely adopted models such as CIPP and Kirkpatrick, the DIPPO framework offers a more explicit linkage between instructional processes and work-oriented outcomes. Similar to findings reported in recent multidimensional evaluation studies, the present results suggest that evaluation models grounded in program logic and outcome relevance are more effective for informing improvement-oriented decisions than outcome-only assessments (Alsalamah & Callinan, 2022; Nuha et al., 2024). The moderate score observed in the Input dimension further illustrates the diagnostic value of DIPPO, as it enables evaluators to identify specific leverage points for intervention rather than attributing effectiveness solely to instructional design.

4.2 Project-based learning (P5) and 21st-century skills development

The significant pre–post gains across all measured 21st-century skills confirm the pedagogical effectiveness of Project-Based Learning within the P5 framework. These findings are consistent with a substantial body of empirical research demonstrating that PBL enhances critical thinking, collaboration, communication, and creativity by engaging learners in authentic, problem-centered activities (Puspitasari, 2020; Martinez, 2022; Busnawir et al., 2025). The strongest improvements observed in creativity and collaboration reflect the collaborative and inquiry-driven nature of P5 projects, reinforcing the theoretical assumptions underlying project-based pedagogy.

In the Indonesian context, previous studies have highlighted P5 as a strategic mechanism for integrating character education with competency development (Amiruddin et al., 2024; Zahara & Nugraha, 2025). The present findings extend this literature by providing empirical evaluation evidence, rather than descriptive implementation accounts. By embedding P5 within the DIPPO evaluation structure, this study demonstrates how learning outcomes can be systematically linked to program design and instructional processes, addressing a gap identified in earlier P5-focused research (Multazam, 2023; Hidayah, 2024; Wati & Saputra, 2024).

4.3 Implications for work readiness and TVET-oriented outcomes

The observed improvement in work readiness underscores the relevance of P5 as a bridge between general education and workforce preparation. This finding aligns with TVET-oriented research emphasizing experiential learning, teamwork, and problem-solving as key predictors of employability (Tumpa, 2023). The DIPPO model’s Outcome dimension proved particularly effective in capturing these competencies, supporting calls for evaluation frameworks that move beyond academic achievement to include transferable and occupationally relevant skills (Ralph et al., 2022; Chen & Yu, 2022).

Moreover, the absence of significant differences across stakeholder groups in most DIPPO dimensions suggests a shared perception of P5 effectiveness, which is critical for sustainable implementation. The significant difference identified in the Input dimension mirrors findings from educational quality and performance evaluation studies, where discrepancies in resource perceptions often signal structural challenges rather than pedagogical failure (Santiko et al., 2022; Rozak et al., 2022). From an evidence-based policy perspective, this insight supports targeted resource optimization rather than wholesale program redesign.

4.4 Evidence-based decision-making and evaluation practice

A key contribution of this study lies in positioning evaluation results as decision-support evidence. Consistent with evidence-based policy and evaluation literature, the integration of quantitative outcomes and qualitative insights enables informed judgments about program continuation, scaling, or refinement (Vanlandingham & Drake, 2012; Garcia & Cerado, 2020). The use of triangulated data and effect size interpretation strengthens the credibility of findings and aligns with best practices in contemporary educational evaluation (Cook et al., 2024; Okoye et al., 2023).

Importantly, the pilot nature of this study demonstrates that even preliminary data can generate actionable insights when structured within a robust evaluation model. This supports recent arguments that evaluation frameworks should prioritize analytic clarity and decision relevance, rather than solely large-scale statistical generalization (Liu, 2022; Spatıotı et al., 2023).

4.5 Limitations and directions for full-scale research

As a pilot study, the findings should be interpreted cautiously. The limited sample size restricts generalizability, and future research should involve larger and more diverse participant groups to validate the observed patterns. Nevertheless, the consistency of results with established theory and prior empirical studies suggests strong potential for scaling the DIPPO-based evaluation approach. Future studies may also integrate advanced analytical techniques, such as clustering or machine learning-based evaluation, to further enhance decision precision (Liu, 2022; Okoye et al., 2022).

The findings of this study offer several important implications for educational policy, instructional practice, and curriculum reform, particularly in the context of implementing Project-Based Learning (P5) within competency-oriented curricula.

4.6 Policy implications

From a policy perspective, the results highlight the importance of adopting evidence-based evaluation models as integral components of curriculum reform. The DIPPO evaluation model proved effective in identifying not only overall program effectiveness but also specific leverage points for improvement, particularly within the Input dimension. This finding supports policy arguments that curriculum success should not be judged solely by learning outcomes, but by the coherence between design, resources, implementation processes, and long-term impact (Vanlandingham & Drake, 2012; Alsalamah & Callinan, 2022). Policymakers may therefore utilize DIPPO-based evidence to prioritize targeted investments in teacher capacity building and learning infrastructure rather than pursuing uniform or generalized reform strategies.

4.7 Practice implications

For educational practitioners, especially teachers and school leaders, the findings affirm the pedagogical value of Project-Based Learning (P5) in fostering 21st-century skills and work readiness. High scores in the Process and Product dimensions indicate that when PBL is implemented with fidelity, it supports active engagement, collaboration, and applied problem-solving. This aligns with prior evidence emphasizing the role of authentic, project-oriented learning environments in improving instructional quality (Puspitasari, 2020; Cook et al., 2024). Practitioners can use DIPPO indicators as reflective tools to continuously monitor and improve instructional practices, ensuring alignment between learning activities and intended competencies.

4.8 Curriculum reform implications

At the curriculum level, this study underscores the need for systematic evaluation frameworks to accompany curriculum innovation. While P5 provides a strong pedagogical foundation, the integration of the DIPPO model ensures that curriculum implementation remains adaptive and responsive to contextual conditions. Similar to recommendations in recent curriculum evaluation studies, embedding evaluation mechanisms within curriculum design enables continuous refinement and sustainability (Nuha et al., 2024; Santiko et al., 2022). Consequently, curriculum reform initiatives may benefit from positioning evaluation not as a post-hoc activity, but as an ongoing process that informs evidence-based curriculum decision-making.

4.9 Conclusion and future research

This study explored the application of an Evidence-Based DIPPO Evaluation Model in assessing the implementation of Project-Based Learning (P5) with a focus on 21st-century skills and work readiness. Drawing on pilot study findings, the results demonstrate that the DIPPO model provides a coherent and decision-oriented framework for evaluating complex educational programs. Strong performance across the Design, Process, Product, and Outcome dimensions indicates that P5 is conceptually sound and pedagogically effective, while the relatively lower Input scores offer actionable insights for targeted improvement.

By integrating evaluation findings into an evidence-based decision-making framework, this research contributes to the advancement of educational evaluation theory and practice, particularly at the intersection of project-based pedagogy and workforce-oriented learning outcomes. The study also extends existing literature by operationalizing a classical evaluation model within a contemporary curriculum reform context, thereby reinforcing the relevance of structured evaluation approaches in modern education systems.

4.10 Future research directions

Given the pilot nature of this study, future research should validate the findings using larger and more diverse samples across different educational contexts. Longitudinal studies are also recommended to examine the sustained impact of P5 on work readiness beyond immediate learning outcomes. Additionally, future investigations may integrate advanced analytical techniques—such as clustering or learning analytics—to further enhance the precision of evidence-based evaluation and decision-making (Liu, 2022; Okoye et al., 2022). Expanding the application of the DIPPO model to other curricular innovations may also provide comparative insights and strengthen its generalizability.