ALL Metrics
-
Views
-
Downloads
Get PDF
Get XML
Cite
Export
Track
Research Article

Evaluating Business Intelligence Success in the Public Sector: Extending the DeLone and McLean Model with Perceived Organizational Support

[version 1; peer review: awaiting peer review]
PUBLISHED 23 Jan 2026
Author details Author details
OPEN PEER REVIEW
REVIEWER STATUS AWAITING PEER REVIEW

Abstract

Background

The effectiveness of information systems in enhancing user performance is influenced by various factors, including system quality, service quality, and user satisfaction. Perceived Organizational Support (POS) has been proposed as a moderator in this relationship, though its impact remains underexplored. This study aims to investigate the moderating role of POS on the relationship between system quality, service quality, and user performance, specifically in the context of Approweb, an information system used by account representatives at the Directorate General of Taxes (DGT) in Jakarta.

Methods

A quantitative research approach with an explanatory design was adopted to examine causal relationships among the variables. The theoretical framework was based on the DeLone and McLean Information System Success Model (2003), with POS incorporated from the socio-technical perspective. Data were collected through a cross-sectional survey from 329 Account Representatives (ARs) using a probability sampling method

Results

The study found that both system quality and service quality significantly influence user satisfaction and user performance. Moreover, POS was found to significantly moderate the relationship between system quality, service quality, and user performance, enhancing the positive effects of both quality dimensions on performance. These results suggest that organizational support is a key factor in maximizing the benefits of information systems.

Conclusions

The findings confirm the critical role of system quality, service quality, and POS in improving user performance. POS was shown to amplify the positive effects of both system and service quality, indicating that users are more likely to perform effectively when they perceive strong organizational support. The study contributes to the DeLone and McLean IS Success Model by incorporating POS as a moderating factor and emphasizes the importance of a holistic approach to system success that integrates both technical and social dimensions.

Keywords

System Quality, Service Quality, Perceived Organizational Support, User Satisfaction, User Performance.

1. Introduction

Taxation is a cornerstone of state revenue, underpinning national development financing, public service delivery, and economic stability. In Indonesia, the Directorate General of Taxes (DGT) is tasked with the pivotal role of mobilizing tax revenues to ensure the country’s fiscal health. However, despite these efforts, Indonesia continues to face significant challenges in meeting its revenue targets, with the national tax ratio remaining below the Asia-Pacific average and far from the International Monetary Fund’s (IMF) recommended minimum of 15% of GDP required for sustainable development. This underperformance highlights structural weaknesses in the tax administration system, particularly in compliance monitoring and taxpayer engagement. These weaknesses necessitate the adoption of more effective administrative processes and technological interventions to enhance tax compliance and revenue generation (Mardiasmo, 2018; Nkwe, 2013; Waluyo, 2020; IMF, 2022).

In response to these challenges, the DGT has introduced various technology-based solutions, including the Approweb system. Approweb is a business intelligence (BI) application designed to support Account Representatives (ARs) in monitoring taxpayer compliance and identifying potential revenue. The system integrates internal and external tax-related data, providing actionable insights that help improve decision-making processes. Despite its potential, the implementation of Approweb faces several obstacles, such as integration issues with other DGT applications, suboptimal management of third-party data, and the need for continuous adaptation to rapidly evolving information technology. These challenges suggest that while technology can enhance tax administration, its success depends on overcoming significant technical and organizational hurdles (Negash, 2004; Chen, 2010; DeLone & McLean, 2003; Van Cauter et al., 2017).

The effectiveness of BI systems like Approweb does not solely rely on technical aspects such as system quality, information quality, and service quality but also on social factors like user satisfaction and perceived organizational support (POS). The DeLone and McLean Information System Success Model (2003) offers a comprehensive framework for evaluating information system success through these dimensions. However, empirical findings regarding the relationships among these factors, especially within public sector contexts, have been inconsistent. Studies have shown mixed results on how system quality, information quality, service quality, and user satisfaction influence performance, particularly in tax administrations (Çelik & Ayaz, 2022; Lutfi, 2023). The need for further research is evident, as understanding these relationships is crucial for improving the success of BI systems in public sector environments, especially in developing economies like Indonesia.

In recent years, BI adoption in the public sector has expanded significantly, with tax administrations worldwide using these technologies to improve transparency, strengthen compliance monitoring, and support evidence-based policymaking. While BI systems can yield substantial benefits, their effectiveness hinges on user acceptance, perceived usefulness, and the organizational environment in which they are implemented. A socio-technical perspective suggests that successful BI implementation requires a balance between technical capabilities and social dynamics, such as user readiness, trust, and organizational support. These factors are critical for increasing system acceptance, reducing resistance to change, and enhancing individual performance. In the context of tax administration, a supportive organizational climate may encourage ARs to utilize BI functionalities more effectively, leading to more precise taxpayer risk profiling and improved enforcement strategies (Omar et al., 2019; Benmoussa et al., 2018; Petter et al., 2008; Trieu, 2017; Trist & Bamforth, 1951; Emery, 2013).

Despite growing interest in the adoption of BI systems within public administration, there is a noticeable gap in empirical studies that integrate the DeLone and McLean IS Success Model with POS as a moderating factor. This gap is particularly evident in the context of tax administrations in developing economies. Empirical studies on BI success in the public sector have yielded inconsistent results, making it important to conduct further investigations to assess the success of these systems, particularly in terms of performance outcomes. This study aims to address this gap by evaluating the success of Approweb implementation among ARs at the Jakarta Regional Office of DGT, and by developing a more comprehensive evaluation model for BI success in the public sector. The findings from this research are expected to contribute to both theoretical advancements and practical strategies for improving tax administration performance (Chen, 2010; Saputra et al., 2023).

As Indonesia continues its journey of technological transformation in tax administration, the introduction of systems like Approweb is a crucial step toward improving tax compliance and revenue generation. However, the integration of technology into tax administration brings with it a range of operational challenges that need careful consideration. Approweb, for instance, is part of a broader effort to modernize Indonesia’s tax system, yet its adoption has not been without difficulties. One of the key challenges is the rate of adoption among various stakeholders, particularly small businesses. These businesses often face challenges in adapting to new technologies due to a lack of digital literacy and limited resources, creating resistance to change. To address this, it is essential to provide support and training to small business owners to improve their understanding of digital tax systems, which in turn could facilitate greater compliance (Priyono et al., 2024).

In addition to adoption challenges, the human element in tax administration remains crucial. While technology can streamline processes, tax officials must be adequately trained to effectively use these new systems. Paroli (2023) emphasizes the importance of performance management for tax officials, noting that without proper training, the new systems may not yield the desired results. Tax officials need to be equipped with the skills and knowledge to help taxpayers navigate these changes. Inadequate training could lead to inefficiencies, inconsistencies in service delivery, and, ultimately, reduced compliance. This underscores the need for the DGT to prioritize training and capacity-building efforts to ensure the successful implementation of Approweb and similar systems.

Financial performance metrics are also essential for evaluating the success of technology adoption in tax administration. Studies have shown mixed results regarding the impact of technology on revenue generation. According to Pasaribu et al. (2022), tax reforms between 2017 and 2021 have shown uneven results, with some regions experiencing success while others struggle. These disparities suggest that the effectiveness of technology adoption depends on various factors, including the quality of implementation strategies, system maintenance, and technical support. The DGT faces significant challenges in ensuring that systems like Approweb run smoothly, requiring ongoing technical support and infrastructure updates.

In conclusion, while the adoption of technology-based systems like Approweb holds great promise for improving tax administration in Indonesia, it also presents numerous challenges that must be addressed systematically. The successful integration of technology into tax administration requires not only technical upgrades but also the management of human and regulatory factors. A long-term perspective is essential, recognizing that initial difficulties may give way to substantial improvements in the future. As highlighted by various scholars, including Hidayat and Inayati (2025), Paroli (2023), and Meilandri (2025), the effective governance of tax administration in Indonesia depends on a balanced approach that considers both the technological and socio-organizational aspects of reform.

2. Literature review and hypothesis development

2.1 System quality on user satisfaction

DeLone and McLean’s IS Success Model (2003) emphasizes system quality as a key dimension of information system success, asserting that systems that are reliable, efficient, user-friendly, and secure lead to positive user experiences and increased satisfaction. From a socio-technical perspective (Trist & Bamforth, 1951; Bostrom & Heinen, 1977; Abbas & Michael, 2023), system success is not solely determined by technical features but also by how users perceive its benefits in everyday interactions, creating an ecosystem that fosters comfort and trust, ultimately boosting user satisfaction.

Empirical research supports this relationship. Chen (2010) found a significant link between system quality and taxpayer satisfaction with online tax-filing systems, showing that system quality directly influences user satisfaction. Similarly, Trisnawati et al. (2017) found a positive correlation between system quality and user satisfaction, suggesting that high system quality leads to better system use and improved organizational performance. Lutfi (2023) also confirmed that system quality significantly impacts user satisfaction in accounting information systems. However, some studies show mixed results. Song et al. (2017) found no significant effect of system quality on user satisfaction in the context of Building Information Modeling (BIM), and Çelik & Ayaz (2022) found no significant relationship between system quality and user satisfaction in university information systems.

In conclusion, while most studies support a positive relationship between system quality and user satisfaction, inconsistent findings highlight the need for further research to explore additional factors that may mediate or moderate this relationship. Thus, the hypothesis can be stated:

H1:

System quality positively affects user satisfaction

2.2 System quality on user performance

System quality is a crucial component of every successful system, according to DeLone and McLean’s (2003) IS Success Model. This model helps to explain the connection between system quality and user performance. While the original model links system quality to user satisfaction and usage, many studies show a direct impact on user performance. Reliable, efficient, and user-friendly systems are believed to directly enhance users’ ability to perform tasks effectively. From a socio-technical systems perspective (Trist & Bamforth, 1951; Bostrom & Heinen, 1977; Abbas & Michael, 2023), system success occurs when technical quality and user interaction are balanced. High-quality systems enable smoother user interactions, boosting productivity and performance.

Several studies support this relationship. Benmoussa et al. (2018) found that improving system quality positively impacted organizational performance. Bahari and Mahmud (2018) also found that system quality significantly influenced individual and organizational performance. Mutiso and Mutuku (2020) confirmed that system quality positively affected user performance, further reinforcing this link across different contexts. Mauye (2024) found similar results with ERP systems, showing that better system performance improved user performance. However, Saputra et al. (2023) found no significant relationship between system quality and user performance in their study of the Correctional Database System, suggesting that user perceptions may mediate the impact. Based on this explanation, the hypothesis can be stated as:

H2:

System quality positively affects user performance

2.3 Service quality on user satisfaction

The relationship between service quality and user satisfaction can be explained through DeLone and McLean’s IS Success Model (2003), which identifies service quality as a fundamental dimension of information system success. The model suggests that good service support enhances the user experience and directly boosts user satisfaction. From a socio-technical system perspective (Trist & Bamforth, 1951; Bostrom & Heinen, 1977; Abbas & Michael, 2023), service quality is not just a technical issue but also a result of the social interaction between service providers and users, shaping positive perceptions of the system.

Empirical studies support this relationship. Chen (2010) found a significant positive effect of service quality on user satisfaction in the context of online tax-filing systems. Similarly, Ameen et al. (2020) in their study “Examining Relationship between Service Quality, User Satisfaction, and Performance Impact in The Context of Smart Government in UAE” showed that service quality significantly influences user satisfaction. Lutfi (2023) also confirmed the significant impact of service quality on user satisfaction in his research on information systems success. Additionally, Song et al. (2017) concluded that service quality significantly affects user satisfaction. However, some studies show differing results. For instance, Çelik and Ayaz (2022) found no significant relationship between service quality and user satisfaction in their study of university information systems, suggesting that other factors may influence this link. Therefore, the hypothesis can be stated as:

H3:

Service quality positively affects user satisfaction

2.4 Service quality on user performance

DeLone and McLean’s IS Success Model (2003), which highlights service quality as a crucial component of system success, helps explain the connection between service quality and user performance. The model assumes that service quality influences user satisfaction and system usage, which in turn affects individual and organizational performance. While the original model suggests an indirect relationship, in practice, particularly with the use of systems like Approweb, service quality can play a more direct role. Fast technical support, attention to user needs, and readiness for emergency service can directly impact the speed and accuracy of task completion by account representatives.

This perspective aligns with the socio-technical systems theory (Trist & Bamforth, 1951; Bostrom & Heinen, 1977; Abbas & Michael, 2023), which highlights how the interaction between technical and social elements can have a direct impact on performance. Service quality, as part of the social aspect, bridges the gap between technology and human productivity, creating a direct effect on user performance. Several studies support this relationship. Bahari and Mahmud (2018) found that service quality, along with system and information quality, positively and significantly influences individual and organizational performance. Similarly, Ameen et al. (2020) found that service quality positively impacts user performance in their research on smart government services, confirming the direct influence of service quality on performance. Riandi et al. (2021) also demonstrated that service quality significantly affects user performance, particularly in the context of e-learning systems.

Most studies suggest that service quality positively influences user performance, with some research indicating that this relationship is direct. Therefore, the hypothesis can be stated as:

H4:

Service quality positively affects user performance

2.5 User satisfaction on user performance

DeLone and McLean’s IS Success Model (2003) explains the connection between user performance and user satisfaction. It states that system quality, information quality, and service quality interact to produce user pleasure, which in turn encourages more intensive system utilization. This increased usage, in turn, leads to improved individual and organizational performance. In other words, user satisfaction acts as a bridge between the system experience and enhanced work performance. When users are satisfied with the system, they are more likely to engage with it fully, which enhances their performance.

The socio-technical systems perspective (Trist & Bamforth, 1951; Bostrom & Heinen, 1977; Abbas & Michael, 2023) further explains this relationship by emphasizing that a system’s success is influenced not only by its technical aspects but also by social factors such as user experience and perceptions. Systems that are technically reliable and easy to use create better working conditions, while user satisfaction arises from social interactions, such as trust, comfort, and organizational support. When the balance between technical and social elements is achieved, user satisfaction increases and, consequently, leads to improved performance.

Several studies support this relationship. Hou (2012), in his study “Examining the Effect of User Satisfaction on System Usage and Individual Performance with Business Intelligence Systems,” found that user satisfaction has a significant impact on user performance. Similarly, Gonzales et al. (2015) concluded in “Measuring the Impact of Data Warehouse and Business Intelligence on Enterprise Performance in Peru” that user satisfaction significantly affects individual performance. Gaardboe et al. (2017) also found a positive and significant relationship between user satisfaction and individual impact in their study of public hospitals in Denmark. Additionally, Al-Okaily et al. (2023) in “Evaluation of Data Analytics-oriented Business Intelligence Technology Effectiveness” showed that perceived benefits, which are influenced by user satisfaction, positively affect organizational outcomes.

Research indicates that user satisfaction positively affects user performance, as satisfied users are more likely to use the system effectively, resulting in enhanced performance. Therefore, the hypothesis can be stated as:

H5:

User satisfaction positively affects user performance

2.6 Perceived organizational support as mediator

POS has been identified as a critical factor in enhancing employee and user performance, particularly in the context of information systems. In the DeLone and McLean IS Success Model (2003), POS plays a vital role in shaping users’ experiences with systems by fostering an environment of trust, support, and resource availability, which can ultimately influence their performance. POS, when high, is believed to strengthen the positive effects of system quality, service quality, and user satisfaction on user performance.

The theoretical justification for this relationship draws on the socio-technical system perspective, which emphasizes the balance between technical and social factors. In this context, POS represents a social factor that can enhance the effectiveness of technical system components, such as system quality and service quality, by providing users with the necessary support, training, and organizational commitment to perform effectively (Trist & Bamforth, 1951; Bostrom & Heinen, 1977; Abbas & Michael, 2023). When users perceive higher levels of organizational support, they are likely to feel more motivated, confident, and equipped to perform their tasks, leading to improved user performance.

Several studies support the moderating role of POS. For example, Yongxing et al. (2017) demonstrated that POS moderates the relationship between work engagement and job performance, making the positive relationship more significant when POS is high. Similarly, Gemilang and Riana (2021) found that POS positively influences employee performance, with work engagement serving as a mediator in the relationship. This suggests that when users feel supported by their organization, their engagement and performance levels are significantly improved. Khan et al. (2022) also showed that POS, mediated by job satisfaction, positively impacts employee performance, further indicating the importance of POS in moderating performance outcomes.

Additionally, Hwang et al. (2012) and Ballaro & Washington (2016) highlighted the critical role of POS in ensuring the success of system implementation, with POS positively influencing system usage and, consequently, user performance. These studies suggest that POS not only impacts user satisfaction but also amplifies the benefits of system use by enhancing the user’s ability to perform tasks efficiently. However, some studies, such as Dewi et al. (2020), have shown inconsistent results, finding that POS does not directly correlate with performance, but rather with employee engagement. This suggests that the impact of POS might vary across different contexts, underscoring the need for further research to understand the precise mechanisms through which POS moderates the relationship between system quality, service quality, user satisfaction, and user performance.

Based on this theoretical and empirical background, the following hypothesis is developed:

H6:

POS moderates the relationship between system quality on user performance

H7:

POS moderates the relationship between service quality on user performance

Based on the literature review and hypothesis development that has been explained previously, the research hypothesis framework can be seen in Figure 1.

3f4f63d4-94e7-4851-9f0b-d1a96cdf6403_figure1.gif

Figure 1. Research hypothesis framework.

3. Research methodology

3.1 Research design

This study adopts a quantitative research approach with an explanatory design, aiming to examine causal relationships between system quality, service quality, user satisfaction, and user performance, with POS as a moderating variable. The DeLone and McLean Information System Success Model (2003) serves as the theoretical foundation, extended by incorporating POS from the socio-technical perspective. A cross-sectional survey method was applied, collecting data from respondents at a single point in time to test the proposed hypotheses.

3.2 Population and sample

The study population consists of all Account Representatives (ARs) working at tax service offices (KPP) within the Regional Office of the Directorate General of Taxes (DGT) Jakarta. ARs were chosen as the research unit of analysis because they are the primary users of Approweb, directly utilizing it for taxpayer compliance monitoring and revenue potential assessment. A probability sampling method using proportional stratified random sampling was employed to ensure representation from various KPP clusters within the region. The sample size determination followed Hair et al.’s (2019) recommendation of at least 10 respondents per indicator in the measurement model, resulting in a minimum sample requirement of 329 respondents.

3.3 Variables and measurement

The variables in this study are defined as follows:

  • System Quality (SQ): Users’ perceptions of the technical performance of Approweb, measured through indicators such as reliability, usability, accessibility, and response time (DeLone & McLean, 2003).

  • Service Quality (ServQ): Users’ assessment of technical support, responsiveness, and assistance provided in using Approweb (Pitt et al., 1995).

  • User Satisfaction (US): The degree to which ARs are satisfied with their experience of using Approweb (Ives & Olson, 1984).

  • User Performance (UP): The perceived effectiveness and efficiency of ARs in monitoring taxpayer compliance and identifying revenue potential using Approweb (Govindan et al., 2013).

  • Perceived Organizational Support (POS): ARs’ perceptions of the extent to which DGT values their contributions and supports their welfare, covering fairness, supervisor support, and organizational rewards (Rhoades & Eisenberger, 2002).

All constructs were measured using multiple-item scales adapted from validated instruments in prior studies. Items were rated on a five-point Likert scale (1 = strongly disagree to 5 = strongly agree).

3.4 Ethical approval

This study involved human participants and was conducted in accordance with the principles of the Declaration of Helsinki (World Medical Association, 2013). Formal ethical approval from an institutional review board (IRB) was not required because this research involved non-invasive survey procedures, posed minimal risk to participants, and was conducted as part of institutional performance evaluation within a government organization. According to national and institutional guidelines applicable in Indonesia, studies based on anonymous questionnaires without medical or psychological intervention do not require formal ethics committee approval. Participation was voluntary, and no personally identifiable information was collected.

3.5 Data collection procedures

Data were collected via a self-administered questionnaire distributed both in printed form and electronically to ARs across tax service offices within the Jakarta Regional Office. The questionnaire consisted of three parts: (1) respondent demographics, (2) measurement items for the research variables, and (3) additional open-ended questions designed to capture qualitative feedback on Approweb usage experiences. Prior to the main survey, a pilot test involving 30 ARs was conducted to ensure clarity, validity, and reliability of the instrument items. The full dataset, including questionnaire items, coding schemes, and descriptive statistics, is publicly available on Figshare (Samiono et al., 2025), accessible at Item - Research Data - figshare - Figshare.

Informed consent was obtained from all participants prior to their participation in the study. Participants were informed about the purpose of the research, the voluntary nature of participation, confidentiality of responses, and their right to withdraw at any time without consequences. Consent was obtained verbally and electronically at the beginning of the questionnaire, as the survey was administered anonymously and did not collect any personally identifiable information. Written consent was not required due to the minimal-risk nature of the study, anonymous nature of the survey, absence of sensitive personal data, and minimal risk to participants, in line with institutional and national research guidelines.

3.6 Data analysis

Data analysis was conducted using Partial Least Squares Structural Equation Modeling (PLS-SEM) with SmartPLS 4 software, which is suitable for complex models with moderating variables and does not require multivariate normality (Hair et al., 2019). The analysis comprised:

  • Measurement Model Evaluation: Including convergent validity (outer loadings, Average Variance Extracted), discriminant validity (Fornell–Larcker criterion, HTMT ratio), and construct reliability (Composite Reliability, Cronbach’s Alpha).

  • Structural Model Evaluation: Assessing path coefficients, coefficient of determination (R2), effect sizes (f2), predictive relevance (Q2), and significance testing through bootstrapping (5,000 resamples).

  • Moderation Analysis: Testing the interaction effects of POS on the relationships between system quality, information quality, service quality, user satisfaction, and user performance.

The statistical significance level was set at p < 0.05, and interpretation followed the guidelines of Hair et al. (2019) and Ubaidillah et al. (2022) for PLS-SEM reporting in academic research.

4. Results

4.1 Demographic respondent

This study involved 329 respondents, all of whom were account representatives at the Jakarta Tax Service Office. The respondent descriptions aim to examine the demographic distribution of the respondents, as shown in Table 1.

Table 1. Demographic respondent.

No.Demographic categoryAttributeFrequency Percentage (%)
1GenderMale22969.60%
Female10030.40%
2Age25 – 2830.91%
29 – 323610.94%
33 – 364413.37%
37 – 405717.33%
41 – 448124.62%
45 – 485316.11%
49 – 523610.94%
53 – 56185.47%
57 – 6010.30%
3Work Duration at DJP4 – 7 years30.91%
8 – 11 years8124.62%
12 – 15 years6620.06%
16 – 19 years5717.33%
20 – 23 years4313.07%
24 – 27 years5215.81%
28 – 31 years195.78%
32 – 35 years72.13%
36 – 39 years10.30%
4Duration as Account Representative1 – 2 years4313.07%
3 – 4 years4613.98%
5 – 6 years5817.63%
7 – 8 years4613.98%
9 – 10 years329.73%
11 – 12 years4313.07%
13 – 14 years175.17%
15 – 16 years319.42%
17 – 18 years133.34%
5Education LevelD3288.51%
D4/S123872.34%
S26319.15%

The demographic data of the respondents in this study in Table 1 reveals several key insights about the workforce at the Jakarta Regional Tax Office. The gender distribution indicates that the majority of account representatives are male, with 69.60% (229 respondents) identifying as male and 30.40% (100 respondents) as female. This suggests a gender imbalance in this particular professional group, with men making up a significantly larger proportion of the workforce.

In terms of age, the data shows that most respondents fall within the productive working age range, with the largest group being between 41–44 years old (24.62%, 81 respondents), followed by those aged 37–40 (17.33%, 57 respondents) and 45–48 (16.11%, 53 respondents). Collectively, respondents between 33 and 48 years of age account for more than 70% of the sample. This indicates that the workforce in this area is mature, with a high level of experience, and likely reflects a stable workforce with considerable expertise in the field. In contrast, the youngest age group, 25–28 years, comprises only 0.91% of the respondents, suggesting that the workforce is dominated by those with more established careers.

Regarding the length of service at the Directorate General of Taxes (DJP), the majority of respondents have worked for the organization for a significant number of years. The largest group has been employed for 8–11 years (24.62%, 81 respondents), followed by those with 12–15 years of service (20.06%, 66 respondents) and 16–19 years (17.33%, 57 respondents). Together, these groups represent over 60% of the sample, indicating that most respondents have substantial experience with DJP. This highlights the stability of the workforce, as many individuals have been in the organization long enough to develop expertise and a deep understanding of their roles. There are fewer respondents in the categories of extreme work durations (4–7 years and 36–39 years), suggesting a high level of retention among mid-tenure employees.

In terms of the duration spent as an account representative, the data reveals that the majority of respondents have held this position for 5–6 years (17.63%, 58 respondents), with similar numbers in the 3–4 years and 7–8 years categories (both 13.98%). These findings suggest that while the account representative role sees moderate turnover, many individuals remain in the position for several years. However, fewer respondents have served in the role for longer periods (13–18 years), which may indicate that individuals tend to move on to other roles or are promoted after gaining significant experience in this capacity.

When it comes to educational background, the majority of respondents hold a bachelor’s degree (D4/S1), making up 72.34% (238 respondents) of the sample. This high percentage reflects the requirement for academic qualifications to hold positions in the tax office. Additionally, 19.15% (63 respondents) have a master’s degree (S2), suggesting that a portion of the workforce holds advanced qualifications. A smaller proportion, 8.51% (28 respondents), have a diploma (D3). The dominance of bachelor’s degree holders indicates that the workforce is well-educated and equipped to perform the tasks required in the tax office, with a notable proportion having the expertise expected of more senior professionals.

Overall, the demographic characteristics of the respondents highlight a well-established, mature, and experienced workforce, with the majority being male, middle-aged, and holding higher education degrees. The workforce is predominantly composed of individuals with significant tenure in both the Directorate General of Taxes and as account representatives. These factors are important to consider when interpreting the survey responses, as they suggest a group with a wealth of experience and education, which may influence their perspectives on the effectiveness and quality of the systems and services in place at the tax office.

4.2 Measurement model evaluation

To make sure that all constructs were assessed in a valid and reliable manner, the measurement model had to be evaluated first. Convergent validity and reliability were the two primary factors evaluated. The Average Variance Extracted (AVE) for each construct and the outer loadings of each indicator were examined in order to test convergent validity. Indicators that have outer loadings more than 0.7 and AVE values greater than 0.5 are said to accurately reflect their latent concept. Cronbach’s Alpha and Composite Reliability (CR) were used to evaluate reliability; values greater than 0.7 are regarded as satisfactory (Hair et al., 2019). Table 2 provides a summary of the validity and reliability metrics for every construct.

Table 2. Measurement model validity.

VariableDimensionsItemsLoadingsAVECR CA
System Quality (SQ)System reliabilitySQ110.8210.9220.8910.923
SQ120.800
System efficiencySQ210.8190.9130.8740.915
SQ220.829
Ease of useSQ310.7830.7890.8460.885
SQ320.815
SQ330.818
System securitySQ410.8350.9120.8320.893
SQ420.821
SQ430.800
Service Quality (ServQ)Technical competenceServQ110.8250.9330.8440.874
ServQ120.828
CareServQ210.8560.9060.8150.858
ServQ220.855
ServQ230.861
Service availabilityServQ310.8600.9100.8080.849
ServQ320.870
ServQ330.864
User Satisfaction (US)Satisfaction with system functionalityUS110.8600.9090.8870.916
US120.873
US130.853
Satisfaction with system performanceUS210.8100.9210.8530.902
US220.848
Satisfaction with support servicesUS310.8340.9150.8610.901
US320.820
General satisfactionUS410.8390.9080.8730.914
US420.860
User Performance (UP)ProductivityUP110.8380.9330.8320.886
UP120.836
Work efficiencyUP210.8320.9120.8680.897
UP220.860
Output qualityUP310.8570.9120.8860.917
UP320.845
UP330.864
Perceived Organizational Support (POS)Managerial supportPOS110.8690.9000.8530.938
POS120.865
POS130.858
Concern for well-being POS210.8750.9120.8970.955
POS220.879
POS230.863
POS240.852
Access to training and developmentPOS310.8380.9140.8520.867
POS320.849
POS330.865

Table 2 indicates that all constructs have convergent validity values that are sufficient. The Heterotrait-Monotrait ratio (HTMT) was then used to assess discriminant validity.

4.3 Discriminant validity

Discriminant validity was assessed using the HTMT to confirm that each construct is unique from others, with values below 0.9 signifying sufficient discriminant validity. The HTMT findings are presented in Table 3.

Table 3. HTMT values.

USUPServQSQ POS
US
UP 0.819
ServQ 0.8240.824
SQ 0.8360.8360.741
POS 0.7970.7970.8020.295

All of the measurement model’s constructs show sufficient discriminant validity, according to the HTMT values shown in Table 3. The measuring model is robust since all constructs meet the necessary criteria for reliability and validity. This is shown by the strong convergent validity, reliability metrics (CA and CR), and discriminant validity (HTMT values). Hence, we can move on to the next part of the study with certainty. It involves looking at the structural model to see if the hypothesized path coefficients hold and to examine the links between the constructs.

4.4 Hypothesis testing

To examine the hypothesized relationships, the structural model was evaluated after the measurement model was confirmed. In order to ascertain the intensity and orientation of the connections between variables, this assessment included looking at the path coefficients (β). After that, we used bootstrapping with 5,000 resamples to see if these paths were statistically significant; this gave us p-values to test our hypothesis. The results are summarized in Table 4 below.

Table 4. Hypothesis Testing.

HypothesisPathβt-value p-value Result
H1SQ ➔ US0.76015.2240.000Supported
H2SQ ➔ UP0.1403.3050.001Supported
H3ServQ ➔ US0.1352.9410.003Supported
H4ServQ ➔ UP0.1345.7960.000Supported
H5US ➔ UP0.75616.1420.000Supported
H6SQ * POS ➔ UP0.1662.8140.007Supported
H7ServQ * POS ➔ UP0.1354.8780.000Supported

The results of the hypothesis testing indicate strong support for all the proposed relationships, demonstrating significant effects between the variables. First, System Quality (SQ) was found to have a highly significant positive impact on User Satisfaction (US), with a path coefficient of 0.760 and a t-value of 15.224, which is well above the threshold for significance. This suggests that a higher quality system directly enhances user satisfaction, confirming that system quality is a key driver of users’ positive experiences.

Similarly, System Quality (SQ) also positively influenced User Performance (UP), albeit with a smaller effect size (β = 0.140) but still statistically significant, as indicated by the t-value of 3.305. This highlights that system quality not only improves user satisfaction but also contributes to better performance, albeit to a lesser extent. The relationship between Service Quality (ServQ) and User Satisfaction (US) was also supported, with a positive effect (β = 0.135) and a significant t-value of 2.941. While the effect of service quality on user satisfaction is somewhat smaller than that of system quality, it still plays an important role in shaping users’ satisfaction with the system.

In terms of User Performance (UP), Service Quality (ServQ) was found to have a strong and statistically significant effect (β = 0.134, t-value = 5.796), indicating that higher service quality significantly enhances user performance. This supports the idea that not just the system’s technical features but also the quality of service provided plays a vital role in improving the effectiveness of users. Furthermore, User Satisfaction (US) was found to have a very strong impact on User Performance (UP) (β = 0.756, t-value = 16.142). This result reinforces the idea that satisfied users are more likely to perform well, linking the emotional or cognitive responses to the system directly with work outcomes.

The moderation effect of POS was also examined, with both System Quality (SQ) and Service Quality (ServQ) showing stronger impacts on User Performance (UP) when POS was higher. For System Quality, the interaction term (SQ * POS) had a significant effect (β = 0.166, t-value = 2.814), while for Service Quality, the moderation effect (ServQ * POS) was even more pronounced (β = 0.135, t-value = 4.878). These findings suggest that when employees feel supported by their organization, both system and service quality have a more pronounced positive effect on their performance.

In conclusion, the results clearly demonstrate the importance of both system and service quality in enhancing user satisfaction and performance. Additionally, the moderation effects of POS emphasize the critical role of organizational support in strengthening the impact of these quality dimensions on user performance.

5. Discussion

5.1 The relationship between system quality and service quality on user satisfaction

The findings of this study confirm that both system quality and service quality significantly influence user satisfaction among account representatives using Approweb. In particular, system quality exhibits a very strong effect on satisfaction, reflecting that when the system is reliable, efficient, easy to use, and secure, users tend to report high levels of satisfaction. This aligns with previous empirical research. For instance, in a study of a governmental financial application system, system quality and service quality were found to positively affect user satisfaction (Rahmawati et al., 2023).

From a socio-technical perspective, the strong effect of system quality suggests that technical attributes (such as reliability and usability) interact with human experience to produce positive perceptions of the system (Trist & Bamforth, 1951; Bostrom & Heinen, 1977). In practice, when the system responds quickly, functions smoothly, and safeguards data appropriately, users feel confident and comfortable using it, which translates into higher satisfaction. A 2025 study of a learning-management system (LMS) also demonstrated system quality and service quality as significant predictors of user satisfaction, highlighting that satisfactory user experience depends on both technical and service dimensions (Simelane-Mnisi & Mthimunye, 2025; Widyaningrum et al., 2024).

On the other hand, service quality also shows a positive and significant influence on satisfaction, although with a lower effect size compared to system quality. This underscores the importance of support services as a complement to technical robustness. Good service quality, including technical competence of support staff, responsiveness, empathy, and availability, helps users overcome difficulties, feel valued, and trust the system more. These social-organizational interactions contribute significantly to satisfaction beyond mere system performance. This is supported by journal studies investigating online registration and e-service systems, which find that service quality significantly affects user satisfaction even when information or system quality may not have a strong effect on system usage (Pratama & Al Rasyid, 2024).

The relatively smaller magnitude of service quality’s effect (compared to system quality) in this study does not diminish its relevance. Rather, it highlights that while a solid technical foundation (system quality) is the backbone of satisfaction, the “human touch” provided through service quality remains essential to address user needs, resolve problems, and maintain trust. In many organizational settings, including public sector agencies, the availability of responsive support and maintenance often determines whether users continue using the system or revert to older, familiar methods.

These results reinforce the relevance and contemporary applicability of the DeLone & McLean IS Success Model within government tax administration contexts. Recent studies including in educational systems, ERP systems, and other public sector applications, continue to validate the model’s claims regarding system quality and service quality as key antecedents to user satisfaction (Rahmawati et al., 2023; Rulinawaty et al., 2024; Simelane-Mnisi & Mthimunye, 2025).

5.2 The relationship between system quality and service quality on user performance

Account representatives using Approweb had their performance significantly affected by system and service quality, according to this study’s findings. System quality plays a pivotal role in enhancing user performance by providing a reliable, efficient, and secure system that allows account representatives to complete their tasks more effectively. This result aligns with the DeLone and McLean IS Success Model (2003), which positions system quality as a key determinant of both system success and user performance. The model suggests that a high-quality system not only contributes to user satisfaction but also directly influences users’ ability to perform tasks efficiently. Furthermore, recent studies have corroborated this notion, showing that system quality impacts performance outcomes in various contexts, such as e-government systems (Lutfi, 2023) and enterprise systems (Khan et al., 2022).

From a socio-technical systems perspective, the study findings underscore that technical quality interacts with social and organizational factors to improve user performance. The system’s reliability ensures stability, which is crucial in maintaining uninterrupted workflow. The efficiency of the system facilitates quicker data processing and navigation, enabling users to complete tasks in a timely manner. Moreover, the ease of use of Approweb fosters user confidence, encouraging account representatives to leverage the system’s features more effectively. Security ensures that users can work without concerns about data vulnerabilities, which builds trust and confidence in the system. These dimensions of system quality act synergistically to enhance user performance.

The findings on service quality also reveal a significant positive effect on user performance. Although service quality is traditionally seen as influencing user satisfaction, this study confirms that it also directly contributes to user performance. In line with the DeLone and McLean IS Success Model, which suggests an indirect effect through user satisfaction, service quality in the context of Approweb appears to have a more direct impact. Technical competence, care, and service availability are key factors in improving user performance. Competent and responsive support staff ensure that users’ issues are resolved quickly, preventing delays in task completion. A caring support system enhances users’ motivation, while the availability of service, especially in critical situations, ensures that users can continue their work without interruptions.

These findings are consistent with previous research, which has demonstrated that service quality improves user performance through various mechanisms. For instance, studies by Bahari and Mahmud (2018) and Ameen et al. (2020) showed that service quality enhances individual and organizational performance, particularly when support is prompt and effective. Similarly, research by Riandi et al. (2021) found that fast, responsive service in e-learning systems contributed to higher user performance in educational settings. These studies emphasize that service quality is not only about resolving technical problems but also about creating an environment in which users feel supported and empowered to perform at their best.

However, despite the strong support for the relationship between service quality and user performance, there are studies that argue the link may not always be direct. For example, Saputra et al. (2023) highlighted that factors such as perceived ease of use and perceived usefulness may play a more dominant role in influencing user performance than service quality itself. This suggests that the influence of service quality might vary depending on the context, the user’s perceptions, and the specific attributes of the system in question.

These findings highlight the importance of a balanced approach that integrates both system quality and service quality to optimize user performance. Approweb’s success in improving account representatives’ performance depends not only on the system’s functionality but also on how well support services are provided. With a holistic focus on both technical and service aspects, the system can significantly contribute to higher individual and organizational performance.

5.3 The relationship between user satisfaction on user performance

User happiness significantly affects user performance, according to this study’s findings. The higher the level of satisfaction reported by users, the greater the improvement in their performance. This relationship can be understood within the framework of the DeLone and McLean IS Success Model (2003), which positions user satisfaction as one of the key dimensions of system success. In the model, satisfaction is the result of interactions between system quality, information quality, and service quality, which encourages more intensive use of the system, ultimately leading to enhanced individual and organizational performance. This suggests that user satisfaction acts as a bridge, connecting users’ experiences with the system to their daily work performance.

Moreover, the socio-technical system theory (Trist & Bamforth, 1951; Bostrom & Heinen, 1977; Abbas & Michael, 2023) adds further insight into this relationship. It emphasizes that system success is not solely determined by technical factors, but also by social elements related to user experience and perception. A technically reliable and easy-to-use system creates better working conditions, while user satisfaction arises from social interactions, such as trust, comfort, and organizational support. When the balance between technical and social aspects is achieved, user satisfaction increases, which in turn impacts performance.

These findings are in line with previous research. For example, Hou (2012) demonstrated that satisfaction enhances individual performance through increased utilization of business intelligence systems, as well as improved user competence and confidence. Similarly, Gonzalez (2015), within the framework of the DeLone and McLean model, emphasized that satisfaction positively contributes to performance by facilitating technology utilization. Gaardboe et al. (2017) found that user satisfaction in public hospitals was significantly correlated with positive individual outcomes. Moreover, Al-Okaily et al. (2023) extended this perspective by identifying system quality, information quality, and training quality as key predictors of user satisfaction, with data quality being the most influential factor in improving perceived benefits and organizational performance.

Logically, high satisfaction drives intrinsic motivation, causing users to be more enthusiastic and committed to completing tasks effectively. Satisfaction also enhances users’ engagement with the system, encouraging them to use it more consistently and intensively. Additionally, satisfaction boosts users’ confidence, making them more willing to make decisions based on the data provided by the system. This combination of factors directly contributes to increased work effectiveness and efficiency.

Descriptive analysis supports this logical progression. Respondents generally expressed satisfaction with the system’s functionality, although operational speed emerged as a significant concern, as delays may disrupt work flow. Moreover, the system’s visual design also plays a role in shaping user experience. In terms of information quality, there is room for improvement in relevance and accuracy, which would better meet users’ specific needs. Support services were also identified as an area for improvement, particularly regarding the speed of response to issues, as delays in support can undermine trust and user comfort.

5.4 The moderating effect of POS on the relationship between system quality and service quality on user performance

According to the study’s findings, POS significantly modifies the relationship between user performance, system quality, and service quality. The findings demonstrate that POS not only strengthens the positive effects of system and service quality on user performance but also highlights the crucial role that organizational support plays in optimizing system usage and improving work outcomes.

System quality and service quality have both been shown to have a direct impact on user performance, as earlier discussed in this study. However, the moderating effect of POS enhances these relationships, meaning that when users perceive high organizational support, the positive impact of system and service quality on performance is significantly strengthened. This finding aligns with the DeLone and McLean IS Success Model (2003), which suggests that system and service quality lead to higher user satisfaction, which in turn impacts user performance. The moderation of POS shows that organizational support amplifies the influence of these quality dimensions on users’ ability to perform effectively.

From a socio-technical system perspective, the findings underscore the importance of balancing technical quality with social factors in ensuring optimal performance. The socio-technical perspective (Trist & Bamforth, 1951; Bostrom & Heinen, 1977) argues that the success of information systems is determined not only by the technical features of the system but also by the social environment in which it operates. In this case, POS, which includes aspects such as managerial support, access to training, and recognition, acts as a social enabler, ensuring that users feel supported in their work, which, in turn, helps them perform better. This is consistent with previous studies indicating that POS can improve employee motivation and commitment, which ultimately enhances performance (Yongxing et al., 2017; Gemilang & Riana, 2021).

Research by Benmoussa et al. (2018) supports the notion that the effectiveness of system quality and service quality in improving organizational performance can be significantly enhanced by organizational support. Their study found that organizational support structures, including adequate training and managerial backing, had a direct positive impact on user performance in information systems. Similarly, Ahmed et al. (2011) demonstrated that POS helped reduce resistance to change and improved the acceptance of technology, thus positively influencing performance outcomes. Therefore, the moderating role of POS in this study aligns with these findings, reinforcing the idea that users who feel supported by their organization are more likely to benefit from system and service improvements.

The findings also underscore that service quality is more likely to influence user performance positively when organizational support is high. As demonstrated in the study by Ameen et al. (2020), service quality contributes to user performance, but this effect is stronger when users feel their organization supports them. In the context of Approweb, this means that when account representatives perceive that they have managerial backing, access to sufficient training, and the resources they need, their ability to use the system efficiently and perform their tasks effectively is enhanced.

This moderation effect of POS also aligns with motivational theories such as Self-Determination Theory (Deci & Ryan, 2000), which suggests that when individuals feel supported, they are more likely to be intrinsically motivated to perform well. In this context, POS acts as a source of external motivation, boosting user confidence and engagement with the system. As user satisfaction with the system and service increases, users are more willing to utilize the system’s full potential, leading to improved user performance.

However, despite these strong findings, it is important to acknowledge that POS does not work in isolation. The moderating effect of POS is influenced by various other factors, including the user’s level of technology readiness and their individual characteristics (e.g., experience, role). This suggests that while POS is an important factor, it interacts with other personal and contextual factors, which may influence its effectiveness as a moderator. Additionally, POS alone cannot compensate for deficiencies in system quality or service quality. For instance, if the system is difficult to use or prone to errors, or if service responses are slow or unhelpful, even the highest level of organizational support may not lead to optimal user performance.

The descriptive analysis further supports these findings, with respondents indicating that while they generally perceived adequate organizational support, they also highlighted areas where support could be improved. These areas included more timely responses to system issues, better training programs, and stronger managerial involvement in facilitating technology adoption. Improving these aspects could enhance the moderating effect of POS on user performance.

These results further emphasize that POS is not just an ancillary factor in the success of information systems but a critical component that can amplify the benefits of well-designed and well-supported systems. Organizations should consider POS as a central element in their strategies for improving system adoption and performance outcomes, ensuring that users feel both technically and socially supported in their use of information systems like Approweb.

6. Conclusion, implications, and limitations

This study provides valuable insights into the significant relationships between system quality, service quality, and user performance, while also highlighting the crucial moderating role of POS. The findings confirm that both system quality and service quality are essential drivers of user satisfaction, which in turn enhances user performance. Specifically, a reliable, efficient, and easy-to-use system, combined with high-quality service support, plays a pivotal role in enabling users to perform their tasks effectively. Furthermore, POS strengthens the positive impact of both system and service quality on user performance, suggesting that organizational support is not merely supplementary but a critical factor in optimizing the benefits of information systems.

The research underscores the importance of a balanced approach, where technical system features and social support mechanisms work in tandem to improve user outcomes. Organizations, particularly in the public sector like the DGT, should focus on enhancing system quality through continuous upgrades and improvements in efficiency, security, and user interface design, while also investing in robust service quality through responsive support, training, and communication. Additionally, fostering a strong Perceived Organizational Support (POS) culture can significantly amplify the impact of these technological and service improvements, ensuring that users not only feel confident in their use of the system but also motivated to perform at their best.

Ultimately, this study contributes to a deeper understanding of the interrelated factors that drive user performance in technology-driven environments. By addressing both technical and social aspects, organizations can create an environment where technology adoption is not only seamless but also leads to meaningful performance improvements, benefiting both individuals and the organization as a whole. Future research could further explore how specific elements of POS interact with other organizational factors to influence user performance in different sectors and contexts.

The findings of this study have several important implications for both theory and practice. From a theoretical perspective, the study reinforces the DeLone and McLean IS Success Model, particularly the role of system quality and service quality in enhancing user performance through user satisfaction. Additionally, it extends the model by highlighting the moderating role of POS, which amplifies the positive effects of both system and service quality on performance. Managers and organizations can use these insights to inform their strategies for designing and implementing information systems, ensuring that both technical and social factors are addressed to maximize user productivity and system success. In practice, organizations like the DGT should focus not only on the technical improvements of systems such as Approweb, but also on strengthening their support infrastructure. Providing responsive service, effective training, and fostering a supportive organizational culture can significantly enhance the performance of users and, ultimately, the organization.

This study has several limitations that provide avenues for future research. Firstly, the research was conducted within the Directorate General of Taxes (DGT), limiting the generalizability of the findings to other sectors or organizations with different user contexts. Future studies could explore whether the relationships between system quality, service quality, and user performance hold across diverse industries, such as healthcare or education, or among different job roles. Additionally, the study relied on self-reported data, which could introduce biases like social desirability or subjective interpretations, and future research could incorporate objective performance measures or triangulate data sources to validate the findings. Furthermore, while this study focused on the direct and moderating effects of Perceived Organizational Support (POS), future research could investigate the mediating role of other factors such as employee engagement, motivation, or job satisfaction in the relationship between system quality and performance. Longitudinal studies examining the sustained effects of POS over time, as well as cross-cultural studies, could offer deeper insights into how organizational support influences system usage and user performance in different cultural and organizational contexts.

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 23 Jan 2026
Comment
Author details Author details
Competing interests
Grant information
Copyright
Download
 
Export To
metrics
Views Downloads
F1000Research - -
PubMed Central
Data from PMC are received and updated monthly.
- -
Citations
CITE
how to cite this article
samiono s, Astuti ES, Worokinasih S and Noerman T. Evaluating Business Intelligence Success in the Public Sector: Extending the DeLone and McLean Model with Perceived Organizational Support [version 1; peer review: awaiting peer review]. F1000Research 2026, 15:109 (https://doi.org/10.12688/f1000research.175003.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status:
AWAITING PEER REVIEW
AWAITING PEER REVIEW
?
Key to Reviewer Statuses VIEW
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions

Comments on this article Comments (0)

Version 1
VERSION 1 PUBLISHED 23 Jan 2026
Comment
Alongside their report, reviewers assign a status to the article:
Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions
Sign In
If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password.

The email address should be the one you originally registered with F1000.

Email address not valid, please try again

You registered with F1000 via Google, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Google account password, please click here.

You registered with F1000 via Facebook, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Facebook account password, please click here.

Code not correct, please try again
Email us for further assistance.
Server error, please try again.