E-learning success evaluation in Lebanon during wartime: An extension of Delone and McLean IS success model

1. Introduction

Although the Lebanese higher education law n°285 does not recognize e-learning studies, since the COVID-19 outbreak, the higher education directorate within the Ministry of Education and Higher Education (MEHE) has imposed a traditional course delivery suspension aligning with the Lebanese government’s decision to gather limitations, and has started progressively working on updating laws and regulations to recognize e-learning with a limited number of online courses or credits per program. However, until today, these efforts have been unfruitful.

The war on Lebanon impacted the higher education sector; for instance, Lebanese citizens, including academic staff and students, had to relocate from warzones to other Lebanese regions or abroad, some campuses of Lebanese higher education institutions in warzones were closed, and campus-based learning turned out to be almost inapplicable and amplified the need for e-learning.

According to available figures, the UN Refugee Agency has reported in December 02, 2024, at least 1.3 million people in Lebanon on displaced. The Lebanese General Security has conveyed that 564 thousand people left Lebanon, which constitutes about 10.5% of Lebanon’s population.

An interview with the higher education general director at MEHE revealed that all Lebanese higher education institutions in warzones (south Lebanon, Bekaa, and Beirut suburbs) were closed, counting 11 campuses of private Lebanese higher education institutions and 14 campuses of Lebanese public university. In addition, six campuses of private Lebanese higher education institutions and 13 campuses of Lebanese university in safe zones (Beirut, Mount Lebanon, Saida, and North Lebanon) were occupied by displaced people. He added that students of all mentioned campuses counted 53000 at private Lebanese higher education institutions and 33000 students at Lebanese public university benefited from e-learning. He mentioned that the rest of the Lebanese higher education institutions were mandatory in providing blended learning for their students since these may be displaced; for instance, 28%, 34%, and 38% are the respective percentages of students who might be displaced and unable to follow campus-based learning at Saint Joseph University in Beirut, American University of Beirut, and Lebanese American University in Beirut. Unfortunately, there were no precise figures for the displaced academic staff.

Since no one knows when the war will stop completely and in rush to overcome this critical situation, on October 3, 2024, MEHE announced the closure of Lebanese higher education institutions campuses located in warzones and granted exceptional permission to conduct all academic activities online, with the possibility of keeping campus-based learning at safe campuses.

Therefore, Lebanese higher education institutions have focused on determining a suitable model to integrate information and communication technologies in learning to guarantee learning continuity. Campus-based, blended and distanced e-learning were employed, and each one requires different learning methods, scheduling, and guidance level.²

As this move to e-learning is quite fast, little attention has been paid to learning outcomes (LeaO); thus, there is a need for empirical studies on how well it works for academic staff and students.

Research on e-learning success is extensive, yet indecisive. Many researchers have confirmed the success of e-learning systems.^3–4 However, many researchers have found that e-learning systems have been unsuccessful.^5,6 Additionally, e-learning success has been evaluated from different perspectives, for instance researchers focused on performance impact,³ actual and continued usage,^7,8 net benefits,^2,5,9 but none of them have considered the evaluation of learning outcomes as e-learning success measure.

Scholars often disregard the relationship between e-learning and its application in unconventional contexts, such as war zones.¹⁰ The organization of e-learning is considered a complex system¹¹ and its assessment involves a deep understanding of knowledge in realistic contexts.¹² Therefore, organizing e-learning in wartime might become more complex and require the joint efforts of students, academic staff, Higher Education Institutions (HEI) administration, and government support from government.¹³ Accordingly, its evaluation should consider the challenges, factors, and context complexity affecting its success in attaining desired outcomes.

There is no consensus among scholars on the success of e-learning in providing efficient higher education during war; thus, the need to be further developed. This study extends the study of e-learning by assessing its success under war risk in Lebanon, which has never been assessed before.

The significance of this study lies in the fact that the MEHE believes that e-learning will not achieve the learning outcomes of traditional learning, whereas Lebanese higher education institutions are encouraged to keep up with international development and increase their positions within the e-learning global market. However, there is little empirical evidence showing that learning outcomes are actually not achieved or that e-learning has not succeeded.

The updated¹ IS Success Model (D&M model) was modified to be used as the base of this conceptual model to assess factors affecting e-learning usage, user satisfaction, and its success in achieving its desired outcomes. The findings of this research provide a theoretical foundation and empirical evidence serving decision makers at MEHE to gain real insight into e-learning usefulness and outcomes and provide important insights for Lebanese higher education institution administrations about investments in e-learning.

2. Literature review

As the demand for e-learning services has increased, with the rapid advancement of information systems (IS) and technology, e-learning systems have become more expensive and require significant time, effort, and money to succeed. Hence, several e-learning success evaluation studies have been conducted by researchers and organizations in developed and developing countries. Despite these efforts, practitioners and researchers are still struggling to identify the issues and factors affecting e-learning success.

Learning outcomes are developed in this research as the e-learning systems’ Net Benefits dimension of the D&M model to be assessed from its users’ perspective.

2.1 Delone and Mclean model

IS success model was introduced as a synthesis of preceding theoretical and empirical IS research conducted between 1970 and 1980, involving management information system success in one coherent model to provide guidance for future researchers.¹⁴ They introduced the success model presented in Figure 1, grouping the six main categories of IS success dimensions. After ten years,¹⁵ reviewed over 150 studies that have empirically validated and applied their model by highlighting recent contributions to IS success measurement and concluded with recommendations for further success measurement. In 2003, they proposed an enhanced model, including six success dimensions, adding service quality as a new dimension for measuring IS success, and merging organizational and individual impacts into a single impact variable called “net benefit”.

Figure 1. Delone and Mclean IS success model (2003).

Although they did not provide the reformulated model as an empirical validation of the previous model, they recommended further development and validation. In response to their recommendation, several researchers have revised and expanded their model in different information system success evaluation contexts, which turned out to be the most predominant IS success evaluation model in IS literature according to Ref. 16 cited in over 300 referenced journals.¹⁷

As for e-learning success evaluation, the D&M model has been employed over the years by various researchers to assess the success of IS.¹⁸ According to a systematic review by Ref. 19, 92 primary studies were identified to systematically compile, review, synthesize, and analyse the D&M model in the e-learning context from 2010 to 2020. Among these studies, only one research²⁰ was conducted from an e-learning outcomes perspective, replacing the net benefits dimension of the D&M model, but in a specific higher education program, namely nursing undergraduate program.

3. Research model and hypotheses

Following the aforementioned research aims, this study adopts the six-dimensions-updated D&M model to determine factors responsible for e-learning success by academic staff and students in Lebanese higher education institutions. As an extension to this model during wartime in Lebanon, the net benefits are replaced by the learning outcomes expected to be achieved by e-learning as an auxiliary medium of campus-based learning, and the war-perceived risk is added as a new dimension affecting the whole success model ( Figure 2).

Figure 2. The research model.

The war circumstances prompted HEI to think of an alternative medium of learning and paved the way towards e-learning use so that they could guarantee education continuity. This significantly increases the use of related technologies and information systems. However, war is likely to lead to a perceived risk affecting the information and communication technology infrastructure in the country; hence, Internet accessibility and e-learning systems are continuously available in a negative way. Therefore, we propose the following hypothesis:

Additionally, e-learning entails a variety of tasks that must be personalized through the learning process. We are talking about the organization and management of information to be provided by academic staff, such as creating online courses and lecture materials, conducting online classes, and recording lectures for asynchronous learning. In addition, assignments and tests should be submitted on time by the students. The fact that war may prevent the timely and up-to-date sharing of information due to displacement or loss of Internet connection, computers, and mobile devices, we propose the following hypothesis:

Likewise, war leads to further setbacks in service and technical support due to the destruction of HEI technical infrastructure and offices, and the displacement of help desk staff responsible for service and technical support. Therefore, we propose the following hypothesis:

In the e-learning context, system quality (SysQ) is defined as the usability and performance characteristics of an information system that can meet users’ needs with maximum ease and minimum problems encountered.¹ This dimension is often mobilized in e-learning literature.^5,7,8,20 However, there is no consensus about its positive effect on the intention to use/use (Int) and user satisfaction (Sat) dimensions. For instance,³ confirmed that system quality affects user satisfaction and the intention to use/use dimensions. However,⁵ found that system quality significantly impacts only the intention to use/use, whereas its effect on user satisfaction is not significant. In contrast, the relationship between e-learning system quality and user satisfaction was not supported by Ref. 20. Therefore, we propose the following hypotheses:

Information quality (IQ) refers to the accuracy of information, information relevance, up-to-date information, and timeliness in an e-learning information system.⁵ Previous studies have found different results, varying from the significant impact of information quality on the intention to use/use and user satisfaction³ to an insignificant impact on user satisfaction,⁵ according to,⁴ information quality does not have a positive effect on intention to use/use. Consequently, the following hypotheses are proposed:

Service quality (ServQ) is an e-learning system requirement for efficient overall service support that can be measured by point responsiveness, empathy, trust, and security.^1,4 In fact, the relationships between service quality, intention to use/use e-learning systems, and user satisfaction have not yet been obtained. According to,⁴ service quality has no significant impact on user satisfaction and intention to use/use. By contrast, these relationships are significant for both.³ Accordingly, the following hypotheses were developed:

According to,¹ intention to use/use is the degree to which the information system is used; it is measured in terms of the duration, nature, and frequency of usage. This definition can also be applied to IS in the context of e-learning.¹ also specified that among the most important guidelines for IS usage is the evaluation of the impact of IS use on endogenous success dimensions. Several papers have examined the effect of the intention to use/use e-learning IS on user satisfaction. Despite a mixed bag of results, there is no consensus on the significance of the relationship between intention to use/use and user satisfaction. Numerous scholars have determined the significance of this relationship.³ However, others have reported the insignificance of this relationship.⁵ Consequently, we propose the following hypothesis:

The net benefits dimension is the second endogenous success dimension of intention to use/use, and its relationship was assessed by e-learning IS scholars.⁵ Adopting the D&M model, e-learning information systems have been assessed from different perspectives, whereby research has replaced the net benefits dimension with another, such as individual impacts,⁴ and performance impact.³ Reference 20 were the only to replace the net benefits with learning outcomes, although they did not assess the impact of intention to use/use on it. Based on previous studies, learning outcomes can be defined as the contributions of the teaching-learning process. It can be measured by acquired generic skills, acquisition of major field-specific knowledge and skills, and course, module, or degree completion. In addition to these traditional learning outcomes, various scholars have mentioned that e-learning helps reduce learning costs and helps students acquire new technical skills.^3,18,26 Thus, we propose the following hypothesis:

IS research clearly shows that user satisfaction is one of the most important factors in assessing the success of a system implementation.¹⁴ User satisfaction with e-learning systems will determine the extent to which it meets user expectations, allows richer interaction between students and academic staff, provides personalized learning, and maintains student motivation. The effect of user satisfaction on achieving successful learning outcomes was only examined by,²⁰ who proposed that e-learning systems could improve perceptions in achieving better learning outcomes when satisfaction is increased. The authors have approved the following hypothesis:

Reference 1 drew feedback that could relate the net benefits dimension to its endogenous dimensions, that is, the intention to use/use and user satisfaction, and the latter to the intention to use/use. In this study, to avoid model complexity, the feedback link and its related reciprocal hypothesis from user satisfaction to intention to use/use, and from learning outcomes to intention to use/use and user satisfaction were excluded.

4. Methodological approach

This study aims to build and test a proposed conceptual model inspired by existing IS and e-learning evaluation models revealed by previous scholars. We chose the appropriate structure for our model after going through a deep literature on IS evaluation and e-learning success assessment in order to choose convenient constructs and to propose related hypotheses. The latter was tested using a quantitative method through a questionnaire-based survey. This methodology enhances the validity of the study results by building a conceptual model, generating related hypotheses, and testing them in the context of higher education in Lebanon.

5. Analysis and results

5.2 Hypothesis testing and structural model

The generated results support eight out of the twelve hypotheses, with the highest coefficient of determination of 69.5%, confirming the model’s strong predictive power, as illustrated in Figure 3.

Table 6. Path coefficients and F² values.

Hypotheses	Path coefficients	T-statistics	P-values	F2 values	Decision
H1	0.194	3.675	0.000	0.042	Supported
H2	0.108	2.064	0.040	0.014	Supported
H3	0.216	4.558	0.000	0.052	Supported
H4	0.522	9.557	0.000	0.251	Supported
H5	0.377	4.642	0.000	0.093	Supported
H6	0.292	5.593	0.000	0.091	Supported
H7	0.256	4.013	0.000	0.056	Supported
H8	0.043	0.850	0.396	0.005	Rejected
H9	0.121	1.814	0.070	0.020	Rejected
H10	0.088	1.562	0.119	0.009	Rejected
H11	-0.020	0.544	0.587	0.003	Rejected
H12	0.847	32.294	0.000	1.397	Supported

Figure 3. Structural model.

The standardized path coefficient (β) is considered proof of evidence for the structural quality of the model. To assess its statistical significance, using resampling bootstrapping procedures, we can generate T-statistics and P-values, whereby T-statistics should be greater than T critical (in this case, 1.96 for a sample size of 429) and P-values should be less than 0.05.

Table 6 shows the path coefficients results; eight out of twelve hypothesized paths were significant, except the relationship path between service quality and intention to use/use (H8: β = 0.043, t < 1.96, p > 0.05), service quality and user satisfaction (H9: β = 0.121, t < 1.96, p > 0.05), intention to use/use and user satisfaction (H10: β = 0.088, t < 1.96, p > 0.05), and intention to use/use and learning outcomes (H11: β = -0.02, t < 1.96, p > 0.05), in which they exhibit weak T-statistics and very high P-values. Therefore, hypotheses (H1, H2, H3, H4, H5, H6, H7, and H12) are supported and hypotheses (H8, H9, H10, and H11) are rejected.

Table 7. Effects of variables in the structural model.

		1	2	3	4	5	6
Direct effect (P-value)	2	0.194 (0)
	3	0.108 (0.045)
	4	0.216 (0)
	5		0.522 (0)	0.292 (0)	0.043 (0.396)
	6		0.377 (0)	0.256 (0)	0.121 (0.070)	0.088 (0.119)
	7					-0.020 (0.587)	0.847 (0)
Indirect effect (P-value)	2
	3
	4
	5	0.142 (0.001)
	6	0.140 (0.001)	0.046 (0.138)	0.026 (0.129)	0.003 (0.494)
	7	0.116 (0.001)	0.348 (0)	0.233 (0)	0.105 (0.066)	0.074 (0.118)
Total effect (P-value)	2	0.194 (0)
	3	0.108 (0.040)
	4	0.216 (0)
	5	0.142 (0.001)	0.522 (0)	0.292 (0)	0.043 (0.396)
	6	0.140 (0.001)	0.423 (0)	0.282 (0)	0.124 (0.061)	0.088 (0.119)
	7	0.116 (0.001)	0.348 (0)	0.233 (0)	0.105 (0.066)	0.054 (0.374)	0.847 (0)

Further, F² values, indicating the effect size of different exogenous variables on endogenous variables, show that user satisfaction and learning outcomes have the highest F² value (1.397), suggesting a very strong impact of user satisfaction on learning outcomes. One other relationship shows moderate effect (F² values between 0.15 and 0.35), namely “system quality -> intention to use/use”. The remaining hypotheses had small or negligible effects (F² values below 0.15), indicating weaker relationships between the latent variables.

Table 7 presents several key findings. First, among the quality dimensions, system quality exerts the strongest effects on intention to use/use and user satisfaction, while service quality has the lowest effect, as explained by a weak T-statistic (< 1.96) and a high P-Value (>0.05) and proved by its F² value of less than 0.02; in this case, service quality has no effect on intention to use/use and user satisfaction. Furthermore, the table reveals the significant indirect effects. For instance, war-perceived risk has a significant indirect effect on the intention to use/use, user satisfaction, and learning outcomes. Among the quality dimensions, system quality exerts the strongest indirect effects on learning outcomes, while service quality has no effect.

The total effects combine direct and indirect paths, providing a comprehensive view of these relationships. For example, the total effects of system quality and information quality on user satisfaction are 0.423 and 0.282, respectively, indicating an overall positive influence, primarily through its impact on intention to use/use. These findings suggest that factors such as system and information quality play pivotal roles in shaping user satisfaction and intention to use/use, ultimately influencing learning outcomes. The model highlights the importance of considering not only direct relationships between variables, but also indirect pathways and the potential influence of contextual factors such as war-perceived risk.

6. Results and discussion

The objective of this study is to propose an e-learning evaluation model as an extension of the D&M model by incorporating war-perceived risk as a dimension affecting the overall quality and subsequently the whole model’s other dimensions. The article investigated war-perceived risk through three items, and the findings revealed that war-perceived risk significantly affects the three quality dimensions, which has never been identified in previous studies.

These findings imply that an increase in war-perceived risk is associated with an increase in the perception of the importance of system, information, and service quality. This suggests that during times of heightened perceived risk, in terms of higher education institutions and housing destruction, students and academic staff displaced to another house, shelters, or even abroad tend to perceive the importance of system quality, information quality, and service quality as higher. The fact that the mass displacement with a significant portion of respondents (35.9%), with the majority (90.3%) residing in another house exacerbated the problems associated with the closure of 63.6% of universities due to war risks and safety concerns, on campus learning became impossible; therefore, e-learning remains the only available solution for educational services provision and forced HEI to make hasty operative changes to technical infrastructure, courses syllabus, and educational service process in order to ease e-learning usage. These findings comply with,²³ who reported that during the war, difficulties prevented proper studies, and the need for significant improvements in e-learning quality has been highlighted.

With regard to quality dimensions, factors other than service quality affect users’ intention to use/use and their satisfaction. The results revealed that the higher the system quality in terms of interactivity, ease, and flexibility, the higher the intention to use/use and user satisfaction. These findings comply with³ deductions, who reported that the system quality to intention to use/use and to user satisfaction hypotheses are both supported. However, it opposes the findings of,⁸ for whom system quality does not exhibit any effect on intention to use/use, and the findings of,⁵ for whom system quality does not exhibit any effect on user satisfaction.

The results show that information quality significantly affects the intention to use/use, and user satisfaction. This implies that users find the information they exchange via the system to be accurate, organized, up-to-date, and relevant to their usage, and suggests that when users perceive the provided information to be of high quality, their satisfaction is significantly improved. The obtained results are consistent with previous studies³ findings, explained by the fact that if information quality is regarded as good, it should promote system use and satisfy its users.

Unexpectedly, H8 and H9 are rejected. Service quality has no significant effect on the intention to use/use. This implies that the quality of the services provided to the academic staff and students does not affect their use of e-learning systems. This can be explained by the fact that during war, they did not have any other option for learning other than using e-learning, so it was compulsory. Therefore, users may perceive that they do not need quality service because they have to use it regardless of their quality. Similarly, service quality had no significant effect on user satisfaction. This can be explained by the fact that since there is no option for learning other than using e-learning, users are expected to receive identical educational services on campus learning, and these expectations may be higher than the existing situation. These results are consistent with those of previous studies.^5,8 Support provided by the help desk to academic staff and students is regarded as insufficient and needs to be improved to increase their satisfaction. In addition, the unavailability and slowness of the system, sometimes because of damaged electrical and network infrastructure, can be reasons behind the dissatisfaction of users and the decrease in their intention to use/use.

Thus, in addition to the direct effect of war-perceived risk on quality dimensions, these risks exert an indirect effect on the user itself in terms of his/her intention to use/use e-learning systems and his/her satisfaction. This was also elucidated by many scholars; for instance,¹³ stated that war brought new challenges to the education sector, such as the destruction of educational institutions and housing, the forced relocation of both students and teaching staff to other regions, and imposing the usage of e-learning as the only available form of education.²³ stated that the consequences of war can demotivate participants in education. In addition, sometimes, during war, not every user has Internet access, or the same level of security; thus, synchronous e-learning may not be acceptable. In such cases, some students may skip their courses, and academic staff may not be able to provide their class, which leads to their dissatisfaction.

The results showed that intention to use/use had no significant effect on user satisfaction and learning outcomes. The insignificance of this relationship between intention to use/use and user satisfaction is contrary to prior similar studies^3,8 but confirms,⁵ which showed that intention to use/use does not affect user satisfaction or the net benefits replaced in this article by learning outcomes. In this context, this result was not surprising. The user is obliged to use the system, and he/she does not have any other means to accomplish the educational process. However, system usage does not necessarily lead users to be satisfied towards the system or significantly affect their learning outcomes. This can be explained by the fact that if the system is not obligatory to be used, we can conclude that students and academic staff are using it because it satisfies their needs or because it leads to higher learning outcomes. Nonetheless, this is not the case for the system in this study; thus, users are dissatisfied due to some limitations of war circumstances, and the learning outcomes they perceive are insufficient.

User satisfaction was found to have a significant effect on learning outcomes. This can be explained by the fact that user satisfaction generates more learning outcomes than intention to use because the user is satisfied when he/she achieves his/her educational goals and success, major knowledge, and generic skills acquisition. Additionally, the Q² value for learning outcomes is greater than 0.35, indicating that the model has a large predictive power for the endogenous construct and that user satisfaction shows a large relative predictive relevance for e-learning. This indicates that overall quality can improve learning outcomes when satisfaction increases.

Finally, the results provide evidence that the perceived risk of war has an indirect effect on the end result of the e-learning process, which is the learning outcome. This explains why even though the intention to use/use does not lead to achieving learning outcomes; the existence of war risks imposes its usage to achieve learning outcomes as the only available solution for learning in such circumstances.

7. Conclusion

This study aims to assess the success of e-learning during the war. To this end, a success model based on the D&M model is developed and tested. Upon literature examination, a lack of e-learning success evaluation was determined in Lebanon during war based on the D&M model. Therefore, this study is of great importance for filling these theoretical gaps. It extends this research field by providing the first empirical attempt based on the D&M model in a developing country context, Lebanon, and under war-perceived risk-exceptional circumstances.

The model was validated through a survey of 429 students and academic staff at Lebanese higher education institutions. Data analysis was performed using PLS-SEM techniques of SmartPLS. The results show that war-perceived risk affects all model factors; with any increase in perceived risk of war, users tend to perceive the importance of overall quality and increase e-learning usage in order to achieve the learning outcomes. It is clear that e-learning usage is an obligation for users, which determines why service quality does not affect their usage and explains users’ dissatisfaction and non-achievement of learning outcomes. War perceived risk positively affects user satisfaction and learning outcomes, but even if e-learning usage is compulsory not under war perceived risk, users should be satisfied with the use of e-learning and feel that it attains more learning outcomes. This article concludes with important theoretical contributions, suggests some practical implications without neglecting all the encountered limitations, and finally recommends future research.

Ethical approval statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the institutional review board (IRB) of Lebanese International University- LIU Institutional Review Board (LIUIRB) to confirm the study meets national and international guidelines for research on humans. (Approval number LIUIRB-250315-LR-396, date 18 October 2024).

Consent statement

All participants were informed in writing about the purpose of the study and provided their voluntary, written informed consent prior to participation.