Data Envelopment Analysis (DEA) is a widely used methodology for evaluating efficiency across decision-making units (DMUs) based on inputs and outputs. However, its “black-box” approach often fails to capture the complexities of multi-stage systems where intermediate outputs are significant ( Färe & Primont, 1984; Seiford & Zhu, 1999). To address this limitation, Network Data Envelopment Analysis (NDEA) extends DEA by modelling the internal structure of DMUs, providing a more granular assessment of efficiency ( Tone & Tsutsui, 2009). NDEA offers unique advantages over classical DEA, including the ability to disaggregate multi-stage processes into sub-processes, enabling a detailed evaluation of inefficiencies ( Färe & Primont, 1984), and improve strategic decision-making by identifying performance bottlenecks across stages ( Kao, 2014). Table 1 summarizes key theoretical advancements of NDEA compared to DEA.

( Färe & Primont, 1984) initiated the exploration of the ‘black-box’ system of classical DEA and was followed by other scholars ( Seiford & Zhu, 1999; Wang et al., 1997). Figure 1 illustrates the interactions among inputs (x _i), outputs (y _r), and intermediate factors (z _kh) in two-stage manufacturing operations. S _(k,h) represents the number of intermediate measures passing from the k ^th process to the h ^th process. Process 1 has input (x ₁) and the output of process 1, which is called intermediate output (z ₁₂), becomes the input in process 2. In addition to z ₁₂, process 2 has a supplementary input (x ₂) to yield output (y ₂). Given that process 2 is the final process, y ₂ is the final output of the manufacturing operations.

While the intermediate and final products are characterized as desirable outputs, a manufacturing process often generates undesirable outputs including rejected products and waste. Figure 1 presents the undesirable outputs produced by both processes (y ₁ ^UD and y ₂ ^UD). Given that desirable outputs are expected to be maximized, undesirable outputs must be minimized. Some authors have incorporated undesirable outputs in the DEA model ( Chung et al., 1997; Scheel, 2001).

(a) Distance, orientation, and scale assumptions

Researchers may choose the NDEA models between the radial and non-radial approaches. The radial approach works under the proportionality assumption in the changes of inputs or outputs. Because in manufacturing operations, production factors such as labour, materials, and capital may not be changed proportionally, we choose the network slacks-based measure (NSBM) that operates the efficiency measurement based on input excess and output shortfall ( Tone & Tsutsui, 2009). The choice of orientation between input and output depends on whether the decision-makers’ focus is on controlling inputs or outputs to improve efficiency. We chose a non-oriented approach to avoid any subjective preferences and merely rely on the mathematical model to assess those production factors or outputs that should be improved. In NDEA, two-scale assumptions are commonly employed: constant returns to scale (CRS) and variable returns to scale (VRS).

In DEA literature, the combination of non-orientation, non-radial, and VRS modelling is often used to enhance the relevance of frontier efficiency studies ( Avkiran, 2011). VRS is often highlighted for accommodating the differing economies of scale among Decision-Making Units (DMUs), where scale is not constant in nature ( Zuniga-Gonzalez et al., 2025). However, this study evaluates a manufacturing line across various production periods where economies of scale are not a concern. Therefore, we selected the combined NSBM, non-oriented, CRS model as the most suitable NDEA approach for multi-stage manufacturing operations within a single plant.

(b) The presence of undesirable outputs

NDEA models maximise the outputs of a system to achieve the system’s optimum efficiency. Manufacturing environments, however, may generate undesirable outputs such as production wastes, rejected products, and pollution. Therefore, NDEA, which incorporates undesirable outputs in its mathematical model, is considered. We applied this undesirability phenomenon to the NSBM approach ( W. B. Liu et al., 2010) as shown in the mathematical model 1 below. The extended model is characterised as NSBM-non-oriented-CRS with undesirable outputs. Min ρ o ∗ = ∑ k = 1 K W k [ 1 − 1 m k ( ∑ i = 1 m k s i k − x io k ) ] ∑ k = 1 K W k [ 1 + 1 R k D + R k UD ( ( ∑ r k D = 1 R k D S r k + D y ro k D ) + ( ∑ r k UD = 1 R k UD S r k + UD y ro k UD ) ) ] Subject to ∑ j = 1 n x ij k λ j k − S i k − ≤ x io k , i = 1 , … , m k , k = 1 , … , K ∑ j = 1 n y r k D j k D λ j k − S r k + D = y r k D o k D , r k D = 1 , … , R k D , k = 1 , … , K ∑ j = 1 n y r k UD j k UD λ j k − S r k + UD = y r k UD o k UD , r k UD = 1 , … , R k UD , k = 1 , … , K ∑ j = 1 n ∑ s ( k , h ) = 1 S ( k , h ) z s ( k , h ) j ( k , h ) λ j k = ∑ j = 1 n ∑ s ( k , h ) = 1 S ( k , h ) z s ( k , h ) j ( k , h ) λ j h , ∀ ( k , h ) , λ j k , S r k + ≥ 0 (1)

The study’s sample size would be n DMUs (j = 1, …, n), which consist of K sub-processes (k = 1, …, K). The objective ρ o ∗ characterises the non-oriented efficiency score of DMUo (the subscript “o” represents the DMU under analysis), while w k in this function is denoted as a subjective weight of the kth sub-process. Because the subjectivity is avoided in this study’s decision-making process, the weights are set as 1.00 for all sub-processes. If ρ o ∗ = 1 and all input and output slacks are equal to zero, then the DMUo is efficient. The method decomposes the overall efficiency into divisional/process efficiency ( ρ k ), given in model (2), where s i k − ∗ and s r k + ∗ are the optimal input-slack and output-slack for model (1). All nomenclatures of models (1) and (2) are described in Table 2. ρ k = 1 − 1 m k ( ∑ i = 1 m k s i k − ∗ x io k ) 1 + 1 r k ( ∑ r = 1 r k s r k + ∗ y ro k ) , k = 1 , … , K (2)

This case study illustrates the transformation of manufacturing operations by applying the NDEA model. It explores the practical implementation of NDEA to establish a multi-stage Performance Measurement System (PMS) within a pharmaceutical production environment. The study aims to achieve two main objectives: developing a practical, process-based performance measurement framework for single-enterprise, multi-stage systems, and examining the trade-off between NDEA model complexity and discrimination power.

The pharmaceutical company occupies approximately 60,000 square meters. It promotes four product groups: intravenous sets (IV Sets), IV Solutions, Therapeutic Drugs, and Clinical Nutrition. The company dominates the market with a 70% share in basic solutions and medicines, catering to both domestic and international markets. Its operations adhere to high regulatory standards and stringent quality requirements, making efficiency and waste minimization crucial for maintaining competitiveness.

This study focuses on the IV Set production line, a medical device used for transferring nutrients intravenously or conducting blood transfer. The strategic choice of this line is based on several reasons: ▪

Strategic Importance: IV sets represent one of the company’s highest-revenue product lines, accounting for nearly 30% of total sales. Ensuring optimal performance in this line has a direct impact on overall profitability.

IV-sets are produced across six interconnected workstations, as depicted in Figure 2. These stages include:

PVC Granulation (WS1): Conversion of raw Polyvinyl Chloride (PVC) resins into medical-grade granules. ▪

Moulding Extruder (WS2): Fabrication of sub-assembled tubes using the granulated PVC.

Each stage is interdependent, with outputs from one stage serving as inputs for the next. This multi-stage structure makes the IV-set line an ideal case for applying NDEA, which accounts for intermediate outputs and interconnected processes.

In NDEA studies, model specification is crucial to list the performance assessment criteria. However, studies that have rationalized the essential variables for performance assessment are scarce ( Cook et al., 2014; Kishore et al., 2024). Once the process map is available, identifying inputs and outputs for each process becomes more straightforward when adopting the NDEA for manufacturing operations.

The initial process involves consuming various PVC resins as raw materials to produce non-toxic medical-grade PVC granules. These granules serve as inputs for the subsequent moulding processes. In the second stage, moulding includes additional materials to create a range of components. The moulding extruder generates an assortment of sub-assembled tubes, whereas the moulding injection yields drip chamber components (e.g., the central part of drip chambers, spikes, needle covers, joints, and seals) for the following assembly process. Both PVC granulation and moulding processes require operators and machines.

The assembly processes combine all the parts produced by the prior workstations. The assembly station comprises three sub-stations: automatic assembly, manual assembly, and final assembly. The automatic assembly combines components from moulding injection into the finished unit of drip chambers. It is characterized as a one-man-one-machine workstation, where machine-hour or man-hour is used interchangeably. As for the manual assembly workstation, sub-assembled tubes from the moulding extruder and additional components from suppliers are manually assembled, requiring only man-hour as the input. The final assembly line involves a combination of machine and manual work, signifying man-hours and machine-hours.

Outputs from the earlier stages, which function as inputs for the subsequent stages, are called intermediate factors. The last stage generates the final outputs. The ultimate product of this production line is the IV-Set in various sizes intended for the transfer of nutrition, medication, and blood. In addition, the IV-Set production line yields waste or rejected outputs known as undesirable outputs. Another undesirable output refers to machine downtime that occurs in PVC granulation, moulding extruder, and moulding injection workstations. Meanwhile, the assembling activities that utilize machines and equipment with lower breakdown risks render machine downtime - an insignificant factor in all assembly processes.

A favourable DEA/NDEA model is typically characterized by its discrimination power. The discrimination power of a DEA model can be compromised when a massive number of inputs and outputs are used, mainly because a particular number of DMUs is considered efficient in certain scenarios ( Cook et al., 2014). As a rule of thumb, the number of DMUs should be at least twice the total number of inputs and outputs. Adhering to this guideline minimizes the correlation between variables and DEA/NDEA outputs, thereby enhancing the discriminating power of the model.

According to ( Castelli et al., 2010), discrimination power is higher in the NDEA model, compared to the classical model. In certain cases, it is possible for most or even all DMUs to be deemed inefficient. However, the idea that adding more stages to the NDEA model improves discrimination power is inconclusive. The existing literature does not provide clear guidance on how to divide a system into multi-stage and interconnected sub-systems, nor does it specify the optimal number of stages needed for particular manufacturing operations. To bridge these gaps, this paper outlines five scenarios aimed at identifying the most appropriate NDEA model for the IV-Set production system (refer to Figure 3). The nomenclature related to Figure 3 can be found in Table 3.

Referring to Figure 3, Model (1) illustrates the classical DEA, where the production line is a ‘black-box’ system and the intermediate factors are dismissed. Model (2) shows the simplest form of NDEA (two-stage NDEA), where the manufacturing process of IV-Set is divided into production and assembly stages. In model (2), the intermediate factors flow between the production and the assembly stages.

Model (3) expands on Model (2) by dissecting the production stage into PVC granulation and moulding after considering the intermediate factors and undesirable outputs that flow between the three stages. Unlike Model (2), Model (3) treats the additional materials required for moulding as a separate variable. Model (4) segregates the production system into four stages based on the responsibilities of the four supervisors in the IV-Set department. Each supervisor oversees one of the following four units: PVC granulation, moulding, assembly, and final assembly.

Model (5) incorporates the six workstations outlined in the process map. It looks into the internal structure of all processes, which consist of the largest number of stages and NDEA variables. The technical correctness of each model was validated by conducting a comprehensive review by the research team and production supervisors. The focus of this validation is to ascertain that there was no significant process change throughout the one-year study period to maintain the high acceptability of the models.

The following sub-section details the data source, the model selections based on five models, and the decision support system facilitated by NDEA for performance evaluation and process improvement. The data from 12 months, including inputs, outputs, and intermediate variables, were extracted from the company. The one-year monthly production period served as the DMUs to meet the requirements of the annual review conducted by the corporate board.

The NDEA efficiency scores were computed for each production month, hence treating the IV-Set production system as a one-, two-, three-, four-, or six-stage production process. The objective is to identify a model that aligns with NDEA requirements and serves as a decision support system for process improvement. The models must exhibit an acceptable level of discrimination power and offer accurate insights for process enhancement. The basic descriptive statistics shown at the bottom of Table 4 demonstrate how effectively the models differentiated the efficiency levels among the DMUs.

Apart from assessing whether an increased number of stages in NDEA can enhance discrimination power, examining the five models determined the most suitable model for the IV-Set production line. The discrimination power of NDEA was assessed by looking into the following statistics: average efficiency score, least efficiency score, number of efficient DMUs, and standard deviation, with a smaller value indicating greater discrimination power.

The descriptive statistics revealed that Model (1) performed best in distinguishing the efficiency scores throughout the production period. However, selecting a single-stage DEA model is unsuitable for a process-based PMS that emphasizes internal processes. This approach provided insufficient information to uncover the specific stages and production network that demands improvements. Model (5) was the favoured model for process enhancement because it offered detailed insights by breaking down the production line into more processes than the other models. However, more stages in an NDEA model introduce additional variables that may affect discrimination power. Based on Table 3, Model (5) exhibited the lowest discrimination power, as indicated by the highest average and the least efficiency values, the smallest standard deviation, and the highest number of efficient DMUs.

Upon comparing Models (1), (2), and (3), the statistical results disclosed that Model (3) had lower discrimination power than Models (1) and (2) for two reasons. First, Model (3) recorded a higher average efficiency score and the lowest efficiency score when compared to Models (1) and (2). Second, Model (3) had a smaller standard deviation value than the other two. Nonetheless, the oversimplification inherent in the single- and two-stage Models (1) and (2) limited their efficacy in comprehending the production system.

After considering the trade-offs, Model (3) was selected as the preferred model for several reasons. Given its moderate number of stages, Model (3) strikes a balance between the discrimination power required by NDEA and the necessary details for process improvement purposes. From the stance of PMS, Model (3) aligns with the parsimony principle, which emphasizes data collection and processing without excessive cost and time implications.

The NDEA non-parametric method measures efficiency by assessing each criterion measure (weighted output/input) and constructing an envelopment frontier across all measures to ascertain that the observed data points lie on or below the frontier. The three-stage NDEA model ( Figure 3C) was deployed in this case study. Scores were computed based on the 12-month production period. The efficiency scores (see Table 5) revealed that 50% of the production period fell below the production frontier. Table 5 presents the inefficient production months and their respective peer groups. A peer group refers to the efficient months with the most similar circumstances to each inefficient month concerning the input and output sets. For example, the peer group of production period 02_23 includes 07_23 and 11_23.

The classical DEA displayed the sources of inefficiency through input and output variables. Taking a step further, the NDEA method evaluated each stage along the network to determine the production process with the most significant impact on the overall efficiency of the manufacturing operations. More insights were captured from the NDEA results, particularly by examining the descriptive statistics (see bottom of Table 4). The PVC granulation stage recorded the lowest efficiency score, which solidified its status as the most inefficient stage and a prominent contributor to the overall inefficiency of the IV-Set manufacturing line. The standard deviation of its efficiency score was the largest, translating into considerable performance fluctuations over the studied production year. On the contrary, the assembly stage displayed the highest average efficiency score and minimal performance variability, further confirming its pivotal role in bolstering the overall production efficiency.

Referring to the efficiency scores, the NDEA produced slacks for each variable in the model to signify the shortfall of outputs or the excess of inputs that rendered a DMU inefficient. Besides, the NDEA offered improvement targets for each production factor (i.e., material, man-hour, and machine-hour) and output (i.e., machine downtime, rejected outputs, and good products). Improvement can manifest as a decrease in inputs and undesirable outputs or an increase in desirable outputs.

For each category in Figure 4, the first, second, and third bars represent inputs consumed or outputs generated at the PVC granulation, moulding, and assembly workstations. The fourth bar depicts the average potential improvement required for each category.

Figure 4 illustrates the imperative need for the company to minimize input materials, man-hours, and machine-hours by 45.08%, 38.16%, and 44.28%, respectively. Moreover, a reduction of 45.83% for machine downtime and 29.51% for rejected outputs/waste appears to be crucial. To achieve 100% efficiency for the entire production system, a comprehensive approach involving cutbacks in all input factors and undesirable outputs, along with an increment of 31.28% in good outputs, is essential.

Managers find the NDEA-based PMS to be effective in analysing the performance fluctuations of production factors and outputs in each production process. This sheds light on the impact of such fluctuations on the overall performance of the production line. Both the production manager and supervisors concurred that the NDEA model comprehensively addressed the essential measures related to medical device manufacturing operations. The model facilitated identifying inefficient processes and pinpointed the production factors or outputs that required enhancement. Decisions associated with process enhancement typically fall in the purview of the manufacturing head department or production line managers and supervisors.

A post-study meeting with the company’s executive board emphasized the need for a generic framework to extend DEA applications in manufacturing. A generalized framework is essential for managers applying similar techniques across production lines or manufacturing companies. Aligning with PMS principles, the proposed framework consists of three main phases: design, implementation, and review (see Figure 5).

In the design phase, the initiation involves process mapping, outlining the systematic flow of the manufacturing process from raw materials to finished products ( Lindsay et al., 2003). A process map visually portrays the systematic flow of sub-processes from start to finish ( Wilson, 2004). According to ( Sinclair & Zairi, 1995a, 1995b) performance measures are classified into inputs, processes, and outputs. The NDEA-based PMS defines input and output factors for each subprocess involved in the production line, enabling efficiency evaluation, benchmarking, and process improvement.

Regular assessments of the production system’s performance—daily, weekly, or monthly—are crucial during the implementation phase. Data collection precedes the calculation of efficiency scores using NDEA software. The model outcomes provide performance scores for each DMU, identifying top performers. For underperforming DMUs, the NDEA model pinpoints sub-processes causing inefficiencies.

In the review phase, two scenarios arise: for major modifications affecting the entire manufacturing process, the cycle resets to the start of the framework. For minor process changes, revisiting the last two stages—implementation and review—is sufficient. Minor alterations involve reviewing improvement targets and benchmarking against peer groups to align with the best performers.