F1000Research F1000Research 2046-1402 F1000 Research Limited London, UK 10.12688/f1000research.153125.2 Research Article Articles Discrepancy in efficiency scores due to sampling error in data envelopment analysis methodology: evidence from the banking sector

[version 2; peer review: 1 approved, 2 approved with reservations]

 L Kishore Conceptualization Data Curation Formal Analysis Investigation Methodology Software Validation Visualization Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0001-9024-0435 a 1 E Geetha Conceptualization Data Curation Formal Analysis Investigation Methodology Validation Visualization Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0002-1346-7641 1 S P Shivaprasad Data Curation Formal Analysis Methodology Validation Visualization Writing – Original Draft Preparation https://orcid.org/0000-0003-2253-6108 2 Chandra Sachin R Data Curation Formal Analysis Software Validation Writing – Review & Editing 1 Jain Raksha Writing – Original Draft Preparation Writing – Review & Editing 1 Department of Commerce, Manipal Academy of Higher Education, Manipal, Karnataka, 576104, India School of Business, RV University, Bengaluru, Karnataka, 560059, India kishore.l@manipal.edu

No competing interests were disclosed.

 25 11 2024 2024 13 890 8 11 2024 Copyright: © 2024 L K et al. 2024 This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Background

Data Envelopment Analysis (DEA) methodology is considered the most suitable approach for relative performance efficiency calculation for banks as it is believed to be superior to traditional ratio-based analysis and other conventional performance evaluations. This study provides statistical evidence on the sampling error that can creep into performance evaluation studies using the DEA methodology. Inferences are drawn based on samples, and various preventive measures must be taken to eliminate or avoid sampling errors and misleading results. This study demonstrates the possibility of sampling error in DEA with the secondary data available in financial statements and reports from a sample set of banks.

 Methods

The samples included 15 public sectors and five leading private sector banks in India based on their market share, and the data for calculating efficiencies were retrieved from the published audited reports. The sample data was collected from 2014 to 2017 because the banking sector in India witnessed a series of mergers of public sector banks post-2017, and the data after that would be skewed and not comparable due to the demonetization policy implementation and merger process-related consolidation implemented by the Government of India. The efficiency measures thus computed are further analyzed using non-parametric statistical tests.

 Results

We found statistically significant discrepancies in the efficiency score calculations using DEA approach when specific outlier values. Evidence is provided on statistically significant differences in the efficiencies due to the inclusion and exclusion of particular samples in the DEA.

 Conclusion

The study offers a novel contribution along with statistical evidence on the possible sampling error that can creep into the performance evaluation of organizations while applying the DEA methodology.

 Sampling Error; Performance Evaluation; Efficiency; Data Envelopment Analysis; Non-parametric Tests; Banking industry The author(s) declared that no grants were involved in supporting this work. Amendments from Version 1

Additional literature supporting the research gap and a summary of the review in a table (Table 1) is added in the literature review section highlighting the gap. Managerial and policy implications are added under the conclusion section. Additional references are also added. A new author was added.

 Introduction

Performance measurement and productivity improvement are vital for any organization as they help assess their performance and compare with benchmark organizations or set benchmarks to identify weak areas and opportunities for performance enhancement. 1 Among the various performance indicators, efficiency is crucial and frequently used as an assessment tool against comparative benchmarks. Banks operate in a highly competitive market, and efficiency evaluation helps identify areas for improvement and develop sustainable strategies for performance enhancement, leading to a better competitive position in the industry. Financial ratio-based evaluation is adopted widely in performance assessment, which can be misleading as the benchmarks are arbitrary. 2 This shortfall has increased the adoption of frontier performance evaluation methods across industries, including the banking sector. 3 , 4 One commonly used non-parametric technique is Data Envelopment Analysis (DEA) in the banking industry. DEA is opined as the aptest approach for relative performance efficiency calculation for banks as DEA methodology is considered superior to traditional ratio-based analysis embraced for conventional performance evaluation. 4 However, inferences drawn with DEA methodology based on samples can lead to possible sampling error. 5 , 6 This study demonstrates the impact of such outlier samples that can skew the efficiency computed using DEA methodology with the secondary data available in financial statements and reports from a sample set of banks. The study offers a novel contribution along with statistical evidence on the possible sampling error that can creep into the performance evaluation of organizations while applying the DEA methodology.

Literature review

As per DEA methodology, the efficiency of any service process or system can be computed as a ratio of output to input. 7 Considering several inputs and outputs, the efficiency of each decision-making unit (DMU) can be written as Efficiency of unit j = u 1 y 1 j + u 2 y 2 j + … … … v 1 x 1 j + v 2 x 2 j + … … … where u 1 is the weight given to the output 1, v 1 is the weight given to the input 1, x 1j is the amount of input from unit j, y 1j is the amount of output from unit j

The model for efficiency computation considering the multiple outputs and multiple inputs based on the above is given by max E m = ∑ j = 1 J v jm y jm ∑ i = 1 I u im x im subject to 0 ≤ ∑ j = 1 J v jm y jn ∑ i = 1 I u im x in ≤ 1 ; n = 1 , 2 , K , N v jm , u im ≥ 0 ; i = 1 , 2 , K , I ; j = 1 , 2 , K , J

Where “E m is the efficiency of the mth DMU”, “y jm is the output of the mth DMU”, “v jm is the weight of the output”, “x im is the input of the mth DMU”, “u im is the weight of the input” and “y jn and x in are the j th output and i th input”

Based on the Linear Programming problem technique, DEA computes efficiency with two assumptions of return to scale - constant and variable, 7 where constant return to the scale-based assumption of DEA is referred to as the overall technical efficiency. At the same time, variable returns to scale decompose overall technical efficiency into two components: pure technical efficiency and scale efficiency. 8

The available literature on the discrepancies that can happen in DEA methodology is very scant. Those discussions on variation in performance efficiency are primarily about sample size and the possible interpretation errors during decision-making. Zhang & Bartels 8 analyzed the variation in efficiency based on sample size using Monte Carlo Simulation. Using data from electricity distribution companies, they evaluated the efficiency and proved that the estimated mean technical efficiency decreases with increased sample size. The impact of sample size on the results of models involving non-discretionary variables was studied by Staat. 8 Smith 5 discussed the model misspecification in DEA, specifically when the sample size is insignificant. Banker and Chang 8 discussed the procedure to detect and remove the outliers in the DEA approach. However, their approach was to remove selected observations or samples. However, the variation that can creep in due to the inclusion and exclusion of a specific set of samples is not considered. While Johnson & McGinnis 6 focused on the mathematical model of inefficient outlier detection, Chen & Johnson 9 developed a mathematical model to detect efficient and inefficient outliers in efficiency computation.

Though the DEA model is a robust tool for assessing the relative efficiency of a group of homogenous DMUs, the key limitation of traditional DEA models is their inability to distinguish between efficient DMUs. As Wichapa et al. 10 highlighted, cross-efficiency measurement offers a valuable extension to the standard DEA approach, enabling a more effective ranking of all DMUs and addressing this limitation. The methodological challenges in comparing different ranking datasets for DEA time-series efficient analysis 11 have been emphasized. Vaninsky 12 found the DEA model advantageous compared to other approaches in his study because the model was non-parametric. To overcome the inherent drawbacks of DEA, Jauhar and Pant 13 tried to develop an efficient system for traditional multi-criteria performance evaluation tools, and the study resulted in a more realistic outcome. Addressing the sampling error in DEA involves dealing with population distribution uncertainties, 14 multicollinearity, 15 and careful model selection. 16 Sampling errors in DEA can significantly impact the accuracy and stability of efficient estimates. This issue arises due to the population distribution uncertainty and input and output variables selection. The accuracy of DEA efficiency scores was questioned due to the uncertainty of the population distribution. This uncertainty can lead to misleading efficiency scores if not adequately addressed. 14 The choice of the DEA model and the selection of input and output variables are critical; selecting the most appropriate model can help reduce the impact of sampling error. 16 The study by Ahn et al. 17 evaluated whether the input-output specification used in DEA for banks aligns with the criteria banks use in decision-making. Four bank behavior models were applied to determine input and output factors in DEA studies. A comparison of these models with standard DEA models reveals pitfalls and the study found that these issues prevent conventional DEA models from accurately capturing bank behavior, making the result less reflective of actual performance. Charles and Díaz 18 have proposed an index that is a non-radial variation of traditional DEA scores, addressing common issues in DEA-based indexes, such as assigning unreasonable weights. Further, the authors recommended using a meta-frontier method to compare the competitive performances across different evaluation periods. Table 1 outlines the prior studies on sampling errors in Data Envelopment Analysis and identifies a few critical gaps in the literature, emphasizing the need for further investigation on whether sampling errors result in misleading results in performance evaluation.

Summary of literature review applying DEA methodology highlighting the research gap.
Source Country of study Sample description Key findings Gaps identified
1 India The study included 42 companies operating in the OGP sector and forming part of the NSE 500 index The study found that about 38% are technically efficient as well as scale efficient, whereas about 62% are pure technically efficient. The study was limited to the OGP sector of India.
5 The UK The study examined 10,000 observations using Monte Carlo simulation to yield a series. The investigation revealed that the risk of misspecification is most consequential when simple models are incorporated with a small sample size. The study examined the impact of using a mis-specified DEA model on efficiency scores. However, significant differences in the efficiency scores were overlooked.
19 India The study included 27 PuSBs and 22 PrSBs, i.e., 49 banks' efficiency, have been measured. The study revealed that PuSBs outperformed PrSBs in all categories of efficiencies. The analysis was limited to measuring the OTE, PTE and SE of PuSBs and PrSBs from 2009 to 2010 and emphasized further empirical research in the field.
20 India The analysis included a sample of 44 banks operating in India. The study results found that slack and radial movements are crucial in efficiently utilizing resources. Lower technical efficiency and low performance lead to higher costs and increased customer service prices. The study focused on assessing the underutilization of public goods & resources and the resulting poor technical efficiency and components.
8 Germany The study included the original data for re-evaluation - which consists of 69 pharmacies. The study disclosed, through reanalysis of the original data, that the efficiency scores are significantly affected by variations in the sample size inherent in the model. The study was limited to demonstrating the effects of sample size on the results of models for non-discretionary variables through a reanalysis of the original study data.
9 Taiwan The analysis encompassed a series of mathematical observations across all four case studies. The model applications in the study indicated that the observations with low-efficiency estimates are not necessarily outliers. The study tried to develop a unified model to identify efficient and inefficient DEA outliers. Furthermore, the study stretched the need for empirical evidence on the DEA model.
11 Germany The study included 70 European universities with budget and staff input data for the period of 2011–2016. The analysis found no evidence of a specific index data issues for the application of ranking data in DEA efficiency analysis The primary attention of the study was analysing the methodological challenges specific to comparing different ranking datasets, particularly index-based versus additive data, for DEA Malmquist index time-series efficiency analysis.

There is a significant amount of research on the performance evaluation of banks based on DEA. 2 – 4 , 21 – 28 These prior studies have given insight into the efficiency of the performance of private and public sector banks in different contexts. These studies using DEA methodology have computed the efficiency based on the sample data collected from financial reports and published reports, and inferences are made according to the efficiency output from DEA methodology. However, the sampling error that can creep into performance evaluation studies using DEA methodology needs to be addressed, as it often gives misleading results regarding the efficiency of sample organizations under study. More precisely, which decision-making units under study must be chosen as the sample in the survey? The DEA methodology is a relative efficiency evaluation technique, and it will lead to misleading extrapolation when proper samples are not considered. Therefore, the problem statement framed is “Do sampling errors result in misleading results in performance evaluation using DEA methodology?

The study aims to critically analyze any significant difference in the efficiency scores due to the inclusion or exclusion of specific sample(s) in DEA methodology-based studies. State Bank of India (SBI) is the outlier to test the hypothesis, as stated below.

“ There is no significant difference in the average efficiency scores between the two groups, SBI included and excluded, respectively.”

“ There is a significant difference in the average efficiency scores between the two groups, SBI included and excluded, respectively.”

Methods

The performance efficiency scores of banks are calculated using DEA methodology. 7 The study sample comprises 20 selected banks in India, which include fifteen public sector and five leading private sector banks based on their market share. The data for analysis for all 20 banks were retrieved from the published audited reports and Profit & Loss statements of banks available on the respective bank websites. 29 The period of data collected is from 2014 to 2017 for all the 20 banks selected, which is before the demonetization policy implementation by the Government of India 30 and before the mergers of many public sector banks in India 31 so that the effects of such a major national-level policy implementation and mergers do not have any impact on the data analysis. Overall technical efficiency based on DEA methodology was computed with the input variables as total and employee expenses and net Non-Performing-Assets (NPAs), considering the two critical variables per the literature review: total income and operating profit as outputs. Efficiency was computed in two groups, with one group of DMUs including SBI (State Bank of India) and another group excluding SBI. SBI is selected for comparison sample as the values of all the variables for SBI considered in the study indicate it as an outlier and, hence, the most suitable sample to compare. The results were analyzed to study the efficiency distribution after the computation of performance efficiency scores of the DMUs. Further, the Mann-Whitney U test is adopted to test the differences in the average distribution between the two groups. 32 The free version of the DEAFrontier tool 32 was used to calculate efficiency scores using DEA methodology, and the open-source R programming software was used to perform the non-parametric hypothesis test. Data visualization was performed using Python open-source programming software.

 Results and Discussion

The performance efficiency score was calculated using DEA methodology, with the SBI sample included and later excluded for comparative analysis. The histogram showing the performance efficiency of the samples – with SBI included and SBI excluded, as shown in Figure 1 and Figure 2, respectively, indicates that the efficiency scores are not normally distributed. The box plot for the performance efficiency scores, with SBI included and SBI excluded, shown in Figure 3, also confirms that the efficiency scores are not normally distributed.

Performance efficiency distribution - SBI included. Performance efficiency distribution - SBI excluded. Box Plot for SBI included and excluded.

Shapiro-Wilk Test is performed to check for normal data distribution, and the test results are shown in Table 2. W-static and p-value for both SBI included and excluded confirm that the distributions are not typical. Skewness and Kurtosis measures also indicate that the efficiency of samples with SBI included having negatively skewed distribution and without SBI included as positively skewed.

Shapiro-Wilk Normality Test result.
Measure SBI included SBI excluded
W 0.77683 0.82856
p-value 2.2e −16 3.8392e −15
Skewness -1.29 1.19
Kurtosis 0.736 0..325

Since the data is abnormal, non-parametric tests are applied to test the hypothesis. The results of the Mann-Whitney U test for independent samples test are shown in Table 3. The p-value is almost equal to 0 at a 0.05 significance level.

Mann-Whitney test for independent samples Test result.
Minimum 25% Median 75% Maximum p-value
SBI included 0.4492 0.8273 0.9800 1.000 1 5.34e −52
SBI excluded 0.0954 0.1986 0.3068 0.4923 1

Interpreting from Table 3, Ho can be rejected, indicating that there is a statistically significant difference in efficiency distribution between the performance efficiency calculated with SBI included and excluded. Wilcoxon signed-rank test with continuity correction 33 , 34 was also performed to test the following hypothesis:

“There is no significant difference in the efficiency score calculated with SBI included and excluded.”

“There is a significant difference in the efficiency score calculated with SBI included and excluded.”

The p-value is almost 0 and is <0.05. Therefore, the authors fail to accept Ho, indicating a statistically significant difference in efficiency values computed with SBI and without SBI being considered in the sample. Spearman’s rank and Kendall Tau correlation tests are also conducted with the following hypotheses to increase the power of a statistical test.

“There is no association between the efficiency values”

“There is a significant association between the efficiency values”

According to Spearman’s rank correlation test, the p-value is almost 0.3671 and >0.05. Also, rho is very negligible, 0.06. Therefore, we fail to reject Ho, indicating no statistically significant association between the efficiency values computed with SBI and those without SBI being considered in the sample. Per the Kendall Tau correlation test, the p-value is almost 0.3165 and >0.05. Also, tau is very negligible, 0.048. Therefore, we fail to reject Ho, indicating no statistically significant association between the efficiency values computed with SBI and those without SBI being considered in the sample. The correlation plot and bar diagram shown in Figure 4 and Figure 5, respectively, also confirm the absence of any association between the efficiency values computed with SBI and those without SBI being considered in the sample.

The correlation plot of SBI included and excluded. Bar diagram of SBI included and excluded.

Therefore, inclusion and exclusion of a sample result in significant variation in the efficiency computed, which in statistical terms is sampling error. Blind consideration of sample units for performance efficiency computation will result in misleading results, flawed interpretations, and erroneous decisions. Before calculation of efficiency using DEA methodology, the box-and-whisker plot for one of the variable Employee Expenses with the outlier SBI included, as shown in Figure 6, along with basic descriptive statistical measures, can help to identify the outliers that can result in efficiency computation errors due to sampling. The box-and-whisker with SBI excluded, as shown in Figure 7, shows how the variables are distributed in a range that allows comparison and efficiency evaluation to be more realistic.

Box-and-whisker plot with SBI included. Box-and-whisker plot with SBI excluded. Conclusion

The relative efficiencies were calculated in two groups; one group included SBI as a DMU, and the other excluded SBI. The efficiencies thus calculated were tested for significant change in the average efficiency between the groups. The study provides statistical evidence on the sampling error that can creep into performance evaluation studies' DEA methodology. In practice, inferences were made based on samples, and various preventive measures must be taken to eliminate or avoid sampling error and related misleading results. The study offers a novel contribution along with statistical evidence on the possible sampling error that can creep into the performance evaluation of organizations due to the inclusion or exclusion of specific samples in the relative performance evaluation computation using DEA. The study also discussed how box-and-whisker can help quickly to identify the possible outlier that can cause skewed efficiency scores due to sampling error. The sampling error in DEA can significantly influence efficiency estimates' accuracy, leading to poorly addressed results. Decision makers should be cognizant of potential biases in efficiency results caused by sampling errors; therefore, considering enhanced models is crucial to achieving more accurate measurements and, thus, enabling effective resource allocation. Policymakers must establish measures to ensure more reliable efficiency evaluations, including measurement error corrections in DEA models. Also, supporting research focused on advanced DEA models that account for sampling errors can lead to better-informed policy decisions and enhance operational efficiency across various sectors. The study was limited only to major and selected sample banks. Future studies can consider all types of banks to get a holistic picture of the banking industry performance benchmark and outlier impacts. Future studies can adopt other relevant non-parametric tests and thus increase the statistical power to confirm the findings.

Ethical approval and consent

Ethical approval and consent were not required.

 Data availability Underlying data

The underlying data of the variables used for performance efficiency score calculation is taken from the audited reports of banks published on the respective bank websites and accessible to the public since the sample companies are registered and listed. For example, the data of SBI Bank is available in the PDF format reports published in the link https://sbi.co.in/web/investor-relations/reports. The user can navigate and select the appropriate year to download the reports in PDF format and retrieve data as required. The links for accessing other sample banks are given below.

https://www.canarabank.com/pages/audited-and-unaudited-financial-results

https://www.iob.in/Financial_perf

https://karnatakabank.com/investors/quarterly-results

https://ucobank.com/financial-results-2014-15

https://centralbankofindia.co.in/en/node/219217

https://www.idbibank.in/idbi-bank-analyst-presentation.aspx

https://www.unionbankofindia.co.in/english/financial-result.aspx

https://bankofindia.co.in/financial-result

https://www.bankofbaroda.in/shareholders-corner/financial-reports/financial-report-q4-2022-23

https://www.hdfcbank.com/personal/about-us/investor-relations/financial-results

https://www.icicibank.com/about-us/qfr

https://www.axisbank.com/shareholders-corner/financial-results-and-other-information/quarterly-results

https://www.kotak.com/en/investor-relations/financial-results/quarterly-results.html?source=website

https://www.yesbank.in/about-us/investor-relations/financial-information/financial-results

For easy access, the data collected from the reports of all the sample banks considered for the study are consolidated and made available in the Figshare repository per the details below.

Figshare: Audited reports data, https://doi.org/10.6084/m9.figshare.26046277. 35

This project contains the following data:

Audited reports data in xlsx file format containing the data required for analysis taken from the banks website. Audited reports data” file contains all the data taken from audited reports that are required for analysis.

Quarterly performance efficiency scores calculated based on the data from audited reports. “Quarterly efficiency score” file contains the calculated performance efficiency scores using DEAFrontier software.

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

 Software availability

The free version DEAFrontier tool, available at https://www.deafrontier.net/deafree.html, was used to calculate performance efficiency scores using DEA methodology. The open-source R programming software, available at https://cran.r-project.org/bin/windows/base/, was used for the non-parametric hypothesis test.

 References Saxena P Saxena RR Sehgal D : Efficiency evaluation of the energy companies in CNX 500 Index of the NSE, India using data envelopment analysis. Benchmarking. 2016;23(1):113–126. 10.1108/BIJ-08-2014-0074 Sathye M : Efficiency of banks in a developing economy: The case of India. Eur. J. Oper. Res. 2003;148(3):662–671. 10.1016/S0377-2217(02)00471-X Ray SC Das A : Distribution of cost and profit efficiency: Evidence from Indian banking. Eur. J. Oper. Res. 2010;201(1):297–307. 10.1016/j.ejor.2009.02.030 Saha A Ravisankar TS : Rating of Indian commercial banks: a DEA approach. Eur. J. Oper. Res. 2000;124(1):187–203. 10.1016/S0377-2217(99)00167-8 Smith P : Model misspecification in Data Envelopment Analysis. Ann. Oper. Res. 1997;73:233–252. 10.1023/a:1018981212364 Johnson AL McGinnis LF : Outlier detection in two-stage semiparametric DEA models. Eur. J. Oper. Res. 2008;187(2):629–635. 10.1016/j.ejor.2007.03.041 Charnes A Cooper WW Rhodes E : Measuring the efficiency of decision making units. Eur. J. Oper. Res. 1978;2(6):429–444. 10.1016/0377-2217(78)90138-8 Staat M : The Effect of Sample Size on the Mean Efficiency in DEA: Comment. J. Prod. Anal. 2001;15(2):129–137. 10.1023/A:1007826405826 Chen W-C Johnson AL : A unified model for detecting efficient and inefficient outliers in data envelopment analysis. Comput. Oper. Res. 2010;37(2):417–425. 10.1016/j.cor.2009.06.010 Wichapa N Khanthirat W Sudsuansee T : A Novel Gibbs Entropy Model Based upon Cross-Efficiency Measurement for Ranking Decision Making Units. Math. Model. Eng. Probl. 2022;9(2):390–396. 10.18280/mmep.090212 Klumpp M : The Index Number Problem with DEA: Insights from European University Efficiency Data. Educ. Sci (Basel). 2018;8(2):79. 10.3390/educsci8020079 Vaninsky AY : Environmental Efficiency of Electric Power Industry of the United States: A data Envelopment Analysis Approach. Int. J. Electr. Power Energy Syst. Eng. 2008;1(1):55–61. Jauhar SK Pant M : Integrating DEA with DE and MODE for sustainable supplier selection. J. Comput. Sci. 2017;21:299–306. 10.1016/j.jocs.2017.02.011 Vaseei M Daneshmand-Mehr M Bazrafshan M : A network data envelopment analysis to evaluate the performance of a sustainable supply chain using bootstrap simulation. J. Eng. Res. 2024;September:1–12. 10.1016/j.jer.2023.10.003 Kalita JK Iyer GR , “ EFFECTS OF MULTICOLLINERITY IN DATA ENVELOPMENT ANALYSIS (DEA) MODELS OF PRODUCT MARKET EFFICIENCY,” Proceedings of the 1995 Academy of Marketing Science (AMS) Annual Conference. Cham: Springer International Publishing;2014; pp.185–191. Öztürk E Bal H : Ranking the Airports with Data Envelopment Analysis and Canonical Correlation Analysis. Gazi Univ. J. Sci. 2017;30(2):237–245. Ahn H Le MH : An insight into the specification of the input-output set for DEA-based bank efficiency measurement. Manag. Rev. Q. 2014;64:3–37. 10.1007/s11301-013-0098-9 Charles V Díaz G : A Non-radial DEA Index for Peruvian Regional Competitiveness. Soc. Indic. Res. 2017;134(2):747–770. 10.1007/s11205-016-1444-9 Puri J Yadav SP : Performance evaluation of public and private sector banks in India using DEA approach. Int. J. Oper. Res. 2013;18(1):91–121. 10.1504/IJOR.2013.055540 Kumar S Raina S : Performance measurement of Indian banking sector: a cross-section perspective. Int. J. Bus. Glob. 2024;37(2):210–236. 10.1504/ijbg.2024.10064190 Geetha E Kishore L Shivaprasad SP : Quarterly performance benchmarking of Selected Banks in India - A DEA approach. Int. J. Pure Appl. Math. 2017;117(20). Special Issue. Gulati R Kumar S : Assessing the impact of the global financial crisis on the profit efficiency of Indian banks. Econ. Model. 2016;58:167–181. 10.1016/j.econmod.2016.05.029 Bapat D : Efficiency for Indian public sector and private sector banks in India: Assessment of impact of global financial crisis. Int. J. Bus. Perform. Manag. 2012;13(3–4):330–340. 10.1504/IJBPM.2012.047299 Narayanaswamy T Muthulakshmi AP : Productivity and cost efficiency of commercial banks in India. Indian J. Financ. Jan. 2016;10(1):8–27. 10.17010/IJF/2016/V10I1/85839 Kaur J Pathak N Aggarwal S : Data Envelopment Analysis (DEA) in measuring the performance of public sector banks in India. Sustainability and Equity: Economic Democracy and Social Empowerment. Jan. 2024; pp.231–243. Khan A Ahmad A Shireen S : Ownership and performance of microfinance institutions: Empirical evidences from India. Cogent Econ. Financ. 2021;9(1). 10.1080/23322039.2021.1930653 Venugopal SK : Loan Portfolio Management and Bank Efficiency: A Comparative Analysis of Public, Old Private, and New Private Sector Banks in India. Economies. Apr. 2024; vol.12(4): 10.3390/ECONOMIES12040081 Kumar S Gulati R : Evaluation of technical efficiency and ranking of public sector banks in India: An analysis from cross-sectional perspective. Int. J. Product. Perform. Manag. 2008;57(7):540–568. 10.1108/17410400810904029 Audited reports data.Accessed: Jun. 16, 2024. Reference Source Rajendra Kumar R : Demonitization and its effect on life of common people in India. Indian J. Econ. Dev. 2016;4(11). Singh S Das S : Impact of post-merger and acquisition activities on the financial performance of banks: A study of Indian private sector and public sector banks. Espacios. 2018;39(26). DEA Software.Accessed: Jun. 13, 2024. Reference Source Hollander M Wolfe DA Chicken E : Nonparametric statistical methods. 2015. 10.1002/9781119196037 Wilcoxon F : Individual Comparisons by Ranking Methods. Biom. Bull. Dec. 1945;1(6):80. 10.2307/3001968 Kishore L Shivaprasad SP Geetha E : Audited reports data.[Dataset]. figshare. 2024. 10.6084/m9.figshare.26046277 10.5256/f1000research.174428.r360607 Reviewer response for version 2 Hadad Yossi 1 Referee Sami Shamoon College of Engineering, Beer-Shvea, Israel

Competing interests: No competing interests were disclosed.

 14 2 2025 Copyright: © 2025 Hadad Y 2025 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

The study addresses performance evaluation using DEA methodology (CCR model): The impact of sampling errors. As highlighted in the article, the focus on statistical evidence and examination of outliers fills a critical gap in the literature. Statistical methods, including the Mann-Whitney U test, Wilcoxon signed rank test, and correlation analyses, are used, adding credibility to the findings. The inclusion of private and public sector banks from India ensures a reasonable sample diversity. At the same time, the period before significant policy changes (e.g., demonetization and mergers) minimizes extraneous noise in the data. Practical contribution: Identifying outliers using visual tools such as box and whisker plots is a practical contribution that practitioners can quickly implement. A discussion of the implications for decision-makers and policymakers highlights the real-world applicability of the study. The article provides new insights into how the inclusion or exclusion of specific decision-making units (DMUs) can bias efficiency calculations.

Comments and Suggestions for Authors:

1) The paper requires rewriting and editing. The contribution of the manuscript was not clear. A strong competitive analysis is needed. What are the contributions to research by writing your manuscript?  Make the exposition tight and clear (right now, it is meandering and redundant, particularly in the first part of the paper).  The background is not well organized. It would be better if the authors first explained the motivation for their study, then discussed the problem statement, and finally reviewed the relevant studies.

2)  In the conclusions section, the authors should provide a general interpretation of the rustles, the unique contributions of the paper, limitations of the research, managerial implications, and the impact that the paper might have on future research and on policy decisions.

3) Page 1- Change equations 1) and 2) to the following formulation:

        

4) It is important to note that:  Model (1) - The CCR model (Charnes et al., 1978) assumes that the production function exhibits constant returns-to-scale (RTS). The BCC developed by Banker, Charnes, and Cooper (BCC model) adds an additional constant variable, , in order to permit variable returns-to-scale.

5) An explanation should be provided as to why you used the CCR model rather than the BCC model.

6) The authors should add the following paragraph: “The study focuses exclusively on Indian banks, which limits the generalizability of findings to other countries or industries with different operational contexts (limited generalizability). Although the study examines key input and output variables (e.g., total revenue, operating profit, total expenses), it is possible that adding variables such as customer satisfaction or adoption of digital banking could provide a more holistic assessment of efficiency (narrow scope of analysis). This is simplistic and mainly involves identifying outliers using descriptive statistics and visualization. It is possible that using a BCC model rather than a CCR model, adding constraints on the weights (a non-Archimedean quantity) could provide deeper insights into the impact of outliers. Although non-parametric tests are appropriate given the distribution of the data, additional statistical techniques could have been applied to strengthen the findings. A broader set of banks in multiple countries or regions can increase the external validity of the findings. Combining advanced DEA techniques: using methods such as bootstrapped DEA or stochastic DEA (see, for example, Olesen & Petersen, 2016). or ranking methods using DEA (see, for example, Hadad, &. Hanani,2011).  Including dynamic variables (e.g., year-over-year growth) and non-financial factors (e.g., environmental and social governance indicators) can add depth to the efficiency analysis."

 Add to the References:

Olesen, O. B., & Petersen, N. C. (2016 [Ref-1]). Stochastic data envelopment analysis—A review.  European journal of operational research,  251(1), 2-21.

‏

Hadad, Y., & Hanani, M. Z. (2011 [Ref-2]). Combining the AHP and DEA methodologies for selecting the best alternative.  International journal of logistics systems and management,  9(3), 251-267.

‏

Nguyen, H. O., Nguyen, H. V., Chang, Y. T., Chin, A. T., & Tongzon, J. (2016 [Ref-3]). Measuring port efficiency using bootstrapped DEA: The case of Vietnamese ports, Maritime Policy and Management. 43(5), 644–659.)

Is the work clearly and accurately presented and does it cite the current literature?

Partly

If applicable, is the statistical analysis and its interpretation appropriate?

Yes

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Partly

Are the conclusions drawn adequately supported by the results?

No

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

Data Envelopment Analysis (DEA), work-study, ranking methods

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

References : Stochastic Data Envelopment Analysis—A review. European Journal of Operational Research .2016;251(1) : 10.1016/j.ejor.2015.07.058 2-21 10.1016/j.ejor.2015.07.058 : Combining the AHP and DEA methodologies for selecting the best alternative. International Journal of Logistics Systems and Management .2011;9(3) : 10.1504/IJLSM.2011.041687 10.1504/IJLSM.2011.041687 : Measuring port efficiency using bootstrapped DEA: the case of Vietnamese ports. Maritime Policy & Management .2016;43(5) : 10.1080/03088839.2015.1107922 644-659 10.1080/03088839.2015.1107922 10.5256/f1000research.167974.r325801 Reviewer response for version 1 Lam Weng Hoe 1 Referee Universiti Tunku Abdul Rahman, Kampar, Malaysia

Competing interests: No competing interests were disclosed.

 9 10 2024 Copyright: © 2024 Lam WH 2024 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve

The study offers a novel contribution along with statistical evidence on the possible sampling error that can creep into the performance evaluation of organizations while applying the DEA methodology. Kindly refer to the following comments for improvement.

1)Please include the managerial and policy implications section

2)Please include the state-of-the-art table in the literature review section to show the research gap

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Yes

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

Statistical modeling, Operations Research

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

L Kishore Department of Commerce, Manipal Academy of Higher Education, Manipal, Karnataka, India

Competing interests: No competing interests were disclosed.

 9 10 2024

Dear reviewer,

Thanks for the review and comments.

We will include implications in addition to tabulating the literature review highlighting the gap in the revision.

Thanking once again.

Regards

Kishore L

 10.5256/f1000research.167974.r325798 Reviewer response for version 1 Čiković Katerina Fotova 1 Referee University North, Koprivnica, Croatia

Competing interests: No competing interests were disclosed.

 30 9 2024 Copyright: © 2024 Čiković KF 2024 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation approve-with-reservations

The topic is well formulated, relevant to the  area of interest and related to an important and current subject on global level.

The abstract is well worded and provides a short, clear and effective introduction to the paper.

Methodology is extensively introduced and each step of the process is presented. There are too many figures. I wonder if they are all beneficial?

However, there is a need for sample clarification. How did you come up with this sample? What was the selection process of the banks entering the sample? The analyzed period is somewhat questionable (it is a period almost 7 years ago, and it may not be relevant anymore). 

Do not use "we", "our" in the text. Instead, use 3rd person singular.

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Yes

Are all the source data underlying the results available to ensure full reproducibility?

Partly

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

Finance, banking sector, M&A, DEA

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

L Kishore Department of Commerce, Manipal Academy of Higher Education, Manipal, Karnataka, India

Competing interests: Nil

 30 9 2024

How did you come up with this sample? What was the selection process of the banks entering the sample?

Response to the above 2 query:

The study sample comprises 20 selected banks in India, which include fifteen public sector and five leading private sector banks. This was done to have representation of public as well as private sector banks in the same. And therefore, as mentioned in the methodology section which I quote " The study sample comprises 20 selected banks in India, which include fifteen public sector and five leading private sector banks based on their market share", the selection was based on their market share. Hope the explanation is satisfactory.

The analyzed period is somewhat questionable (it is a period almost 7 years ago, and it may not be relevant anymore). 

Response to the above query:

The period of data collected is from 2014 to 2017 for all the 20 banks selected. This period was selected because there was two national-level major policy implemented in India - one on Demonetization and another was the merger of many banks. Due to demonetization and mergers there was an overhaul in the banks and also after the mergers the balance sheet is consolidated due to which the data after these policy implementations will not be suitable to empirically prove our hypothesis. And therefore, as mentioned in the methodology section which I quote "The period of data collected is from 2014 to 2017 for all the 20 banks selected, which is before the demonetization policy implementation by the Government of India [23] and before the mergers of many public sector banks in India [24] so that the effects of such a major national-level policy implementation and mergers do not have any impact on the data analysis", we decided to use the data before these major reforms. Hope the explanation is satisfactory.

Do not use "we", "our" in the text. Instead, use 3rd person singular.

Response to the above suggestion:

Completely agree to your suggestion. The correction will be incorporated in the revision. Thanks for the review.