Performance Efficiency Measurement in Managing Elementary Schools by Using a Robust Data Envelopment Analysis

Efficiency is one of the determinants of the quality of education in schools. This study aimed to measure the performance of Indonesian provinces in managing elementary schools by using Data Envelopment Analysis (DEA). This research is evaluation-research using a quantitative approach. As many as 34 provinces in Indonesia were defined as DMUs by using six input variables, namely the number of schools, the number of teachers, the number of teachers certified more than the degree of bachelor, the number of students, the number of classes, and the number of libraries. Meanwhile, there were three output variables, namely the number of graduates, the number of schools that were A-accredited, and the average national exam. The robust data envelopment analysis (RDEA) applied the number of replications B=100, 500, and 1000. The results indicated that RDEA provided better accuracy and preciseness. Overall, all 34 provinces in Indonesia were found to have good performances in managing elementary schools where their efficiency scores were above 90% for both the traditional DEA and RDEA approaches. The implications of this research can be used as a study for stakeholders in making policies to improve the quality of education.


INTRODUCTION
School-based management was the autonomy of education management in the education unit; in this case, the principal and teacher were assisted by the school committee in managing educational activities.Furthermore, school-based management in Indonesia did not only aim to improve the quality of education but to meet national education standards (Bryk, 2010;Gregory et al., 2010;Kanokorn et al., 2013).The management of education units at the level of primary and secondary education implemented school-based management as indicated by independence, partnership/cooperation, participation, openness, and accountability (Lyon et al., 2021;Pramungkas, 2020).It was important to point out that in Indonesia; the national education standard was defined as a minimum education standard that must be

A R T I C L E I N F O
Rokhmaniyah / Performance Efficiency Measurement in Managing Elementary Schools by Using a Robust Data Envelopment Analysis fulfilled by schools to create quality education.Likewise, the framework for creating effective and efficient school management must be based on established national education standards.Furthermore, there were eight aspects of national education standards, namely graduate competency standards, content standards, process standards, assessment standards, educator standards and education personnel, standards of facilities and infrastructure, management standards, and financing standards (Egeberg et al., 2016;Made Sudana et al., 2019).Having good school management was one of the indicators of the success of schools in managing educational institutions.As many as 81% of successful students stated school environment contributed positively to academic success.Furthermore, schools that applied high discipline tended to have better achievements.Other than that, the main key to managing a good school was to have a strong school administration, teachers, principals, and a strong school support system.Good management tended to have a positive impact on improving the quality of schools.Good management played an important role in achieving academic achievement (Abbi, 2012;Ates & Artuner, 2013).School management, especially teacher management, educational programs, facilities, and infrastructure, was one of the most important factors that influenced the achievement and success of students.Good school management could not be separated from good leaders (Daryanto et al., 2017;Kolodziejczyk, 2015;Ng & Szeto, 2016).The leader was the principal, which had a big responsibility to ensuring that all available resources were managed to establish an efficient and effective learning environment therefore the quality of teaching is well maintained to obtain a good educational output.Responsibility and commitment from the school principal were needed to improve successful school management (Arhipova et al., 2021;Epstein, 2018).Management in leadership was an effort to creating, strengthening, developing, and uniting the school community.Therefore, the principal was responsible for the entire community in the school especially for the success of students as well as improving the quality of the school.In creating good school management, a collaboration between the principal, the teachers, and all those with an interest in school was needed.Good school management encouraged organizational members to carry out teamwork.Teamwork was a promising way to build positive relationships and bring healthy collaboration in achieving the education unit's vision and mission (Hassan et al., 2019;Pitsoe & Isingoma, 2014).In fact, Indonesia has a low education efficiency index and even ranks at the bottom of 40 countries (tribunews).The education efficiency index measures the level of quality and quality of an educational unit.A school is said to be efficient if it can produce the maximum level of student achievement with the resources available at the school.This efficiency is also an indicator of good school management.Indonesia, which had 34 provinces, namely DKI Jakarta, West Java, Banten, Central Java, Yogyakarta, East Java, Aceh, North Sumatra, West Sumatra, Riau, Kepulauan Riau, Jambi, South Sumatra, Bangka Belitung, Bengkulu, Lampung, West Kalimantan , Central Kalimantan, South Kalimantan, East Kalimantan, North Kalimantan, North Sulawesi, Gorontalo, Central Sulawesi, South Sulawesi, West Sulawesi, Southeast Sulawesi, Maluku, North Maluku, Bali, West Nusa Tenggara (NTB), East Nusa Tenggara (NTT) , Papua, and West Papua with a total of 147,513 elementary schools consisting of 132,609 public elementary schools and 14,904 private elementary schools.Indonesia with a large number of elementary schools was expected to have good quality and quality, both in terms of products (results/achievements) but also the formation of the character of elementary school students (Fatimah & Mahmudah, 2017).
Education at the elementary school level was the first foundation or basis for education to improve the quality of education.An elementary school is an education unit that organizes the basic education process that underlies the next educational process, namely by providing basic skills in reading, writing, and arithmetic.Further, education in elementary schools was an essential foundation informing students' character (Murniyetti et al., 2016;Pala, 2011).Therefore, elementary school education was the most important part for children in starting cognitive, affective, and psychomotor development, as well as character education.Therefore, efforts were needed to measure the efficiency of the performance of all provinces in Indonesia in school management, especially elementary schools.Efficiency measurement was a very important aspect in the management of educational units to improve the quality of education (Martínez-Caro et al., 2015;Munoz, 2016).Schools were said to be efficient if they found ways to produce maximum levels of student achievement with some available resources (Misut & Pribilova, 2015;Ray, 1991;Wößmann & Schütz, 2006)..The meaning of education efficiency was the ability to do well and deliver desired results without losing resources and effort, time, and money.The most popular method for evaluating technical efficiency was data envelopment analysis (DEA), which was a method to evaluate the productivity of a decision-making unit (DMU) by using several inputs to obtain a targeted output.There had been many studies that examined performance evaluation in elementary schools.Previous study conducted a study in 140 elementary schools in Taiwan to investigate relative efficiency using a two-stage DEA (Zhang, 2010).School size and student characteristics were used as environmental variables in the second stage.The results revealed that small schools had the highest efficiency scores even though they had more resources (Zhang, 2010).Other study implemented 10 elementary schools in Kutahya, a province in Turkey, to measure relative efficiency using the DEA approach with three inputs (students/teachers, students/classes, and students/parts) and two outputs (total scores and the number of graduates) (Kecek & Demirag, 2016).The results showed that there were four elementary schools with efficient performance while the remaining schools were needed to improve their performance to reach the efficiency point.Previous study analyzed the efficiency level of elementary school performance in all provinces in Indonesia using the DEA method (Fatimah & Mahmudah, 2017).Empirical results revealed that the VRS (variable returns to scale) model from DEA provided an efficiency score that was better than the CRS (constant returns to scale) model for measuring the performance of elementary schools in Indonesia.Besides, the CRS model produced 35.29% of provinces that performed efficiently while the VRS model produced around 47.06% of provinces with elementary schools functioning efficiently.
This study applied the robust approach to obtain better results of technical efficiency scores as well as to overcome the weakness in the traditional DEA, which tended to produce over-estimate efficiency scores.However, some studies used this method in DEA for analyzing the efficiency score of DMUs in various fields of studies (Bertsimas & Sim, 2003;Mahmudah & Lola, 2018;Mustakim et al., 2019).
Mahmudah & Lola applied RDEA in estimating employee's performance when they received profit sharing from the company by using 102 samples in a shipping company in Malaysia.The results suggested that the robust approach in data envelopment analysis provided better accuracy of the bias-corrected efficiency scores than the traditional DEA (Besstremyannaya et al., 2015;Mahmudah & Lola, 2018).There were 34 provinces in Indonesia with a total of 147,513 elementary schools which consisted of 132,609 public elementary schools and 14,904 private elementary schools.Therefore, Indonesia was expected to have good quality in education, especially at the level of elementary schools, both in terms of products (results/achievements) and the formation of the character of elementary school students (Fatimah & Mahmudah, 2017;Marini et al., 2018).To produce good quality education at the level of elementary school, a good and planned school organization was needed.The main purpose of this study was to measure the performance efficiency of all provinces in Indonesia in organizing elementary schools based on the input and output variables by using robust data envelopment analysis (RDEA).Further, each province as compared to others to determine which provinces had the best performance.Therefore, the provinces performed efficiently in organizing elementary schools could be used as a role model for other provinces to improve their performances.It was expected to provide a very good framework to improve quality education in Indonesia, especially at the level of elementary schools.Furthermore, it will be analyzed which variables most influence the efficiency of elementary schools.

METHOD
This research is evaluation-research using a quantitative approach (Nurindarwati et al., 2022).As many as 34 provinces in Indonesia were defined as DMUs by using six input variables, namely the number of schools, the number of teachers, the number of teachers certified more than the degree of bachelor, the number of students, the number of classes, and the number of libraries.Meanwhile, there were three output variables, namely the number of graduates, the number of schools that were A-accredited, and the average national exam.The robust data envelopment analysis (RDEA) applied the number of replications B=100, 500, and 1000.DEA was the most popular approach in measuring technical efficiency, which was a non-parametric method therefore there was no need to do distribution assumptions.Basically, DEA compared input and output variables in providing the efficiency scores of each decision-making unit (DMU).There were two models of DEA, constant return to scale (CRS) and variable return to scale (VRS) models.This model assumed that the increasing inputs and outputs had similar ratios and DMUs performed at an optimal scale.Meanwhile, the second model was the development of the first model.Unlike the CRS model, this model assumed that most DMUs did not perform at an optimal scale.In the traditional DEA, a unit was considered to perform efficiently when its efficiency score equaled 1.On the other hand, when the score of performance efficiency was less than 1 then the unit was considered to be inefficient.Meanwhile, RDEA provided better estimates by considering the uncertainty of the input and output variables.Therefore, there was no perfect efficiency score, which was equal to 1.However, the results of RDEA were always consistent with the traditional DEA.
As mentioned before, DEA was a powerful method to measure technical efficiency but this method needed precise and accurate data to produce an unbiased score of technical efficiency of DMUs.In fact, because of the uncertainty of the variables of input and output then it was very difficult to get real data that had a high level of accuracy and preciseness.In order to deal with this problem then the bootstrap method, which was a re-sampling method to approximate the estimator sampling distributions was applied.This approach allowed correcting the estimation bias because the estimated boundary  ̂() of the input variable unsuccessful to include the most DMUs with efficient performances.

Result
This research was conducted by analyzing the secondary data of elementary schools in Indonesia based on the Central Statistics Agency and the Indonesian Ministry of Education and Culture.As many as 34 of Indonesia's major political subdivisions were defined as decision-making units (DMUs).Further, the input and output variables were analyzed by using the R program in order to obtain the technical efficiency scores for each province in organizing elementary schools in Indonesia.This study used six input variables, i.e., the number of schools (X1), the number of permanent teachers (X2), the number of teachers certified more than the degree of bachelor (X3), the number of students (X4), the number of classes (X5), and the number of libraries (X6).Whereas there were three output variables were used, i.e., the number of elementary school graduates (Y1), the number of schools is A-accredited (Y2), and the average national exam (Y3).The selection of input and output variables as well as the DMUs was under the quality standards of education in Indonesia.Table 1 showed a description of the variables of input and output which were used in this study.In producing the efficiency scores, these input and output variables were analyzed to compare the efficiency performance in organizing the elementary school of each province.In order to apply the RDEA, this study used the number of bootstrap replications B = 100, 500, and 1000.The confidence interval size for corrected scores, which was the size of a confidence interval for the bias-corrected DEA score based on robust data envelopment analysis, alpha = 0.01, 0.02, and 0.05.Furthermore, this study applied the inputoriented model under the variable return to scale (VRS) model.The empirical results indicated that the bias-corrected DEA scores for the input-oriented model always followed the technical efficiency scores of the traditional DEA.It was important to note that blue lines represented the efficiency scores by using the traditional DEA.Whereas efficiency scores of RDEA were presented by the red line (for alpha=0.01), the green line (for alpha =0.02), and the purple line (for alpha =0.05).In order to see the comparison of the empirical results from the traditional DEA and RDEA approaches it could be seen in Figure 2. By using 100 replications (B=100), Figure 2 indicated that the efficiency scores of RDEA followed the scores of the traditional DEA continuously which means that the RDEA produced consistent results.Further, the result also revealed that the average technical efficiency of the bias-corrected score for alpha = 0.01 was equal to 0.926 and its standard deviation was 0.035.The highest efficiency score was 0.967 (DI Yogyakarta) and the lowest score was 0.828 (Central Sulawesi).In other words, the province of DI Yogyakarta was expected to be able to support its activity by using only as many as 96.7% of the available resources.Meanwhile, the province of Central Sulawesi was predicted to be able to support its activity by using only as many as 82.8% of the resources.Furthermore, the average bias-corrected score for alpha = 0.02 was equal to 0.926 with a standard deviation of 0.035.The highest bias-corrected score was 0.965 (DI Yogyakarta) and the lowest bias-corrected score was 0.828 (Central Sulawesi).Furthermore, the average biased-corrected score for alpha = 0.05 was equal to 0.926 and its standard deviation was 0.035.The highest biased-corrected score was 0.966 (DI Yogyakarta) and the lowest biased-corrected score was 0.829 (Central Sulawesi).Further, the empirical result also showed a narrow confidence interval which indicated a higher accuracy of the results.Furthermore, the results also indicated that when the values of alpha increased the average confidence interval was smaller.The values were 0.091, 0.086, and 0.074 when alpha = 0.01, 0.02, and 0.05, respectively.Moreover, the results also provided small bias estimates which allowed us to make valid conclusions.Traditional DEA VS RDEA is show in Figure 3. Figure 3 showed RDEA results by using 500 replications which provided a similar average of biascorrected technical efficiency scores for all values of alpha, i.e., 0.926.The values of bias estimates were quite small, which was 0.038.The averages confidence intervals were 0.091, 0.083, and 0.075 for the values of alpha = 0.01, 0.02, and 0.05, respectively.The results also indicated that when alpha was greater, the average confidence interval went down.Meanwhile, figure 4 showed RDEA results when 1000 replications were used, which provided the average of the bias-corrected score as many as 0.926.Further, the results also provided small bias estimates whereas the averages of confidence interval went down when the values of alpha went up.The result of traditional DEA VS RDEA (B=1000) is show in Figure 4.
Furthermore from Figure 4 it was apparent that the technical efficiency scores of RDEA followed the scores of the traditional DEA continuously.It indicated there was a linear relationship, when the efficiency scores of the traditional DEA increased then the bias-corrected scores of RDEA also went along.On the other hand, when the efficiency scores of the traditional DEA went down then the bias-corrected scores of RDEA were also expected to decrease.These figures also indicated that the traditional DEA always provided higher scores of technical efficiency compared to the bias-corrected efficiency scores based on RDEA for all the number of replications.This was because the traditional DEA had a tendency to produced over-estimate scores of efficiency.It was also important to note that the highest efficiency score of the traditional DEA was close to the values of upper bounds of confidence intervals of RDEA.Meanwhile, Table 2 showed the general description of efficiency scores for both the traditional DEA and RDEA where the average efficiency score of the traditional DEA was 0.964 while the average efficiency score of RDEA provided a similar value for all the numbers of replications, which was 0.926.Based on Table 2, the average efficiency score of the traditional DEA was 0.964, which means that all the provinces in Indonesia were supposed to be able to support their activities by using only 96.4% of the available resources.Furthermore, the results also revealed that the traditional DEA produced as many as 16 provinces (37.21%) that performed efficiently where their efficiency scores equaled 1 whereas the lowest efficiency score was 0.852, which means the worst province in managing the elementary school in Indonesia, i.e.Central Sulawesi.

Discussion
The results of the analysis show that RDEA was more appropriate to be applied to measure technical efficiency performance in managing elementary schools in Indonesia due to its produce better accuracy of the results as well as provide less uncertainty.Other than that, this method was expected to be able to overcome the problem of over-estimates that usually occurred in the traditional DEA.The results of this study prove the findings that school performance efficiency is usually measured using DEA.Through RDEA get more accurate results.These results are relevant to the research of previous study found that analysis of efficiency using traditional DEA produces a large estimated score (Fatimah & Mahmudah, 2017).So that elementary schools that should have efficient performance show inefficiency.Another study by other study measuring the efficiency of elementary schools in Jakarta also found that only 14 elementary schools using the DEA VRS model had efficient performance out of the 103 elementary schools measured (Mahmudah et al., 2018).This analysis can be measured using a more accurate approach, namely by using RDEA.
The results of the analysis show that the bias-corrected DEA scores for the input-oriented model always followed the technical efficiency scores of the traditional DEA.In order to see the comparison of the empirical results from the traditional DEA and RDEA approaches it could be seen in figure 2, figure 3, and figure 4 for all the numbers of replication.DEA model of input-oriented indicated that inefficient units needed to reduce their inputs proportionally to achieve efficient performance when the output variables were constant (Mahmudah & Lola, 2018;Simar & Wilson, 2007).RDEA produced a similar value of the average efficiency score for all the number of replications, which was 0.926.This means that in order to support the activity, the provinces in Indonesia were able to use only 92.6% of their resources.Besides, it was also common that its value was smaller than the average efficiency score based on the traditional DEA because the traditional method tended to provide over-estimate results (Mahmudah et al., 2018;Simar & Wilson, 2007).These results were also consistent with previous studies regarding RDEA that reported that RDEA produced a smaller average score than the traditional DEA in measuring technical performances of decision-making units (Bertsimas & Sim, 2003;Mahmudah et al., 2018;Mustakim et al., 2019).Furthermore, the results of RDEA also revealed that the lowest efficiency scores for all the number of replications did not show significant differences (Bertsimas & Sim, 2003).Small wonder that for all the number of replications based on RDEA, the province of Central Sulawesi also had the lowest rank in organizing elementary schools.This statement indicated that RDEA did not contradict the traditional DEA results, but it provided better estimation where the efficiency scores were not over-estimates.Furthermore, for all the number of replications in RDEA, the values of standard deviation were smaller than the traditional DEA, which also indicated that RDEA produced better accuracy in obtaining efficiency scores (Bertsimas & Sim, 2003;Mahmudah et al., 2018;Simar & Wilson, 2007).All of the provinces in Indonesia had a very good performance in managing elementary schools based on the input and the output variables used in this study.It was indicated by the average efficiency scores of both the traditional DEA and RDEA approaches, which were 0.964 and 0.926, respectively.In other words, their performance efficiency in organizing elementary schools reached as many as 96.4% and 92.6%, respectively.Furthermore, RDEA results provided narrow confidence intervals so that it was able to reduce the uncertainty of the analysis results based on the traditional DEA.It was important to point out that a narrower confidence interval had a positive relationship with accuracy and preciseness.The results of RDEA indicated that the technical efficiency bias-corrected scores were always in the range of confidence interval and it was important to note that these values followed the technical efficiency scores of the traditional DEA continuously.
School efficiency is an important aspect of the success of an educational unit.Previous study found that the efficiency of elementary schools in Indonesia as measured by the DEA model (CRS and VRS) showed that teacher qualifications, average national exam scores, and average number of students had a significant effect on elementary school efficiency (Fatimah & Mahmudah, 2017).Teacher qualifications have a major impact on student learning outcomes and the quality of education.Previous study stated that teacher competence and the quality of teacher teaching in class had a positive effect on student learning outcomes (Morrar et al., 2017).Other study explains the big impact if the teacher does not pay attention to his competence (Lozano-Peña et al., 2021).The findings show that teachers who do not develop their competence or have other professions besides being teachers greatly affect the quality of education.The decline in teacher competence has a major impact on the quality of education.This study was also corroborated by study that show teachers who have good competence have a positive impact on student development, especially for children's cognition (Sutisna, 2020).
Apart from teachers, students have an important role in improving the quality of education.High student competence will also produce good results in the world of education.Even with the high ability of students will be able to compete at the international level.Because so far the results of the PISA and TIMSS surveys, Indonesian students always get low scores.The existence of good management of the school in order to develop students' potential is the key in order to increase student competence.Previous study states that student management has a strategic position in developing the quality of education (Ates & Artuner, 2013).The existence of good student management can improve student achievement.Overall, while there may still be some challenges in managing elementary schools in Indonesia, the country has made significant progress in recent years, and all provinces in Indonesia has good performances in managing elementary schools.This is due to the government's commitment to improving education, collaboration between the government, local communities, and private sector, as well as assistance from NGOs and international organizations.
Furthermore, the results also revealed that RDEA obtained better accuracy as well as preciseness of the bias-corrected efficiency scores.Meanwhile, the traditional DEA produced over-estimates scores of efficiency.This is because RDEA also uses bias-correction techniques that can adjust for the potential bias in the estimated efficiency scores.This can further improve the accuracy and precision of the efficiency scores.In summary, RDEA obtains better accuracy and preciseness of the bias-corrected efficiency scores compared to traditional DEA, as it uses robust optimization techniques that can reduce the impact of outliers and other data disturbances, and bias-correction techniques that can adjust for potential bias in the estimated efficiency scores, as the results are not skewed by outliers or other data disturbances.Consequently, it was better to apply RDEA rather than the traditional DEA to measure technical performance in managing elementary schools to avoid over-estimate results.

CONCLUSION
This study applied data envelopment analysis in measuring the technical efficiency of Indonesian provinces in organizing its elementary schools, which was the most popular method in efficiency measurement.However, a robust approach was used to deal with the complexity of the traditional DEA regarding the uncertainty of the input and output variables.The empirical results indicated that all provinces in Indonesia had good performances in managing elementary schools based on both traditional DEA and RDEA.It is important to note that Indonesia has made significant progress in recent years in terms of improving the management of its elementary schools.One of the key factors contributing to this success is the government's commitment to improving education at the national level.Another important factor is the collaboration between the government, local communities, and private sector in managing elementary schools.This has enabled the development of local-level policies and programs that are tailored to the specific needs and resources of each region.This approach has helped to ensure that all provinces in Indonesia have good performances in managing elementary schools, regardless of their individual circumstances.