Renewable Energy Utilization Analysis of Highly and Newly Industrialized Countries Using an Undesirable Output Model

Abstract

In the fight against climate change, the utilization of renewable energy resources is being encouraged in every country all over the world to lessen the emissions of greenhouse gases. However, not all countries are able to efficiently utilize these resources, and instead of providing solutions, the inefficient use of renewable energy may lead to even more damage to the environment. Data from eight countries belonging to the highly industrialized countries (HIC) group and nine from newly industrialized countries (NIC) group were used to evaluate the energy utilization of these groups. Factors such as total renewable energy capacity, the labor force, and total energy consumption were considered to be the input factors, while, CO₂ emission and gross domestic product are the output factors. These factors were used to calculate efficiency scores of every country from 2013 to 2018 using the undesirable output model of Data envelopment analysis (DEA). The grey prediction model was also used to measure the forecasted values of the input and output factors for the year 2019 to 2022, and measure again the future efficiency scores of the HICs and NICs. The combination of grey prediction and DEA undesirable output model made this paper unusual and the most appropriate method in dealing with data that contains both desired and undesired outputs. The results show that the United Kingdom, Germany, France, and the United States continuously top the efficiency ranking among the HIC group, with a perfect 1.0 efficiency score from 2013 to 2022. Russia demonstrates the lowest score of 0.1801 and is expected to perform the same low-efficiency score in the future. Within the NIC group, Indonesia can be highlighted for performing with perfect efficiency starting from the year 2015 and even through 2022. Other NICs are performing at a very low-efficiency, with scores ranging from 0.2278 to 0.2734 on average, with Turkey displaying the lowest rank. This study recommends some useful strategies to improve the utilization of renewable energy resources such as improvements in the political and legal structure surrounding their use and regulation, tax incentives or exemptions to private power producers to encourage shifting away from conventional energy production, partnerships with non-governmental and international organizations that can provide assistance in managing renewable energies, strengthening of the energy sector’s research and development activities and long-term strategic plans for the development in renewable energy with considerations to the social, environmental, and economic impact on each country.

1. Introduction

Global warming, as it is called, is simply the heating up of the atmosphere attributed mainly to the greenhouse effect caused by the rising concentrations of carbon dioxide, chlorofluorocarbons (CFCs), and other air pollutants. This extreme phenomenon is the primary reason for the climate change that is considered to be an environmental crisis. During the United Nations Framework Convention on Climate Change, several state parties agreed to pursue efforts to reduce this century’s average increase of the Earth’s temperature from 2 degrees Celsius to 1.5 degrees Celsius. The consensus entered force last November 2016 and is referred to as the Paris Agreement [1]. One foreseen solution is to slowly transform from conventional energy production based on fossil fuel combustion to renewable energy resources. Fuel combustion is a traditional way to generate electric power through the burning of coal and fossil fuels. Many countries are dependent on this practice. Even some developed ones are major contributors to the presence of CO₂ in the atmosphere caused by this process [2].

In the fight against climate change, other nations are now making use of different renewable energy sources. In this way, they can lessen the use of CO₂-emitting methods in the production of electricity which is very essential to human living standards. Renewable energies are sources of energy which are continuously being replenished naturally by the Earth itself. These energies can be obtained straight from the Sun (solar power and thermal), wind power, hydroelectric power, tidal or wave energy, and geothermal and biomass, but the transition to these energy sources can be difficult and costly. Though many potential benefits can ensue, there are also some technical limitations that must be considered. Also, instead of providing an additional solution to the climate change problem, the improper utilization of renewable energy may lead to more damage to the environment. These environmental damages may be caused by the disadvantages of using renewable energies such as air pollution caused by biomass burning, corrosion problems when using geothermal energy, risk of flooding in the communities surrounding a hydropower plant, negative impact on marine wildlife in using marine energy, and impact on the environmental landscape of using solar and wind energies [3].

There are international agencies and organizations that aim to provide guidance and assistance to those countries that are making their way to the use of renewable energies. One is the International Renewable Energy Agency (IRENA) which is an association of world governments that provides support to countries as they shift to a future with sustainable energy. They also function as a primary platform for global collaboration and as an archive of policies, resources, technologies, and economic know-how on renewable energy [4]. This study will use data from IRENA, Enerdata, and the World Bank.

To evaluate the relative efficiencies of the renewable energy utilization of seventeen (17) nations belonging from highly industrialized countries group (HIC—Russia, Canada, the United States, Japan, the United Kingdom, Italy, Germany, and France) which are also being referred to as the G8 or the Group of Eight Industrialized Nations [5] and newly industrialized countries group (NIC—Turkey, Thailand, Malaysia, Indonesia, India, China, Brazil, Mexico, and South Africa) [6] is the main goal of this study. These HICs and NICs are expected to have highly developed and developing economies. The HICs and NICs are chosen by the authors to be the subject countries for this study because of their high potential in investing in renewable energies since they have more developed economies. The aim is to identify which of the countries are performing efficiently as they progress in the use of their renewable energy resources. Energy consumption has an essential effect to the country’s gross domestic product (GDP) as the ratio between the two factors affects the economic output of several countries since energy is a major input in continuous consumption of goods from energy-demanding sectors such as in production and manufacturing [7].

Three input and two output factors during the six-year periods will be considered for forecasting future values using the grey forecasting method or GM (1,1). The data from the past and the future will then be analyzed using the data envelopment analysis (DEA) undesirable output model.

The combination of these two models makes this study different from other papers, especially the use of the undesirable output model in the energy sector. This model will give consideration to the presence of good and bad outputs, treating the bad or undesired outputs as less important contrary to good outputs. This paper will make use of these two methods to evaluate the past to future efficiencies of seventeen countries.

The whole paper is divided into five sections. Reviews of previous literature related to the study are found in the Section 2. The proposed approach in forecasting future values and evaluation of efficiencies is in the Section 3. The Section 4 presents the interpretations and analyses of data gathered using GM (1,1) and the results of the DEA undesirable model. Concluding statements are described in the Section 5.

2. Literature Review

The energy sector has been a very important aspect of human life and has a strong impact on the economic, social, institutional, and environmental conditions of every country. Cirstea et al. [8] calculated the renewable energy sustainable index (RESI) using the normalization and multivariate analysis which affects the said conditions. The goal of the index is to provide a framework that can be used by the renewable energy sector’s potential investors to aid their decision making. Another study conducted by Iddrisu and Bhattacharyya [9] made use of the arithmetic mean of the four normalized indicators for the measurement of the sustainable energy development index (EDI) which was devised to evaluate, rate, and rank countries according to the calculated energy indices. Lee and Zhong [10] by using min-max normalization combined with multivariate analysis were able to draft the renewable energy responsible investment index (RERII) wherein the primary intention is to help energy investors to decide effectively and proactively and also, to establish an investment framework for energy stakeholders in developing or revising current approaches for investing in the renewable energy field. The ecological factor was considered by Schlör et al. [11] to form the sustainable development index (SDI). In their study, the methods of selecting variables, normalization, and weighting to analyze whether the German energy sector is on a sustainable development track even under the pressure of sustainability goals. A general sustainability indicator for the consumption of renewable energy resources was established by Liu [12] using weighing, quantification, and evaluation of theoretical criteria. The framework incorporated a multicriteria decision-making model (MCDM) called the analytic hierarchy process (AHP) to provide a precise measurement of sustainability. The same index model was developed beforehand by Doukas et al. [13] applying a multivariate technique called the principal component analysis (PCA) for the analysis of nine different indicators to quantitatively measure the energy sustainability of rural communities. Due to the integration of MCDM techniques in various efficiency analysis, the method has become popular to evaluate the energy sector. Štreimikienė et al. [14] combined AHP with the additive ratio assessment technique (ARAS) to analyze the environmental impact criteria and rate the electricity generation technologies in Lithuania. Another method popularly known as the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) was merged with AHP to provide a comparison and rank the five low carbon energy resources in China in a study conducted by Ren and Sovacool [15] in which they found that wind and hydroelectric power have the most potential for improvement. Troldborg et al. [16] make use of the Preference Ranking Organization Method of Enrichment Evaluation (PROMETHEE) method to formulate an assessment model of sustainability in a national-level and ranks Scotland’s technological capabilities for the renewable energy sector. The complex proportional assessment (COPRAS) technique is used by Yazdani-Chamzini et al. [17] for an effective selection of the most pertinent renewable energy project in comparison to the current available options. Another project selection method for renewable energy programs in Spain was applied by San Cristóbal [18] by using the compromise ranking method which allows the Spanish government decision-makers to provide weights of importance to different criteria according to their own preferences. Kabak and Dağdeviren [19] used a hybrid MCDM framework—Benefits, Opportunities, Costs, and Risks (BOCR)—combined with an analytic network process (ANP) method, to aid Turkish policymakers in decision-making in terms of choosing which of five alternative energy sources should be given priority for development.

Aside from the studies which make use of advance statistical methods, several past studies have used DEA as a mathematical method to evaluate the energy production and consumption of several countries. The method is very essential to the assessment since it can provide a comparison of systems with multiple input and output factors.

Halkos and Tzeremes [20] used total capital stock and total labor as input variables and GDP as desirable output and CO₂ emissions as undesirable output for the analysis of 110 countries using data from the year 2007. In this study, DEA constant return-to-scale (CRS) scores provided an analytical result showing that only five countries most likely performed efficiently. However, the outcome of the bias-corrected scores are larger than the scores of the standard deviation, so the authors were able to rank the countries accordingly to get a list of ten countries with the highest and the ten lowest scores. The authors then concluded that the environmental efficiencies of each country have shown positive effects over the first six years after signing the Kyoto Protocol Agreement, but this performance did not last long, as the efficiencies of the countries appeared to decline in the following years. This can be reflected from the countries’ avoidance of compliance with the impositions of the agreement and their inability to accordingly adjust their CO₂ emission reduction to a value that is relative to the growth rate of their economies.

Wang et al. [21] make use of combined DEA models, the super slack-based model, and the Malmquist productivity index for evaluation and selection of sustainable logistics providers in the US. Using financial indicators such as the equity, liabilities, operating expense, and assets as input factors and revenue, net income, and earnings-per-share as outputs, their paper established a list of rankings among different decision making units (DMUs). With the combined methods used, the paper was able to determine the nine perfectly efficient logistic providers among the sixteen DMUs.

Oggioni et al. [22] measure the ecological efficiency of the cement industry from 21 countries. Using CCR (Charnes-Cooper-Rhodes) and BCC (Banker-Charnes-Cooper) DEA models, the study was able to determine which of these countries performed efficiently in terms of disposability of undesirable outputs. Two outputs were used, cement production and CO₂ emission, first being the desirable one and the second otherwise. These data—including the total labor, installed capacity, energy consumption, and raw materials—were collected from the years 2005 to 2008 as inputs for the application of DEA. Analyzing the scores for every country, the outcome points out that during the period under study there are countries like Canada, Brazil, Turkey, and the United States that performed the very worst in terms of eco-efficiency. This is due to the absence of strong environmental protection regulations. Emerging countries like China and India, at that time, were showing high-efficiency levels in cement production. This is attributed to their investment in more efficient technologies and the production of low-quality cement which emits lower CO₂ levels.

Woo et al. [23] applied the DEA-based Malmquist productivity index (MPI) to evaluate the environmental efficiency of renewable energy of 31 country members of the Organisation for Economic Co-operation and Development (OECD). This study makes use of total labor, total capital, and total renewable energy supply as input factors, while renewable electricity generation and GDP are used as output factors. Carbon emissions are also part of the study as they were considered to be the undesirable output for the analysis. The MPI model is deployed to measure the technical efficiency change, frontier change, and productivity of the involved countries with and without consideration of carbon emissions. The results of this study concluded that in the evaluation of efficiencies for the energy sector, the undesirable output must always be considered as they have a significant relationship to energy performance. The paper also encourages policymakers to support the development of technologies related to the use of renewable energy in their own country as it also has a significant impact on the energy market.

The same DEA model, MPI, was used by Zhou et al. [24] to measure the carbon emission performance from the world’s top 18 CO₂-producing countries. Capital stock, total energy consumption, and total labor force were used as the input variables while GDP and CO₂ emissions were the outputs. The MPI scores display the performance of these countries in terms of carbon emission productivity, efficiency change, and technological change over the period of 1997 to 2004. The results showed that there was a 24% increase in carbon emissions during the study period and cites technological progress as the primary reason. Germany turned out to be the number one carbon emitter while Indonesia and China displayed negative productivity.

Wang et al. [25] made a forecast of energy efficiency or the period from 2018 to 2023 using data from 25 countries. The study method used the DEA Slack-Based Model (SBM) to determine the efficiencies using historical data from 2008 to 2017 and then applied grey forecasting to aid the SBM to produce future efficiency scores. The countries were chosen according to the sufficiency of data that is available from different sources. Two commonly used economic indicators (labor force and capital stock) and one energy-related factor (energy consumption) were used as inputs. The desirable output is GDP and the undesirable one is CO₂ emissions. This combination of variables used for the study is enough to provide appropriate results to achieve the goal of the paper. After the result analysis, the authors found out that during the past period, only eight out of 25 countries performed efficiently and this performance will be maintained for the future period. This indicates the proper balance in their growing economies while protecting the environment due to a deliberate reduction of CO₂ emissions. The findings also suggest that European countries have higher efficiency scores compared to those in Asia and America. The paper further recommended a substantial government policy intervention for every country that should focus on improving the energy production and environmental sectors.

More studies have suggested different considerations for input and output factors to be used for the analysis of countries’ energy efficiency scores, as listed in

Table 1. List of commonly used input and output factors for several past literature.

Authors, Year [References]	Factors		Method/s	No. of Counties
	Inputs	Outputs
Zofio and Prieto, 2001 [26]	Energy Consumption Capital stock Labor	GDP CO2 emission	DEA	18
Xie et al., 2014 [27]	Labor Installed capacity Fuel and nuclear	Power generation CO2 emission	DEA-SBM	26
Cicea et al., 2014 [28]	GDP capita Energy intensity Investment to renewables	CO2 emission	DEA	22
Wang et al., 2018 [29]	Energy Consumption Population	GDP CO2 CH4 methane N2O nitrous oxide	DEA-Undesirable model	42
Chien and Hu, 2009 [30]	Capital stock Energy consumption Labor	GDP	DEA	45

below.

These studies are proof that DEA is an effective way to measure the efficiencies of the energy sector from different countries using diverse combinations of inputs and outputs. This method has played a very important role in efficiency analyses since it was introduced by Charnes et al. [31] in 1978. The Charnes, Cooper, Rhodes (CCR) model became the first traditional method to calculate the relative efficiency scores of several numbers of DMUs which represents the technical efficiency. Banker et al. [32] in 1984 presented another model called Banker, Charnes, Cooper (BCC) to evaluate efficiencies of the DMUs that are not or not yet operating at an optimum scale in which CCR is incapable of. Another DEA-based model was proposed in 1982 by Caves et al. [33] and is called the Malmquist Productivity Index (MPI). It was later been split into two segments by Fare et al. [34] to represent catch-up and frontier-shift efficiencies. Furthermore, non-radial DEA types such as the additive (ADD) [35] and slacks-based measure (SBM) [36] models were also introduced.

These developments to DEA involve the introduction of a model that will be able to consider the presence of some bad outputs or unwanted factors. Cooper et al. [37] modified the SBM model to be able to provide a more precise measurement of efficiency and is called the undesirable outputs model.

DEA models are used to measure the efficiency coming from data currently available. There can be no measurement of future efficiencies but only can measure the past up to the present scores. Wang et al. [38] use GM (1,1) along with DEA to measure the future efficiency performances of some Vietnamese ICT firms. The grey forecasting method is a time-series prediction model that does not require enormous amounts of data to be able to produce highly accurate results [39]. This capability of GM (1,1) have made it become a popular forecasting tool for many studies which this paper will also utilize and combine with the DEA undesirable output model.

3. Materials and Methods

3.1. Research Process

To be able to reach the goal of this paper, this study is divided into four stages as shown in Figure 1. This will serve as the guide for the authors in finalizing the study.

• Stage 1. Collection of Data

Data were collected through IRENA, Enerdata, and World Bank. Based on several pieces of cited literature, the authors selected the appropriate input and output factors suitable for this study. These factors are also commonly used by previous studies related to this paper.

• Stage 2. Grey Forecasting Method

The grey forecasting method is used to predict the values of the input and output factors for the future period. The method uses historical data. The mean absolute percentage error (MAPE) determines whether the predicted value is acceptable or not. Lower values of MAPE means higher accuracy.

• Stage 3. Pearson Correlation

To check if the selected input and output factors and the predicted values are suitable for DEA processing, the calculation of the Pearson coefficient of correlation is very necessary. This method was widely used in previous studies. It is used to confirm the isotonic relationship between factors and a positive correlation is a requirement for DEA.

• Stage 4. Data Analysis and Conclusion

Since this paper focuses on the efficiency of renewable energy programs, the presence of carbon emissiosn suggests the use of the DEA undesirable output model. The DEA result will show which countries performed efficiently and those did not. The ranking will be based on the output efficiencies. This is to determine which countries among HICs and NICs have better renewable energy capabilities. The conclusions will provide a summary and addresses the objective of the study. The authors will specify some recommendations and information valuable for decision-making by policymakers.

3.2. GM (1,1) Grey Prediction Model

The GM (1,1) grey prediction model is a widely used forecasting method associated with time series and differential equations. One advantage of this method is the requirement of few historical data, at least four successive data with intervals that are equally distributed in a timely manner, to generate an acceptable prediction and calculated efficiently as discussed by Julong [39] and supported by Tseng et al. [40]. The procedure for grey prediction is shown in Figure 2 below.

Given the variable primitive series $X^{\left(0\right)}$ in Equation (1), a more detailed procedure of prediction using the GM (1,1) grey model is discussed:

$$X^{\left(0\right)}=\left[ X^{\left(0\right)}\left(1\right), X^{\left(0\right)}\left(2\right),\dots , X^{\left(0\right)}\left(n\right) \right] ,n\ge 4$$

(1)

where $X^{\left(0\right)}$ is a positive sequence and n is the total number of historical observations [39].

One very necessary property of a grey model is the accumulating generation operator (AGO) that is used for elimination of uncertainties from the primitive data. The equation for AGO is presented in Equation (2):

$$X^{\left(1\right)}=\left[ X^{\left(1\right)}\left(1\right), X^{\left(1\right)}\left(2\right),\dots , X^{\left(1\right)}\left(n\right) \right] ,n\ge 4$$

(2)

where $X^{\left(1\right)}\left(1\right)=X^{\left(0\right)}\left(1\right), X^{\left(1\right)}\left(k\right)={\displaystyle \sum }_{i=1}^k{X}^{\left(0\right)}\left(i\right),$ and k = 1, 2,…, n [39].

The partial data series is described in Equation (3):

$$Z^{\left(0\right)}=\left[ Z^{\left(1\right)}\left(1\right), Z^{\left(1\right)}\left(2\right),\dots , Z^{\left(1\right)}\left(n\right) \right]$$

(3)

where $Z^{\left(1\right)}\left(k\right)$ is the value of the mean from the adjacent data described in Equation (4):

$$Z^{\left(1\right)}\left(k\right)=\frac{1}{2}\times \left[X^{\left(1\right)}\left(k\right)+X^{\left(1\right)}\left(k-1\right)\right], k=2, 3,\dots ,n,$$

(4)

Through $X^{\left(1\right)}$, the first order differential equation $X^{\left(1\right)}\left(k\right)$ of grey prediction model can be derived from Equation (5) [39]:

$$\frac{d{X}^{\left(1\right)}\left(k\right)}{dk}+a{X}^1\left(k\right)=b$$

(5)

wherein a is the developing coefficient and b is the grey input.

In general, Equation (5) does not generate the values for parameters a and b. The above equation is solved through the least square methods (Equation (6)):

$${\widehat{X}}^{\left(1\right)}\left(k+1\right)=\left(X^{\left(0\right)}\left(1\right)-\frac{b}{a }\right)e^{-ak}+\frac{b}{a}$$

(6)

where ${\widehat{X}}^{\left(1\right)}\left(k+1\right)$ depicts the prediction value of X at a k + 1 point in time. Using the method of ordinary least square (OLS), the values of [a,b]^T can be acquired as described by Equations (7)–(9) [39]:

$${\left[a,b\right]}^T={\left(B^{T}B\right)}^{-1}{B}^{T}Y$$

(7)

$$Y=\left[\begin{array}{c} x^{\left(0\right)}\left(2\right) \\ x^{\left(0\right)}\left(3\right)\\ \dots \dots \dots \\ \dots \dots \dots \\ x^{\left(0\right)}\left(n\right) \\ \end{array}\right]$$

(8)

$$B=\left[\begin{array}{cc}-z^{\left(1\right)}\left(2\right)& 1\\ -z^{\left(1\right)}\left(3\right)& 1\\ \dots \dots \dots & \dots \\ \dots \dots \dots & \dots \\ -z^{\left(1\right)}\left(n\right)& 1\end{array}\right]$$

(9)

wherein ${\left[a,b\right]}^T$ is referred to as the parameter series, Y is the data series and B is the data matrix.

The values for ${\widehat{X}}^{\left(1\right)}\left(k\right)$ will be calculated by letting ${\widehat{X}}^{\left(0\right)}$ become the predicted series:

$${\widehat{X}}^{\left(0\right)} X^{\left(0\right)}\left(1\right), {\widehat{X}}^{\left(0\right)}\left(2\right), \dots ,{\widehat{X}}^{\left(0\right)}\left(n\right)$$

(10)

where ${\widehat{X}}^{\left(0\right)}\left(1\right)=X^{\left(0\right)}\left(1\right)$

Equation (11) is obtained through the application of inverse accumulated generation operation [39]:

$$X^{\left(0\right)}\left(k+1\right)=\left(X^{\left(0\right)}\left(1\right)-\frac{b}{a }\right)e^{-ak}\left(1-e^a\right)$$

(11)

The accuracy of the predicted values can be measured using the actual and predicted data. The measurement is called the mean absolute percentage error or MAPE and is described in the formula below:

$$MAPE=\frac{1}{n} {{\displaystyle \sum }}^{ }\left(\frac{x^{\left(0\right)}\left(k\right)-{\widehat{x}}^{\left(0\right)}\left(k\right)}{x^{\left(0\right)}\left(k\right)}\right)\times 100\%$$

(12)

The acceptability of the predicted data depends on the values of the MAPE. Small values for MAPE means a higher rate of accuracy. Ju-long [41] also categorized the reliability classes into four as listed in

Table 2. Equivalent forecast category for every MAPE percentage score.

MAPE	Forecast Categories
<10%	High Accuracy
10–20%	Good
20–50%	Reasonable
>50%	Inaccurate

.

3.3. Data Envelopment Analysis—Undesirable Output Model

The undesirable output model is one of the many widely used DEA models. One thing that makes this model special and different from the others, is that this model considers the presence of bad output factors in the data set. Cooper et al. [37] modified the slack-based model (SBM) to be able to take account of the undesirable outputs during efficiency analysis. This research contains data involving the presence of a bad output making it more suitable for the study. The undesirable model will be described in the following paragraphs.

The input and output matrix of the DMUs will be denoted as (x₀,y₀). The output parameters of the matrix y will be decomposed into two: the desirable outputs are Y^g (good matrices) and the undesirable outputs are Y^b (bad matrices). The whole decomposition will become $\left(x_0, y_0^g, y_0^b\right)$.

The set for production possibility is described by:

$$P=\left\{ \left(x, y^g, y^b\right) | x \ge X\lambda , y^g\le Y^g\lambda , y^b\ge Y^b\lambda , L \le e\lambda \le U, \lambda \ge 0\right\}$$

(13)

wherein the intensity factor is $\lambda$, L is the lower boundary and U is the upper boundary for $\lambda$.

In a presence of bad output, a DMU is efficient if there is no vector $\left(x, y^g, y^b\right) \in P$ in such $x_{0 }\ge x, y_0^g\le y^g, y_{0 }^b\ge y^b$ having at least one inequality.

The modification of SBM to attain the objective of the undesirable model is described as:

$${\rho }^{*}=\min \frac{1-\frac{1}{m} {{\displaystyle \sum }}_{i=1}^m\frac{s_{io}^{-}}{x_{io}}}{1+\frac{1}{s} \left({{\displaystyle \sum }}_{r=1}^{s_1}\frac{s_r^g}{y_{ro}^g}+{{\displaystyle \sum }}_{r=1}^{s_2}\frac{s_r^b}{y_{ro}^b}\right)}$$

(14)

subject to $x_0=X\lambda +s^{-}; y_0^g=Y\lambda -s^g; y_0^b=Y\lambda +s^b;L\le e\lambda \le U; s^{-}, s^g, s^b, \lambda \ge 0.$ The excesses in inputs are denoted by the vector $s^{-}$ and bad outputs is $s^b$. In contrast, $s^g$ denotes the lack of good outputs. $s_1$ and $s_2$ express the number of elements in $s^b,$ and $s^g,$ and s = $s_1+s_2.$

According to Cooper et al. [37], the DMU $\left(x_0, y_0^g, y_0^b\right)$ is efficient even under a condition of any undesirable outputs if ${\rho }^{*}=1$. An inefficient DMU, ${\rho }^{*}<1$, can be enhanced by removing the excesses in inputs and bad outputs and intensifying the deficiencies in good outputs with the following projection:

$$\begin{array}{c}{x}_0\Leftarrow x_0-s^{-*}\\ y_0^g\Leftarrow y_0^g-s^{g*}\\ y_0^b\Leftarrow y_0^b-s^{b*}\end{array}$$

(15)

Through the Charnes–Cooper transformation method as described by Tone in 2001 [36], the fractional formula can be converted into a linear program with the following variables for the constant return to scale.

Whereas:

$$\begin{array}{r}v, u^g,u^b\\ L=0, U=\infty \\ \max u^g{y}_o^g-v-u^b{y}_o^b\end{array}$$

(16)

subject to:

$$u^g{Y}^g-vX-u^b{y}^b\le 0$$

(17)

$$v\ge \frac{1}{m}\left[\frac{1}{x_o}\right]$$

(18)

$$u^g\ge \frac{1+u^g{y}_o^g-v{x}_o-u^b{y}_o^b}{s} [\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$y_o^g$}\right.]$$

(19)

$$u^b\ge \frac{1+u^g{y}_o^g-v{x}_o-u^b{y}_o^b}{s} [\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$y_o^b$}\right.]$$

(20)

The $v$ and $u^b$ variables, are respectively referred to as the values of inputs and bad outputs while $u^g$ refers to the value of good outputs.

Cooper et al. [37], set out the weights for bad and good outputs to be encode before running the undesirable output model. The weight variables must satisfy the $w_1,w_2\ge 0$ conditions for the bad and good outputs so that the calculated relative weights will become $W_1=s{w}_1/\left(w_1+w_2\right)$ and $W_2=s{w}_2/\left(w_1+w_2\right)$. The final function will be transformed into:

$${\rho }^{*}=\min \frac{1-\frac{1}{m} {{\displaystyle \sum }}_{i=1}^m\frac{s_{io}^{-}}{x_{io}}}{1+\frac{1}{s} \left(W_1{{\displaystyle \sum }}_{r=1}^{s_1}\frac{s_r^g}{y_{ro}^g}+W_2{{\displaystyle \sum }}_{r=1}^{s_2}\frac{s_r^b}{y_{ro}^b}\right)}$$

(21)

The default value for $w_1$ and $w_2$ is 1. To give importance to the degree of emphasis for the evaluation of bad outputs, the value of $w_2$ can be larger than $w_1$ or vice versa. In this model, the heading (O) refers to good outputs while (OBAD) is for bad outputs.

4. Results

4.1. Data Analysis of the Input and Output Factors

The mere difference between the NIC and HIC is that NICs are those countries in which their economic development is said to be in between those classified to be as developing and highly developed. Substantial growth in their own gross domestic product (GDP) is a key indicator in transitioning from one classification to another. However, the authors choose to process the data combining NICs and HICs due to the small number of HICs in the world. For DEA to come up with a highly reliable result, there should be sufficient number of DMUs for comparative analysis.

Table 3. Name of the countries in their respective DMU number and group.

DMU No.	Highly Industrialized Countries (HIC)	DMU No.	Newly Industrialized Countries (NIC)
CTRY1	France	CTRY9	South Africa
CTRY2	Germany	CTRY10	Mexico
CTRY3	Italy	CTRY11	Brazil
CTRY4	United Kingdom	CTRY12	China
CTRY5	Japan	CTRY13	India
CTRY6	United States	CTRY14	Indonesia
CTRY7	Canada	CTRY15	Malaysia
CTRY8	Russia	CTRY16	Thailand
		CTRY17	Turkey

below lists down the names of countries belonging to NICs and HICs with their respective DMU representation.

The primary objective of data envelopment analysis is to calculate the efficiency using multiple inputs and outputs. From the diverse combination of factors used by previous studies as presented in Table 1, the authors decided to use total renewable energy consumption (TREC), the labor force (LF) and total energy consumption (TEC) as input factors, while carbon dioxide emission (CO₂) and gross domestic product (GDP) are the outputs.

Table 4. Descriptive statistics and Pearson correlation coefficients of the year 2015.

	Input Factors			Output Factors
	Total Renewable Energy Capacity in GW (TREC)	Labor Force In millions (LF)	Total Energy Consumption in Mtoe (TEC)	Carbon Dioxide Emission in MtCO2 (CO2)	Gross Domestic Product in $ Million (GDP)
	Descriptive Statistics
Max	479.11	785,372.42	2993.90	9061.26	18,219.3
Min	3.43	14,589.35	86.33	228.53	296.64
Average	80.98	124,337.42	560.54	1401.99	3229.5
SD	110.48	197,968.12	780.22	2226.36	4468.96
	Correlation Scores
TREC	1	0.8274	0.9191	0.9467	0.7080
LF	0.8274	1	0.8104	0.8515	0.4637
TEC	0.9191	0.8104	1	0.9888	0.8581
CO2	0.9467	0.8515	0.9888	1	0.7941
GDP	0.7080	0.4637	0.8581	0.7941	1

Note: The scores of the correlation coefficient are all positive values from 2013–2018. 2015 data is used to represent the other year periods.

below summarizes the descriptive statistical values and the coefficient of correlation from the 2015 period with reference to the input and output factors.

4.2. GM (1,1) Grey Prediction Model Results

Acquiring a positive correlation using the data from 2013 to 2018, indicates that the input and output factors used complied with the homogeneity and isotonicity requirement of DEA. For this reason, the data is suitable for Grey prediction to obtain the future factors for 2019 to 2022 periods.

Table 5. Summary of the average mean absolute percentage error (MAPE) of HICs and NICs.

DMU No.	Country	TREC	LF	TEC	CO2	GDP	Average
CTRY1	France	0.21%	0.09%	0.39%	1.02%	4.81%	1.30%
CTRY2	Germany	0.34%	0.20%	0.85%	0.99%	4.60%	1.39%
CTRY3	Italy	0.37%	0.25%	0.59%	0.71%	4.65%	1.31%
CTRY4	United Kingdom	2.85%	0.06%	0.32%	0.72%	3.03%	1.40%
CTRY5	Japan	2.28%	0.19%	0.35%	0.44%	2.51%	1.15%
CTRY6	United States	0.77%	0.14%	1.00%	1.37%	0.46%	0.75%
CTRY7	Canada	1.24%	0.14%	0.80%	0.63%	4.57%	1.48%
CTRY8	Russia	0.13%	0.21%	1.42%	1.29%	12.82%	3.17%
CTRY9	South Africa	5.17%	0.43%	1.14%	1.38%	5.29%	2.68%
CTRY10	Mexico	1.51%	0.09%	0.62%	1.39%	4.51%	1.62%
CTRY11	Brazil	0.31%	0.14%	0.89%	1.26%	8.10%	2.14%
CTRY12	China	0.52%	0.07%	0.94%	0.83%	2.12%	0.90%
CTRY13	India	1.06%	0.08%	0.47%	1.07%	2.14%	0.96%
CTRY14	Indonesia	0.40%	0.28%	0.96%	1.49%	2.01%	1.03%
CTRY15	Malaysia	3.33%	0.08%	1.21%	0.53%	5.40%	2.11%
CTRY16	Thailand	1.96%	0.17%	0.34%	0.55%	2.84%	1.17%

Source: Calculated by the authors.

provides the summary of gained mean absolute percentage error (MAPE) from 2013 to 2022.

The results above show that most of the MAPE from HIC and NIC are below 10%, aside from Russia which gained 12.82% for the GDP factor. This can be a result of a tremendous decline in their GDP output from the year 2015 to 2018. However, this score is still considerably “good” as far as a grey prediction is concerned. Since most of the average MAPE scores are less than 10%, this study can proceed to the next phase using the predicted values for 2019 to 2022.

Table 6. Descriptive statistics and Pearson correlation coefficients using the projected values for the year 2021.

	Input Factors			Output Factors
	Total Renewable Energy Capacity in GW (TREC)	Labor Force In millions (LF)	Total Energy Consumption in Mtoe (TEC)	Carbon Dioxide Emission in MtCO2 (CO2)	Gross Domestic Product in $ Million (GDP)
	Descriptive Statistics
Max	1025.678	788,934.32	3265.25	9,616.34	22,840.18
Min	9.79	16,459.70	94.97	240.41	347.94
Average	142.74	130,441.84	604.62	1475.28	3965.78
SD	233.78	203,616.31	839.58	2342.19	5920.01
	Correlation Scores
TREC	1	0.8531	0.9110	0.9512	0.7194
LF	0.8531	1	0.8212	0.8597	0.5379
TEC	0.9110	0.8212	1	0.9888	0.8774
CO2	0.9512	0.8597	0.9888	1	0.8226
GDP	0.7194	0.5379	0.8774	0.8226	1

Note: The scores of the correlation coefficient are all positive values from 2019–2022. The data from 2021 is used to represent the other year periods.

below displays no negative coefficient using the projected data for the year 2021. This also indicates that the other projected data for the year 2019, 2020, and 2022 do not contain any negative coefficients. Thus, the authors can use these data for further analysis using DEA.

4.3. Results of the DEA Undesirable Model for the Period 2013–2018

4.3.1. Efficiency Scores of HICs and NICs

Since the requirement for DEA has been met from the previous analysis, the DEA undesirable output model will be used to calculate the efficiencies of NICs and HICs as well as their rankings according to their corresponding country category.

Table 7. Efficiency scores of countries and group rankings from period from 2013 to 2018.

Countries	Year Periods and Rankings
	2013	Rank	2014	Rank	2015	Rank	2016	Rank	2017	Rank	2018	Rank
Highly Industrialized Countries (HICs)
CTRY1	1.0000	1	1.0000	1	1.0000	1	1.0000	1	1.0000	1	1.0000	1
CTRY2	0.7030	6	0.6277	5	0.5518	6	0.6498	6	0.7326	5	1.0000	1
CTRY3	0.7783	4	0.6829	4	0.5860	4	0.6794	5	0.7358	4	0.7325	5
CTRY4	1.0000	1	1.0000	1	1.0000	1	1.0000	1	1.0000	1	1.0000	1
CTRY5	0.7341	5	0.5928	6	0.5647	5	0.7187	4	0.7153	6	0.6770	6
CTRY6	1.0000	1	1.0000	1	1.0000	1	1.0000	1	1.0000	1	1.0000	1
CTRY7	0.5117	7	0.4115	7	0.3648	7	0.3945	7	0.4341	7	0.4157	7
CTRY8	0.2215	8	0.1887	8	0.1417	8	0.1531	8	0.1852	8	0.1904	8
Newly Industrialized Countries (NICs)
CTRY9	1.0000	1	1.0000	1	0.3000	3	0.2600	4	0.2760	4	0.2748	4
CTRY10	0.3574	2	0.3439	2	0.3455	2	0.3518	2	0.4020	2	0.3873	2
CTRY11	0.3362	4	0.2718	5	0.2050	6	0.2338	5	0.2791	3	0.2266	6
CTRY12	0.1338	8	0.1315	8	0.1500	8	0.1668	8	0.1811	8	0.1840	8
CTRY13	0.1001	9	0.0993	9	0.1150	9	0.1351	9	0.1537	9	0.1411	9
CTRY14	0.2993	5	0.2877	3	1.0000	1	1.0000	1	1.0000	1	1.0000	1
CTRY15	0.2285	6	0.2185	6	0.2049	7	0.2309	6	0.2568	7	0.2810	3
CTRY16	0.2055	7	0.1876	7	0.2104	5	0.2291	7	0.2626	6	0.2716	5
CTRY17	0.3363	3	0.2793	4	0.2634	4	0.2806	3	0.2653	5	0.2152	7

Source: Calculated by the authors.

below shows how every country performed in terms of technical efficiency as well as which country is on top for every year in the period.

As seen in the table, three countries—France, the United Kingdom, and the United States—are the most efficient among the HICs which recorded a score of 1 in all year periods. Germany was able to follow through in 2018. While countries from NICs have lower efficiency scores compare to HICs, one country—Indonesia—has shown improvement by obtaining a score of 1 starting from the year 2015 to 2018. South Africa was recorded to be the most efficient NIC from 2013 to 2014.

Since the United Kingdom, the United States, and France have shown consistency in obtaining an efficiency score of 1, there is no need to include them in the line graph as shown in Figure 3 below. It can be observed that Russia has the lowest scores among the groups. Russia’s lowest point score of 0.14 efficiency was during 2015, wherein most HIC countries (except the US, UK, and France) also experienced the same decline.

It can also be noticed that all of them have declining efficiencies from 2013 to 2015. After this period, it can be observed that most of these countries have increased their efficiencies from 2015 to 2017. However, Japan’s score dropped again until 2018. Germany shows a huge increase in their technical efficiency, achieving a score of 1 at the end of 2018.

South Africa started in a high score during the 2013 to 2014 period as seen in Figure 4 below. Unfortunately, the country’s efficiency dropped from 2015 and this trend continued until 2018. In contrast, Indonesia improved at the same time South Africa’s score fell. Indonesia was able to maintain a score of 1.0 efficiency until 2018, making the country the most efficient among the NICs. India performed with the lowest efficiency. India scores only 0.09 in 2014 and reached its highest point of 0.15 in 2017. With almost the same performance with India, China placed second lowest in terms of efficiencies during the whole 2013 to 2018 period. Some other NICs performed very low, with efficiency scores below 0.5.

4.3.2. Average Efficiency Scores and Overall Ranking

Table 8. Average Efficiency Scores and Overall Rankings of HICs and NICs.

DMU No.	Countries	Average Efficiency Score	Overall Ranking
Highly Industrialized Countries
CTRY1	France	1.0000	1
CTRY4	United Kingdom	1.0000	1
CTRY6	United States	1.0000	1
CTRY2	Germany	0.7108	4
sCTRY3	Italy	0.6992	5
CTRY5	Japan	0.6671	6
CTRY7	Canada	0.4221	7
CTRY8	Russia	0.1801	8
Newly Industrialized Countries
CTRY14	Indonesia	0.7645	1
CTRY9	South Africa	0.5185	2
CTRY10	Mexico	0.3647	3
CTRY17	Turkey	0.2734	4
CTRY11	Brazil	0.2588	5
CTRY15	Malaysia	0.2368	6
CTRY16	Thailand	0.2278	7
CTRY12	China	0.1579	8
CTRY13	India	0.1241	9

Source: Calculated by the authors.

below arranges the HICs and NICs according to their average efficiency scores and ranking. The average efficiency scores were calculated from the values of efficiency scores per individual year 2013 to 2018 as shown in Table 7 from the previous section.

Since France, the UK and the US are consistently obtaining a 1.0 score for the whole year periods, their average scores are the highest among the others and therefore, the three countries ranked first while Germany, with a score of 0.711 is not too far from acquiring the highest score in the future. Russia remains the least efficient in the HIC group. With the Indonesia obtaining a score of 1.0 from 2015 to 2018, the country was able to take the lead in terms of efficiency among the NIC group with an average score of 0.765. Not too far from Indonesia’s score is South Africa in second with 0.519 efficiency. With the rest of the countries from the NIC group obtaining an efficiency score below 0.5, India appeared to be the most inefficient, ranking 9th with a very low score of 0.124.

4.4. Projected Efficiency Scores for the Period 2019–2022

The projected efficiency scores are calculated using the values obtained from the grey prediction method. These values are used as inputs and outputs for the computation of technical efficiencies using the undesirable output model of DEA.

Table 9. Projected efficiency scores of countries and group rankings from period from 2019 to 2022.

Countries	Year Periods and Rankings
	2019	Rank	2020	Rank	2021	Rank	2022	Rank
Highly Industrialized Countries (HICs)
CTRY1	1.0000	1	1.0000	1	1.0000	1	1.0000	1
CTRY2	1.0000	1	1.0000	1	1.0000	1	1.0000	1
CTRY3	0.7687	5	0.8030	5	0.8208	5	0.8381	6
CTRY4	1.0000	1	1.0000	1	1.0000	1	1.0000	1
CTRY5	0.7577	6	0.7977	6	0.8207	6	0.8493	5
CTRY6	1.0000	1	1.0000	1	1.0000	1	1.0000	1
CTRY7	0.3909	7	0.3686	7	0.3482	7	0.3295	7
CTRY8	0.1580	8	0.1498	8	0.1423	8	0.1356	8
Newly Industrialized Countries (NICs)
CTRY9	0.2342	5	0.2152	6	0.1983	7	0.1834	7
CTRY10	0.3958	2	0.3717	2	0.3495	2	0.3288	2
CTRY11	0.2332	6	0.2308	5	0.2286	5	0.2261	5
CTRY12	0.2101	8	0.2126	7	0.2149	6	0.2173	6
CTRY13	0.1627	9	0.1639	9	0.1651	9	0.1664	8
CTRY14	1.0000	1	1.0000	1	1.0000	1	1.0000	1
CTRY15	0.2576	4	0.2532	4	0.2489	4	0.2447	4
CTRY16	0.2720	3	0.2728	3	0.2737	3	0.2747	3
CTRY17	0.2248	7	0.1999	8	0.1748	8	0.1534	9

Source: Calculated by the authors.

shows the efficiency scores for the period of 2019 to 2022.

The result of the forecasted efficiencies shows that in the HIC group, France, Germany, United Kingdom, and the United States will continue topping the ranks, while Canada and Russia remain in the 7th and 8th rank, respectively. Japan and Italy can be seen switching their ranks in the last period of 2022 with the former going up from 6th to 5th.

Figure 5 does not include the countries that obtain an efficiency score of 1 through the whole period of 2019 to 2022 as they are understood to be highly efficient already. It can be observed that Italy and Japan display a quite positive trend, with a slight increase in efficiency, while Canada and Russia have a slightly negative trend.

Efficiency scores of the NICs are mostly around of below a 0.5 level, except for Indonesia which garnered a score of 1.0 from 2019 to 2022. None of the countries show any remarkable positive trends. Instead, some countries like China, India, and Thailand are expected to display stable performance or very little improvement in efficiency. Mexico, Thailand, and Turkey will have declining efficiencies during the projected period, as seen in Figure 6.

As presented in Figure 7, the calculated average efficiencies of HICs during the past and projected periods are comparatively higher than the NICs. However due to the existence of the very low efficiency scores of Russia and Canada, the effect to the average efficiency scores of the HIC group reached the 0.7099 level for the past period and is expected to increase by 7.76% to reach the 0.765 projected efficiency level. A different scenario is expected from the NIC group which will exhibit a decline of 1.23% from a 0.325 score down to a projected average level of 0.321.

5. Conclusions

With the implementation of the Paris Agreement signed by the HICs and NICs, the results show different effects on each country. Some countries have efficiently utilized their renewable energy resources, but some are not doing so, while others have maintained their levels and it seems they are not moved by the Agreement terms. The use of the undesirable model has successfully calculated each country’s position in terms of the utilization of their resources while considering greatly a very bad factor, which is the carbon emission.

Knowing the economic capabilities of the highly industrialized nations, the results show that Russia is the least efficient among them. A mean efficiency score of 0.1801 verifies the country’s lack of attention to its renewable energy program. In fact, in 2015, renewable energy only comprised 20% of the total installed power capacity, which mostly came from bioenergy and hydropower. The country must focus on other sources such as geothermal, wind, and solar photovoltaic (PV) [42]. Canada, with a mean efficiency score of 0.4221 manages to place second to the last among the HICs. A report from 2015 reveals that Vancouver alone sourced its 69% of energy from fossil fuels and 31% from renewable sources [43]. The situation in Japan is a little different, as due to the nuclear disaster from a major earthquake years ago, the country aims their energy to be completely 100% supplied by renewable energy, especially in their local regions, by 2020 [44]. Additionally, a report states that there is a decrease in the use of fossil fuels in 2018, 78% from 81% share. However, a dependency of the country on nuclear energy still exists and is evident from the increase of its generation from 2.8% to 4.7% of the same year [45]. This can be one of the reasons for Japan’s efficiency score drop from 2016 to 2018 and its acquisition of an average of 0.6671 throughout the studied period. The same goal is seen in Italy’s energy program. With the introduction of the 20-20-20 EU goals, the Italian government aims to decrease the emission of greenhouse gases by 20%, improve energy savings to up to 20%, and attain a 20% generation of renewable energy by 2020 [46]. The result of this study shows that Italy has quite improved its renewable energy utilization with increased and maintained efficiency scores from 2015 to 2018. While reaching a perfect efficiency of 1.0 in 2018, the average score becomes 0.7108 due to lower scores acquired from previous years. These scores imply that the German government has effectively improved its renewable energy utilization throughout the study period. Gaining an efficiency score of 1.0 throughout the study period, France, the United Kingdom, and United States, have been consistently utilized their renewable energies even before or after the Paris agreement. Their energy and sustainability programs can be a benchmark for other HICs that aim to improve their use of own renewable resources.

In contrast with the efficiency scores of the HICs, none from the list of NICs is able to get a consistent perfect efficiency. In spite of this, Indonesia manages to acquire perfect efficiency during the period of 2015 to 2018 rising from low-efficiency scores. Getting the highest mean efficiency score of 0.7645 among the NICs, this can be due to the fact that the country is considered to have the biggest resources of geothermal energy in the world [47]. South Africa demonstrates an interesting efficiency score as it can be seen that during the period of 2013 to 2014, the country was able to gain a perfect efficiency. This performance is attributable to their noticeable increase in renewable energy production in previous years [48]. However, this performance by South Africa was not maintained since the country’s efficiency dropped tremendously in succeeding years resulting in an average efficiency score of only 0.5185. Mexico, Turkey, Brazil, Malaysia, and Thailand are performing consistently at a low-level efficiency, with a minimum of 0.2278 and a maximum of 0.2734 mean scores. These results can be caused by the following factors: Mexico’s dependency on fossil fuels wherein 85% of their total power was produced in 2012 [49]; Turkey’s power sector industry as the biggest contributor to CO₂ emissions and a high reliance to coal-powered electricity generation accounting to 37.2% of their total electricity [50]; the old structure of Brazil’s energy sector which limits their capacities in handling renewable energy demands, and other political factors that hinder the country’s development to successfully execute their national energy programs [51]; and, Malaysia and Thailand’s high proportional use of crude oil, coal, and natural gas, giving less importance to renewable energies [52,53].

The HIC group is expected to keep increasing the efficiency level by 7.76% from the past period to the projected period. This can be due to the expected development to the future renewable energy developments by Japan and Italy. However, the NIC group is expected a bit decline in the average efficiency with negative 1.23% due to the efficiency drop performance by Turkey and Mexico along with the low efficiency performances by other countries.

Future implications from the result of combined data gathered using grey prediction and the undesirable DEA model shows that the HICs and NICs will continuously follow the trend of the efficiency scores for all these countries with Germany joining the group of perfectly efficient ones together with Indonesia, France, United Kingdom, and the United States. All other HICs and NICs will perform otherwise, especially if they will not concede significant importance to renewable energy. Most of the developments in the use of the renewable energy begin with the improvement in the political and legal structure surrounding its use and regulation, providing incentives or tax exemptions to private power producers to shift in using the renewable energy. Countries can also build strong partnerships with non-government and international organizations that are focusing on providing assistance to countries that want to achieve sustainable and renewable energy production. Strengthening the support to the research and development sector to determine the suitable sites to deploy renewable energy sources such as solar, wind and wave energy. Long-term strategic plans for renewable energy development that will also consider the economic, social, and environmental impacts to the country.

This study contributes as a method to mathematically evaluate the energy utilization efficiency of HICs and NICs based on available public data. The DEA undesirable model treatment of the CO₂ emission factor as a less important factor made this study different from the others. Hence, this model is the most appropriate method to evaluate the energy sector that uses data with undesired factors. The results of this research may guide each country for improvement of its production and consumption towards sustainable renewable energy development. This can also help policymakers, government agencies and the energy sectors to address the problem in the existence of bad outputs such as CO₂ emissions. For future studies involving quantitative measurements, the authors recommend combining DEA with a qualitative evaluation approach such as the one described in Wang et al. [54], which uses the fuzzy analytical hierarchy process for analysis to improve studies of this kind.