Next Article in Journal
Investigating the Relative Performance of Bricks-and-Mortar, Clicks-and-Mortar, and Pure-Click Firms in Taiwan
Previous Article in Journal
Production of High Added-Value Chemicals in Basfia succiniciproducens: Role of Medium Composition
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sustainability
Volume 13
Issue 6
10.3390/su13063515
sustainability-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Multi-Criteria Decision Analysis for Renewable Energies: Research Trends, Gaps and the Challenge of Improving Participation

by
Rodrigo A. Estévez
1,2,3,*,
Valeria Espinoza
3,
Roberto D. Ponce Oliva
2,4,5,
Felipe Vásquez-Lavín
2,4,5,6 and
Stefan Gelcich
2,3,5
1
Centro de Investigación e Innovación para el Cambio Climático, Facultad de Ciencias, Universidad Santo Tomás, Santiago 8370003, Chile
2
Instituto Milenio en Socio-Ecología Costera (SECOS), Santiago 8320000, Chile
3
MERIC, Marine Energy Research & Innovation Center, Santiago 7550268, Chile
4
School of Business and Economics, Universidad del Desarrollo, Concepción 7550000, Chile
5
Center of Applied Ecology and Sustainability (CAPES), Departamento de Ecología, Pontificia Universidad Católica de Chile, Santiago 8320000, Chile
6
Center for the Socioeconomic Impact of Environmental Policies (CESIEP), Pontificia Universidad Católica de Chile, Santiago 8320000, Chile
*
Author to whom correspondence should be addressed.
Sustainability 2021, 13(6), 3515; https://0-doi-org.brum.beds.ac.uk/10.3390/su13063515
Submission received: 28 February 2021 / Revised: 9 March 2021 / Accepted: 11 March 2021 / Published: 22 March 2021
(This article belongs to the Section Energy Sustainability)
Browse Figures
Versions Notes

Abstract

:
The global increase in renewable energy initiatives has been followed by the need to include the social impact of any project as a core element. Significant challenges for renewable energy development include uncertainty in assessing social impacts at local scales, participation and social acceptance. Multi-criteria decision analysis (MCDA) approaches have been widely used in energy planning to address these challenges. This article reviews how social criteria and participation mechanisms have been incorporated into decision-making processes for renewable energy projects. A total of 184 articles were analyzed. A total of 490 indicators that estimated social impacts were identified and organized into nine criteria: employment, social acceptance, social development, health impact, governance, visual impact, knowledge and awareness, cultural value and social justice. Most research included analytical hierarchy process methodologies, and the articles were geographically concentrated in Asia and Europe. Most articles included a participative component (92.3%), and the majority of them were based on expert consultation (75.4%). Of the articles that exclusively considered experts, almost 40% did not provide any description of the expert elicitation process. Results revealed advances in the use of MCDA but highlighted important challenges—related to improving expert consultation methodologies and broadening the participation of stakeholders—when developing renewable energy initiatives and policies.

1. Introduction

The constant increase in average global temperature in recent decades is an unequivocal signal of the impact of greenhouse gases [1]. Aiming to keep the increase in the average global temperature below 2 °C, the international community has adopted the Paris Agreement, establishing mandatory targets for signatory countries [2,3]. To achieve the goals set by the agreement, a global energy transition toward efficient renewable energies and low-carbon energy services is required [2]. This is in line with the 2030 Sustainable Development Goals of the United Nations, which seek to “increase substantially the share of renewable energy in the global energy mix by 2030” (goal seven). Historically, energy transitions have been related to social transformations, e.g., industrialization, urbanization and the emergence of a consumer society [4]. Accordingly, the international community faces substantial knowledge and action gaps which go beyond technological and economic barriers. Closing these gaps will depend on structural social patterns of energy production and utilization, including mechanisms for citizen participation and social acceptability [3]. Unfortunately, in renewable energy development, social dimensions have not received the attention they deserve [5], despite their importance in determining public engagement and local community acceptability [6].
The substantial increase in renewable energies at a global scale calls for the systematic incorporation of social impact as a core aspect considered during the development of any energy project [7,8]. However, measuring social attributes is a difficult task [9,10]. Social impacts are not objectively negative or positive; rather, they depend on subjective perceptions and the scale of analysis [11,12]. In pursuit of an operational consensus, the International Committee on Guidelines and Principles for Social Impact Assessment (ICGP) defined social impacts as “the consequences to human populations of any public or private actions that alter the ways in which people live, work, play, relate to one another, organize to meet their needs and generally cope as members of society” [13].
Uncertainty regarding social impacts at local scales and the consequent social acceptance are significant barriers for renewable energy projects [14,15]. Exploring the feasibility of energy policies requires in-depth knowledge of social expectations for participation in new energy models, characterized by low-carbon sources and more sustainable individual practices [3,4,16]. In renewable energy, social acceptability is associated not only with the perception of direct benefit, but also with the values involved in the decision and the potential consequences for the environment [17,18]. The participation of key actors is a critical step in the definition, estimation and prioritization of social impacts and their management [5,8]. Here, participation is understood as a mechanism through which interested citizens are active actors in one or more stages of the decision-making process [19].
The development of energy projects requires methodological approaches to incorporate social, environmental and economic criteria into decision-making models, including stakeholder participation [11,20,21]. Cost–benefit analyses and life cycle cost analyses are some of the most common methods used to support decision-making processes in the energy sector [22,23]. Although these are valid and widely used techniques, their capacity to capture complex social impacts and express them in monetary terms is not straightforward—and can often be unclear [11,24]. Multi-criteria decision analysis (MCDA) is a family of decision-making protocols that evaluate and prioritize multi-objective problems [25,26]. MCDA approaches have been widely used in sustainable energy planning and decision-making [27], presenting advantages over traditional cost–benefit analyses [6,27,28,29]. MCDA methods can incorporate quantitative and qualitative data in decision models, assessing impacts when monetary analyses are inappropriate [11,30,31]. Furthermore, participatory MCDA approaches provide a systematic and structured process to incorporate stakeholder participation in the decision cycle [5,32]. As a consequence, MCDA can clarify subjective value judgments—facilitating social learning, consensus and acceptability issues among stakeholders [33,34,35].
Though a wide variety of multi-criteria methodologies exist, all share major stages in their theoretical formulation [26] (Figure 1). This process is not unidirectional, and in practical applications, researchers may consider one or more of the following steps: (1) Formulation of a decision problem: in this stage, the range of values that comprise the decision-making problem are explored. (2) Establishment of objectives and indicators: in this stage, values are converted into objectives and indicators, allowing operationalization of the decision-making problem [36]. (3) Development of alternatives: in this stage, strategies for the achievement of objectives are explored and defined [37]. (4) Calculation of weights: a major step in multi-criteria analysis is the establishment of the relative importance of objectives. The relative importance is generally represented as weight factors [26]. (5) Estimation of the consequences: this stage establishes the potential impact of alternatives on the objectives, utilizing quantitative models or qualitative evaluations, generally based on experts [30]. The most commonly used MCDA methods are multi-attribute value theory (MAVT), analytical hierarchy process (AHP), outranking and goal programming [26].
MAVT, as well as its variant multi-attribute utility theory (MAUT), assigns a numerical value to each alternative under evaluation [34]. As a result, MAVT constructs a preference order of the alternatives based on the decision maker’s value judgment [26]. A central component in MAVT is the construction of partial value functions, which are subsequently aggregated, generally in the form of an additive value function [38]. AHP has many connections with the MAVT approach, and it is also based on an additive preference function [26]. However, unlike MAVT, AHP uses pairwise comparisons in evaluating alternatives with respect to a set of criteria, based on converting verbal statements into preference scores (e.g., how important objective x is with respect to objective y) [39]. The analytic network process (ANP) is a variant of AHP that allows greater sophistication in the interdependence of criteria, offering a solution to complex multi-criteria problems [40]. AHP and MAVT are widely used methods, in part because of the transparency and simplicity of their aggregation methods. However, the same simplicity may generate doubt regarding their validity [41]. AHP and MAVT construct preference orders among the alternatives that require a set of axioms, which are not applied in a literal and rigid form in practice [26]. Outranking methods do not consider the aggregation of value functions. Instead, outranking approaches use pairwise comparisons of alternatives, focusing on protocols that faithfully capture how decision makers think [42]. Therefore, the result is not a performance score for each alternative, but rather an outranking relationship between a set of alternatives [43]. Although the mechanisms for establishing preferences between alternatives are intuitive, the algorithms for integrating various inputs into the decision-making model are complicated to understand in a participative process [26]. ELECTRE and PROMETHEE are the most common variants in outranking [26]. Goal programming provides mechanisms to solve complex multi-criteria quantitative problems. This category includes different optimization approaches, and the achievement of satisfactory levels for each criterion, which are generally defined in terms of maximizing and minimizing aspiration goals [26].
Despite the potential and growing international interest in MCDA to inform renewable energy projects and policy [27], no review has yet clarified the scope, possible challenges and research gaps for MCDA to effectively embrace social dimensions and participation in the decision-making cycle. Accordingly, a systematic review is important to identify the following aspects: (1) the types of decision reported in MCDA applications; (2) the variety of social criteria considered by authors, and the methods to estimate their impacts and relevance; (3) the level of stakeholder participation, and the role of experts in the decision-making; and (4) the type of indicators used to evaluate the social acceptability of renewable energy projects. In this article, we undertook a comprehensive approach to analyze how social criteria and participative mechanisms have been incorporated into decision-making processes for renewable energy development. We analyze to what extent the literature reports the use of social criteria and mechanisms for public participation in the decision-making process.

2. Materials and Methods

We performed a systematic search of peer-reviewed articles in the Web of Science (ISI) database (SCI-EXPANDED, SSCI, A&HCI, CPCI-S, CPCI-SSH, BKCI-S, BKCI-SSH, ESCI, CCR-EXPANDED) from 01/01/1989 to 31/12/2019, restricting the search to English articles, and excluding book chapters and proceedings papers. We selected the following terms as keywords: renewable energ*, bioenerg*, geothermal, photovoltaic, solar energ*, wind energ*, wave energ*, marine energ*. Each term was combined (AND) with the following keywords: multi-criteri*, multiple criteri*, outranking, analytic hierarchy process, structured decision-making, MCDA, MAVT, MAUT, ELECTRE, AHP, analytic network process, PROMETHEE. Results were combined (OR) to avoid duplication of articles. These queries resulted in 853 articles, which were screened by the following criteria: (1) The article reports a decision-making problem related to the following renewable energies: wind, solar, geothermal, bioenergy or marine. (2) The article reports an unequivocal application of MCDA to support the decision-making process. (3) The article includes social criteria in the decision-making process. We identified a subset of 184 articles that meet these criteria (see Supplementary Materials).
We reviewed each of the 184 articles, classifying them into the following categories: (1) year of publication; (2) source of energy; (3) MCDA method (AHP, ANP, MAVT, outranking, ELECTRE, PROMETHEE, goal programming); (4) region/country; (5) decision objective; (6) social criteria; (7) participation process; (8) level of decision (local, regional, national, international); (9) reason for engagement (expert elicitation, stakeholder participation); (10) method of engagement (questionnaire, interviews, workshops); and (11) stage of participation in the decision cycle (establishing objectives, identifying alternatives, estimating consequences, weighting objectives).
We also examined the indicators used in the articles to estimate social acceptability. In MCDA, social indicators are a mechanism for estimating fundamental values represented in objectives [44]. We classified indicators in the following categories: (1) comparative indicators, which use descriptions to compare the relative importance of objectives; (2) natural indicators, which directly measure the objective in question [36]; (3) proxy indicators, which indirectly estimate the objective under consideration [36]; and (4) constructed indicators, which use ordinal scales to describe degrees of consequences, linking narrative descriptions to an impact [30]. In constructed indicators, the risk of ambiguity of the estimation increases, due to possible different interpretations of the meaning of each level in the ordinal scale [45]. Therefore, it is highly recommended to clearly define each level of the ordinal scale [11].
In the identification of social criteria, we did not include articles that only report distance to the facility or type of land use for a geographic information systems (GIS) analysis. We also excluded economic criteria (e.g., electricity demand, economic growth) and indicators that only refer to facilities operation, such as existing logistics, maintaining a leading position as an energy supplier, maintenance convenience and storage load bearing capacity.

3. Results

The number of publications that consider social impact in MCDA for renewable energy decisions has increased markedly in the last decade, with 93% of all papers published since 2010. In particular, a significant increase was observed in bioenergy, solar and wind energies. Geothermal and marine energies have a lower incidence in the literature (Figure 2).
Table 1 presents a description of the 184 articles included in this review, categorized by continent and MCDA method. The articles consider several sources of renewable energies and technologies. However, wind, solar and bioenergies represented 80.6% of the studies. Pairwise comparison was the most used MCDA approach (61.1%), particularly AHP, which allows direct comparison of qualitative criteria based on expert or stakeholder judgments. Outranking, compensatory and fuzzy environment were also commonly applied. Studies were concentrated in Asia and Europe (79.9% altogether), and we found that developing countries lead in the inclusion of social impacts in decision-making for renewable energies, such as Turkey (n = 18), Greece (n = 17), China (n = 14) and Iran (n = 13).

3.1. Types of Decision Objectives

In relation to the decision objective, most of the articles seek to select the most appropriate energy alternative (n = 76) (Table 2). Energy alternatives were evaluated considering outcomes such as sustainability, cost–benefit, efficiency of producing energy and compatibility with national polices, among other criteria (e.g., [46,47]). Selecting the most appropriate location for energy plants was the second common decision objective (n = 40). Selecting the most appropriate technology was also a relevant decision category (n = 30). In this category, authors focused on technology transfer capacity, suitable micro-grid system and assessment of energy flow, among others (e.g., [48]). MCDA approaches were also commonly used to evaluate scenarios/alternatives for energy policy or project development (n = 23). Of the 184 articles reviewed, 57.5% consider decisions at a national level, 29.6% at regional/local level and 13.4% at international level.

3.2. Social Indicators

We identified 490 indicators that estimate social impacts, which were organized into nine criteria (Table 3). The majority of the papers (59%) included one or two social indicators in the MCDA process (mean = 2.6). Table 3 presents the social indicators grouped according to the criteria. The indicators identified cover a wide range of topics, providing a map to explore potential social consequences. The criteria employment (n = 97) and social acceptance (n = 95) represented 39.2% of the indicators. Social development was another important criterion considered in the MCDA process. Here, we included a variety of indicators, such as local development (n = 21), energy access (n = 10) and quality of life (n = 9), among others. Health impacts were also commonly considered in MCDA, particularly noise (n = 22), safety and risk (n = 20) and pollution (n = 15). Indicators that represent governance were also relevant, such as policy compatibility (n = 29), participation (n = 9) and law and regulation (n = 9), among others.

3.3. Stakeholder Participation in the MCDA Process

In the MCDA process, estimating social impacts can include stakeholder involvement, but this is not compulsory. Of the articles reviewed, 9.7% did not report any stakeholder or expert participation. In these cases, the authors directly selected, evaluated and weighted the social indicators, using their own knowledge or secondary data. Of the 134 studies that did include a mechanism for participation, 75.4% included experts in the MCDA process, and 24.6% included stakeholders in one or more stages of the MCDA cycle. Experts mostly focused on evaluating consequences, including social and cultural impacts, and estimating weights for objectives. Of the articles that exclusively considered experts in the MCDA process, 39.6% did not provide any description of the expert elicitation process (n = 40) (Figure 3). Of the articles that did include stakeholders, 87.9% provided a methodological description of the participative process (n = 29) (Figure 3). Articles that incorporated experts and stakeholders generally used a mix of qualitative and quantitative tools, such as surveys (48.6%), interviews (32.7%) and workshops (18.7%).
Stakeholder participation varied across the reviewed articles (Figure 4). Participation was concentrated in estimating weights for objectives. Other stages have a relatively low degree of stakeholder involvement. Identifying objectives and indicators was considered in 40% of the articles that included stakeholder participation (n = 13). Selecting alternatives (n = 8) and estimating consequences (n = 10) were less frequent.

3.4. Social Acceptability Indicators

Social acceptability was one of the most frequent social indicators considered in the articles (Table 3) (n = 82). Of the indicators that evaluated social acceptability, 82% focused on public acceptability, and 18% on political or government support. Public acceptability articles only included expert assessments. The remaining 18% that did consider stakeholders included decision makers, ex-presidents, NGOs, potential investors, local communities, environmental groups, producers and social actors (without describing what groups of social actors). In 36% of cases, the authors did not include any description regarding the construction of indicators or methodologies used, nor a definition of social acceptability. Figure 5 presents the types of indicators used to evaluate social acceptability. The results shows that 64% of the articles used comparative indicators, associated with pairwise methods. Of this group, 41% did not present a description of the variable used as a basis for the comparison. Moreover, 25.2% of the articles used ordinal scales to evaluate the level of social acceptability of each of the alternatives. Of these, 38% did not provide details regarding the construction of the scale or the definition of the levels. Eleven articles established a short linguistic variable for each level in the ordinal scale, generally descriptions such as low, medium and high. Only four articles provided scales with detail descriptions for each level [49,50,51,52].

4. Discussion

Academics and practitioners using MCDA have increasingly considered social criteria for evaluating renewable energy policies and projects. In a seminal study, Wang et al. highlighted the relevance of MCDA for sustainable energies [27]. In 2011, Ribeiro et al. identified 101 articles that integrated social criteria in energy planning [53]. In this study, we found more than 180 articles that focus particularly on renewable energy development. The social impacts associated with renewable energies vary according to countries, regions, geographical scales and stakeholders, with stable patterns of public acceptability, social consequences and institutional barriers being difficult to identify. These results are consistent with a variety of studies [4,54].
AHP was the MCDA methodology most frequently used to include social impacts of renewable energies, used in almost 50% of cases. This value exceeds 60% if ANP, an AHP variant, is included. AHP contains features that make it especially useful for participatory applications [26]. It is based on pairwise comparisons, which facilitated the participatory process in the case studies [39]. In addition, AHP uses an additive preference function, which is a simple algorithm understandable for decision makers. In the case studies, one of the difficulties in applying AHP to group decision-making is the need to review the consistency of the set of comparisons made by the participants, which presents methodological challenges to be addressed during the workshops [55]. Outranking methodologies represented 12.4% of the case studies, including PROMETHEE, ELECTRE and other variants. The relatively low incidence of outranking methodologies can be explained by the complexity of their algorithms to define outranking relationships in participatory processes [26]. Similar to outranking, MAVT methodologies were used in about 12% of the case studies. MAVT, like AHP, uses an additive preference function that facilitates participants’ interpretation of the results [30]. However, its relatively low utilization compared with AHP can be explained by the need to build value functions for each of the objectives under evaluation, which is a demanding exercise to perform in a participatory approach [26]. The different optimization approaches were grouped in the goal programming category, representing almost 17% of the case studies. In the review, these approaches were especially useful for incorporating multiple alternatives and social criteria, based mostly on proxy indicators [56]. This can lead to challenges in including constructed indicators that are generally appropriate for measuring social impacts [11].
In the review, we identified over 450 indicators to measure the social impacts of renewable energy projects. We found two important methodological gaps associated with the use of social indicators. First, in more than 35% of cases, the authors did not detail the protocol used to assess social acceptability, or the definitions used to make comparisons between alternatives. Second, in cases where ordinal scales were used, most papers did not define the scale levels accurately, as best practices recommend [11,36]. The proper construction of social indicators requires a systematic process. Rigorousness in the construction of social indicators allows an increase in the validity of the results obtained in a decision-making process [11,53,57]. This is a key methodological issue which needs to be urgently addressed.
Our results highlight the key role that experts have played in MCDA processes. We found that more than 75% of the cases that established a participation mechanism exclusively considered experts in the decision-making process. In these cases, the experts primarily estimated the consequences of the alternatives, and evaluated the relative importance of the objectives. While experts provided relevant information for decision-making, our results show a gap in terms of effectively incorporating stakeholders into the decision-making process. Additionally, in 40% of cases, the authors did not include any reference to the protocols used for the expert estimates. This element constitutes a major gap in the literature. Expert elicitation, if not based on proven protocols, can generate serious problems in the validity and robustness of results [9,58], which can then affect the legitimacy of MCDA processes.
The use of experts within MCDA requires the use of specially designed methodological guidelines. Comprehensive protocols have been proposed by authors to improve the estimates made by experts [59,60]. Expert elicitation applications have increased their relevancy in different fields of policy-making [61,62,63]. The protocols are particularly concerned with minimizing the multiple biases in experts’ assessments. These biases have been widely described in psychology, considering the recurrent errors produced by overconfidence, generalization of results and the anchor effect, among many other biases [64,65]. Protocols have focused on establishing iterative processes, combining group discussion spaces, individual estimates and delimitation of the estimate ranges [60,66]. Recently, protocols have also been evaluated to aggregate the estimates of numerous experts [67].
Some of the limitations of this study should be addressed in future reviews. First, we focused on conventional renewable energies, such as solar, geothermal, bioenergy, wind and marine energy. It is important in future studies to analyze the use of social criteria and mechanisms for public participation in the development of other energy sources, such as hydrogen, nuclear and other non-renewable sources [68,69]. Second, in this review we circumscribed the analysis to the use of MCDA methodologies. However, other decision-making methodologies, such as cost–benefit analysis, life cycle cost and contingent valuation, include mechanisms to represent non-market value in monetary terms and deserve an in-depth review of advances and gaps [22,23,70].
This review identified a critical gap in terms of the effective engagement of stakeholders in MCDA methodologies. This creates a challenge for the application of MCDA methods in the development of renewable energies, in particular because participatory mechanisms contribute to building consensus among actors with diverse interests. Despite the greater social acceptance of green energy compared with energy from polluting sources, renewable energy projects are not exempt from situations of social conflict [71,72]. Socio-environmental problems related to renewable energy development are becoming more common [73,74]. Lack of participation and referencing could affect the legitimacy of renewable energy decision-making processes, jeopardizing their successful implementation. Recent studies confirm the importance of communication and citizen participation in addressing conflicts and protests due to power grid expansion [74,75]. Therefore, improving participatory processes in MCDA is key for the sustainable expansion of renewable energies [76].

5. Conclusions

The results of this review reveal significant advances, challenges and gaps in terms of the effective inclusion of social impacts and broader participation in renewable energy development, providing critical insights for public policy in the energy sector. Articles were concentrated in Asia and Europe, with a clear absence of articles in Latin America and Africa. In the future, it is expected that developing countries with high potential for energy generation from renewable sources will face challenges in including mechanisms for participation and evaluation of potential social impacts. Accordingly, there is a responsibility for public policy to learn from the experiences presented in this review, strengthening mechanisms for public participation, and improving methodologies that include experts as valid sources of information on social impacts. The creation of regional learning platforms where capacity building and experiences can be shared could provide an important way to avoid undesirable outcomes of renewable energy transitions. As renewable energy sources increase in the energy matrix of developing countries, attention must be placed on enabling a participatory process and correctly assessing social impacts. To this end, MCDA has potential, but must still advance toward embracing best practices that allow for the inclusion of multiple stakeholders and their values in a structured and transparent manner.

Supplementary Materials

The following are available online at https://0-www-mdpi-com.brum.beds.ac.uk/2071-1050/13/6/3515/s1, Table S1: List of articles included in the review.

Author Contributions

Conceptualization, R.A.E.; methodology, R.A.E.; formal analysis, R.A.E., R.D.P.O., F.V.-L., V.E. and S.G.; data curation, R.A.E., R.D.P.O., F.V.-L., V.E. and S.G.; writing—original draft preparation, R.A.E., S.G.; writing—review and editing, R.D.P.O., V.E., F.V.-L.; visualization, R.A.E.; project administration, R.A.E., S.G.; funding acquisition, R.A.E., S.G. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Acknowledgments

The authors thank Marine Energy Research & Innovation Center (MERIC), ANID/FONDECYT 11170333, ANID/PIA BASAL FB0002 and ANID—Millennium Science Initiative Program—ICN 2019_015. The ideas in this paper are the responsibility of the authors and do not necessarily represent those of the host institutions.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. National Centers for Environmental Information. State of the Cimate. 2020. Available online: https://www.ncdc.noaa.gov/sotc/ (accessed on 27 February 2021).
  2. Gielen, D.; Boshell, F.; Saygin, D. Climate and energy challenges for materials science. Nat. Mater. 2016, 15, 117–120. [Google Scholar] [CrossRef]
  3. Van Vuuren, D.P.; Stehfest, E.; Gernaat, D.; van den Berg, M.; Bijl, D.L.; de Boer, H.S.; Daioglou, V.; Doelman, J.C.; Edelenbosch, O.Y.; Harmsen, M.; et al. Alternative pathways to the 1.5 degrees C target reduce the need for negative emission technologies. Nat. Clim. Chang. 2018, 8, 391–397. [Google Scholar] [CrossRef]
  4. Bridge, G.; Bouzarovski, S.; Bradshaw, M.; Eyre, N. Geographies of energy transition: Space, place and the low-carbon economy. Energy Policy 2013, 53, 331–340. [Google Scholar] [CrossRef]
  5. Grafakos, S.; Enseñado, E.M.; Flamos, A. Developing an integrated sustainability and resilience framework of indicators for the assessment of low-carbon energy technologies at the local level. Int. J. Sustain. Energy 2017, 36, 945–971. [Google Scholar] [CrossRef]
  6. Wang, C.-N.; Thanh, N.V.; Su, C.-C. The Study of a Multicriteria Decision Making Model for Wave Power Plant Location Selection in Vietnam. Processes 2019, 7, 650. [Google Scholar] [CrossRef] [Green Version]
  7. Wüstenhagen, R.; Wolsink, M.; Burer, M.J. Social acceptance of renewable energy innovation: An introduction to the concept. Energy Policy 2007, 35, 2683–2691. [Google Scholar] [CrossRef] [Green Version]
  8. Gallego Carrera, D.; Mack, A. Sustainability assessment of energy technologies via social indicators: Results of a survey among European energy experts. Energy Policy 2010, 38, 1030–1039. [Google Scholar] [CrossRef]
  9. Assefa, G.; Frostell, B. Social sustainability and social acceptance in technology assessment: A case study of energy technologies. Technol. Soc. 2007, 29, 63–78. [Google Scholar] [CrossRef]
  10. Byrka, K.; Jȩdrzejewski, A.; Sznajd-Weron, K.; Weron, R. Difficulty is critical: The importance of social factors in modeling diffusion of green products and practices. Renew. Sustain. Energy Rev. 2016, 62, 723–735. [Google Scholar] [CrossRef] [Green Version]
  11. Estévez, R.A.; Walshe, T.; Burgman, M.A. Capturing social impacts for decision-making: A Multicriteria Decision Analysis perspective. Divers. Distrib. 2013, 19, 608–616. [Google Scholar] [CrossRef]
  12. Vanclay, F. Conceptualising social impacts. Environ. Impact. Assess. Rev. 2002, 22, 183–211. [Google Scholar] [CrossRef]
  13. International Committee on Guidelines and Principles for Social Impact Assessment (ICGP). Principles and guidelines for social impact assessment in the USA. Impact. Assess. Proj. Apprais. 2003, 21, 231–250. [Google Scholar] [CrossRef]
  14. Diógenes, J.R.F.; Claro, J.; Rodrigues, J.C.; Loureiro, M.V. Barriers to onshore wind energy implementation: A systematic review. Energy Res. Soc. Sci. 2020, 60, 101337. [Google Scholar] [CrossRef]
  15. Longstaff, H.; Secko, D.M.; Capurro, G.; Hanney, P.; McIntyre, T. Fostering citizen deliberations on the social acceptability of renewable fuels policy: The case of advanced lignocellulosic biofuels in Canada. Biomass. Bioenergy 2015, 74, 103–112. [Google Scholar] [CrossRef]
  16. Sovacool, B.K. An international comparison of four polycentric approaches to climate and energy governance. Energy Policy 2011, 39, 3832–3844. [Google Scholar] [CrossRef]
  17. Abotah, R.; Daim, T.U. Towards building a multi perspective policy development framework for transition into renewable energy. Sustain. Energy Technol. 2017, 21, 67–88. [Google Scholar] [CrossRef]
  18. Okello, C.; Pindozzi, S.; Faugno, S.; Boccia, L. Appraising Bioenergy Alternatives in Uganda Using Strengths, Weaknesses, Opportunities and Threats (SWOT)-Analytical Hierarchy Process (AHP) and a Desirability Functions Approach. Energies 2014, 7, 1–22. [Google Scholar] [CrossRef]
  19. Ansell, C.; Gash, A. Collaborative governance in theory and practice. J. Publ. Adm. Res. Theory 2008, 18, 543–571. [Google Scholar] [CrossRef] [Green Version]
  20. Büyükozkan, G.; Karabulut, Y. Energy project performance evaluation with sustainability perspective. Energy 2017, 119, 549–560. [Google Scholar] [CrossRef]
  21. Pohekar, S.D.; Ramachandran, M. Application of multi-criteria decision making to sustainable energy planning—A review. Renew. Sustain. Energy Rev. 2004, 8, 365–381. [Google Scholar] [CrossRef]
  22. Strantzali, E.; Aravossis, K. Decision making in renewable energy investments: A review. Renew. Sustain. Energy Rev. 2016, 55, 885–898. [Google Scholar] [CrossRef]
  23. Bottero, M.; Dell’Anna, F.; Morgese, V. Evaluating the Transition towards Post-Carbon Cities: A Literature Review. Sustainability 2021, 13, 567. [Google Scholar] [CrossRef]
  24. Gregory, R.; Slovic, P. A constructive approach to environmental valuation. Ecol. Econ. 1997, 21, 175–181. [Google Scholar] [CrossRef]
  25. Estévez, R.; Gelcich, S. Participative multi-criteria decision analysis in marine management and conservation: Research progress and the challenge of integrating value judgments and uncertainty. Mar. Policy 2015, 61, 1–7. [Google Scholar] [CrossRef]
  26. Belton, V.; Stewart, T.J. Multiple Criteria Decision Analysis: An Integrated Approach, 2nd ed.; Kluwer Academic Publishers: Boston, MA, USA, 2003. [Google Scholar]
  27. Wang, J.-J.; Jing, Y.-Y.; Zhang, C.-F.; Zhao, J.-H. Review on multi-criteria decision analysis aid in sustainable energy decision-making. Renew. Sustain. Energy Rev. 2009, 13, 2263–2278. [Google Scholar] [CrossRef]
  28. Stagl, S. Multicriteria evaluation and public participation: The case of UK energy policy. Land Use Policy 2006, 23, 53–62. [Google Scholar] [CrossRef]
  29. Elghali, L.; Clift, R.; Sinclair, P.; Panoutsou, C.; Bauen, A. Developing a sustainability framework for the assessment of bioenergy systems. Energy Policy 2007, 35, 6075–6083. [Google Scholar] [CrossRef]
  30. Gregory, R.; Failing, L.; Harstone, M.; Long, G.; McDaniels, T.; Ohlson, D. Structured Decision Making: A Practical Guide to Environmental Management Choices; Willey-Blackwell: Oxford, OH, USA, 2012. [Google Scholar]
  31. Munda, G. Social multi-criteria evaluation for urban sustainability policies. Land Use Policy 2006, 23, 86–94. [Google Scholar] [CrossRef]
  32. Wilkens, I. Transdisciplinary Evaluation of Energy Scenarios for a German Village Using Multi-Criteria Decision Analysis. Sustainability 2012, 4, 604–629. [Google Scholar] [CrossRef] [Green Version]
  33. Elkarmi, F.; Mustafa, I. Increasing the utilization of solar energy technologies (SET) in Jordan: Analytic hierarchy process. Energy Policy 1993, 21, 978–984. [Google Scholar] [CrossRef]
  34. Estevez, R.; Alamos, F.; Walshe, T.; Gelcich, S. Accounting for Uncertainty in Value Judgements when Applying Multi-Attribute Value Theory. Environ. Model. Assess. 2017, 23, 87–97. [Google Scholar] [CrossRef]
  35. Florio, P.; Munari Probst, M.C.; Schüler, A.; Roecker, C.; Scartezzini, J.-L. Assessing visibility in multi-scale urban planning: A contribution to a method enhancing social acceptability of solar energy in cities. Sol. Energy 2018, 173, 97–109. [Google Scholar] [CrossRef]
  36. Keeney, R.L.; Gregory, R.S. Selecting attributes to measure the achievement of objectives. Oper. Res. 2005, 53, 1–11. [Google Scholar] [CrossRef]
  37. Gregory, R.; Keeney, R.L. Creating policy alternatives using stakeholder values. Manag. Sci. 1994, 40, 1035–1048. [Google Scholar] [CrossRef]
  38. Von Winterfeldt, D.; Edwards, W. Decision Analysis and Behavioral Research; Cambridge University Press: New York, NY, USA, 1986. [Google Scholar]
  39. Chopin, P.; Guinde, L.; Causeret, F.; Bergkvist, G.; Blazy, J.M. Integrating stakeholder preferences into assessment of scenarios for electricity production from locally produced biomass on a small island. Renew. Energy 2019, 131, 128–136. [Google Scholar] [CrossRef]
  40. Hsueh, J.T.; Lin, C.Y. Constructing a network model to rank the optimal strategy for implementing the sorting process in reverse logistics: Case study of photovoltaic industry. Clean. Technol. Environ. Policy 2015, 17, 155–174. [Google Scholar] [CrossRef]
  41. Harper, M.; Anderson, B.; James, P.; Bahaj, A. Assessing socially acceptable locations for onshore wind energy using a GIS-MCDA approach. Int. J. Low-Carbon Technol. 2019, 14, 160–169. [Google Scholar] [CrossRef]
  42. Gamboa, G.; Munda, G. The problem of windfarm location: A social multi-criteria evaluation framework. Energy Policy 2007, 35, 1564–1583. [Google Scholar] [CrossRef]
  43. Stewart, T.J.; Losa, F. Towards reconciling outranking and value measurement practice. Eur. J. Oper. Res. 2003, 145, 645–659. [Google Scholar] [CrossRef]
  44. Gregory, R. Using stakeholder values to make smarter environmental decisions. Environment 2000, 42, 34–44. [Google Scholar] [CrossRef]
  45. Pascoe, S.; Bustamante, R.; Wilcox, C.; Gibbs, M. Spatial fisheries management: A framework for multi-objective qualitative assessment. Ocean. Coast Manag. 2009, 52, 130–138. [Google Scholar] [CrossRef]
  46. Kambezidis, H.D.; Kasselouri, B.; Konidari, P. Evaluating policy options for increasing the RES-E penetration in Greece. Energy Policy 2011, 39, 5388–5398. [Google Scholar] [CrossRef]
  47. Petrillo, A.; De Felice, F.; Jannelli, E.; Autorino, C.; Minutillo, M.; Lavadera, A.L. Life cycle assessment (LCA) and life cycle cost (LCC) analysis model for a stand-alone hybrid renewable energy system. Renew. Energy 2016, 95, 337–355. [Google Scholar] [CrossRef]
  48. Kumar, A.; Sah, B.; Deng, Y.; He, X.; Kumar, P.; Bansal, R.C. Application of multi-criteria decision analysis tool for design of a sustainable micro-grid for a remote village in the Himalayas. J. Eng. 2017, 2017, 2108–2113. [Google Scholar] [CrossRef]
  49. Steinhilber, S.; Geldermann, J.; Wietschel, M. Renewables in the EU after 2020: A multi-criteria decision analysis in the context of the policy formation process. EURO J. Decis. Process. 2016, 4, 119–155. [Google Scholar] [CrossRef]
  50. Baris, K.; Kucukali, S. Availibility of renewable energy sources in Turkey: Current situation, potential, government policies and the EU perspective. Energy Policy 2012, 42, 377–391. [Google Scholar] [CrossRef]
  51. Tsoutsos, T.; Drandaki, M.; Frantzeskaki, N.; Iosifidis, E.; Kiosses, I. Sustainable energy planning by using multi-criteria analysis application in the island of Crete. Energy Policy 2009, 37, 1587–1600. [Google Scholar] [CrossRef]
  52. Patlitzianas, K.D.; Ntotas, K.; Doukas, H.; Psarras, J. Assessing the renewable energy producers’ environment in EU accession member states. Energy Convers. Manag. 2007, 48, 890–897. [Google Scholar] [CrossRef]
  53. Ribeiro, F.; Ferreira, P.; Araújo, M. The inclusion of social aspects in power planning. Renew. Sustain. Energy Rev. 2011, 15, 4361–4369. [Google Scholar] [CrossRef] [Green Version]
  54. Painuly, J.P. Barriers to renewable energy penetration; a framework for analysis. Renew. Energy 2001, 24, 73–89. [Google Scholar] [CrossRef]
  55. Zelt, O.; Kruger, C.; Blohm, M.; Bohm, S.; Far, S. Long-Term Electricity Scenarios for the MENA Region: Assessing the Preferences of Local Stakeholders Using Multi-Criteria Analyses. Energies 2019, 12, 3046. [Google Scholar] [CrossRef] [Green Version]
  56. Eichhorn, M.; Tafarte, P.; Thrän, D. Towards energy landscapes—“Pathfinder for sustainable wind power locations”. Energy 2017, 134, 611–621. [Google Scholar] [CrossRef]
  57. Brooks, K.; Schirmer, J.; Pascoe, S.; Triantafillos, L.; Jebreen, E.; Cannard, T.; Dichmont, C.M. Selecting and assessing social objectives for Australian fisheries management. Mar. Policy 2015, 53, 111–122. [Google Scholar] [CrossRef]
  58. Sutherland, W.J.; Burgman, M. Policy advice: Use experts wisely. Nature 2015, 526, 317–318. [Google Scholar] [CrossRef] [Green Version]
  59. Cooke, R.M. Experts in Uncertainty: Opinion and Subjective Probability in Science; Oxford University Press: Oxford, UK, 1991. [Google Scholar]
  60. Burgman, M.A. Trusting Judgements: How to Get the Best Out of Experts; Cambridge University Press: Cambridge, UK, 2016. [Google Scholar]
  61. Estévez, R.A.; Mardones, F.O.; Álamos, F.; Arriagada, G.; Carey, J.; Correa, C.; Escobar-Dodero, J.; Gaete, Á.; Gallardo, A.; Ibarra, R.; et al. Eliciting expert judgements to estimate risk and protective factors for Piscirickettsiosis in Chilean salmon farming. Aquaculture 2019, 507, 402–410. [Google Scholar] [CrossRef]
  62. Hemming, V.; Walshe, T.V.; Hanea, A.M.; Fidler, F.; Burgman, M.A. Eliciting improved quantitative judgements using the IDEA protocol: A case study in natural resource management. PLoS ONE 2018, 13, e0198468. [Google Scholar] [CrossRef]
  63. McBride, M.F.; Garnett, S.T.; Szabo, J.K.; Burbidge, A.H.; Butchart, S.H.M.; Christidis, L.; Dutson, G.; Ford, H.A.; Loyn, R.H.; Watson, D.M.; et al. Structured elicitation of expert judgments for threatened species assessment: A case study on a continental scale using email. Methods Ecol. Evol. 2012, 3, 906–920. [Google Scholar] [CrossRef]
  64. Gilovich, T.; Griffin, D.; Kahneman, D. Heuristics and Biases: The Psychology of Intuitive Judgment; Cambridge University Press: Cambridge, UK, 2003. [Google Scholar]
  65. Tversky, A.; Kahneman, D. Judgment under Uncertainty: Heuristics and Biases. Science 1974, 185, 1124–1131. [Google Scholar] [CrossRef]
  66. Hemming, V.; Burgman, M.A.; Hanea, A.M.; McBride, M.F.; Wintle, B.C. A practical guide to structured expert elicitation using the IDEA protocol. Methods Ecol. Evol. 2018, 9, 169–180. [Google Scholar] [CrossRef] [Green Version]
  67. Hemming, V.; Hanea, A.M.; Walshe, T.; Burgman, M.A. Weighting and aggregating expert ecological judgments. Ecol. Appl. 2020, 30, e02075. [Google Scholar] [CrossRef] [PubMed]
  68. Dincer, I.; Acar, C. A review on clean energy solutions for better sustainability. Int. J. Energy Res. 2015, 39, 585–606. [Google Scholar] [CrossRef]
  69. Evans, A.; Strezov, V.; Evans, T.J. Assessment of sustainability indicators for renewable energy technologies. Renew. Sustain. Energy Rev. 2009, 13, 1082–1088. [Google Scholar] [CrossRef]
  70. Stigka, E.K.; Paravantis, J.A.; Mihalakakou, G.K. Social acceptance of renewable energy sources: A review of contingent valuation applications. Renew. Sustain. Energy Rev. 2014, 32, 100–106. [Google Scholar] [CrossRef]
  71. Van der Horst, D. NIMBY or not? Exploring the relevance of location and the politics of voiced opinions in renewable energy siting controversies. Energy Policy 2007, 35, 2705–2714. [Google Scholar] [CrossRef] [Green Version]
  72. Koh, L.P.; Ghazoul, J. Biofuels, biodiversity, and people: Understanding the conflicts and finding opportunities. Biol. Conserv. 2008, 141, 2450–2460. [Google Scholar] [CrossRef]
  73. Cowell, R.; Bristow, G.; Munday, M. Acceptance, acceptability and environmental justice: The role of community benefits in wind energy development. J. Environ. 2011, 54, 539–557. [Google Scholar] [CrossRef]
  74. Agha, M.; Lovich, J.E.; Ennen, J.R.; Todd, B.D. Wind, sun, and wildlife: Do wind and solar energy development ‘short-circuit’ conservation in the western United States? Environ. Res. Lett. 2020, 15, 075004. [Google Scholar] [CrossRef]
  75. Kamlage, J.H.; Drewing, E.; Reinermann, J.L.; de Vries, N.; Flores, M. Fighting fruitfully? Participation and conflict in the context of electricity grid extension in Germany. Util Policy 2020, 64, 101022. [Google Scholar] [CrossRef]
  76. Aitken, M. Wind power and community benefits: Challenges and opportunities. Energy Policy 2010, 38, 6066–6075. [Google Scholar] [CrossRef]
Sustainability 13 03515 g001 550
Figure 1. Stages of participation in multi-criteria decision analysis (MCDA) methodologies.
Figure 1. Stages of participation in multi-criteria decision analysis (MCDA) methodologies.
Sustainability 13 03515 g001
Sustainability 13 03515 g002 550
Figure 2. Year of publication for reviewed articles (n = 184).
Figure 2. Year of publication for reviewed articles (n = 184).
Sustainability 13 03515 g002
Sustainability 13 03515 g003 550
Figure 3. Reviewed articles that considered experts or stakeholders in the participative process, and percentage of articles that included a description of the participative methods.
Figure 3. Reviewed articles that considered experts or stakeholders in the participative process, and percentage of articles that included a description of the participative methods.
Sustainability 13 03515 g003
Sustainability 13 03515 g004 550
Figure 4. Levels of stakeholder participation reported in the reviewed articles (n = 33).
Figure 4. Levels of stakeholder participation reported in the reviewed articles (n = 33).
Sustainability 13 03515 g004
Sustainability 13 03515 g005 550
Figure 5. Types of indicators used to evaluate social acceptability in the MCDA process.
Figure 5. Types of indicators used to evaluate social acceptability in the MCDA process.
Sustainability 13 03515 g005
Table
Table 1. Source of energy MCDA method and geography of reviewed articles (n = 184). Regions/country with more than 5 articles are included.
Table 1. Source of energy MCDA method and geography of reviewed articles (n = 184). Regions/country with more than 5 articles are included.
Source of Energy%MCDA Method%Continent%
Wind30.1AHP48.6Europe45.7
Solar25.8ANP10.3Asia34.2
Bioenergy24.7PROMETHEE I, II8.1America13.0
Geothermal11.0ELECTRE I, II, III1.6Africa3.3
Marine3.6Outranking (others)2.7Oceania2.2
Renewable energies4.7MAVT11.9Global1.6
Goal Programming16.8
Table
Table 2. Total number of decisions reported in the reviewed articles (n = 184).
Table 2. Total number of decisions reported in the reviewed articles (n = 184).
Decisionn%
Selecting the most appropriate energy alternative7641.1
Selecting the most appropriate location for an energy plant4021.6
Selecting the most appropriate technology3016.2
Evaluating scenarios/alternatives for energy policy or project development2312.4
Evaluating sustainability/impacts of using renewable energies73.8
Evaluating strengths, weaknesses, opportunities and threats for energy development52.7
Evaluating barriers for energy project development32.2
Table
Table 3. Number of social indicators evaluated in the 184 articles for each social criterion (n = 490). Some authors formulated two or more social criteria in the same case study. Some articles estimated the same indicator more than once. Lists of articles for each social criterion are included in Supplementary Materials (Table S1).
Table 3. Number of social indicators evaluated in the 184 articles for each social criterion (n = 490). Some authors formulated two or more social criteria in the same case study. Some articles estimated the same indicator more than once. Lists of articles for each social criterion are included in Supplementary Materials (Table S1).
CriterianIndicatorsnReferences in Supplementary Materials
Employment97Creation of new jobs97[3,5,6,10,11,13–16,17,19,21,22,24,26–29,33,34,35,39–42,45,46,47,51,56–59,62,64,66–68,70,72,73,76,79,80,81,84–88,91,93,95,96,100,101,103–106,108,109,112,115,116,121,122,124,125,132,135,137,139,142,143,146,147,150,159,160,163,166,168,169,170,171,173,175–177,179,182–184]
Social acceptance95Public acceptability70[1–3,6,11,13,14,20,22,26–28,33,35,38,45,48,51,53,54,57,59,62,64,69,71,81,85,86,88,93,95,103–106,116,117,120–122,124,125,127,129,130,132,134,135,137,138,140,141,146,147,150,152,155,157,159,167,170,172,174,176,180,182]
Political acceptability17[4,5,19,29,31,69,81,82,85,90,105,119,122,125,137,149,178]
Public resistance8[7,61,66,80,145]
Social development86Local development21[9,15,29,30,39,48,50,87,112,121,123,127,137,146,150,155,159,174,184]
Social benefit18[6,25–27,28,35,37,56,69,72,77,86,107,111,127,158,170,180]
Energy access10[19,39,66,100,104,118,121,147,174]
Quality of life9[14,28,57,75,76,95,178,184]
Negative social impact8[28,32,49,60,65,164,183]
Local economic benefit7[36,47,54,87,109,142,143]
Number of people benefited6[12,15,21,131,168,169]
Regional/national development5[6,52,93,128,129]
Human rights2[72,76]
Health impact84Noise impact22[15,18,25,40,52,54,56,58,64,66,86,87,92,93,123,126,134,137,144,168,174,176]
Safety and risk20[24,28,39,41,69,74,78,103,118,137,140,141,156,164,166,168,173,177]
Health impact19[5,15,18,19,67,79,84,87,95,104,123,127,154,159,161,162,181,184]
Pollution15[8,58,59,64,74,87,93,95,135,137,156,177]
Mortalities5[66,91,114,161,162]
Odor3[87,112,168]
Governance62Policy compatibility29[5,6,14,16,19,26–29,34,38,51,54,58,75,81,95,102,105,122,136,137,145,147,159,160]
Participation9[28,67,93,94,112,124,158]
Law and regulation9[2,44,120,137,147,159,165,166]
Collaboration7[28,54,72,120,145]
Conflicts6[87,98,99,125,161,162]
Transparency2[61,158]
Visual impact22Aesthetic impact19[15,23,40,52,54,56,58,63,64,66,86,87,92,124,134,137,144,151,174]
Landscape value3[67,93,168]
Knowledge and awareness20Public awareness9[2,34,61,89,121,145,148,158,171]
Education increase7[12,28,45,100,113,120,121]
Corporate social responsibility4[28,75,76,156]
Cultural value13Cultural concerns9[28,43,44,107,110,127–129,133]
Archaeological impact4[55,83,97,153]
Social justice11Equality9[50,94,95,100,102,118,148]
Social justice2[93,113]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Estévez, R.A.; Espinoza, V.; Ponce Oliva, R.D.; Vásquez-Lavín, F.; Gelcich, S. Multi-Criteria Decision Analysis for Renewable Energies: Research Trends, Gaps and the Challenge of Improving Participation. Sustainability 2021, 13, 3515. https://0-doi-org.brum.beds.ac.uk/10.3390/su13063515

AMA Style

Estévez RA, Espinoza V, Ponce Oliva RD, Vásquez-Lavín F, Gelcich S. Multi-Criteria Decision Analysis for Renewable Energies: Research Trends, Gaps and the Challenge of Improving Participation. Sustainability. 2021; 13(6):3515. https://0-doi-org.brum.beds.ac.uk/10.3390/su13063515

Chicago/Turabian Style

Estévez, Rodrigo A., Valeria Espinoza, Roberto D. Ponce Oliva, Felipe Vásquez-Lavín, and Stefan Gelcich. 2021. "Multi-Criteria Decision Analysis for Renewable Energies: Research Trends, Gaps and the Challenge of Improving Participation" Sustainability 13, no. 6: 3515. https://0-doi-org.brum.beds.ac.uk/10.3390/su13063515

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop