Methodological Framework to Select Evaluation Criteria for Multi-Criteria Decision Analysis of Road Transportation Fuels and Vehicles

Abstract

Studies applying Multi-Criteria Decision Analysis (MCDA) to evaluate Road Transportation Fuels and Vehicles (RTFV) rely on a wide variety of evaluation criteria and appear to lack a structured and consistent way of criteria selection. This leads to non-transparent and not easily comparable evaluation results. To address this issue, a methodological framework is developed to systematically identify and select relevant MCDA-evaluation criteria for the assessment of RTFV. The methodological framework is based on Life Cycle Sustainability Analysis (LCSA) and considers environmental, economic, and social criteria that are complemented with a technical pillar. The scope of the analysis is further enlarged by considering positive and negative externalities. The first part of the framework follows the LCSA approach and requires the analyst to clearly define the context of the analysis. The second part is to decompose the problem by developing criteria categories along the relevant life cycle for each of the evaluation dimensions. This decomposition process helps decision makers to easily identify and select relevant criteria with clear added value within the context of the analysis. In an exemplary application, the developed methodological framework is used to identify relevant criteria for the evaluation of RTFV alternatives for an island aiming at energy self-sufficiency.

1. Introduction

A wide variety of road transportation fuels and vehicles (RTFV) exists. The options available range from fossil fuels, such as diesel, to biofuels, such as biodiesel, as well as different drive technologies, such as electric engines. This variety confronts decision makers (DMs) with the problem of identifying the best RTFV alternative suitable for their specific interests. As more and more stakeholders consider the inclusion of multiple sustainability criteria mandatory in the relevant decision-making process, this decision problem gets even more complicated.

A helpful tool in this regard is Multi-Criteria Decision Analysis (MCDA), a decision making approach capable of generating rankings of alternatives subject to conflicting and qualitatively different criteria [1,2]. It is often used in the literature to rank RTFV alternatives under specific consideration of sustainability criteria [3,4,5,6]. One of the major difficulties faced by MCDA researchers is the selection of appropriate assessment criteria as these greatly determine the outcome of the MCDA evaluation [7]. Research looking at the pool of criteria used for energy related MCDA making finds a wide range of potential criteria [1,8,9,10]. By analyzing the literature applying MCDA methods to the evaluation of RTFV alternatives, Kügemann and Polatidis [11] similarly found that researchers rely on a large variety of different criteria for the evaluation process. This results from the lack of a systematic way for selecting the criteria. An apparent reason for this lack is that MCDA evaluations are often concerned with quite different problems. Nevertheless, assessing the sustainability dimensions of the different alternatives against a more similar set of criteria would increase comparability among the various studies and lead to more structured and standardized assessments. To this end, Life Cycle Sustainability Analysis (LCSA) can be an analytical method to assist in the standardization of the criteria selection process of relevant MCDA evaluations. LCSA is a systematic tool to assess the overall sustainability of technologies and products by taking into account the entire life cycle of the respective product, process or system under the environmental, economic and social dimensions [12].

The intention of this paper is to standardize the criteria selection process and produce more comparable MCDA-rankings of RTFV alternatives. To this end, a methodological framework is proposed that combines MCDA with LCSA to enable DMs to systematically identify and select a number of sustainability related evaluation criteria. The remainder of this paper is structured as follows.

Section 2 reviews relevant literature. It includes an introduction to the relevance of MCDA and LCSA, a summary of the general guidelines of LCSA implementation, and a presentation of the basic principles of establishing a consistent set of criteria used in MCDA studies. In Section 3, the methodological framework for criteria selection is developed. Subsequently, this framework is applied on an exemplary case study of an island seeking energy independence in Section 4. The section also discusses further research possibilities and potential additional applications of the developed framework. Finally, Section 5 summarizes and concludes.

2. Literature Review

LCSA and MCDA exhibit a number of characteristics that complement each other: LCSA offers a useful pool of relevant sustainability-related evaluation criteria for MCDA assessments, and MCDA provides a powerful tool to aggregate different indicators to one result [13] and, therefore, to evaluate decision problems considering the three dimensions of sustainability defined in LCSA [14]. LCSA assesses the environmental dimension under the relatively standardized Life Cycle Assessment (LCA), the economic dimension under the Life Cycle Costing (LCC), and the social dimension under the Social Life Cycle Assessment (SLCA). Following the principle standards defined in the ISO 14040 series, LCSA follows four main steps [12,15]:

• Define the goal and scope of the study, including the target audience, the functional unit, the system boundaries, and the unit processes.
• Compile the inventory of the items considered for each of the three dimensions. This captures all relevant emissions and extracted resources, and economic costs and benefits, as well as social implications as a consequence of a product or activity [16].
• Assess the life cycle impacts of this inventory for each of the three dimensions:
  • The environmental LCA can be based on midpoint or endpoint indicators. Midpoint indicators are situated within the cause effect-chain, e.g., ozone depletion, human toxic effects, or climate change. Endpoint indicators are situated closer to the final effect on so-called areas of protection, such as human health, resource depletion, or natural environment. Standardized characterization models are used to calculate these impacts, e.g., Ecoindicator’99 or Recipe [17].
  • LCC requires to calculate and aggregate all costs and benefits along the life cycle of the product [18].
  • The social pillar is the least developed dimension within LCSA. So far, there are no standardized impact assessment methods available for SLCA [18,19].
• Interpret the results on a combined basis.

The selection of MCDA-criteria should follow certain principles [8,20]. The most relevant of these principles are listed below with a brief indication on how LCSA adheres to them:

• Systemic principle: The criteria should entail the “essential characteristic” and “whole performance” [8] in order to provide a complete picture of the assessed alternatives. LCSA provides a useful tool for a comprehensive sustainability evaluation.
• Consistency principle: The evaluation criteria should be consistent with the objective of the DM [8]. To this end, the main steps for LCSA analysis as defined above require the user to clearly define the goal and scope of the study.
• Independency and/or minimality principle: Different criteria should reflect different dimensions of the alternatives and not be double counted or redundant in order to avoid excessive overlap. Double counting is also a risk under LCSA [12] and should be avoided under the methodological framework developed below.
• Measurability and operationality principle: The criteria should be measurable and operational, i.e., be meaningfully used in the analysis. Ideally, the criteria are measured in quantitative terms, otherwise qualitative terms—also in form of relative comparisons—can be used. LCSA provides both quantitative and qualitative data, and MCDA can process both [13].

Arce et al. [21], Strantzali and Aravossis [22], and Wang et al. [8] review MCDA decision making papers concerned with renewable or sustainable energy and summarize the evaluation criteria applied. They find a variety of different criteria under the three sustainability dimensions as well as under a technical dimension that is often considered in MCDA evaluation papers. Kügemann and Polatidis [11] review studies that apply MCDA to evaluate RTFV and analyze and categorize the evaluation criteria used in the various papers. Although the studies deal with similar issues, a relatively large number of different evaluation criteria are used. While there seems to be more or less broad agreement on some environmental criteria—GHG in particular is chosen by most authors—this is less true for economic and social criteria. The economic benefits are often neglected. Finally, and as to be expected, the social criteria appear to be least standardized. It seems that no structural approach is used by the authors for criteria selection, although the selection of criteria is a crucial part of the decision-making process. Furthermore, no underlying consistent pool of criteria seems to exist.

Only a few authors explicitly reference LCSA or similar sustainability concepts for the selection of the evaluation criteria. Ammenberg and Dahlgreen [23] develop a MCDA method to evaluate the sustainability of public bus technologies. They identify 12 indicators in an iterative and cooperative process with stakeholders and experts. For some of the indicators, e.g., greenhouse gas emissions or LCC, they explicitly refer to the life cycle approach.

Ekener et al. [14] use MCDA to integrate the results of LCA, LCC, and SLCA to a LCSA study on alternative transportation fuels. Ekener-Petersen et al. [24], compare different fuel alternatives from different countries by applying a SLCA methodology based on the criteria proposed by Benoît et al. [19].

Hirschberg et al. [25] provide a set of sustainability criteria for the integrated evaluation of advanced electricity generation technologies with MCDA techniques. Their final selection of the criteria is based on the three pillars of sustainability and confirmed by a stakeholder survey.

Keller et al. [18] develop a methodology called Integrated Life Cycle Sustainability Assessment (ILCSA) as a practical approach for decision support and conduct an exemplary analysis of a biorefinery. ILCSA extends LCSA by including ex-ante assessments like specific barriers that can lead to failures in scenario realization.

Onat et al. [26,27] compare alternative vehicle technologies based on their macro-level social, economic, and environmental impacts by using data from input-output tables. They base their criteria selection on “data availability, prioritized indicators in national policies, and ease of integration with the current LCSA framework” [26].

Osorio-Tejada et al. [28] conduct a multi-criteria sustainability assessment for biodiesel and liquefied natural gas by considering environmental, economic and social evaluation criteria based on a thorough literature review.

Ren et al. [4] assess bioethanol production pathways and rely on stakeholders to choose the most appropriate criteria based on the LCSA approach.

Santoyo-Castelazo and Azapagic [29] propose a generic methodology and decision-support framework for the sustainability assessment of energy systems using MCDA. Their methodology enables an integrated life cycle sustainability assessment of future energy scenarios based on environmental, economic, and social criteria.

Sharma and Strezov [30] take environmental and economic criteria into account in order to evaluate alternative and conventional transport fuels with a simple normalized score. They rely on the Recipe impact assessment method to assess the environmental life cycle impacts.

Valente et al. [31] select environmental and social indicators following the guidelines of LCA and SLCA for the assessment of the production of second-generation bioethanol and biochemicals. The results for environmental and social indicators are interpreted separately.

Von Doderer and Kleynhans [32] evaluate lignocellulosic bioenergy systems based on the life cycle approach under financial-economic, socio-economic and environmental criteria. They combine the criteria by applying a MCDA method.

Wulf et al. [33] propose a new approach to exclude irrelevant data from LCSA based decision making. To this end, they propose a specific MCDA method to identify and quantify thresholds that can be used to exclude irrelevant data. They conduct a case study on finding the best location for sustainable hydrogen production.

Zhou et al. [3] use a limited number of criteria for their MCDA evaluation of alternative fuels, but mention LCSA as rationale for their criteria selection.

The LCSA-based papers discussed above show more consistency in their evaluation criteria than other MCDA papers, but still rely on a relatively wide variety of different evaluation criteria. The criteria used in these papers are summarized in Appendix A. The methodological framework defined in the next section aims at providing a more structured and comparable way to generate meaningful and relevant sustainability evaluation criteria.

3. Methodological Framework for MCDA-Criteria Selection

In this section, a new methodological framework for the selection of evaluation criteria for MCDA of RTFV alternatives is developed. The selection process is based on the notion of LCSA and combines MCDA and LCSA in a specific way that is consistent with the principles outlined in the literature review above. Instead of providing a predefined fixed pool of potential criteria from which the DMs can select their evaluation criteria, the methodological framework sets the frame within which criteria can be developed. Not all criteria typically included in LCSA have to be relevant for MCDA analyses. Instead, only criteria with significant differences between the alternatives add value to MCDA analyses, otherwise they have limited or no impact on the MCDA result. It may also be the case that the impacts along the entire life cycle—as is done in LCSA—do not have to be considered for all criteria. A useful method to identify irrelevant data is proposed by Wulf et al. [33].

The analysts should provide the DMs with the relevant information that is needed to choose the adequate criteria. This is based on extended literature review, user experience, the context of the case-study, expert knowledge, and the structure provided by the methodological framework. The parts of the methodological framework developed in the next sub-sections are summarized in Figure 1.

The methodological framework supports the analysts or study designer in identifying a pool of relevant criteria that can be finally selected by the DMs.

3.1. Define the Context of the Analysis

First, the analysts should clearly define the context of the analysis, which forms the basis for the criteria selection process. This definition contains the following steps:

• Define the decision makers, their interest and evaluation goals, and the regional background

The DMs, their interests, and the evaluation goals should be defined from the outset of the analysis. It is also important to specify the regional background in which the analysis is embedded, since different regional conditions such as climatic factors can influence the spectrum of relevant criteria.

2.

Specify the relevant alternatives

All alternatives should be described in detail as certain characteristics might determine the applicability of certain evaluation criteria.

3.

Draw the system boundary

The system boundary, e.g., well-to-wheel or cradle-to-gate, defines which steps of the life cycles are included in the analysis [16]. The drawing of the boundary depends on the specific interests of the DMs and might be limited to the consideration of only a part of the life cycle [12].

4.

Set the functional unit

The functional unit is the reference unit in LCSA [12]. Ideally, the criteria are expressed relative to the functional unit, thereby improving the comparability of the study [16]. The functional unit depends on the system boundary: a simple well-to-tank analysis, for instance, might have a functional unit like the impact caused by the production of ‘1 MJ fuel’ [16]. For some of the criteria, specifically social criteria, it might be difficult to evaluate the criteria in the functional unit and simpler forms of evaluation might suffice, e.g., relative comparisons like A is better or worse than B in terms of X.

5.

Define the unit processes

Unit processes are the relevant parts of the production cycle within the system boundary, i.e., the smallest unit of the production process for which data are selected [12]. In life cycle assessment, the breakdown into unit processes is typically quite detailed. For the purpose of an MCDA evaluation, it may be applicable to merge some of the unit processes. For example, a unit process called ‘fuel production’ might be sufficient for the MCDA analysis, although this step would typically consist of many different unit processes under the LCSA framework. In case complete life cycle results for specific criteria are available, a split is not required any more, as the criteria already contain all the necessary information along the value chain (e.g., all the GHG emissions along the value chain). Otherwise, the breakdown into unit processes could show where the differences between alternatives are most pronounced and which unit processes could be neglected in the evaluation.

6.

Enlarge the scope of the analysis if necessary

Life cycle analyses are limited to the impacts within the predefined system boundary. Under certain circumstances, such restriction could omit information that is relevant for the decision process. For instance, a comprehensive comparison of electric and internal combustion vehicles should not only include an assessment of fuel production and usage, but also the differences in the vehicle life cycle and ideally also the requirements for additional refueling infrastructure. In addition to the usage phase, the vehicle life cycle should also contain the manufacturing and disposal phase. In these cases—and depending on the interest of the DM—the system boundary needs to be enlarged in order to include these impacts as well. The functional unit considering the vehicle production could be for instance the impact of the “production and operation of vehicle and fuel for an operational time of ten years with an average mileage of 10,000 km/year”.

7.

Consider externalities if applicable

In some cases it might be applicable to also consider indirect impacts on sectors outside the original scope of the assessment [18] and to expand the system boundaries accordingly. Outside sectors can be negatively or positively affected. In this paper we refer to these effects as positive or negative externalities. The raw material wood could, for instance, be used to produce cellulosic diesel. Similarly, it could be utilized as construction or fuel material, which could ultimately be a more efficient and environmentally friendly use [34]. The increased competition for the same resource is a negative externality and could be included in the analysis. An example for a positive effect on another sector is the additionally available battery storage power from a EV-fleet which could be used as flexible load to balance fluctuating renewable energy power production [35]. This expansion of the system boundary is not necessarily relevant for all evaluation dimensions. For instance, increased competition for a specific resource as in the wood example above could potentially be considered within the economic or social dimension.

3.2. Develop Relevant Criteria under the Evaluation Dimensions

In the second part of the framework a decomposition of the problem for each of the evaluation dimensions is conducted [36]. LCSA considers environmental, economic, and social dimensions. Many researchers add a technical dimension to the evaluation of energy related alternatives such as RTFV [8], which we also include in the analysis, especially as many problem-specific criteria are included in this dimension.

The decomposition process is performed by establishing main criteria categories for each of the evaluation dimensions. For each criteria category, specific evaluation criteria are to be developed along the relevant life cycle and assigned to the respective unit processes. Such structured accounting makes it easier to identify and select relevant criteria that add value to the assessment and to eliminate redundant criteria. It also allows the analysts to decide whether certain criteria are only relevant under specific unit processes or along the entire life cycle. The final criteria selection is restricted by data availability and the specific requirements of each DM. As often done in MCDA studies, insufficient data availability can be remedied by qualitative assessments by experts [6,37].

Each criterion should be classified under only one of these dimensions, thus not be double counted. This classification of criteria is often not carried out consistently [38]. Mineral and fossil depletion, for example, could be labelled both as economic and environmental problem. Attributing a criterion to more than one specific dimension can have significant consequences on the overall evaluation as many MCDA methods attach a weighting not only to the sub-criteria of the different dimensions but also to the main dimensions [28].

The sustainability related criteria developed under the four predominant dimensions are to be complemented with problem-specific criteria; for instance, the sense of comfort or the vehicle capacity of alternative transportation buses [39].

The main criteria categories under each of the four evaluation dimensions and the related decomposition process are discussed in the following.

3.2.1. Environmental Dimension

The environmental criteria categories are based on the concept of LCA which considers environmental impacts on the inventory level (i.e., all arising emissions and resources like CO₂, nitrogen oxides, etc.), or via impact assessment methods on mid- or endpoint level. Impact assessment methods translate the inventory into environmental impacts via specific characterization methods. Environmental midpoint impact categories may include climate change, ozone depletion, and eutrophication, whereas endpoint indicators encompass human health, resource depletion, and ecosystem quality, among others [12]. As endpoint indicators require more information and are less transparent than midpoint indicators, it is recommended to rely on midpoint indicators as criteria categories. Other environmental indicators that are often not included in LCA analysis, e.g., biodiversity, environmental risks, or recyclability, can be added to the pool of relevant criteria categories [14,17] and developed along the life cycle for each unit process. From the resulting list, the analysts can easily determine the main impacts and decide whether to rely on the inventory level or midpoint level indicators as evaluation criteria. The inventory level is easier to be accounted for and can be sufficient for a relative comparison of alternatives.

3.2.2. Economic Dimension

The economic criteria categories are based on the concept of LCC. LCC considers all economic costs and benefits over the entire product life cycle on a private or societal level [12]. Societal LCC [12,40] aims at monetizing all external costs like environmental or social costs. Conventional or private LCC considers private costs and benefits and is accounted for at market or factor prices. Factor prices are defined as market prices adjusted for transfers [40], i.e., redistributions between stakeholders like taxes and subsidies. The different levels of LCC analysis are summarized in Figure 2.

The analyst should first decide on the level of LCC and whether to take market or factor prices into account. External costs are already often accounted for by environmental and social criteria in the overall MCDA assessment. Therefore, societal LCC could potentially double count these effects by considering them as external effects in LCC and in the environmental or social dimension. Nevertheless, double counting criteria should be avoided, and, thus, societal LCC is not recommended for MCDA analysis. At a private user perspective, market prices are potentially most relevant. A policy maker, however, might be interested in the net impact on society and therefore adjust market prices for certain taxes and subsidies, which would be closer to looking at the factor price level.

The next step is to develop the selected economic criteria categories along the life cycle by considering the arising cost or benefit, etc., under each unit process. From the resulting list, the analysts can identify the categories along the life cycle of interest that add value to the MCDA analysis. For the purposes of most MCDA studies, the inclusion of the costs of each alternative is certainly essential. Benefits should be considered if relevant differences exist between the alternatives. Taxes and subsidies should be considered depending on the level of interest. All relevant criteria should be aggregated to a comparable indicator like the net present value (NPV), the internal rate of return (IRR) or levelized cost of energy.

Economic externalities measure the impact of the alternatives on sectors outside of the original scope of the analysis, for instance price effects on competing sectors through increased competition for resources [40]. These effects can be monetized and integrated under the economic dimension as additional criteria categories. Another possibility is to include them as stand-alone criteria under the social or technical dimension, especially if it is difficult to monetize their impact.

3.2.3. Social Dimension

The selection of social criteria is considered to be the major challenge in triple-bottom-line sustainability analysis: there is still no common agreement on the relevant parameters [26] and the number of possible social indicators seems unlimited [38]. The approach of the developed methodological framework is to first determine the relevant stakeholder groups and second to identify relevant criteria categories. The five main stakeholder groups proposed by Benoît et al. [19] are consumers, workers, local communities, the society, and value chain actors.

Table 1. Stakeholder categories and potential criteria categories adopted from Benoît et al. [19] and Ekener et al. [41].

Stakeholder Categories	Potential Criteria Categories
Worker	Freedom of association and collective bargaining, child labor, fair salary, working hours, forced labor, equal opportunities/discrimination, health and safety, social benefits, social security, health care, education
Consumer	Health and safety, feedback mechanism, consumer privacy, transparency, end of life responsibility
Local community	Access to material resources, access to immaterial resources, delocalization and migration, cultural heritage, safe and healthy living conditions, respect of indigenous rights, community engagement, local employment, secure living conditions, job creation, rural development
Society	Public commitments to sustainability issues, contribution to economic development, prevention and mitigation of armed conflicts, technology development, corruption
Value chain actors (not including consumers)	Fair competition, promoting social responsibility, supplier relationships, respect of intellectual property rights

lists the five main stakeholders groups and potential criteria categories, adopted from Benoît et al. [19] and Ekener et al. [41]. Other categories might be added to this list.

Once the relevant criteria categories are identified, their respective impacts are to be defined along the life cycle for the relevant unit processes. Finally, the DMs can pick the relevant evaluation criteria from this list. The final evaluation of the specific criteria is ideally based on stakeholder consultation and will often be qualitative. MCDA’s ability to process qualitative evaluations is particularly helpful in this regard [41].

3.2.4. Technical Dimension

The inclusion of the technical dimension goes beyond the notion of LCSA but can provide added value, although some typical technical criteria are, to a certain degree, already incorporated in the environmental, economic, or social dimension. For instance, the efficiency of the fuel production process is inherent in the economic cost and the environmental impact, as well as the resources required for the fuel production. Nevertheless, a comparison by energy efficiency allows a more detailed analysis by asking which process manages energy—and thus resources—most efficiently. Additionally, more problem-specific technical criteria could be considered, e.g., the reliability of a fuel or vehicle under certain weather conditions, the fuel range [5,42,43], or the passenger capacity [44].

4. Exemplary Application of the Methodological Framework

In the following, the methodological framework outlined in the previous section is exemplarily applied on a hypothetical evaluation problem of alternative fuel vehicles and the respective selection of evaluation criteria. The exemplary application serves to illustrate the appropriateness of the developed framework. The data generation and application is the next step after the criteria selection and is not part of this case study. It is assumed that research analysts support the DMs in the selection process by setting up the following study design.

4.1. Define the Context of the Analysis

The fictitious DMs are policy makers on an island with approximately 100,000 inhabitants, relatively cold climate and sufficient agricultural land to potentially produce energy crops. They are supposed to place more focus on the local impacts on the island than on the overall sustainability impact. Their aim is to replace the existing private car fleet by alternative fuel vehicles and to increase the island’s energy self-sufficiency. Thus, they evaluate the alternatives under the assumption that the fuel and the respective feedstock are produced on the island. They use MCDA to determine which alternatives should be preferred under consideration of environmental, economic, social, and technical criteria. To this end, four different alternatives are considered: Biodiesel Vehicles (BDV), Biogas Vehicles (BGV), Battery Electric Vehicles (BEV), and Hydrogen Fuel Cell Vehicles (HFV). The system boundary is set to a well-to-wheel perspective as the DMs focus the evaluation on the impact caused on the island. The vehicle production cycle is not considered by the DMs as no vehicle manufacturing takes place on the island.

When possible, the criteria are expressed in the functional unit, i.e., the island’s car fleet of 40,000 passenger vehicles with a lifetime of ten years and an average annual mileage of 15,000 km per year.

The following unit processes are considered within the defined system boundary:

-

Feedstock production for fuels;

-

Production process of the fuel, considering potential by- or co-products;

-

Operational phase of the vehicle, i.e., final use of the fuel.

The environmental and social implications of the required additional refueling infrastructure are assumed to be small, only the economic implications are considered in the decision process. The DMs decide to enlarge the scope of the well-to-wheel boundary to also take into account the economic costs of the refueling infrastructure. Further, the acquisition costs of the vehicles are considered within the unit process Operational phase of the vehicle.

Table 2. Crucial factors for each unit process and alternative.

Alternatives	Unit Processes
	Feedstock Production	Fuel Production	Refueling Infrastructure	Operational Phase (Including Acquisition of Vehicle)
BDV	Rapeseed produced on island	Biodiesel plant, based on transesterification process (by-product animal feed)	Gas stations (already available)	Existing vehicles can be partly used with a relatively simple modification of the motor
BGV	Natural waste	Biogas plant	CBG fueling stations (already available)
BEV	n.a.	Wind parks	Electric charging terminals (only few available)	Mainly new vehicles to be acquired (few existing vehicles on island)
HFV	Groundwater (for electrolysis)	Wind parks Electrolysis facility	Hydrogen fueling stations (none available)	New vehicles to be acquired (no existing vehicles on island)

summarizes the crucial factors for each unit process and alternative.

The DMs also decide to consider the impact on sectors outside the directly affected system boundary. Potential negative and positive externalities on other industry sectors are summarized in

Table 3. Potential negative and positive externalities by alternative.

	Negative Externalities	Positive Externalities
BDV	Increased competition for agricultural land	By-product of biodiesel production can be used for animal feed
BGV	Increased competition for waste resources and biogas produced on the island	Unused gas (if any) could be used for electricity production and for grid balancing
BEV	Increased competition for electric power	Batteries could serve as energy storage for grid balancing
HFV		Hydrogen could be used as energy storage for grid balancing

.

Negative externalities are caused by increased competition for the same resources and products which could lead to price increases for consumers. Positive externalities arise in case of synergy effects with the electric power system, e.g., hydrogen or batteries can be used for grid balancing. There are several methods to consider these externalities: for instance, they could be monetized and used within LCC in the economic dimension, or they could be considered as stand-alone criteria within other dimensions. The DMs decide to neglect the elusive price implications of increased competition for resources and fuel, but to consider increased competition for agricultural land and potential price increases in food under the social dimension. The positive externality of grid balancing is considered as a separate criterion within the technical dimension, whereas the economic impact of the by-product of biodiesel production is assumed to be included in the market price of biodiesel.

4.2. Develop Relevant Criteria under the Evaluation Dimensions

The criteria selection process is presented in the following sub-sections by each sustainability dimension, as described in Section 3.2.

4.2.1. Environmental Dimension

The environmental criteria are established for a well-to-wheel perspective based on the most relevant Recipe life cycle impact assessment methods (

Table 4. Environmental criteria for a well-to-wheel analysis.

	Feedstock Production	Fuel Production	Operational Phase
Climate change GHG emissions (CO2, CH4, N2O)	GHG emitted by production of biomass (fertilizers, pesticides, herbicides, energy for cultivation)	GHG set free during production of fuel and for construction of fuel conversion facilities	GHG set free during operation of vehicle
Terrestrial acidification	NOx and SO2 (acid rain) [45], NH3 (from fertilizers) [46]	NOx SO2	NOx SO2
Eutrophication	Phosphor (P) and nitrogen from fertilizing [47]	NOx	NOx
Photochemical oxidant formation	NOx (from fertilizers) [47] Volatile Organic Compounds (VOC) [48]	NOx VOC CH4 [45]	NOx VOC CO [45]
Ozone depletion	N2O (especially from agricultural cultivation)	N2O	N2O (especially cars with catalyst [44]); VOC [48]
Particulate matter formation	From fertilizer (NH3—ammonia) [49] From burning fuel (for cultivation): VOC [48] and NOx and SOx [50]	From fuel combustion (for fuel production): VOC [48] and NOx and SOx [50]	From fuel combustion: VOC [48] and NOx and SOx [50]
Agricultural land occupation	Land used for feedstock production	Land used for additional production facilities (biogas, biodiesel, electrolyzer)	n.a.

).

The breakdown in Table 4 helps the DMs to decide whether to rely on the inventory or the midpoint impact level and which unit processes to include in the process. It shows that CO₂, CH₄, N₂O, NO_x, VOC, SO_x, NH₃, and phosphor are relevant emissions under the considered impact categories. To simplify the analysis, the DMs decide to rely on the inventory level and the emissions that are considered to have most impact, i.e., GHG, NO_x, and SO_x emissions along the well-to-wheel life cycle. Moreover, the land used for feedstock production is considered for the analysis. Taking the vehicle manufacturing process into consideration would make other criteria like water pollution or solid waste more important in the analysis.

4.2.2. Economic Dimension

Market prices of fuels at the refueling station represent the life cycle costs of fuels. They contain the feedstock and production costs and implicitly the costs of the refueling infrastructure as well as the taxes and subsidies along the production cycle. The economic life cycle benefit generated by the different alternatives is the additional economic value from feedstock and fuel production on the island. This example illustrates the inherent contradiction in LCC analysis between user goals (minimization of cost) and social goals (maximization of value added) [51]: on the one hand, fuel costs should be minimized, but on the other hand, economic value added from feedstock and fuel production on the island should be maximized. The relevant components of LCC are summarized in

Table 5. Life cycle costs and benefits, subsidies and taxes under the relevant unit processes.

	Feedstock Production	Fuel Production	Refueling Infrastructure	Operational Phase
Life cycle costs	Life cycle costs of fuels based on market prices			Life cycle costs of vehicles
Life cycle benefits	Additional economic value generated on the island by feedstock production	Additional economic value generated on the island by fuel production	n.a. (assumed to be negligible)
Subsidies	Subsidies for feedstock production	Subsidies for fuel production facilities	Subsidies for refueling stations	Subsidies for purchase of vehicles
Taxes	Taxes paid by feedstock producer	Taxes paid by fuel producer	Taxes paid by refueling infrastructure	Taxes paid for purchase of vehicle and during operational phase of vehicle (by vehicle owner)

along the unit processes.

The DMs decide to rely on market prices for fuels and vehicles when calculating LCC. Taxes and subsidies related to buying and owning a vehicle, however, are to be explicitly considered in the LCC as they are significantly different between the alternatives and not contained in the market prices. Thus, the final economic criterion is the present value of the future discounted life cycle costs of the vehicles and fuels at market prices and the subsidies received and the taxes paid by the vehicle owners.

4.2.3. Social Dimension

Social criteria are developed along the unit processes for the stakeholder groups Consumers, Local communities, Society, and Value chain actors [19] and the respective criteria categories, as listed in

Table 6. Social criteria sorted by stakeholder group and criteria categories along the unit process.

		Feedstock Production	Fuel Production	Refueling Infrastructure	Operational Phase
Consumers	Safety issues	n.a.	n.a.	Flammability during refueling	Flammability of vehicles (e.g., batteries in EVs) [52]
	Social acceptability	Acceptance of feedstock (increased competition for land, food vs. fuel) [53]	n.a.	n.a.	Acceptance of vehicle technology [54]
	Sense of comfort	n.a.	n.a.	n.a.	Comfort of vehicle
Local community	Access to resources (food security)	Food vs. fuel: agricultural feedstock in competition to food production	n.a.	n.a.	n.a.
	Local employment	Additional employment by feedstock production	Additional employment by fuel production	n.a.	n.a.
Society	Resource and energy security	Availability of feedstock	Availability/reliability of production facilities	n.a.	Availability of vehicles
	Noise level	n.a.	n.a.	n.a.	Noise level of vehicle
Value chain actors	Fair competition (level playing field)	Regulation of feedstock production	Regulation of fuel production	Regulation of refueling stations	Environmental regulations, quota for specific vehicles, regulations on specific fuels (e.g., blending rates or emission regulations)

. The group Workers is not specifically considered as working conditions on the island are assumed to be fair and regulated whereas working conditions for workers outside the island are excluded from the analysis.

From the decomposition in Table 6, the DMs make the following decisions about the evaluation criteria:

Safety issues as well as Sense of comfort are performing similarly for all alternatives and are therefore not considered in the analysis.

Social acceptability consists of the acceptance of the feedstock as well as the acceptance of the respective vehicle technology. The acceptance of the feedstock is assumed to cover the negative externality arising from the increased competition and potentially elusive price implications on food. The DMs consider the acceptance of the feedstock and of the respective vehicle technology as related and decide to consider them as one criterion called Social acceptability of feedstock and vehicle. The criterion Access to resources (food security) is not considered in order to avoid double counting with the acceptance of the feedstock.

Local employment measures the net effect on the employment on the island arising from feedstock and fuel production and is taken into account as a criterion.

Resource and energy security comprises the availability of the feedstock, the reliability of the production facilities, and the availability of the respective vehicles. Pending on data availability, each of these factors could be considered as separate criteria. DMs in this example application decide to aggregate them to the criterion Resource and energy security.

The noise levels especially of electric and combustion engines are quite different. Noise level is thus taken into account.

The category Fair competition is defined by the respective regulations for the feedstock and fuel production, for the refueling stations as well as fuel- and vehicle-specific regulations. The DMs regard the regulations for the fuel production as the most important factor and take them into account in the analysis.

4.2.4. Technical Dimension

Table 7. Technical criteria categories developed along the unit processes.

	Feedstock Production	Fuel Production	Refueling Infrastructure	Operational Phase
Energy efficiency	Energy needed for feedstock production	Energy needed for fuel production	Loss during storage and refueling/charging [55]	Efficiency in operation, e.g., tank-to-wheel efficiency [30]
Fuel range	n.a.	n.a.	Density of refueling station network	Mileage per tank/fuel range
Specific fuel characteristics	n.a.	n.a.	Storage properties	Specific fuel properties in operation (e.g., operability in low temperature)
Time needed for implementation	Time needed to grow sufficient feedstock	Time needed to construct required production facilities	Time needed to build sufficient refueling stations	Time needed to replace old and acquire new vehicles
Balancing effect (positive externality)	n.a.	n.a.	n.a.	Use of fuels for grid balancing

shows the technical criteria categories considered by the analysts and developed along the unit processes.

Again, some of the criteria categories can be aggregated along the unit processes to represent one criterion. Energy efficiency measures the energy needed along the entire value chain from feedstock and fuel production and distribution, to refueling and tank-to-wheel efficiency. Furthermore, the Time needed for implementation of the alternative is supposed to include the time needed to ramp up feedstock and fuel production, as well as the time needed for constructing sufficient refueling stations, and replacing the vehicle fleet. Further, the Mileage per tank or Fuel range is considered important as well as specific fuel properties like the Operability in low temperature conditions, whereas Storage properties are deemed a minor issue and are not taken into account. Finally, the Use of fuels for grid balancing is taken into account as a positive externality.

4.3. Results and Discussion

The resulting evaluation criteria selected by the DMs in this exemplary application are summarized in

Table 8. Final evaluation criteria by dimension.

Environmental	Economic	Social	Technical
GHG NOx SOx Land use	Net Present Value of life cycle costs of the vehicles life cycle fuel costs at market prices subsidies received by vehicle owners taxes paid by vehicles owners	Social acceptability of feedstock and vehicle Net effect on employment Resource and energy security Noise level of vehicles Regulation for fuel production	Energy efficiency Fuel range Operability in low temperatures Time needed for implementation Use of fuel for grid balancing

. The aim of the exemplary application is to illustrate how evaluation criteria can be selected based on the methodological framework. The next steps in an actual MCDA evaluation would be to collect and normalize the data for all criteria and alternatives within the specified context. This step does not form part of the discussion in this paper. Potential problems arising from data availability could be remedied by the advantageous property of MCDA to process both quantitative and qualitative data [1]. If quantitative data is not available or cannot be collected, MCDA offers the option to use qualitative assessments by experts or stakeholders.

This exemplary application shows how the developed methodological framework helps to structure the criteria selection process and distinguish relevant from irrelevant criteria. It is important to note that the definition of the context of the analysis has a huge impact on the outcome of the analysis. For example, widening the system boundary to include the impacts from vehicle production might have a significant impact on the criteria selection and, hence, the final study results. The exemplary case study was designed in a way to highlight the discrepancy between a local focus of the assumed DMs and a more holistic sustainability point of view. It could be interesting to enrich the analysis of selecting RTFV alternatives by showing the results obtained under different contexts and a resulting different set of evaluation criteria. Such scenarios might increase the awareness of DMs for the overall impact of their decision making and how a different scope could impact their decision. In this way, it could finally lead to more sustainable results.

At the same time, since the framework outlined above is based on LCSA, it suffers from the same problems as LCSA, in particular its insufficient standardization. More research work is required on a standardized pool of relevant criteria especially for the social dimension. Similarly, LCSA is mainly focused on negative impacts and the consideration of more positive criteria could enrich the analysis [31,41].

The methodological framework as developed above is aimed at the evaluation of RTFV. Other energy related problems have a similar scope of analysis and should also incorporate sustainability considerations [25,29]. Thus, the methodological framework can potentially be modified in order to be used beyond RTFV for decision making on alternative sustainable energy systems in general. In particular, the main steps of the framework could provide the baseline for other sector analyses.

5. Conclusions

In this paper, a methodological framework is developed to systematically identify and select relevant evaluation criteria for the assessment of RTFV with MCDA methods under consideration of LCSA-based sustainability aspects. Environmental, economic, and social criteria are complemented with a technical pillar and the consideration of economic externalities, thereby enlarging the scope of the analysis. The first part of the framework requires the analyst to clearly define the context of the analysis and to select only relevant criteria based on the specific problem-definition. The context of the analysis is set by clearly defining the DMs, their interests and evaluation goals, the regional background, all relevant alternatives, the system boundaries, the functional unit, and the unit processes. The second part is to decompose the problem by developing criteria categories along the relevant life cycle for each of the evaluation dimensions. This allows the DMs to easily determine the relevance of the criteria and to select the criteria most appropriate for the evaluation. In this way a standardized, predefined and comparable criteria selection process increases the transparency and comparability of studies, even though each study might still be evaluated against a different set of final evaluation criteria. Going through this structured and detailed process of criteria selection also sharpens the problem awareness of all stakeholders involved in the decision-making process. An exemplary application of the framework is provided to highlight its practical aspects and demonstrate its applicability. The developed methodological framework is expected to support analysts and DMs to better understand and structure the decision-making process and make more informed decisions.