Uncertainty of Preferences in the Assessment of Supply Chain Management Systems Using the PROMETHEE Method

Abstract

The use of Supply Chain Management (SCM) systems allows for the improvement of an organization’s operations. Companies use many Enterprise Resource Planning (ERP) systems that also include SCM functionalities. As a result, the selection of the right system to be used in the enterprise is a complex problem. The use of multi-criteria decision aid (MCDA) methods provides the possibility of system ordering in a ranking, based on an asymmetric preference relation, symmetric indifference and incomparability relations. The aim of the article is to evaluate ERP systems in terms of their support for SCM. The scientific contribution of the article is the study of the impact of various degrees of uncertainty of the decision-maker’s preferences on the evaluation results and the analysis of the impact of various approaches to the preferences of alternatives on the final ranking. An approach based on MCDA Preference Ranking Organization Method for Enrichment Evaluation (PROMETHEE) was used for the study. The decision model uses 12 criteria, three different preference functions and two variants of the PROMETHEE method (I and II). In this way, a total of six rankings were built, and each of them includes seven systems, supporting the management of the supply chain. As a result of the study, it was found that the highest functionality in logistics is characterized by the Oracle E-Business Suite system, which is more functional than SAP ERP and JD Edwards EnterpriseOne. The remaining analysed systems offer much less functionality. The applied approach, which was possible with the use of various preference functions, allowed three different levels of uncertainty in the preferences of decision-makers to be taken into account in the study. Moreover, the application of two different variants of the PROMETHEE method made it possible for the obtained solution to take into account the uncertainty of positions taken by individual ERP systems in the final rankings.

1. Introduction

In recent years, issues related to logistics and the application of information technology (IT) have become the basic factors determining the success of an organization. Currently, no company is able to compete on the market without efficient logistics facilities, usually supported by an information system for management purposes. The use of the information system is aimed at broadly understood improvement in the organization’s operation by supporting the management of knowledge and information. Knowledge and information are of particular importance for logistics and management, allowing for the implementation of such management functions as: planning, organizing, leading and controlling [1]. The use of IT is especially important now, when more and more enterprises operate on a global scale and a large part of their resources are dispersed. Thus, the integration of all aspects of knowledge about the operation of an organization without the use of IT is very difficult or even impossible.

Enterprises operate on the market by cooperating with each other in creating an extensive supply network [2]. Their goal is to carry out the flow of goods as efficiently as possible in order to meet the customer’s needs. Therefore, software for Supply Chain Management (SCM) is of great importance. IT, and in particular, the relevant software, significantly influences the SCM structure of any organization, as it can integrate many procedures internally and also greatly assist the integration with suppliers and customers. IT improves the communication between the parties to the SCM process, the acquisition and transfer of data, allowing for effective policy making and, thus, improving the overall performance of the SCM [3]. SCM software is used to integrate the links of the supply chain of a single company and its partners into one coherent and stable entity, which is more competitive and flexible due to the combination of forces. Choosing the right SCM software is a difficult and complex decision-making problem due to the existence of many SCM software packages, with a wide range of functionalities. An additional problem is the decision-maker’s limited knowledge of the possibilities offered by individual software. On the other hand, the multiplicity of evaluation criteria and the contradictions between them require the use of a multi-criteria approach when selecting the SCM system. It is important that in many enterprises, the mosaic of various management systems is replaced by the Enterprise Resource Planning (ERP) system, which has become a standard in business. Dedicated SCM systems are, therefore, largely replaced by ERP, which also integrate SCM functionalities [4]. Since ERP systems have never been designed to support SCM only, it is important to choose an ERP system that can handle SCM function satisfactorily.

The aim of this article is to research and evaluate ERP systems in terms of the implementation of the SCM functions and to identify the system that best deals with supply chains, and at the same time, universal and useful for companies operating on the Polish market. For this reason, only systems available on the Polish software market and adapted to the provisions of Polish law were considered. As expert judgement is usually subject to uncertainty, and so it should be carried out using methods that take into account the complexity of the assessment, conflicting criteria, different scenarios, preferences of decision makers, sources of uncertainty, etc. Multi-criteria decision aid (MCDA) methods can capture all these aspects of the assessment [5]. Therefore, an approach based on MCDA was used to study the systems, or more precisely, the Preference Ranking Organization Method for Enrichment Evaluation (PROMETHEE) was used. The use of PROMETHEE allows the ordering of the systems into a partial or full ranking. The full ranking is based on an asymmetric preference relation and a symmetric indifference relation, while the partial ranking also uses a symmetric incomparability relation. The uncertainty of recommendations was captured by the use of preference functions characterized by a different degree of uncertainty of the decision-maker preferences and by using two variants of the PROMETHEE method, differently relating to the uncertainty of preferences between the alternatives in the final ranking. The study of the impact of various degrees of uncertainty in the decision-maker’s preferences on the evaluation results and the analysis of the impact of various approaches to the preferences of alternatives on the final ranking constitute the scientific contribution of the article. The practical contribution is the analysis of ERP systems and the recommendation of the system that works best with SCM and supports Internet and communication technologies. The structure of the article is as follows. Section 2 provides an overview of the literature on the evaluation of ERP systems and other systems supporting the functioning and management of organizations. Section 3 considers the criteria for the evaluation of SCM systems and presents the calculation procedure used in the PROMETHEE method. Section 4 contains the research results in the form of rankings obtained using the various parameters of the PROMETHEE method. Section 5 compares the results of the assessment with the results of applying the AHP method. The Section 6 focuses on the summary of the results and conclusions.

2. Literature Review

The literature relatively often deals with the issue of multi-criteria analysis of systems supporting the functioning of enterprises and other organizations. Among the studies using MCDA methods, there are analyses that directly refer to ERP systems, as well as studies on systems supporting a narrower range of functions, such as SCM, Customer Relationship Management (CRM), etc. Wei et al. defined a framework for selecting an ERP system based on the AHP method, and then selected an ERP system for an electronic company [6]. Cebeci applied the Fuzzy AHP method to the problem of choosing an ERP system for textile companies [7]. Gürbüz et al. proposed an ERP system assessment framework based on ANP, Choquet Integral (CI) and Measuring Attractiveness by a Categorical-Based Evaluation Technique (MACBETH) [8]. Baki and Cakar did not use any advanced MCDA method but used a simple arithmetic mean to determine the validity of the ERP system selection criteria [9]. On the other hand, Salmeron and Lopez used the AHP method to assess the risk factors of ERP systems maintenance [10]. Razmi et al. proposed a framework for fuzzy assessment of the company’s readiness to implement an ERP system. The framework is based on the ANP method [11]. Xu applied the AHP method to ERP assessment (weighting) of criteria and sub-criteria in the problem of ERP sandtable simulation [12]. Parthasarathy and Sharma also used the AHP method to identify the needs of ERP system and enterprise organization customization in order to align the enterprise structure and ERP system [13]. Büyüközkan and Güler assessed the analytical tools supporting SCM [14]. Deepu and Ravi analysed and assessed inter-organizational information systems for an electronic supply chain [15]. Gaur et al. proposed to use the AHP method to assist managers in making decisions about the closed-loop SCM problem [16]. Dev et al. used the Fuzzy ANP and Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) methods to analyse the operational factors of the supply chain [17]. Büyüközkan et al. presented the application of MCDA methods in the problem of choosing CRM business partners. The authors used the methods of Intuitionistic Fuzzy Decision Making Trial and Evaluation Laboratory (IF-DEMATEL) and Intuitionistic Fuzzy ANP (IF-ANP) [18]. On the other hand, Öztaysi et al. presented an assessment on the efficiency of CRM systems in enterprises using the ANP method [19]. Similarly, Meira et al. proposed the AHP method to select an appropriate Computerised Maintenance Management System (CMMS) meeting the needs of the organization [20]. Yang et al. presented the possibility of choosing the optimal Smart Healthcare Management System (SHMS) using the following methods: DEMATEL, ANP and Zero-One Goal Programming (ZOGP) [21]. These studies are characterized in

Table 1. Applications of MCDA methods in research related to systems supporting the work of an organization.

Research Goal	MCDA Method	Approach to Uncertainty	Reference
Choosing a comprehensive ERP system with a defined selection framework	AHP	Group assessment	[6]
Choosing the right ERP system for the textile industry	Fuzzy AHP	Triangular fuzzy numbers	[7]
Choosing the right ERP system to meet the company’s requirements	ANP, CI, MACBETH	Fuzzy measures integrated in CI	[8]
Determining the importance of the criteria for ERP system selection	Likert scale, Arithmetic mean	-	[9]
Investigate ERP maintenance risk factors	AHP	-	[10]
Measuring the company’s readiness to implement an ERP system	Fuzzy ANP	Triangular fuzzy numbers	[11]
ERP sandtable simulation evaluation	AHP	-	[12]
Identify feasible customization choices for the ERP implementing	AHP	-	[13]
Evaluation and selection of the most appropriate SCA tool in logistics	HFL AHP, HFL MULTIMOORA, HFL VIKOR	Hesitant fuzzy linguistic term sets	[14]
Choosing the best IOIS alternative in an electronic supply chain	AHP, TOPSIS	-	[15]
Help supply chain managers with improved decision making for closed loop SCM	AHP	-	[16]
Analyzing the big data on operational factors of the SCM	Fuzzy ANP, TOPSIS	Triangular fuzzy numbers, Sensitivity analysis	[17]
Evaluating CRM partner selection	IF-DEMATEL, IF-ANP	Group assessment, Intuitionistic fuzzy sets	[18]
CRM performance evaluation	ANP	Sensitivity analysis	[19]
Choosing the right CMMS to meet the needs of the organization	AHP	-	[20]
Choosing the optimal SHMS system	DEMATEL, ANP, ZOGP	-	[21]

ERP—Enterprise Resource Planning; SCA—Supply Chain Analytics; IOIS—Inter-Organizational Information System; SCM—Supply Chain Management; CRM—Customer Relationship Management; CMMS—Computerized Maintenance Management System; SHMS—Smart Healthcare Management System; AHP—Analytic Hierarchy Process; ANP—Analytic Network Process; CI—Choquet Integral; MACBETH—Measuring Attractiveness by a Categorical-Based Evaluation Technique; HFL—Hesitant Fuzzy Linguistic; MULTIMOORA—Multi-Objective Optimization By Ratio Analysis; VIKOR—ViseKriterijumska Optimizacija I Kompromisno Resenje; TOPSIS—Technique for Order Preference by Similarity to Ideal Solution; IF—Intuitionistic Fuzzy; DEMATEL—Decision-Making Trial and Evaluation Laboratory; ZOGP—Zero-One Goal Programming.

.

The analysis in Table 1 shows that in a large part of the research [9,10,12,13,15,16,20,21], the uncertainty related to the evaluation of the systems is not taken into account. In some articles cited in Table 1 [7,8,11,14,17,18], the uncertainty of weights and alternatives was captured using fuzzy numbers and measures. In these studies, fuzzy sets make it possible to indicate a range instead of a single value, and this allows one to indicate the area of fluctuations, uncertainty, and imprecision of the input data. In two articles [17,19], sensitivity analysis was used, which makes it possible to examine the uncertainty of the solution in terms of variable weights in the criteria. In this way, various variants of solutions are examined, obtained depending on the weights of the criteria specified at the entrance to the decision problem. Further, in two articles [6,18] group assessment was used, which allows one to objectify the results to some extent. Group assessment reduces the imprecision of assessments in the case of non-measurable data by averaging them based on the opinions of many experts, aggregated into a single value.

Preference uncertainty was not taken into account in any of the cited studies. This is a research gap in management system evaluation and an interesting area of research into the uncertainty and imprecision of evaluations. The uncertainty of preferences lies in the fact that the decision-maker is not able to unequivocally determine whether the differences between the alternatives, in terms of a given criterion, make one of the alternatives unquestionably better than the other. The uncertainty of preferences reflects the zones of uncertainty, imprecision, indecision, as well as conflicts and contradictions in the mind of the decision-maker [22]. Indifference (q) and preference (p) thresholds are used to describe the uncertain preferences of the decision-maker. These thresholds make it possible to distinguish the relationships: weak preference, strict preference and indifference [23]. The preference function using both thresholds is presented in Figure 1. The q and p thresholds allow for the imprecision, uncertainty or ill determination of preferences [24], making the relations of preferences fuzzy and, thus, expressing uncertainty [25].

The outranking relations methods, such as ELECTRE and PROMETHEE, are able to capture preference uncertainty precisely through the use of q and p thresholds. With the PROMETHEE method, the results obtained with different degrees of uncertainty can be directly compared. It is possible due to the fact that PROMETHEE is more universal [26] and enables the use of several preference functions, differing in the degree of uncertainty [27]. Moreover, in contrast to ELECTRE, the PROMETHEE method does not use a veto threshold [28], has a clearer calculation procedure that is easier to understand for a decision-maker [29] and gives more stable results [30]. These are the main reasons for using the PROMETHEE method in this study.

3. Materials and Methods

3.1. Selection of Evaluation Criteria for the Needs of SCM Systems

When analysing the criteria used in the assessment of ERP systems, the primarily focused on the articles listed in Table 1 [6,7,8,9]. In these studies, the criteria were usually divided into at least two groups, i.e., software and vendor criteria. Additionally, Cebeci [7] distinguished a group of criteria related to financial and time costs, and Gürbüz et al. [8] distinguished criteria related to the customer. On the other hand, Baki and Cakar [9] did not divide the criteria into groups. The criteria used in these studies are presented in

Table 2. Criteria used in research on the selection of an ERP system.

Criterion	Group of Criteria	Reference
Total costs/Cost	SSF/IF/SRC	[6,7,8,9]
Implementation time/Implementation	SSF/IF/SRC	[6,7,8,9]
Functionality	SSF/SC/SRC	[6,7,8,9]
User friendliness	SSF/SC	[6,7]
Flexibility/Ease in customizing the system (Flexibility)/Ease of customization	SSF/SC/CRC	[6,7,8,9]
Reliability/System reliability	SSF/SC/SRC	[6,7,8,9]
Reputation/Vendor reputation	VF/VC/VRC	[6,7,8]
Technical capability/R&D capability/Technical aspects	VF/VC/SRC	[6,7,8,9]
Service/After sales service (Consultancy services)/Support and service	VF/VC/VRC	[6,7,8,9]
Better fit with company’s business processes	SC	[7]
Ability for upgrade in-house	SC	[7]
Compatibility with other systems/Compatibility	SC/SRC	[7,8,9]
Terms and period of guarantee	VC	[7]
Vision	VRC	[8,9]
Market position/Market position of the vendor	VRC	[8,9]
Domain knowledge/Domain knowledge of the vendor	VRC	[8,9]
Methodology of software	VRC	[8,9]
Better fit with organizational structure	CRC	[8,9]
Fit with parent/allied organizational system	CRC	[8,9]
Cross module integration	CRC	[8,9]
References of the vendor		[9]
Consultancy		[9]

SSF—System Software Factors, VF—Vendor Factors [6]; IF—Investment Factors, SC—System Characteristics, VC—Vendor Criteria [7]; SRC—Software-Related Criteria, CRC—Customer-Related Criteria, VRC—Vendor-Related Criteria [8].

.

According to the literature, functionality is defined as the most important criterion for evaluating ERP systems [8]. Confirmation of this fact can be found, for example, in the articles by Wei et al. [6] and Cebeci [7], in which the functionality criterion was given the highest importance, by far dominating the other criteria related to the evaluation of ERP systems. Further, in other studies, functionality, also referred to as capabilities, is considered one of the most important criteria for assessing various types of systems [9,15,20]. In the context of ERP systems, it is considered that the solution should have enough modules related to the basic activities of the company [8]. If the system implemented in the organization has insufficient functionalities, further adjustments and improvements to the functionality of the ERP system may be required. Such constant changes and improvements can negatively affect the stability of the system, increasing the initial budget and planned implementation time [10]. On the other hand, the functionality and reliability of an ERP system largely depends on the degree of customization; therefore, personalization is also one of the most important factors for a successful system implementation [13]. Based on the above analysis, it was decided that in this study the basic criteria for assessing ERP systems performing SCM functions should refer to the functionality and personalization of the systems.

The primary task of the SCM system is integration between key supply chain processes such as planning, procurement, production, distribution and retail. These key processes are aimed at developing, manufacturing and delivering a product to meet market demand [31]. SCM software is a complex information system that supports a number of areas of logistics chain management. The literature distinguishes 10 basic SCM functions [32]:

• Supplier management—a strategic method that allows companies to plan, manage and enhance their relationships with suppliers.
• Purchasing management—a business activity that allows businesses to manage the actions and relations that constitute the purchasing functions.
• Order management—a business process that entails receiving, tracking and completing customer orders.
• Customer relationship management—the strategies, methods and tools that businesses employ to satisfy, keep and acquire customers.
• Warehouse/inventory management—a variety of business tasks, such as predicting, ordering, receiving and allocating goods.
• Handling—transportation, protection and storage of materials and products during the production, warehousing, distribution stages.
• Transportation—transfer of commodities and goods from one point to another.
• Packaging—a process of preparing, enclosing and protecting products for distribution, storage, sale and usage.
• Insuring—a contract that protects resources and goods from a variety of dangers they may face during storage, production and transportation.
• Inspection and customs clearance—the process of checking and passing goods and products through customs at the point of entry or exit from a country.

Modern logistics systems should also enable the use of the Electronic Data Interchange (EDI) [33] standard and ensure automatic identification of goods by: adapting to the requirements of GS1 [34], supporting radio frequency identification (RFID) technology and barcodes [35].

Based on the presented requirements for ERP and SCM systems, criteria were defined for the evaluation of ERP systems in terms of supporting the management of the supply chain. They are arranged in a hierarchical structure of criteria and groups. The structure of the criteria is presented in

Table 3. The structure of the criteria for the evaluation of SCM systems.

Criterion	Name	Functionality in the Criterion	Reference
C1	SCM functions
C1.1	Distribution	links between plants, wholesalers and customers; replenishment planning in related entities and across the network; supply chain service; bar codes; possibility to use GS1	[14,32,34,35]
C1.2	Distribution networks	managing contacts with SRM suppliers; cooperation with CRM customers; transport management; renovation economy; quality control; advanced APS planning and scheduling	[14,15,32]
C1.3	Trade	support for POS points of sale; handling returns; handling of returnable packaging; handling sales and settlement procedures in accordance with Polish tax regulations; technical service as well as warranty and post-warranty service of products; use of bar codes; use of RFID	[14,15,32,35]
C1.4	International company service	multilingualism; multi-currency; a uniform labelling system for goods	[4,6]
C1.5	Customer relationship management	own database; access via own website; planning, supervision and evaluation of marketing campaigns; collection of marketing data; collecting data in a database about customers, potential customers and markets; correspondence service; issuing commercial documentation	[32]
C2	Internet and communication
C2.1	Use of the Internet and electronic commerce	own website; B2B and B2C cooperation	[4,6]
C2.2	Electronic information exchange	Polish version; foreign language version; access to and application of Internet techniques; use of the XML format	[4,6,14,33]
C2.3	Service processing model	remote work; compiling software from components from different SOA suppliers; work with software made available in the ASP mode; IT service by an external unit—SAAS; Cloud Computing	[6,14]
C3	Versatility
C3.1	Categories of supported enterprises	small, medium, large	[6,8,9]
C3.2	Support for the specific requirements of various industry categories	heavy; automotive—final production; automotive—manufacturing and delivery of components; electromechanical; production of building and ceramic material; precise; electronic; food; chemical; pharmaceutical; light; furniture; other	[6,8,9]
C4	Personalization and polonization
C4.1	Personalization	CASE; program modification; workflow; adaptation to GS1 requirements; personalization of screens; automation of data import to the system; other	[9,13]
C4.2	Polonization	documentation; assistance; screens and printouts; instructions and implementation procedures	[9,13]

.

When analysing the functionalities included in individual criteria, it is easy to notice that some functionalities are related to the adaptation of systems to the Polish market. Particularly, it is about adjusting the systems to the Polish legal environment and tax regulations, as well as the Polonization of the system and its documentation.

The criteria describing the ease of use, interface and similar subjective criteria were not considered. Instead, criteria related to software functionality were used. Therefore, the assessment is more objective, as there is no room for opinions or qualitative assessments. The individual criteria and related ratings inform about how many and what specific functionalities the SCM software has within the criterion, e.g., whether the Polish version is available for the ‘Polonization’ criterion: documentation, assistance, screens and printouts, as well as implementation instructions and procedures. These criteria were used as the basis for the evaluation of ERP/SCM software using the PROMETHEE method.

3.2. The PROMETHEE Method

The PROMETHEE method is a popular MCDA method that employs pairwise comparison and outranking flows to produce a ranking of decision alternatives [36]. PROMETHEE considers a finite set of alternatives $A=\left\{a,b,\dots ,m\right\}$ with M alternatives considered in terms of $n$ criteria belonging to a set $C=\left\{c_1,c_2,\dots ,c_n\right\}$. The PROMETHEE method consists of 5 steps [37]. These steps are presented in Figure 2.

• Determination of deviations based on pair-wise comparisons according to Formula (1):

  $$d_j\left(a,b\right)=c_j\left(a\right)-c_j\left(b\right)$$

  (1)

  where $d_j\left(a,b\right)$ denotes the difference between the evaluations of a and b on each criterion.
• Application of the preference function using the Formula (2):

  $$P_j\left(a,b\right)=F_j\left[d_j\left(a,b\right)\right]$$

  (2)

  where $P_j\left(a,b\right)$ denotes the preference of alternative a with regard to alternative b on each criterion, as a function F of $d_j\left(a,b\right)$ [38]. The preference functions represented by F are described by the following Formulas:
  • Usual (true) Criterion (3):

    $$P_j\left(a,b\right)=\{\begin{array}{ll}0& for d_j\left(a,b\right)\le 0\\ 1& for d_j\left(a,b\right)>0\end{array}$$

    (3)
  • U-shape Criterion (semi-criterion) (4):

    $$P_j\left(a,b\right)=\{\begin{array}{ll}0& for d_j\left(a,b\right)\le q_j\\ 1& for d_j\left(a,b\right)>q_j\end{array}$$

    (4)
  • V-shape Criterion (pre-criterion) (5):

    $$P_j\left(a,b\right)=\{\begin{array}{ll}0& for d_j\left(a,b\right)\le 0\\ \frac{d_j\left(a,b\right)}{p_j}& for 0<d_j\left(a,b\right)\le p_j\\ 1& for d_j\left(a,b\right)>p_j\end{array}$$

    (5)
  • Level Criterion (6):

    $$P_j\left(a,b\right)=\{\begin{array}{ll}0& for d_j\left(a,b\right)\le q_j\\ \frac{1}{2}& for q_j<d_j\left(a,b\right)\le p_j\\ 1& for d_j\left(a,b\right)>p_j\end{array}$$

    (6)
  • V-shape with indifference Criterion (pseudo-criterion) (7):

    $$P_j\left(a,b\right)=\{\begin{array}{ll}0& for d_j\left(a,b\right)\le q_j\\ \frac{d_j\left(a,b\right)-q_j}{p_j-q_j}& for q_j<d_j\left(a,b\right)\le p_j\\ 1& for d_j\left(a,b\right)>p_j\end{array}$$

    (7)
  • Gaussian Criterion (8):

    $$P_j\left(a,b\right)=\{\begin{array}{ll}0& for d_j\left(a,b\right)\le 0 \\ 1-exp\left(\frac{-d_j{\left(a,b\right)}^2}{2r_j{}^2}\right)& for d_j\left(a,b\right)>0\end{array}$$

    (8)

    where q, p, s are: q—indifference threshold, p—preference threshold, s—gaussian threshold, respectively [27].
• Calculation of an overall or global preference index based on Formula (9):

  $$\pi \left(a,b\right)={\displaystyle \sum }_{j=1}^n{P}_j\left(a,b\right)w_j$$

  (9)

  where $\pi \left(a,b\right)$ of a over b (from 0 to 1) is defined as the weighted sum $p\left(a,b\right)$ for each criterion, and $w_j$ is the weight associated with j-th criterion.
• Calculation of outranking flows (the PROMETHEE I partial ranking) according to Formulas (10) and (11):

  $${\varphi }^{+}\left(a\right)=\frac{{{\displaystyle \sum }}_{i=1}^M\pi \left(a,b_i\right)}{M-1}$$

  (10)

  $${\varphi }^{-}\left(a\right)=\frac{{{\displaystyle \sum }}_{i=1}^M\pi \left(b_i,a\right)}{M-1}$$

  (11)

  where ${\varphi }^{+}\left(a\right)$ and ${\varphi }^{-}\left(a\right)$ denote the positive outranking flow and negative outranking flow for each alternative, respectively. Partial ranking is constructed using the following rules:
  • alternatives $a$ and $b$ are indifferent when ${\varphi }^{+}\left(a\right)={\varphi }^{+}\left(b\right) \wedge {\varphi }^{-}\left(a\right)={\varphi }^{-}\left(b\right)$,
  • alternatives $a$ and $b$ are incomparable when ${\varphi }^{+}\left(a\right)<{\varphi }^{+}\left(b\right) \wedge {\varphi }^{-}\left(a\right)<{\varphi }^{-}\left(b\right)$ or ${\varphi }^{+}\left(a\right)>{\varphi }^{+}\left(b\right) \wedge {\varphi }^{-}\left(a\right)>{\varphi }^{-}\left(b\right)$,
  • alternative $a$ is preferred over $b$ when ${\varphi }^{+}\left(a\right)>{\varphi }^{+}\left(b\right) \wedge {\varphi }^{-}\left(a\right)<{\varphi }^{-}\left(b\right)$ or ${\varphi }^{+}\left(a\right)>{\varphi }^{+}\left(b\right) \wedge {\varphi }^{-}\left(a\right)\le {\varphi }^{-}\left(b\right)$ or ${\varphi }^{+}\left(a\right)\ge {\varphi }^{+}\left(b\right) \wedge {\varphi }^{-}\left(a\right)<{\varphi }^{-}\left(b\right)$ [39].
• Calculation of net outranking flow (the PROMETHEE II full ranking) using Formula (12):

  $${\varphi }_{net}\left(a\right)={\varphi }^{+}\left(a\right)-{\varphi }^{-}\left(a\right)$$

  (12)

  where ${\varphi }_{net}\left(a\right)$ denotes the net outranking flow for each alternative. Full ranking is constructed using the following rules:
  • alternatives $a$ and $b$ are indifferent when ${\varphi }_{net}\left(a\right)={\varphi }_{net}\left(b\right)$,
  • alternative $a$ is preferred over $b$ when ${\varphi }_{net}\left(a\right)>{\varphi }_{net}\left(b\right)$ [39].

4. Results

The analysis covered seben ERP systems containing SCM support functions, available on the Polish software market:

• A1—Comarch ERP XL,
• A2—Epicor ERP 10,
• A3—Infor M3,
• A4—JD Edwards EnterpriseOne,
• A5—Microsoft Dynamics 365 Business Central Essentials,
• A6—Oracle E-Business Suite,
• A7—SAP Business One.

The systems were analysed in terms of 12 criteria and included in four groups. In order to determine the weights of the criteria, weights were first assigned to groups of criteria, and then local weights of the criteria were determined within each group. Therefore, the global weight of a criterion was influenced by the significance of a given group of criteria in relation to other groups and the local weight of a given criterion in relation to others within a given group [40]. More precisely, the individual (global) weight of each criterion was calculated as the product of the local weight and the group weight. When determining the weights, it was guided by the fact that the most important are the SCM functionalities provided by the software (C1). Moreover, in the study, particular emphasis was placed on Internet and communication technologies that characterize selected solutions (C2). This is due to the globalization of enterprises and the creation of virtual organizations; therefore, it is important to take into account the communication and integration solutions offered by individual systems in the assessment. Slightly less importance was assigned to the criteria of universality (C3) and software personalization (C4). However, it should be emphasized that these criteria are also important, hence, their presence in the evaluation model.

Table 4. Evaluation model of systems performing SCM functions.

Group	Group Weight	Criterion	Local Weight	Global Weight	Preference Direction	Preference Function	Indifference Threshold	Preference Threshold	Max Value
C1	0.55	C1.1	0.3	0.165	Maximum	Usual/ V-shape/ V-shape with indifference	1	2	5
		C1.2	0.3	0.165			1	3	6
		C1.3	0.2	0.11			1	3	7
		C1.4	0.1	0.055			0	1	3
		C1.5	0.1	0.055			1	3	8
C2	0.25	C2.1	0.3	0.075			0	1	2
		C2.2	0.3	0.075			1	2	4
		C2.3	0.4	0.1			1	2	5
C3	0.1	C3.1	0.3	0.03			0	1	3
		C3.2	0.7	0.07			2	4	13
C4	0.1	C4.1	0.6	0.06			1	3	7
		C4.2	0.4	0.04			1	2	4

presents the weights of the criteria and other elements of the preference model, i.e., the functions and directions of preferences, as well as the thresholds. Additionally, the maximum possible number of functionalities in a given criterion is marked.

The evaluation of alternatives is presented in

Table 5. Assessment of ERP systems performing SCM functions.

Criterion	Comarch ERP XL	Epicor ERP 10	Infor M3	JD Edwards EnterpriseOne	Microsoft Dynamics 365 Business Central Essentials	Oracle E-Business Suite	SAP Business One
C1.1	3	4	3	5	5	5	5
C1.2	5	6	6	6	4	6	6
C1.3	6	5	5	6	6	7	7
C1.4	3	3	3	3	3	3	3
C1.5	7	7	7	7	7	7	7
C2.1	2	2	2	2	2	2	2
C2.2	4	4	4	4	2	4	4
C2.3	4	5	3	2	2	5	3
C3.1	3	2	3	3	2	3	2
C3.2	10	10	6	13	13	12	12
C4.1	4	3	5	5	4	6	6
C4.2	4	2	2	4	4	3	4

. It is easy to see that all systems offer the same functionality in terms of the criteria: C1.4 (support for multinational enterprise), C1.5 (customer relationship management) and C2.1 (Internet use and e-commerce). As a result, the indicated criteria are redundant as they do not differentiate between alternatives; however, they were kept due to the need to retain the possibility of reusing the decision model.

The PROMETHEE I and PROMETHEE II rankings were generated on the basis of the evaluation model and criteria evaluation, in order to identify the ERP system that best deals with supply chains. For each of these methods, three different preference functions were used: usual criterion, V-shape criterion, and V-shape with indifference criterion.

Table 6. Results of the application of the PROMETHEE method.

Alternative	True Criterion				V-Shape Criterion				V-Shape with Indifference Criterion
	${\mathit{\varphi }}^{+}$	${\mathit{\varphi }}^{-}$	${\mathit{\varphi }}_{\mathit{n}\mathit{e}\mathit{t}}$	Rank	${\mathit{\varphi }}^{+}$	${\mathit{\varphi }}^{-}$	${\mathit{\varphi }}_{\mathit{n}\mathit{e}\mathit{t}}$	Rank	${\mathit{\varphi }}^{+}$	${\mathit{\varphi }}^{-}$	${\mathit{\varphi }}_{\mathit{n}\mathit{e}\mathit{t}}$	Rank
A1	0.200	0.432	−0.232	6	0.131	0.248	−0.117	5	0.086	0.132	−0.046	5
A2	0.218	0.362	−0.144	4	0.154	0.217	−0.063	4	0.105	0.107	−0.002	4
A3	0.146	0.403	−0.257	7	0.085	0.315	−0.230	7	0.046	0.258	−0.212	7
A4	0.310	0.140	0.170	3	0.201	0.086	0.115	3	0.138	0.050	0.088	3
A5	0.208	0.420	−0.213	5	0.136	0.295	−0.159	6	0.092	0.224	−0.132	6
A6	0.438	0.050	0.388	1	0.302	0.019	0.282	1	0.213	0.000	0.213	1
A7	0.380	0.093	0.287	2	0.238	0.068	0.171	2	0.145	0.053	0.091	2

shows the preference flows and rankings of alternatives generated by the PROMETHEE II method with the use of individual preference functions. Figure 3, Figure 4 and Figure 5 show the rankings obtained using the PROMETHEE I and PROMETHEE II methods and the given preference functions.

The results presented in Table 6 and Figure 3, Figure 4 and Figure 5 show that A6—Oracle E-Business Suite—is the best ERP system in terms of SCM functionalities. It occupies the first position in the rankings, regardless of the used preference function and the variant of the PROMETHEE method. The second position in most rankings is taken by A7—SAP Business One—although when using the V-shape with indifference criterion, A4—JD Edwards EnterpriseOne—achieves very similar performance. This is because SAP Business One has very similar functionality to JD Edwards EnterpriseOne and only slightly dominates. This advantage is so slight that, taking into account the uncertainty resulting from the use of indifference and preference thresholds, the systems rank the same. The fourth position in all rankings is taken by A2—Epicor ERP 10. When it comes to systems offering the lowest SCM functionality, the last place in the rankings is almost always taken by A3—Infor M3 system. In turn, the preceding ones, A1 and A5—Comarch ERP XL and Microsoft Dynamics 365 Business Central Essentials, respectively—depending on the preference function used, change places in the PROMETHEE II rankings, and according to the PROMETHEE I method, they are most often incomparable.

5. Discussion

The literature review (see Section 2) showed that the MCDA method most often used in the evaluation of ERP systems is AHP. Therefore, in order to verify the obtained results and compare them with the commonly used method, the presented decision problem was also solved using the AHP method [41]. It should be noted that AHP, like the true criterion in the PROMETHEE method, does not take into account the preference uncertainty and is, therefore, based on the utility theory and model with strict preference and indifference [23]. In order to reliably compare the results obtained in Section 4 with the results of the AHP method, the study used the criteria weights presented in Table 4. For the same reason, in order to calculate the priorities, the direct data from Table 5 were used as input instead of the AHP fundamental scale [40,42]. The use of direct data has the additional advantage of eliminating the problem of matrix inconsistency [43]. The results obtained using the AHP method are presented in

Table 7. Results of the application of the AHP method.

Alternative	Priority	Rank
A1	0.1352	5
A2	0.143	4
A3	0.1308	6
A4	0.1467	3
A5	0.129	7
A6	0.1627	1
A7	0.1527	2

.

A comparison of the results obtained using the PROMETHEE II and AHP methods shows that all rankings are consistent in positions 1–4. In addition, the AHP ranking coincides with the PROMETHEE II rankings with the V-shape criterion in position 5. This means that in the case under study, the AHP ranking based on certain preferences is more similar to the PROMETHEE II rankings assuming the uncertainty of preferences, than to the PROMETHEE II ranking assuming the certainty of preferences. This contradiction is not surprising and is due to the fact that the AHP and PROMETHEE methods are based on different methodological bases and, therefore, they are fundamentally different from each other. AHP is based on a utility theory and uses a single synthesizing criterion. PROMETHEE, on the other hand, is based on the outranking relation [44]. In the AHP method, the preferences of the decision-maker are expressed only through criteria weights, while in the PROMETHEE method, along with the weights, thresholds and functions are also used to provide additional information about the preferences of the decision-maker. These thresholds and preference functions allow the introduction of uncertainty and imprecision into the decision problem.

6. Conclusions

The article deals with the problem of the evaluation of ERP systems supporting supply chain management in an organization. When selecting the solutions to be assessed, the emphasis was primarily on the SCM and communication functions. This is especially important in view of the constant globalization of enterprises and cooperation between enterprises. Based on the defined criteria, seven ERP systems were assessed using three different preference functions and two variants of the PROMETHEE method, as well as the AHP method.

The proposed approach, based on the use of different preference functions and different PROMETHEE variants, allowed us to capture the uncertainty of preferences when assessing systems and the uncertainty of the position of systems in the final rankings. The use of three different preference functions allowed us to take into account different degrees of preference uncertainty between the alternatives. The true criterion assumes that preferences are certain, so even the slightest advantage of one alternative over another on a given criterion means that this alternative is more preferred on the indicated criterion [45]. In turn, indifference and preference thresholds used in the V-shape criterion allow for taking into account the sources of arbitrariness, imprecision, uncertainty, or ill determination [24]. As a result, these thresholds allow for the fuzzification of binary preference relations [25]. As for the use of PROMETHEE I and II methods, it should be emphasized that the PROMETHEE II method gives a full ranking, so the final preferences between the alternatives are relationships of a certain nature. There is no doubt about the advantages between the alternatives here. In turn, the PROMETHEE I method generates a partial ranking, allowing for doubts as to the preference of one alternative over another. These doubts are represented by the incomparability relation. Such a complex approach to the uncertainty of preferences and rankings enables the decision-maker to analyse the obtained solution in a broader manner and to take into account factors related to the imprecision or ill determination of the decision problem.

The basic limitation of the research carried out is related to taking into account only seven ERP systems, while there are many more such systems available on the market. Another limitation of the research is related to the types of uncertainties appearing in decision problems. The study took into account the uncertainty of preferences without addressing the issue of uncertainty in the data. This is due to the approach used to study the functionality of ERP systems, which consists of examining the number of functions implemented in individual systems. In this case, the data (assessing alternatives on the criteria) were precise and reliable. If the quality of individual functionalities was additionally tested, e.g., based on the subjective opinions of users, the data would be less precise and there might be a need to use fuzzy numbers, representing linguistic assessments. For example, Pythagorean fuzzy sets [46] or hesitant fuzzy sets could be used here [47,48]. Such an assessment is also an interesting direction for further research.