Next Article in Journal
Trade Volume Prediction Based on a Three-Stage Model When Arctic Sea Routes Open
Next Article in Special Issue
Cubic M-polar Fuzzy Hybrid Aggregation Operators with Dombi’s T-norm and T-conorm with Application
Previous Article in Journal
The Magnetic Hooke-Newton Transmutation in Momentum Space
Previous Article in Special Issue
Representations of a Comparison Measure between Two Fuzzy Sets
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

On the Analytic Hierarchy Process Structure in Group Decision-Making Using Incomplete Fuzzy Information with Applications

1
Department of Mathematics, COMSATS University Islamabad, Lahore Campus, Punjab 53710, Pakistan
2
Research Team on Intelligent Decision Support Systems, Department of Artificial Intelligence and Applied Mathematics, Faculty of Computer Science and Information Technology, West Pomeranian University of Technology in Szczecin, ul. Żołnierska 49, 71-210 Szczecin, Poland
3
Department of Mathematics, University of Lahore, Lahore 53710, Pakistan
4
Department of Mathematics, Virtual University of Pakistan, Lahore 54000, Pakistan
*
Author to whom correspondence should be addressed.
Submission received: 23 March 2021 / Revised: 3 April 2021 / Accepted: 5 April 2021 / Published: 6 April 2021
(This article belongs to the Special Issue Uncertain Multi-Criteria Optimization Problems)

Abstract

:
The multi-criteria decision-making (MCDM) problem has a solution whose quality can be affected by the experts’ inclinations. Under essential conditions, the fuzzy MCDM method can provide more acceptable and efficient outcomes to select the best alternatives. This work consists of a consensus-based technique for selecting and evaluating suppliers in an incomplete fuzzy preference relations (IFPRs) environment utilizing T L -transitivity (Lukasiewicz transitivity). The suggested method is developed based on the criteria of the Analytical Hierarchy Process (AHP) Fframework, and the decision matrix is construtced using consistent fuzzy preference relations (FPRs). We use the symmetrical decisional matrix approach. A variety of numerical explanations and an analysis of quantitative results illustrate the suggested methodology’s logic and effectiveness.

1. Introduction

The supply chain includes divisions responsible for developing new services and products, acquiring raw substances, transforming them into a finished form and delivering them to target consumers. The process of evaluating and selecting appropriate vendors appears to play an important role in the long-term performance and effectiveness of supply networks throughout business corporations. Consequently, a systematic and efficient strategy/method for choosing the most appropriate supplier ultimately reduces the risk of procurement, which tends to increase the number of in-time suppliers available and enhance manufacturing quality [1,2,3,4]. Swaminathan and Tayur [5] identified significant problems in conventional supply chain management (SCM) and obtained an insight into the related theoretical frameworks for use during e-business and supply chain sectors. Subsequently, works can be found involving the selection of an effective and appropriate approach for evaluating potential suppliers [6,7,8,9,10].
A well-established manufacturing company puts together a team of specialists to obtain suitable vendors to procure raw materials and important components to manufacture new products. The team of specialists consider a set of factors to evaluate the best alternatives and may change the criteria and philosophy related to different products and services. It is critical which criteria are considered to be sufficient to assess vendors in the context of decision-making problems. A review of the literature regarding the criteria used to select the best alternatives takes us back to Dickson [11], who analyzed and identified 23 criteria for selecting a supplier in 1966, including price, distribution and success experience as the most important considerations, and [12,13,14,15] contributed a great deal to strengthening this area. Different organizations that have different corporate and social histories may influence the procurement process for suppliers.
Fundamentally, the multi-criteria decision-making (MCDM) problem is associated with the supplier preference challenge in the group decision-making (GDM) framework [10,16]. The MCDM problem carries a solution whose quality can be affected by the inclinations of the experts. Under essential conditions, the fuzzy MCDM method can provide more acceptable and efficient outcomes to select the best alternatives [17,18]. Numerous methodologies have been proposed to address fuzzy MCDM [19,20,21,22,23,24,25]. All of these methods are based on comprehensive information on preferences.
Besides this, in some instances [26,27], experts can have only partial recommendation data for various factors, such as time constraints, a lack of experience or evidence or limited abilities in the problem area. In [27,28], Gong and Xu presented the least square procedure and two-goal-programming models based on incomplete fuzzy preference relations (IFPRs) to evaluate priority weights in GDM. Herrera-Viedma et al. [29] suggested an additive consistency-based recursive method to assess all unknown elements in IFPR. Then, the authors adopted a fuzzy consensus-based procedure to choose the best alternative. In [30], Alonso et al. borrowed the optimization technique given by [29] to propose the framework for estimating unknown values in various formats, including multiplicative, fuzzy, linguistic and interval valued preference relations. Furthermore, Xu deliberated on GDM with four templates of flawed pairwise comparisons [31] to produce a reliable vector of priority weights; first, related optimization techniques to translate various preference formats to FPRs were constructed by the author, and afterwards, the model parameters were obtained by addressing the defined optimization technique. The key concern with this approach is that it does not consider consensus or examine consistency. Encouraged by [29], Lee [32] developed a new GDM methodology based on additive and order consistencies using IFPRs. Later on, Chen et al. [33], presented an improved version of this method. In [34], Rehman et al. proposed a T-transitivity and order consistency-based technique to evaluate the GDM problem. Kerre et al. [35] proposed the GDM model based on multiplicative consistency using incomplete reciprocal fuzzy preference relations.
As mentioned above, several procedures to handle MCDM situations have been proposed in the literature under the condition of complete information. This inspired us to establish a multi-criteria group decision-making (MCGDM) procedure that uses the Analytical Hierarchy Process (AHP) model [36,37], which has already provided significant outcomes in a variety of domains with limited information [38,39,40]. The use of a consensus-based method consisting of several consensus stages is the best way to address GDM problems. Experts agree that several views are exchanged at a reasonable point; however, an unquestionable or full consensus is not conceivable in reality. The research presented above on GDM in an incomplete environment did not use the AHP model with consensus measure. The consensus-based MCGDM approach using the AHP model in an incomplete environment is the main novelty of this work.
This paper provides a framework for building consensus in MCGDM based on T L -consistency in the IFPR context. Since consistency has been a crucial problem to be addressed once information from experts is presented, the developed model will approximate relatively rational and consistent values for IFPRs. Transitivity is synonymous with consistency, and therefore a variety of useful types of transitivity is proposed in the FPR literature [41]. T L -transitivity—i.e., r i k max ( r i j + r j k 1 , 0 ) —is the most suitable and weakest type of transitivity used for fuzzy ordering [42]. In the first step, the missing IFPRs’ preferences are evaluated using the T L -transitivity property. The customized T L -consistent relations are constructed and maintain the degree of consistency. The degrees of importance are allocated to experts based on the weights of consistency. The suggested approach provides us with a powerful way to achieve consensus in MCGDM using T L -transitivity with IFPRs.
This paper is structured as follows. In Section 2, some of the preliminary findings used during the paper are mentioned. The recommended MCGDM process is detailed in Section 3. Numerical examples and a comparative analysis are provided in Section 4 and Section 5, respectively, to highlight the rationality and feasibility of the proposed technique. Section 6 presents some conclusions.

2. Preliminaries

In 1965, Zadeh developed the concept of fuzzy set theory [43], which shows how an entity is more or less linked with a specific group to which we want to adjust.
Definition 1
([36]). A relation H with a finite set A of alternatives characterized by function H : A × A [ 1 / 9 , 9 ] , H h i , h j = h i j , satisfying h i j · h j i = 1 for i , j { 1 , 2 , , m } is called MPR.
Definition 2
([44]). A relation R with a finite set A = { a 1 , a 2 , , a m } of alternatives characterized by mapping R : A × A [ 0 , 1 ] , satisfying: r i j + r j i = 1 (additive reciprocity) for 1 i m and 1 j m , r i j [ 0 , 1 ] shows preference degree of alternative a i over a j y b a
Remark 1
([45]). For H = ( h i j ) m × m , a related FPR R = ( r i j ) m × m is constructed as follows:
r i j = f ( h i j ) = 1 2 1 + log 9 h i j
Function (1), as bijective mapping, allows the notions defined for the FPR to be transferred to the MPR and vice versa.
Definition 3
([29]). If FPR R = ( r i j ) m × m carries at least one missing pairwise comparison r i j of alternative a i over a j , then it is an IFPR.
Definition 4.
If r i j max ( r i k + r k j 1 , 0 ) ( T L -transitivity ) , i j k { 1 , 2 , , m } , is satisfied; then, R is the T L -consistent symbolized by R ˜ = ( r ˜ i j ) m × m .
Definition 5.
The ranking values v ( a i ) of alternatives a i , i = 1 , 2 , . . , n , for R ˜ = ( r ˜ i j ) n × n are determined as
v ( a i ) = 2 n ( n 1 ) m j = 1 , j i r ˜ i j
with i = 1 m v ( a i ) = 1 .

3. Proposed Procedure for the MCGDM Problem

The proposed procedure for the MCGDM problem consists of various phases. The problem is first put to a group of experts E = { E 1 , E 2 , , E l } , with given sets of alternatives A = { a 1 , a 2 , , a m } and criteria C = { c 1 , c 2 , , c n } . The experts measure their own preferences { R 1 , R 2 , , R l } regarding the criteria and { R 1 C , R 2 C , , R l C } of alternatives for each criterion using FPRs based on their evaluation of an issue. A few of the values for preferences in FPRs might be missing, considering the time pressure and lack of information. Following this, the Łukasiewicz transitivity property ( T L -transitivity) with max aggregated operator is used to fill the missing places. Once the FPRs have been completed, the transitive closure formula plays a significant role in building entirely consistent FPRs. A consistency study is conducted to compute consistency-based indices of preference matrices to make the final result more trustworthy. The consensus level among the experts is estimated based on the set preferences: if a sufficient level is reached, the entire decision process undergoes the selection phase; otherwise, experts will be asked to revise existing values. Several levels are discussed in detail below.

3.1. Estimating Missing Values

Here, the subsection includes a method for estimating missed preferences inside an IFPR to build an FPR with full understanding that relies on T L -consistency. It should always be observed that each IFPR can only be completed based on the T L consistency when every alternative has a comparison among the known values of preferences values, at least once. Therefore, the system encourages an expert to define a sufficient number of parameters, where every alternative is evaluated at least once to allow the IFPR to become a complete FPR. Additionally, the order of measurement of the missing preference values affects the final result. To evaluate missing values of preferences in R = ( r i j ) m × m , the following sets of known and unknown preference values to identify pairs of alternatives are defined:
K = { ( i , j ) r i j is known } ,
U = { ( i , j ) r i j is unknown } ,
where r i j [ 0 , 1 ] and represents the degree of preference for alternative a i to a j . Based on the T L -transitivity r i j T L ( r i k , r k j ) , the following set can then be established for estimating a missing value r i j of the preference for alternative a i over a j .
Q i j = { k i , j ( i , k ) K , ( k , j ) K and ( i , j ) U } ,
where i , j , k { 1 , 2 , , m } . Therefore, based on the set defined in (4), r i j is estimated using
r i j = max k Q i j ( T L ( r i k , r k j ) ) , if | Q i j | 0 0.5 , otherwise ,
r j i = 1 r i j ,
where | Q i j | shows the number of elements in set Q i j . Now, we present the following newer sets K and U
K = K { ( i , j ) } , and U = U { ( i , j ) } .
Once FPR R = ( r i j ) m × m has been completed, we can construct a T L -consistent FPR R ˜ = ( r ˜ i j ) m × m using the following expression:
r ˜ i j = max k i , j ( r i j , T L ( r i k , r k j ) ) with r ˜ i j + r ˜ j i = 1 .
There are many decision-making procedures in the real world that take place in a multi-person framework, since the increasing difficulty and volatility of the socio-economic setting makes it less feasible for an individual to understand all the aspects of a decision-making problem.

3.2. Consistency Measures

In this subsection, some consistency measures are defined: the consistency index of a pair of alternatives, the consistency index of alternatives and the consistency index of FPRs. The term consistency index ( C I ) stands for a consistency degree whose value lies within the ragne [ 0 , 1 ] .
Let R p be an IFPR given by expert E p ( 1 p l ) ; then, after evaluating the missing preferences, (8) helps us to construct T L -consistent FPRs R ˜ p . It is then possible to estimate the consistency level for the FPR R p on the basis of its likeness to the correlating relation R ˜ p by measuring their distances [46].
  • The T L -consistency Index ( T L C I ) of each pair of alternatives is computed using the following expression:
    T L C I ( r i j p ) = 1 d ( r i j p , r ˜ i j p ) ,
    where d ( r i j p , r ˜ i j p ) = r i j p r ˜ i j p . Obviously, the greater the value of T L C I ( r i j p ) , the more acceptable r i j p is with respect to the remaining preference values of a i and a j .
  • T L C I values for the alternatives a i and 1 i n are determined using
    T L C I ( a i ) = 1 2 ( m 1 ) m j = 1 ( T L C I ( r i j p ) + T L C I ( r j i p ) ) .
  • T L C I for an FPR R p is therefore evaluated by calculating the mean of T L C I against all alternatives a i :
    T L C I ( R p ) = 1 m m i = 1 T L C I ( a i ) .
  • After evaluating T L C I in three stages (9)–(11), higher weights are assigned rationally to the experts with higher consistency degrees. Consistency weights may therefore be allocated to experts in the sense of the following relation:
    w p = T L C I ( R p ) p = 1 l T L C I ( R p ) .

3.3. Consensus Measures

The subsection includes several measures to assess a global consensus of experts to decide whether the decision process should be moved into the selection phase or not.
When the FPRs carry complete information, it us quite important to calculate the level of consensus among experts. In this context, the similarity relations S q r = ( s i j q r ) m × m with each pair of experts ( E q , E r ) , 1 q l 1 and q + 1 r l , need to be established. A similarity matrix, also known as a distance matrix, helps us to understand how close or far apart a pair of factors is from the participants’ perspective. Therefore, we define s i j q r by
s i j q r = 1 r i j q r i j r .
The aggregation of all similarity matrices results in a cumulative similarity matrix S = ( s i j ) m × m as follows:
s i j = 2 l ( l 1 ) l 1 q = 1 l r = q + 1 s i j q r .
The degree of consensus among experts is the result of the following three phases of the process [46]:
  • First, the degree of consensus for every pair ( a i , a j ) of alternatives, referred to as c o d i j , is determined:
    c o d i j = s i j .
  • At level 2, the degree of consensus among the experts on each alternative a i , referred to as C o D i for 1 i m , is established as
    C o D i = 1 2 ( m 1 ) m j = 1 , j i ( s i j + s j i ) .
  • The third level includes the global consensus degree, symbolized by C o D , among all experts on their observations:
    C o D = 1 m m i = 1 C o D i .
Once a global level of consensus has been reached among all experts, it is necessary to compare this with a threshold degree of consensus η , usually pre-determined based on the problem at hand. If C o D η , it indicates that a sufficient degree of consensus is achieved, and so the decision-making process begins. However, if the degree of consensus is not secure, experts may be asked to update their priorities. When the consensus is not sufficiently strong, the input process provides the experts with ample knowledge to adjust their views and increase the degree of consensus. The following identifier is therefore defined in order to recognize the preference values that need to be modified:
I p = ( i , j ) c o d i j < C o D and r i j p is known .
The corresponding experts are then advised to increase the value if it is lower than the average value of the other experts’ valuations and to reduce it if it is higher than the average.
For the hierarchical problem on the basis of GDM, consider a set { a 1 , a 2 ,.., a m } of m alternatives; a set { c 1 , c 2 , , c n } of n criteria and a team { E 1 , E 2 , , E l } of l experts with priority weights λ = ( λ 1 , λ 2 , λ 3 , , λ l ) T , so that p = 1 l λ p = 1 . The MCGDM procedure using the AHP structure is described as follows:

3.4. Final Priority Weights of the Experts

The final priority weights of the experts are measured by considering the consistency weights and predefined weights, respectively, as
β p = λ p × w p p = 1 l λ p × w p
where λ p and 1 p l are the predetermined priority weights of the experts and p = 1 l β p = 1 . In the absence of a predetermined priority weight vector, the consistency weights are taken as final weights for the experts.

3.5. Ranking of Criteria

In this subsection, priority weights of criteria are evaluated and ranked according to their importance under the following steps.
In step 1, the experts E p ( p = 1 , 2 , 3 , , l ) make pairwise comparisons of the criteria and may provide their evaluations in the form of the following IFPRs, R ( p ) = [ r i j ( p ) ] m × m :
R ( p ) = [ r i j ( p ) ] m × m = c 1 c 2 . . c n 0.5 r 12 ( p ) . . r 1 n ( p ) r 21 ( p ) 0.5 . . r 2 n ( p ) . . . . . . . . r n 1 ( p ) r n 2 ( p ) . . 0.5 , c 1 c 2 . . c n
and r i j ( p ) [ 0 , 1 ] shows the degree of preference of criterion c i compared to criterion c j , evaluated by expert E p , r i j ( p ) + r j i ( p ) = 1 , 1 i , j n , 1 p l .
In step 2, (4)–(8) allow the estimation of all missing preference degrees for R ( p ) , and T L -consistent R ˜ ( p ) = [ r ˜ i j ( p ) ] n × n , 1 p l values are constructed.
In step 3, since consistent FPRs have been constructed, the consistency degree for each FPR of the criteria are calculated using (9)–(11). Usually, an FPR is called consistent to some extent if the level of consistency is higher than 0.5 , while it is fully consistent when that level is 1. The degree of consensus regarding the criteria by all experts is measured with the use of (13)–(17).
In step 4, we construct the aggregated relation R c as follows:
R c = r i j c n × n = p = 1 l β p r ˜ i j ( p ) n × n ,
where 1 i n , 1 j n and 1 p l .
In step 5, using definition 6, we evaluate the ranking values of criteria as follows:
v ( c i ) = 2 n ( n 1 ) n i = 1 , i j r i j c ,
for i = 1 n v ( c i ) = 1 .

3.6. Ranking of Alternatives Regarding Each Criterion

In this subsection, priority weights of alternatives regarding each criterion are evaluated to rank them according to their importance in the following steps.
In step 1, experts E p ( p = 1 , 2 , 3 , , l ) make pairwise comparisons of the alternatives regarding each criterion q ( q = 1 , 2 , 3 , n ) and may provide their evaluations in the form of IFPRs q R ( p ) = [ q r u v ( p ) ] m × m .
In step 2, (4)–(8) are used to estimate all missing preference degrees for q R ( p ) , and T L -consistent q R ˜ ( p ) = [ q r ˜ u v ( p ) ] m × m values are constructed.
In step 3, since consistent FPRs have been constructed, the consistency degree for each FPR for each criterion is calculated using (9)–(11). The degree of consensus among all experts is measured with the use of (13)–(17).
In step 4, we construct the aggregated relation q R c as follows:
q R c = [ q r i j c ] m × m = l p = 1 β p q r ˜ u v ( p ) m × m ,
where 1 u m , 1 v m and 1 p l .
In step 5, using definition 6, we evaluate the ranking values of alternatives regarding each criterion as follows:
q v ( a u ) = 2 m ( m 1 ) m v = 1 , v u q r u v c ,
where 1 u m and u = 1 m q v ( a u ) = 1 .

3.7. Final Ranking of Alternatives

In order to evaluate the final ranking order of alternatives, we have to perform a simple matrix multiplication of the matrix of the priority scores of alternatives corresponding to each criterion and the column matrix for the priority weights of criteria. Suppose the score matrix of alternatives regarding each criterion is symbolized by A and the column matrix for the priority weights of criteria is denoted by w c ; then, the priority weight vector w x of alternatives is determined using
w x = A · w c .
In order to better understand the proposed technique, we therefore take the following fuzzy MCDM problem within the IFPR setting.

4. Example

In order to procure important components for new brands, a high-tech manufacturing corporation chooses an appropriate material supplier. After the initial selection, four candidates ( a 1 , a 2 , a 3 , a 4 ) proceed to some final analysis. In order to find the most appropriate supplier, a group including experts ( E 1 , E 2 , E 3 ) is established, with the priority weights λ = ( 1 / 3 , 1 / 3 , 1 / 3 ) T . Five benefit criteria are considered: (1) technical abilities and leadership ( c 1 ); (2) social responsibility ( c 2 ); (3) competitive pricing ( c 3 ); (4) quality and safety ( c 4 ); (5) delivery ( c 5 ). The pre-established threshold level η of consensus to the set of criteria is 0.75 .
The hierarchical structure for this decision problem can be seen in Figure 1 below.
After the pairwise comparison of the five criteria, the experts provide the following IFPRs:
R ( 1 ) = 0.5 0.6 r 13 1 0.4 0.7 0.4 0.5 r 23 1 0.6 0.7 r 31 1 r 32 1 0.5 0.3 0.4 0.6 0.4 0.7 0.5 r 45 1 0.3 0.3 0.6 r 54 1 0.5 ; R ( 2 ) = 0.5 0.4 r 13 2 0.6 r 15 2 0.6 0.5 0.6 0.5 r 25 2 r 31 2 0.4 0.5 0.3 0.7 0.4 0.5 0.7 0.5 0.2 r 51 2 r 52 2 0.3 0.8 0.5 ; R ( 3 ) = 0.5 0.3 0.4 0.7 0.7 0.7 0.5 r 23 3 r 24 3 r 25 3 0.6 r 32 3 0.5 0.2 0.8 0.3 r 42 3 0.8 0.5 0.3 0.3 r 52 3 0.2 0.7 0.5 .
After the pairwise comparison of the four alternatives for each criterion, the experts provide the following IFPRs:
Technical abilities and leadership:
1 R ( 1 ) = 0.5 0.8 1 r 13 1 1 r 14 1 0.2 0.5 0.4 0.6 1 r 31 1 0.6 0.5 0.7 1 r 41 1 0.4 0.3 0.5 ; 1 R ( 2 ) = 0.5 0.7 1 r 13 2 0.6 0.3 0.5 0.5 0.7 1 r 31 2 0.5 0.5 1 r 34 2 0.4 0.3 1 r 43 2 0.5 ; 1 R ( 3 ) = 0.5 1 r 12 3 0.7 1 r 14 3 1 r 21 3 0.5 1 r 23 3 0.6 0.3 1 r 32 3 0.5 0.5 1 r 41 3 0.4 0.5 0.5 .
Social responsibility:
2 R ( 1 ) = 0.5 0.7 0.5 2 r 14 1 0.3 0.5 0.4 0.6 0.5 0.6 0.5 0.8 2 r 41 1 0.4 0.2 0.5 ; 2 R ( 2 ) = 0.5 0.8 2 r 13 2 2 r 14 2 0.2 0.5 0.3 0.5 2 r 31 2 0.7 0.5 0.3 2 r 41 2 0.5 0.7 0.5 ; 2 R ( 3 ) = 0.5 0.9 2 r 13 3 0.6 0.1 0.5 0.4 0.4 2 r 31 3 0.6 0.5 2 r 34 3 0.4 0.6 2 r 43 3 0.5 .
Competitive pricing:
3 R ( 1 ) = 0.5 0.6 3 r 13 1 0.7 0.4 0.5 0.3 3 r 24 1 3 r 31 1 0.7 0.5 0.4 0.3 3 r 42 1 0.6 0.5 ; 3 R ( 2 ) = 0.5 3 r 12 2 3 r 13 2 0.6 3 r 21 2 0.5 3 r 23 2 0.2 3 r 31 2 3 r 32 2 0.5 0.4 0.4 0.8 0.6 0.5 ; 3 R ( 3 ) = 0.5 3 r 12 3 0.3 0.5 3 r 21 3 0.5 3 r 23 3 0.3 0.7 3 r 32 3 0.5 3 r 34 3 0.5 0.7 3 r 43 3 0.5 .
Quality and safety:
4 R ( 1 ) = 0.5 0.2 4 r 13 1 0.7 0.8 0.5 4 r 23 1 4 r 24 1 4 r 31 1 4 r 32 1 0.5 4 r 34 1 0.3 4 r 42 1 4 r 43 1 0.5 ; 4 R ( 2 ) = 0.5 0.4 0.7 4 r 14 2 0.6 0.5 0.7 4 r 24 2 0.3 0.3 0.5 0.6 4 r 41 2 4 r 42 2 0.4 0.5 ; 4 R ( 3 ) = 0.5 4 r 12 3 0.3 0.4 4 r 21 3 0.5 0.2 0.6 0.7 0.8 0.5 4 r 34 3 0.6 0.4 4 r 43 3 0.5 .
Delivery:
5 R ( 1 ) = 0.5 5 r 12 1 5 r 13 1 0.1 5 r 21 1 0.5 0.4 5 r 24 1 5 r 31 1 0.6 0.5 0.7 0.9 5 r 42 1 0.3 0.5 ; 5 R ( 2 ) = 0.5 0.2 5 r 13 2 0.4 0.8 0.5 5 r 23 2 0.3 5 r 31 2 5 r 32 2 0.5 0.3 5 r 41 2 0.7 0.7 0.5 ; 5 R ( 3 ) = 0.5 0.3 0.4 5 r 14 3 0.7 0.5 0.6 5 r 24 3 0.6 0.4 0.5 5 r 34 3 5 r 41 3 5 r 42 3 5 r 43 3 0.5 .
The use of the suggested method results in the cumulative FPR for five criteria and the corresponding priority weights, as seen in Table 1, whereas the level of consistency and the degree of consensus for the experts are as follows:
T L C I ( R 1 ) = 1 ; T L C I ( R 2 ) = 0.98 ; T L C I ( R 3 ) = 0.95 , and C o D = 0.7933 .
The aggregated FPRs for the four suppliers corresponding to the criteria and the priority scores can be seen in Table 2, and the consistency as well as consensus levels for each criterion are calculated as follows:
Technical abilities and leadership:
T L C I ( 1 R 1 ) = 1 ; T L C I ( 1 R 2 ) = 1 ; T L C I ( 1 R 3 ) = 1 , and C o D = 0.8189 .
Social responsibility:
T L C I ( 2 R 1 ) = 1 ; T L C I ( 2 R 2 ) = 1 ; T L C I ( 2 R 3 ) = 1 , and C o D = 0.8111 .
Competitive pricing:
T L C I ( 3 R 1 ) = 1 ; T L C I ( 3 R 2 ) = 1 ; T L C I ( 3 R 3 ) = 1 , and C o D = 0.8445 .
Quality and safety:
T L C I ( 4 R 1 ) = 1 ; T L C I ( 4 R 2 ) = 1 ; T L C I ( 4 R 3 ) = 1 , and C o D = 0.7933 .
Delivery:
T L C I ( 5 R 1 ) = 1 ; T L C I ( 5 R 2 ) = 1 ; T L C I ( 5 R 3 ) = 1 , and C o D = 0.7877 .
The last column of Table 3 is used to show the final ranking values of the four suppliers, which are w a 1 = 0.2060 , w a 2 = 0.2071 , w a 3 = 0.2896 and w a 4 = 0.2973 . As w a 4 > w a 3 > w a 2 > w a 1 ; therefore, the ranking order of the four suppliers a 1 , a 2 , a 3 and a 4 is a 4 > a 3 > a 2 > a 1 .

5. Comparison

To validate the productivity of the proposed scheme, we compare the results after concluding the problem taken from [22] with our proposed technique.

Problem Statement

A funds, and five potential candidates (loan users) are in competition for the remaining funds. The problem is to rank the applicants and allocate the loan following the principle of loan allocation until the funds are completely used. A team of five decision makers (DMs) participate in the ranking: the President of the Fund Council (DM1), a senior advisor to the Fund (DM2), the fund manager (DM3), an external expert advisor (DM4) and an expert representative of the Ministry. Three criteria are considered by the team: (1) Service, (2) Loan History (LOANH) and (3) Insurance. After pairwise comparison, the MPRs provided by the DMs are given as presented in Table 4, Table 5, Table 6, Table 7 and Table 8.
Srdevic et al. [22] determined the ranking oder of five applicants as U 3 > U 2 > U 1 > U 5 > U 4 . However, the priority weights of applicants obtained by the proposed model after transforming the above problem in a fuzzy environment, with the help of Remark 1, are U 1 = 0.17781, U 2 = 0.23454, U 3 = 0.29778, U 4 = 0.10693, U 5 = 0.18294, which lead to a ranking order of
U 3 > U 2 > U 5 > U 1 > U 4
The result shows that the ranking positions for applicants U 2 , U 3 and U 4 are the same as in [22], while the order of applicants U 1 and U 5 are interchanged, but applicant U 3 is the first preference in both models to get the desired loan. The similarities of these two rankings are very high, where the r w and W S coefficients of the both rankings are equal 0.9167 [47]. There may be two factors that resulted in a different ranking order in few places: (i) different techniques were used to determine the ranking order, and (ii) the corresponding parameters regarding different models could have been evaluated in various ways, and the results may be affected. We think that the proposed model can equally handle complete and incomplete information and provides better results.

6. Conclusions

This paper provides a clear and effective methodology for selecting and ranking suppliers based on a consensus-derived and consistent model for MCGDM in an incomplete AHP environment. The T L -transitivity property plays a main role in evaluating unknown preference values, as it symbolizes one of the most suitable means to model consistent FPRs. T L C I was defined to determine the consistency level of the information provided by each expert. The proposed method was used in three steps: firstly, we evaluated the priority weight vector of criteria; secondly, we estimated the priority ratings of each alternative against each criterion; finally, we determined the priority weight vector for alternatives, which allowed us to obtain the best alternative. At the end of this study, we successfully applied the proposed procedure to select a suitable supplier in SCM by illustrating a numerical example to highlight the practicability and efficacy of the method.
In summary, the proposed method has the following major advantages: (i) in this manuscript, T L -transitivity was considered to measure the FPRs’ unspecified preference values. T L -transitivity is more suitable to model consistent FPRs compared with other consistency-based techniques; (ii) the consistency degrees of experts’ opinions were measured to strengthen the final decision; (iii) after reaching the required level of consensus among the experts, first, a ranking of the criteria was established, and then alternatives were prioritized under each criterion based on the consistent information.
To the best of our knowledge, a similar method to deal with MCGDM problems using the AHP model has not previously been proposed. From our perspective, this procedure is an efficient and reliable way to gain a greater insight to solve MCGDM problems in the current environment. This study suffers from limitations that should be addressed in future work: (i) experts might exhibit a degree of hesitancy while providing their preferences—it will be interesting to develop procedures to deal with MCGDM in AHP under hesitant fuzzy preference relations; (ii) the threshold consensus degree η has a direct influence on the consensus round but is typically evaluated in advance—it will be interesting to observe how this parameter can be estimated based on different factors; (iii) there are some risks that some experts may provide their information dishonestly or refuse to make changes with the preferences. Thus, some mechanism may be introduced to handle non-cooperative activities in consensus-building. We will try to work in the above-mentioned directions to face future challenges; these will contribute to the acceptance of this research area.

Author Contributions

Conceptualization, A.u.R., A.S., N.R., S.F. and W.S.; methodology, A.u.R., A.S., N.R., S.F. and W.S.; software, A.u.R., A.S., N.R. and S.F.; validation, S.F. and W.S.; formal analysis, A.u.R. and N.R.; investigation, A.u.R., A.S., N.R., S.F. and W.S.; resources, A.S. and S.F.; data curation, A.u.R., N.R. and W.S.; writing—original draft preparation, A.u.R., A.S., N.R. and S.F.; writing—review and editing, A.S. and W.S.; visualization, S.F.; supervision, W.S.; project administration, W.S.; funding acquisition, W.S. All authors have read and agreed to the published version of the manuscript.

Funding

The work was supported by the National Science Centre, Decision number UMO-2018/29/B/ HS4/02725 and by statutory funds from the Research Team on Intelligent Decision Support Systems, Department of Artificial Intelligence and Applied Mathematics, Faculty of Computer Science and Information Technology, West Pomeranian University of Technology in Szczecin.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

The authors would like to thank the editor and the anonymous reviewers, whose insightful comments and constructive suggestions helped us to significantly improve the quality of this paper.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

The following abbreviations are used in this manuscript:
AHPAnalytical Hierarchy Process
FPRFuzzy preference relation
IFPRIncomplete fuzzy preference relation
MPRMultiplicative preference relation
SCMSupply chain management
GDMGroup decision-making
MCDMMulti-criteria decision-making
MCGDMMulti-criteria group decision-making

References

  1. Durmić, E.; Stević, Ž.; Chatterjee, P.; Vasiljević, M.; Tomašević, M. Sustainable supplier selection using combined FUCOM—Rough SAW model. Rep. Mech. Eng. 2020, 1, 34–43. [Google Scholar] [CrossRef]
  2. Sałabun, W.; Ziemba, P. Application of the Characteristic Objects Method in Supply Chain Management and Logistics. In Recent Developments in Intelligent Information and Database Systems; Springer: Berlin/Heidelberg, Germany, 2016; pp. 445–453. [Google Scholar]
  3. Badi, I.; Pamucar, D. Supplier selection for steelmaking company by using combined Grey-MARCOS methods. Decis. Mak. Appl. Manag. Eng. 2020, 3, 37–48. [Google Scholar] [CrossRef]
  4. Chakraborty, S.; Chattopadhyay, R.; Chakraborty, S. An integrated D-MARCOS method for supplier selection in an iron and steel industry. Decis. Mak. Appl. Manag. Eng. 2020, 3, 49–69. [Google Scholar]
  5. Swaminathan, J.M.; Tayur, S.R. Models for supply chains in e-business. Manage. Sci. 2003, 49, 1387–1406. [Google Scholar] [CrossRef] [Green Version]
  6. Chen, C.T.; Lin, C.T.; Huang, S.F. A fuzzy approach for supplier evaluation & selection in supply chain management. Int. J. Prod. Econ. 2006, 102, 289–301. [Google Scholar]
  7. De Boer, L.; Labro, E.; Morlacchi, P. A review of methods supporting supplier selection. Eur. J. Purch. Supply Manag. 2001, 7, 75–89. [Google Scholar] [CrossRef]
  8. Demirtas, E.; Üstün, Ö. An integrated multiobjective decision making process for supplier selection and order allocation. Omega 2008, 36, 76–90. [Google Scholar] [CrossRef]
  9. Hwang, C.L.; Yoon, K. Multiple Attributes Decision Making Methods & Applications; Springer: New York, NY, USA, 1981. [Google Scholar]
  10. Ng, W.L. An efficient & simple model for multiple criteria supplier selection problem. Eur. J. Oper. Res. 2008, 186, 1059–1067. [Google Scholar]
  11. Dickson, G.W. A analysis of vender selection systems & decisions. J. Suppl. Chain Manag. 1966, 2, 5–17. [Google Scholar]
  12. Barbarosoglu, G.; Yazgaç, T. An application of the analytic hierarchy process to the supplier selection problem. Prod. Inventory Manag. J. 1997, 38, 14–21. [Google Scholar]
  13. Goffin, K.; Szwejczewski, M.; New, C. Managing suppliers when fewer can mean more. Int. J. Phys. Distrib. Logist. Manag. 1997, 27, 422–436. [Google Scholar] [CrossRef]
  14. Swift, C.O. Preference for single sourcing & supplier selection criteria. J. Bus. Res. 1995, 32, 105–111. [Google Scholar]
  15. Weber, C.A.; Current, J.R.; Benton, W.C. Vender selection criteria & method. Eur. J. Oper. Res. 1991, 50, 2–18. [Google Scholar]
  16. Chai, J.; Liu, J.N.K.; Ngai, E.W.T. Application of decision-making techniques in supplier selection: A systematic review of literature. Expert Syst. Appl. 2013, 40, 3872–3885. [Google Scholar] [CrossRef]
  17. Faizi, S.; Sałabun, W.; Rashid, T.; Wątróbski, J.; Zafar, S. Group decision-making for hesitant fuzzy sets based on characteristic objects method. Symmetry 2017, 9, 136. [Google Scholar] [CrossRef]
  18. Faizi, S.; Sałabun, W.; Rashid, T.; Zafar, S.; Wątróbski, J. Intuitionistic fuzzy sets in multi-criteria group decision making problems using the characteristic objects method. Symmetry 2020, 12, 1382. [Google Scholar] [CrossRef]
  19. Chen, P.C. A fuzzy multiple criteria decision making model in employee recruitment. Int. J. Comput. Sci. Net. Secur. 2009, 9, 113–117. [Google Scholar]
  20. Hadi-Vencheh, A.; Mokhtarian, M.N. A new fuzzy MCDM approach based on centroid of fuzzy numbers. Expert Syst. Appl. 2011, 38, 5226–5230. [Google Scholar] [CrossRef]
  21. Önüt, S.; Efendigil, T.; Kara, S.S. A combined fuzzy MCDM approach for selecting shopping center site: An example from Istanbul, Turkey. Expert Syst. Appl. 2010, 37, 1973–1980. [Google Scholar] [CrossRef]
  22. Srdevic, Z.; Blagojevic, B.; Srdevic, B. AHP based group decision making in ranking loan applicants for purchasing irrigation equipment: A case study. Bulg. J. Agric. Sci. 2011, 17, 531–543. [Google Scholar]
  23. Sun, C.C.; Lin, G.T.R.; Tzeng, G.H. The evaluation of cluster policy by fuzzy MCDM: Empirical evidence from Hsin Chu Science Park. Expert Syst. Appl. 2009, 36, 11895–11906. [Google Scholar] [CrossRef]
  24. Wu, H.Y.; Tzeng, G.H.; Chen, Y.H. A fuzzy MCDM approach for evaluating banking performance based on balanced scorecard. Expert Syst. Appl. 2009, 36, 10135–10147. [Google Scholar] [CrossRef]
  25. Bashir, Z.; Rashid, T.; Wątróbski, J. Sałabun, W.; Malik, A. Hesitant probabilistic multiplicative preference relations in group decision making. Appl. Sci. 2018, 8, 398. [Google Scholar] [CrossRef] [Green Version]
  26. Kim, S.H.; Ahn, B.S. Interactive group decision making procedure under incomplete information. Eur. J. Oper. Res. 1999, 116, 498–507. [Google Scholar] [CrossRef]
  27. Xu, Z.S. Goal programming models for obtaining the priority vector of incomplete fuzzy preference relation. Int. J. Approx. Reason. 2004, 36, 261–270. [Google Scholar] [CrossRef] [Green Version]
  28. Gong, Z.W. Least-square method to priority of the fuzzy preference relations with incomplete information. Int. J. Approx. Reason. 2008, 47, 258–264. [Google Scholar] [CrossRef] [Green Version]
  29. Herrera-Viedma, E.; Chiclana, F.; Herrera, F.; Alonso, S. Group decision-making model with incomplete fuzzy preference relations based on additive consistency. IEEE Trans. Syst. Man Cybern. Part B Cybern. 2007, 37, 176–189. [Google Scholar] [CrossRef]
  30. Alonso, S.; Chiclana, F.; Herrera, F.; Herrera-Viedma, E. A consistency based procedure to estimate missing pairwise preference values. Int. J. Intell. Syst. 2008, 23, 155–175. [Google Scholar] [CrossRef]
  31. Xu, Y.J. On group decision making with four formats of incomplete preference relations. Comput. Ind. Eng. 2011, 61, 48–54. [Google Scholar] [CrossRef]
  32. Lee, L.W. Group decision making with incomplete fuzzy preference relations based on the additive consistency and the order consistency. Expert Syst. Appl. 2012, 39, 11666–11676. [Google Scholar] [CrossRef]
  33. Chen, S.M.; Lin, T.E.; Lee, L.W. Group decision making using incomplete fuzzy preference relations based on the additive consistency and the order consistency. Inf. Sci. 2014, 259, 1–15. [Google Scholar] [CrossRef]
  34. Rehman, A.; Kerre, E.E.; Ashraf, S. Group decision making by using incomplete fuzzy preference relations based on T-Consistency Order Consistency. Int. J. Intell. Syst. 2015, 30, 120–143. [Google Scholar] [CrossRef] [Green Version]
  35. Kerre, E.E.; Rehman, A.U.; Ashraf, S. Group decision making with incomplete reciprocal preference relations based on multiplicative consistency. Int. J. Comput. Intell. Syst. 2018, 11, 1030–1040. [Google Scholar] [CrossRef] [Green Version]
  36. Saaty, T.L. The Analytic Hierarchy Process: Planning, Priority Setting & Resource Allocation; ; McGraw-Hill: New York, NY, USA, 1980. [Google Scholar]
  37. Saaty, T.L. How to make a decision: The analytic hierarchy process. Eur. J. Oper. Res. 1990, 48, 9–26. [Google Scholar] [CrossRef]
  38. Lin, C.T.; Hsu, P.F. Adopting an analytic hierarchy process to select internet advertising networks. Mark. Intell. Plan. 2003, 21, 183–191. [Google Scholar] [CrossRef]
  39. Omasa, T.; Kisshimoto, M.; Kawase, M.; Yagi, K. An attempt at decision making in tissue engineering: Reactor evaluation using the analytic hierarchy process (AHP). Biochem. Eng. J. 2004, 20, 173–179. [Google Scholar] [CrossRef]
  40. Wei, C.C.; Chien, C.F.; Wang, M.J. An AHP-based approach to ERP system selection. Int. J. Prod. Econ. 2005, 96, 47–62. [Google Scholar] [CrossRef]
  41. Tanino, T. Fuzzy preference relations in group decision making. In Non-Conventional Preference Relations in Decision Making; Lecture Notes in Economics and Mathematical Systems; Springer: Berlin/Heidelberg, Germany, 1988; Volume 301, pp. 54–71. [Google Scholar]
  42. Venugopalan, P. Fuzzy ordered sets. Fuzzy Sets Syst. 1992, 46, 221–226. [Google Scholar] [CrossRef]
  43. Zadeh, L.A. Fuzzy sets. Inf. Control 1965, 8, 338–353. [Google Scholar] [CrossRef] [Green Version]
  44. Tanino, T. Fuzzy preference orderings in group decision making. Fuzzy Sets Syst. 1984, 12, 117–131. [Google Scholar] [CrossRef]
  45. Ureña, R.; Chiclana, F.; Alonso, S.; Morente-Molinera, J.A.; Herrera-Viedma, E. On incomplete fuzzy and multiplicative preference relations in multi-person decision making. Procedia Comput. Sci. 2014, 31, 793–801. [Google Scholar] [CrossRef] [Green Version]
  46. Zhang, X.; Ge, B.; Jiang, J.; Tan, Y. Consensus building in group decision making based on multiplicative consistency with incomplete reciprocal preference relations. Knowl.-Based Syst. 2016, 106, 96–104. [Google Scholar] [CrossRef]
  47. Sałabun, W.; Urbaniak, K. A new coefficient of rankings similarity in decision-making problems. In International Conference on Computational Science; Springer: Cham, Switzerland, 2020; pp. 632–645. [Google Scholar]
Figure 1. Hierarchical structure of the decision problem.
Figure 1. Hierarchical structure of the decision problem.
Symmetry 13 00609 g001
Table 1. Accumulated fuzzy preference relation (FPR) of criteria and ranking values.
Table 1. Accumulated fuzzy preference relation (FPR) of criteria and ranking values.
c 1 c 2 c 3 c 4 c 5 Priority Weights
c 1 0.500000.43580.36480.56420.46580.1830
c 2 0.56420.50000.36790.50170.46890.1903
c 3 0.63520.63210.50000.43170.63000.2329
c 4 0.43580.49830.56830.50000.26660.1769
c 5 0.53420.53110.37000.73340.50000.2169
Table 2. Accumulated FPRs for suppliers in relation to each criterion and ranking values.
Table 2. Accumulated FPRs for suppliers in relation to each criterion and ranking values.
a 1 a 2 a 3 a 4 Priority Ratings
c 1
a 1 0.50000.66670.36670.40000.2389
a 2 0.33330.50000.36670.63330.2222
a 3 0.63330.63330.50000.43330.2833
a 4 0.60000.36670.56670.50000.2556
c 2
a 1 0.50000.80000.30000.40000.2500
a 2 0.20000.50000.36670.50000.1778
a 3 0.70000.63330.50000.46670.3000
a 4 0.60000.50000.53330.50000.2722
c 3
a 1 0.50000.40000.26670.60000.2111
a 2 0.60000.50000.13330.20000.1556
a 3 0.73330.86670.50000.33330.3222
a 4 0.40000.80000.66670.50000.3111
c 4
a 1 0.50000.23330.50000.46670.2000
a 2 0.76670.50000.40000.46670.2722
a 3 0.50000.60000.50000.40000.2500
a 4 0.53330.53330.60000.50000.2778
c 5
a 1 0.50000.33330.16670.33330.1389
a 2 0.66670.50000.50000.16670.2222
a 3 0.83330.50000.50000.36670.2833
a 4 0.66670.83330.63330.50000.3556
Table 3. Final priority weights of the four suppliers.
Table 3. Final priority weights of the four suppliers.
c 1 c 2 c 3 c 4 c 5 Priority Weights
Criteria weights0.18300.19030.23290.17690.2169
a 1 0.23890.25000.21110.20000.13890.2060
a 2 0.22220.17780.15560.27220.22220.2071
a 3 0.28330.30000.32220.25000.28330.2896
a 4 0.25560.27220.31110.27780.35560.2973
Table 4. Comparison matrices provided by President of the Fund Council (DM1) [22].
Table 4. Comparison matrices provided by President of the Fund Council (DM1) [22].
CriteriaServiceLOANHInsurance
Service 29
LOANH 5
Insurance
ServiceLOANHInsurance
U 1 U 2 U 3 U 4 U 5 U 1 U 2 U 3 U 4 U 5 U 1 U 2 U 3 U 4 U 5
U 1 31/284 1/91/91/41/6 251/21/6
U 2 1/373 132 51/31/8
U 3 96 32 1/71/9
U 4 1/4 1/2 1/3
U 5
Table 5. Comparison matrices provided by Senior advisor of the Fund (DM2) [22].
Table 5. Comparison matrices provided by Senior advisor of the Fund (DM2) [22].
CriteriaServiceLOANHInsurance
Service 59
LOANH 6
Insurance
ServiceLOANHInsurance
U 1 U 2 U 3 U 4 U 5 U 1 U 2 U 3 U 4 U 5 U 1 U 2 U 3 U 4 U 5
U 1 41/365 1/61/61/31/3 241/21/4
U 2 1/453 122 41/41/6
U 3 98 22 1/81/9
U 4 1/3 1/2 1/2
U 5
Table 6. Comparison matrices provided by Fund manager (DM3) [22].
Table 6. Comparison matrices provided by Fund manager (DM3) [22].
CriteriaServiceLOANHInsurance
Service 1/51/3
LOANH 6
Insurance
ServiceLOANHInsurance
U 1 U 2 U 3 U 4 U 5 U 1 U 2 U 3 U 4 U 5 U 1 U 2 U 3 U 4 U 5
U 1 21/376 1/51/51/21/4 361/21/7
U 2 1/553 122 21/41/7
U 3 97 22 1/61/9
U 4 1/4 1/2 1/5
U 5
Table 7. Comparison matrices provided by External expert advisor (DM4) [22].
Table 7. Comparison matrices provided by External expert advisor (DM4) [22].
CriteriaServiceLOANHInsurance
Service 37
LOANH 5
Insurance
ServiceLOANHInsurance
U 1 U 2 U 3 U 4 U 5 U 1 U 2 U 3 U 4 U 5 U 1 U 2 U 3 U 4 U 5
U 1 31/385 1/81/81/51/6 461/31/5
U 2 1/463 132 21/41/7
U 3 95 32 1/61/9
U 4 1/4 1/3 1/4
U 5
Table 8. Comparison matrices provided by Expert representative of the Ministry (DM5) [22].
Table 8. Comparison matrices provided by Expert representative of the Ministry (DM5) [22].
CriteriaServiceLOANHInsurance
Service 79
LOANH 5
Insurance
ServiceLOANHInsurance
U 1 U 2 U 3 U 4 U 5 U 1 U 2 U 3 U 4 U 5 U 1 U 2 U 3 U 4 U 5
U 1 31/575 1/91/91/51/7 361/31/5
U 2 1/665 175 41/51/7
U 3 97 75 1/71/9
U 4 1/3 1/4 1/4
U 5
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Rehman, A.u.; Shekhovtsov, A.; Rehman, N.; Faizi, S.; Sałabun, W. On the Analytic Hierarchy Process Structure in Group Decision-Making Using Incomplete Fuzzy Information with Applications. Symmetry 2021, 13, 609. https://0-doi-org.brum.beds.ac.uk/10.3390/sym13040609

AMA Style

Rehman Au, Shekhovtsov A, Rehman N, Faizi S, Sałabun W. On the Analytic Hierarchy Process Structure in Group Decision-Making Using Incomplete Fuzzy Information with Applications. Symmetry. 2021; 13(4):609. https://0-doi-org.brum.beds.ac.uk/10.3390/sym13040609

Chicago/Turabian Style

Rehman, Atiq ur, Andrii Shekhovtsov, Nighat Rehman, Shahzad Faizi, and Wojciech Sałabun. 2021. "On the Analytic Hierarchy Process Structure in Group Decision-Making Using Incomplete Fuzzy Information with Applications" Symmetry 13, no. 4: 609. https://0-doi-org.brum.beds.ac.uk/10.3390/sym13040609

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop