Prioritizing Barriers to Be Solved to the Implementation of Reverse Logistics of E-Waste in Brazil under a Multicriteria Decision Aid Approach

Abstract

This study aims to identify the main barriers to fully implement the reverse logistics of e-waste in Brazil and prioritize them under a Multicriteria Decision Aid approach (MCDA) according to the perceptions of small and medium-sized companies, consumers, and the Brazilian government. Hence, the study was conducted in two steps: i) Systematic literature review to identify the main barriers, with data collection carried out through a documental analysis, and ii) MCDA application, through self-administered questionnaires. In order to deal with the uncertainty in subjective measures, the method of Composition of Probabilistic Preferences (CPP) was applied to analyze the results of each group of decision maker, opting for the progressive-pessimistic scenario to prioritize the barriers to be solved. As main results, a priority classification of the internal and external barriers considering each category of decision maker was obtained. Moreover, it was verified that the government and micro and small companies agreed that internal barriers with an organizational nature or related to infrastructure management are the main obstacles to the implementation of reverse logistics. On the other hand, consumers consider the managerial or organizational barriers as a priority.

1. Introduction

In Brazil, 1.4 million tons of electronic waste was generated during the year 2014 (7.1 kilograms per capita), which represented 52% of electronic waste generated in Latin America during the same period [1]. In 2016, this number increased to 1.534 tons, 7.4 kg per inhabitant [2], an increase of approximately 9.57%. The increase in waste generated was greater than the increase related to population in Brazil, which in two years (2014–2016) grew by approximately 1.97% [3,4].

Electro-Electronic Equipment (EEE) contains heavy metals in its composition such as mercury, lead and chromium, and greenhouse gases, among other harmful substances that cause damage to health and the environment [5]. The increase of manufacturing and marketing of these products, combined with the reduction of their life cycle generated by the planned obsolescence, have caused more disposability and, consequently, more urban waste. These facts enable the emergence of reverse logistics as a means of disposal and conscious use of these products, both after-sales and post-consumption [6]. Reverse Logistics (RL) is the process in which the value is recaptured or the appropriate destination of products, used or not, is ensured, this being its main operation [7].

In Brazil, the implementation of reverse logistics for Waste of Electrical and Electronic Equipment (WEEE), or simply e-waste, is different from other countries, especially those in development, as it has operated under a legal requirement since 2010, the Brazilian Policy of Solid Waste—BPSW (Law No. 12’305⁄2010) [8]. Among its principles and instruments, the BPSW determines: i) The obligation of implementation of reverse logistics of several categories of residues; ii) the extended and shared responsibility of the product, involving the main actors of supply chains (manufacturers, retailers, consumers, government); iii) the inclusion of waste pickers of recyclable materials in the process of reverse logistics; iv) the environmental awareness and education of citizens and, iv) the sectoral agreements. With the consideration and application of these principles and instruments, its implementation may generate several benefits, such as income generation, formalization of jobs, reduction of environmental impact, and environmental protection [8,9].

According to the report “A New Circular Vision for Electronics—Time for a Global Reboot” [10], only 67 countries have legislation related to the generation and management of their electronic waste. To implement the reverse logistics of WEEE in Brazil, the BPSW defines that it must happen through sectoral agreements [11]. Stakeholders are organized through sectoral agreements, making a reverse logistics system [12]. However, for these agreements to happen, it is necessary to reach a consensus between the parties involved [13].

Organizations face several complexities and challenges when implementing reverse logistics practices [14]. The implementation of an extended product responsibility in the management of electronic waste is still seen as costly and faces several barriers [15]. The barriers can be ordered according to four main factors. They can be internal to companies, such as management, finance and infrastructure, or external, in the case of the policies (rules) employed [16]. Each country’s context and particularities can influence the importance/relevance of each barrier [17]. The study of Bouzon et al. [18], which deals with barriers in the implementation of reverse logistics in Brazil, proposes further studies focusing on small and medium-sized companies.

Some studies have been developed using Multiple Criteria Decision Aid (MCDA), such as the Analytic Hierarchy Process (AHP), the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS), the Decision Making Trial and Evaluation Laboratory Method (DEMATEL), and the Interpretive Structural Modeling (ISM) [17,19,20,21]. In addition to these MCDA methods, there is the Composition of Probabilistic Preferences (CPP) approach [22], which is a recent method that takes into account imprecision, which is intrinsic to subjectivity, present in the whole decision process [22], [23]. This method employs a probabilistic view in order to reduce the effects of errors in the measurements of the criteria, allowing to simplify the modeling and to take into account the inherent uncertainty in the measurements [22]. Besides, this method does not consider the assignment of weights or scale constants on the analyzed criteria, which sometimes could conduct to arbitrary values. To the best of our knowledge, no study was identified analyzing the reverse logistics barriers applying CPP in the analysis. Thus, considering the above, the objective of this study is to analyze the barriers identified by the stakeholders (final consumer, government) during the implementation of reverse logistics of e-waste, under the perspective of the MCDA approach, specifically the CPP approach, in order to prioritize the identified barriers. The evaluation of the barriers using the perspective of several stakeholders can contribute to reduce the uncertainties identified in the implementation of reverse logistics, since the strategies can be considered together [18].

In this way, the study intends to analyze the perspective of three groups of stakeholders who are involved in the implementation of reverse logistics in the Brazilian context: (a) Government, (b) micro and small companies, and (c) consumers of electronics. This analysis can provide the point of view of policy makers (government—Ministry of Environment), consumers, which uses the EEE and generates the waste, and the retailers and technical assistances, which, considering the shared responsibility related to waste, hold and should manage it at the end of the life cycle. Thus, analyzing these stakeholders is important to understand how RL works in the context of used EEE.

In the systematic review carried out by Rezaei [24] on the application of the multicriteria approach to decision making on reverse logistics, the author notes that the use of the MCDA approach is very limited in the scope of reverse logistics and, because of its complexity and the coverage of multiple criteria, needs more attention from academics and practitioners.

The article is structured in five sections besides this introduction. The theoretical framework with the main constructs is presented in Section 2. The third section sets out the methods and instruments applied in this research. Subsequently, in Section 4, the results are presented and are then discussed in Section 5. Finally, Section 6 presents the final considerations, the limitations of the study, and suggestions for further research.

2. Theoretical Framework

2.1. Brazilian Environmental Policies

The Brazilian Federal Constitution of 1988, which is the major law in the country, determines in art. 225 the universal right to have an ecologically balanced environment because it is a common good for the people and necessary for the quality of life [25]. In 1999, the Brazilian Policy of Environmental Education was enacted (Law No. 9’795/1999, which stipulates in art. 5 the objective to encourage participation, individual and collective, permanent in preserving the balance of the environment, which is recognized as an exercise of citizenship [26]. Seven years later, the Federal Decree No. 5’/2006 was issued, which instituted the separation of solid waste discarded by public bodies and entities [27]. Finally, after 20 years being discussed in Brazilian National Congress, Law No. 12’305/2010 established the Brazilian Solid Waste Policy (BPSW) [28,29].

The BPSW establishes the shared responsibility to manage several categories of solid waste. In this legislation, the supply chains of various segments are required to implement reverse logistics systems, such as: i) Pesticides, their residues and packaging; ii) batteries and batteries; iii) tires; iv) lubricating oils, their residues and packaging; v) fluorescent lamps, sodium and mercury vapor and mixed light; vi) electronics products and their components; and vii) packaging in general [11].Based on this concept, the law determines that the entire supply chain, from the production chain to the final consumer, from the entrepreneurs to the servers, is responsible for their participation in the use and proper final destination of the waste. The country has some of the most advanced legislation in relation to the environment in Latin America and the Caribbean countries [30].

Even the performance of a solid waste policy can be related to political barriers, as can be inferred from reading the study by Muchangos, Tokai, and Hanashima [31], which identified that institutional weakness and the lack of cooperation between stakeholders contributed to the poor performance of that policy in Mozambique.

The BPSW demonstrates the need of sharing responsibilities for waste generated in post-consumption, as the proper disposal of waste depends on the work of everyone involved [32]. However, for Heber and Silva [33] there is still a lack of consensus and interaction between stakeholders, which was perceived when the authors analyzed the institutionalization of BPSW.

The legislation, in art. 33, makes mandatory the structuration and implementation of reverse logistics systems for manufacturers, importers, distributors, and retailers of electronic products and their components [11]. Extended Producer Responsibility (EPR) has been used in many countries as a strategy for achieving objectives—including the adoption of RL. An important feature of BPSW, besides considering the EPR, is to demand the implementation of reverse logistics through a sectoral agreement, arising from the consensus among stakeholders involved in each supply chain, reflecting the above-mentioned categories of residues [13].

2.2. Reverse Logistics

The Green Supply Chain Management (GSCM) covers, among other processes, the activities of reinserting waste into the production process, including manufacturing activities, green design, green operations, and reverse logistics, among others [34].

Reverse logistics differs from waste management due to its focus on adding value to a product that has already been used, while waste management aims to collect and carry out treatment without producing a product [35]. Reverse post-consumption logistics is an environmental practice and, therefore, can be considered as a part of the circular economy cycle, since the waste must leave one production cycle and enter another, immediately after its generation, being reused [36].

Thus, reverse logistics is the area of business logistics that aims to plan, operationalize and control the flow and return of goods—after-sales and post-consumption—to the production cycle, adding value to them [6]. The main reverse logistics activity is the return of the products used or unused to the appropriate destination, made possible by the growth of quantities and the varieties of products to be processed or returned at the end of their life cycle in accordance with the environmental requirements [37,38]

Products at the end of their life cycle can be reused, recycled, and recovered when properly disposed of, with their value reinstated [11,38]. With the collected waste, cooperatives of waste pickers of recycled materials will sort and sell these materials to recycling industries. Consequently, besides extending the life cycle of the residue, this initiative will also promote the social inclusion of waste pickers, which in general are vulnerable workers and devalued during the process of waste management conducted by the municipalities [32,39].

2.2.1. Waste of Electrical and Electronic Equipment (WEEE)

The Council of the European Community [40] (p. 129) art. 1 art. 1 defined waste as “any substance or object that the holder discards”. Thus, the European Union [41] used this definition and added that Waste of Electrical and Electronic Equipment (WEEE) includes everything that is part of the product at the moment of its disposal, such as components, sub-assemblies, and materials that can be consumed. This type of waste comes from several sources, such as the government, companies, and households [10].

According to the report [10], the inadequate management of WEEE is causing an annihilation of raw materials, which are scarce and valuable, mainly the precious metals gold, silver, copper, aluminum, etc. WEEE contains substances such as aluminum, barium, lead, and mercury, among others [5], all considered as carcinogenic elements [10]. Consequently, they can generate three types of risks: i) Contamination of consumers, ii) contamination of the environment, and iii) contribution to global warming [42].

According to the Brazilian Industrial Development Agency (BIDA) [43] electrical and electronic equipment in Brazil can be classified into four categories called: 1) White line—stoves, clothes washers, dryers; 2) brown line—monitors, televisions, camcorders; 3) blue line—mixers, electric irons, drills; and 4) green line—notebooks, printers, cell phones.

While the BPSW defines the shared responsibility in relation to electronic products, authors identified that there are conflicting points of view between the parties—manufacturers, retailers, and commerce—that use the gaps in the legislation to hinder the fixing of the costs involved in the operationalization of reverse logistics of these residues [44]. For this reason, an in-depth analysis of the perspectives of the different stakeholders is essential for the sustainable implementation of the RL.

2.2.2. Stakeholders

Stakeholders are denominated as any association of persons or identifiable individual that may influence or be influenced in the fulfillment of the purposes of an organization or are committed for these purposes [45]. In the context of reverse logistics, the stakeholders are the agents involved in all activities that allow the revalorization of the residues, such as suppliers, manufacturers, retailers, wholesalers, consumers, service providers, and government [6]. Bouzon et al. [18]. identified the possible stakeholders involved in reverse logistics and categorized them into eight groups, as shown in

Table 1. Reverse logistics stakeholders.

Stakeholders	Description
Consumers	Customers and buyers.
Society	Society, community, and Non-Governmental Organizations (NGOs).
Marketplace	Market and competitors.
Providers	Upstream party in the supply chain.
Organization	Focal company (shareholders are included).
Employees	Focal company workforce.
Media	Information dissemination means (traditional and social).

Source: Adapted from Bouzon et al. [18].

.

In Brazil, the socio-productive inclusion of waste pickers’ cooperatives is considered in the RL process as a BPSW instrument to enable the implementation of Law 12’305/ 2010. In the study by Cardoso, Xavier, Gomes, and Adissi [46], it was identified that there are differences in attractiveness for different decision makers depending on the alternative chosen. Thus, stakeholders involved in WEEE reverse logistics need to discover how to cooperate and work together in order to meet the requirements of the BPSW [44].

In addition, an implementation of reverse logistics faces different barriers that cross as inconstant variables, being of a cultural, territorial, and technological nature [44]. According to Souza and Vieira [47], practices such as environmental regulations, ecological aspects, and space enhancement are what motivated Brazilian companies to implement RL. An implementation difficulty is making an estimate of the participants involved in the process [47].

2.2.3. Barriers to the Reverse Logistics Implementation

As previously mentioned, companies face complexities and barriers during the implementation of reverse logistics practices [14], which prevent products at the end of their life cycle from being reused or recovered. Logistics can be seen as a barrier when analyzing the implementation of the circular economy due to the need for integration between members—internal and external—of the supply chain [48].

There are nine categories of barriers that occur during the implementation of reverse logistics, which can be classified as: Economic and financial, legal, environment, policies (rules), management, knowledge, market, technique, and technology [49]. The lack of skilled workers for the practice of reverse logistics can also be understood as a barrier, especially in developing countries [17].

For other authors, barriers can be ordered according to four main factors, which can be internal to companies, such as management, finance, and infrastructure, or external, in the case of the policies employed [16]. Barriers vary between authors due to the perspective analyzed [18,19,20] and the reverse logistics recovery product, as in the study of Gardas, Raut, and Narhede [21]. For these authors, there are 10 barriers for the automotive oil sector: Lack of knowledge and awareness of environmental impacts, high cost of legal disposal, inadequate government policies, lack of integration between vehicle manufacturers and refiners, and inconsistent quality oil, among others.

2.3. Multiple Criteria Decision Aid—MCDA

A multicriteria decision problem comes down to a situation that involves at least two criteria and two alternatives to be chosen and that decision is driven by the desire to satisfy several objectives, which can be conflicting [50]. Thus, Multiple Criteria Decision Aid (MCDA) or Multiple Criteria Decision Making (MCDM) consists of analyzing decision problems that present different opinions that must be considered [51]. Its results are not intended to propose a single solution for the decision maker(s), as opposed to the concept of optimization, commonly used in hard operational research [52].

The choice of method depends mainly on four factors: (a) Problem, (b) context examined, (c) core of preference of the decision makers, and (d) problematic involved [53]. There is no consensus regarding the best method to be employed, and the rationality (whether compensatory or non-compensatory) of the decision maker must also be considered when choosing the method, as this will involve the existence or not of trade-off relationships between the criteria [54].

To understand the preference of the decision makers, it is necessary to assess the individual subjectivity and perceptions, describe the details of the decision context, check the points of view in common, and analyse if there are other decision makers or stakeholders involved [52]. The decision maker’s rationality can be additive, allowing a trade-off between criteria, or non-additive, which does not allow trade-offs. In the non-additive form, the alternatives are evaluated according to the relations of preference: Strong preference, weak preference, indifference, and incomparability [54].

In addition, another element should be emphasized, the MCDA has some approaches (schools of thought): (1) Multi-attribute utility methods (American school), (2) interactive methods, and (3) outranking approach (French school) [54]. Depending on the rationality of the decision makers [52], each approach presents options of methods, also considering the problematic involved: Description (detail of the decision problem); choice (choose one or more alternatives from a wide set); ranking (rank the alternatives according to performance), and categorization (classify the alternatives by groups, considering similarity) [52].

Composition of Probabilistic Preferences—CPP

The composition of probabilistic preferences was proposed by Sant’Anna and Sant’Anna [55]. It consists of an MCDA method that applies a probabilistic view that considers the imprecision present in the measurement of attributes or preferences [56], to minimize the effects of errors in the criteria measurement [57].

The traditional composition of probabilistic preferences is developed in four steps: (a) Identification of the comparison criteria and the variables used, (b) evaluation of the alternatives (options) under each criterion, (c) definition of the composition of the criteria, and (d) calculation of the probabilities of preferences [56]. Thus, the first step involves, besides defining the criteria, the alternatives (options) that will be compared and who will be the decision maker [56,58].

The data collection could be conducted through the elicitation process, by interviews or questionnaires. The responses of the evaluations of each criterion represent, in the CPP, position parameters of statistical distributions [56]. Then, the alternatives (options) are compared according to each criterion, previously established, which results in the preference probabilities [56]. To obtain the probability scores together, the CPP analyzes two aspects: (1) The view adopted by the decision maker in the construction of the joint probabilities, and (2) the joint modeling of the disturbances that affect the assessments, being divided into two axes [59].

The evaluations may be categorized as conservative or progressive. Sant’Anna [59], explains that the term “conservative” refers to the vision of avoiding losses and that the term “progressive” refers to having a focus on obtaining gains. The second aspect (modeling) has an optimistic–pessimistic axis [59]. The optimistic extreme considers acceptable that a criterion is met as the best or the worst [57], [59]. An extreme pessimist, on the other hand, shows a preference for measurement by the probability of maximizing or not all criteria [59].

An important aspect of this method is that not only criteria evaluated in linguistic terms, which generally employ a Likert scale, but also those evaluated using numerical measures cover some inaccuracy that can be modeled [56].

The CPP is developed in two steps: 1) Define the probability of being the best and/or the worst option; 2) carry out the composition of the probabilistic preferences. In the first moment, an ordering of the set of options is established through the determination of preferences according to the type of criteria. After determining this ranking, the maximum (Mij) and/or minimum (mij) probabilities of each option (alternative) i must be calculated in relation to the others, combining the preferences obtained for each criterion j [60]. To obtain the probability of an option i being the best (Mij) or being the worst (mik), according to each criterion j, one must assume statistical independence to simplify the calculation of the joint probabilities by the product of the marginal functions of probability, according to the following equations.

$$M_{ij}=\underset{D_{X_i}}{{\displaystyle \int }}\left[\prod F_{X_{-i}}\left(x_{-i}\right)\right]f_{X_i}\left(x_i\right)d{x}_i$$

(1)

$$m_{ij}=\underset{D_{X_i}}{{\displaystyle \int }}\left[\prod (1-F_{X_{-i}}\left(x_{-i}\right))\right]f_{X_i}\left(x_i\right)d{x}_i$$

(2)

Regarding the first aspect, depending on the approach considered, different joint probabilities can be used. In this sense, such approaches can be divided in terms of the choice between positions among two basic orientations, optimistic versus pessimistic and progressive versus conservative, as shown in

Table 2. Different approaches for combining probabilistic assessment.

Optimistic	Pessimistic
It considers satisfactory to meet at least one criterion.	It seeks optimization according to all criteria.
Progressive	Conservative
Consider the probabilities of maximizing preferences.	Consider the concern only in avoiding negative extremes; chances of not minimizing preferences.

Adapted from [61].

[60].

With regard to the second aspect, by combining the positions at the extremes of these two orientations, four different measures are generated [60]:

1)

Optimistic / Progressive: OP = 1–π (1-Mij);

2)

Optimistic / Conservative: OC = 1–πmij;

3)

Pessimistic / Progressive: PP = πMij;

4)

Pessimistic / Conservative: PC = π (1-mij).

All the probabilities described are based on the premise of independence of the criteria and π means the product operator with m terms is obtained by varying the options (i) over all criteria (j) [60].

3. Methods

This study brings a quali-quantitative approach, as it uses the Multiple Criteria Decision Aid (MCDA) approach to analyze the data related to the perception of decision makers, namely: The Brazilian Government (Ministry of Environment), micro and small companies that act in the reverse logistics processes to revalorize e-waste, and consumers. The MCDA modeling is similar to those used in operational research and consists of the following steps: (i) Describe the decision context; (ii) identify the criteria (variables) and alternatives to be used; (iii) define the parameters of the model chosen; (iv) evaluate the alternatives under criteria; (v) make recommendations. The MCDA approach was chosen because it does not present a single (optimal) solution to the research problem, but rather offers the indication of the set of best alternatives for the decision makers’ choice. This approach also analyses the multiple perspectives of decision makers, based on multiple criteria [62]. The method from the MCDA approach applied was, specifically, the Composition of Probabilistic Preferences (CPP) [22]. This method was selected because it considers the imprecision present in the elicitation process and the inherent subjectivity—to avoid errors that occur during individual and/or group decision making [22,23,57].

Data collection was performed in two steps: (i) We considered secondary data, such as papers published in international and Brazilian journals, in order to identify the main barriers related to the implementation of reverse logistics of WEEE, which contributed to the development of the questionnaires’ items; and (ii) the elicitation process, which consisted in the analysis of the perceptions of the decision makers and the establishment of values (direct ones or through a built scale) to evaluate the alternatives. The data related to this step was collected through questionnaires, including Likert-type scales (5 points). The content analysis, proposed by Bardin [63], was carried out to analyze the papers to develop the questionnaires [64], and the MCDA method and CPP were used to evaluate the preferences of decision makers related to the prioritization of barriers to be solved for the implementation of reverse logistics. Thus, Figure 1 synthetizes the methodological design of the research.

The study was set in two main parts: (1) Systematic literature review to gather barriers to the implementation of reverse logistics; (2) the application of the CPP methods to prioritize the barriers to be solved.

3.1. Study 1: Systematic Literature Review (SLR)

Four systematic literature reviews were carried out in order to gather the main barriers in the implementation of reverse logistics, besides their main characteristics. Considering that the decision problem involved in this paper occurs in Brazil, two SLR considered Brazilian bases as scope and the other two considered international bases.

The protocol from Cronin, Ryan, and Coughlan [64] was applied to select and filter the papers, which consists of five steps: (a) Formulation of the research question, (b) establishment of inclusion and exclusion criteria, (c) selection and access of literature, (d) evaluation of the quality of literature, and (e) analysis, synthesis, and dissemination of results.

The barriers identified in the SLR are shown in

Table 3. Reverse logistics barriers.

Internal	Category	Barriers		Reference
	Financial	IB 1—	There is a high initial and operational cost for implementing RL	Abdulrahman et al. [16], Prakash e Barua [20], Bouzon et al. [17], Gardas et al. [21] and, Sirisawat e Kiatcharoenpol [19].
		IB 2—	Investments are high and return on investment is low
		IB 3—	There is a high financial burden of the tax
		IB 4—	Expenses with collection and storage of used products
		IB 5—	Lack of economic incentives and investments
		IB 6—	Need to create / have a training fund
		IB 7—	Lack of funding to implement RL
		IB 8—	Uncertainty related to economic issues (Risks, market uncertainties, among other issues)
	Management or organizational	IB 9—	When implementing RL, there is no sharing of costs and responsibilities	Abdulrahman et al. [16], Prakash and Barua [20]), Bouzon et al. [17] Demajorovic et al. [13], Souza et al. [65], Deus et al. [66], Gardas et al. [21] and, Sirisawat and Kiatcharoenpol [19].
		IB 10—	Difficulty in Extended Producer Responsibility between countries
		IB 11—	Lack of establishment and classification of the indicators of the processes involved in RL
		IB 12—	Lack of coordination, support and sharing of RL practices
		IB 13—	Lack of interest by companies in implementing RL
		IB 14—	Lack of specialized staff
		IB 15—	Lack of strategic planning
		IB 16—	Lack of internal policies for RL practices
		IB 17—	Lack of waste management practice
		IB 18—	Importance of LR in relation to other issues (cultural, economic, among others)
		IB 19—	The policy of organizations
	Infrastructure	IB 20—	Need to adapt the system of organizations	Prakash e Barua [20], Bouzon et al. [17], Gardas et al. [21] and, Sirisawat and Kiatcharoenpol [19].
		IB 21—	Increased logistical and processing capacity
		IB 22—	Complexity in adapting operations when implementing RL
		IB 23—	Lack of indoor facilities
		IB 24—	Inconvenience or lack of routine service collection
		IB 25—	Organization of reverse channels that enable reuse, recycling, and remanufacturing
		IB 26—	Limited forecasting and planning
		IB 27—	Underdevelopment of recycling technologies
	Technological	IB 28—	Low security of data and information within the supply chain	Prakash e Barua [20], Bouzon et al. [17], Deus et al. [66], E. D. Souza et al. [65], and Sirisawat e Kiatcharoenpol [19].
		IB 29—	Shortage of specific and latest equipment
		IB 30—	Lack of technical assistance to reverse logistics partners
		IB 31—	Lack of technical knowledge to support RL practices and
		IB 32—	Lack of information system, technology, and performance measurement.
External	Cultural	EB 1—	Lack of education of the population regarding reverse logistics	Abdulrahman et al. [16], Prakash e Barua [20], Souza and Vieira [47], Demajorovic e Sencovici [13], Aurélio and Ueno [67], Bouzon et al. [17], Demajorovic et al. [44], Deus et al. [66], E. D. Souza et al. [65], Gardas et al. [21], Sirisawat e Kiatcharoenpol [19].
		EB 2—	Lack of knowledge and awareness of the environmental impacts generated by RL
		EB 3—	Lack of knowledge about taxation of returned products
		EB 4—	Resistance to changing current models to include reverse logistics processes
		EB 5—	Cultural issues (other issues that have not been addressed)
	Political or legal	EB 6—	More informal waste management practices than formal ones	Abdulrahman et al. [16], Demajorovic e Sencovici [13], Prakash e Barua [20], Aurélio and Ueno [67], Bouzon et al. [17], Demajorovic et al. [44], Deus et al. [66], Frota et al. [68], Gardas et al. [21], Ribeiro et al. [69], Silva et al. [70] and Sirisawat e Kiatcharoenpol [19].
		EB 7—	Absence of government oversight of reverse logistics practices
		EB 8—	Lack of sectoral agreement for reverse logistics
		EB 9—	Lack of inter-ministry communication
		EB 10—	Lack of an applicable law and a systematization of all the rules that deal with RL
		EB 11—	Presence of loopholes in the regulations (responsibilities, costs)
		EB 12—	Standardization (excess of standards)
		EB 13—	Legal issues (other legal and political issues)
	Related to the market and competitors	EB 14—	Absence of marketing for remanufactured product	Demajorovic et al. [71], Mota et al. [72], Prakash e Barua [20], Bouzon et al. [17], Deus et al. [66], Gardas et al. [21] and Silva et al. [70].
		EB 15—	The customer’s perception of RL is negative
		EB 16—	Difficulty entering as a supplier of recycled material to industries
		EB 17—	Market to recover undeveloped product
		EB 18—	Little recognition of competitive advantage
		EB 19—	Quality and quantity of return are considered uncertain
		EB 20—	Competition issues (other issues that have not been addressed)
	related to the supply chain process and product	EB 21—	Inadequate organizational acceptance and cooperation	Abdulrahman et al. [16], Demajorovic et al. [71], Souza and Vieira [47] Mota et al. [72] Prakash e Barua [20] Bouzon et al. [17], Gardas et al. [21], Ribeiro et al. [69], Silva et al. [70] and Sirisawat e Kiatcharoenpol [19].
		EB 22—	Difficulty of integration between companies when implementing RL
		EB 23—	Lack of support from supply chain partners
		EB 24—	Lack of structuring of the cooperatives, which receive the material
		EB 25—	Lack of adequate performance management
		EB 26—	Process feasibility

.

Abdulrahman et al. [16], Prakash and Barua [20]), Bouzon et al. [17] Demajorovic et al. [13], Souza et al. [65], Deus et al. [66], Gardas et al. [21] and, Sirisawat and Kiatcharoenpol [19].

3.2. Study 2: Application of the CPP Method

For the application of the CPP method to prioritize the barriers found in literature, three categories of decision makers were chosen: i) Government; ii) micro and small companies acting as technical assistance to repair and recondition EEE; iii) final consumers. Considering the research gaps pointed to by Bouzon et al. [17], we decided to consider micro and small companies (technical assistances), instead of manufacturers and retailers, because they were already the focus of former studies. We also decided to include the government (Ministry of Environment), considering that it is in charge of the creation of policies and rules to be followed by organizations and consumers. Finally, we decided to include consumers of repaired or refurbished electro-electronic equipment (EEE) because former studies focus mainly on consumers of new EEE.

It is also important to mention that this study did not have a quantitative approach. The questionnaires were used to conduct the process of elicitation (get the preferences from decision makers), which is characteristic in MCDA methods, such as CPP. As mentioned before, the MCDA methods were considered the soft part of the operational research, so the nature of the study was quali-quantitative. For this reason, the definition of the sample does not follow the regular quantitative sampling requisites (i.e., probabilistic or not). Thus, the intention of the study was not to demonstrate a characteristic or reality of the universe. Instead, it was limited to the judgments and preferences of the respondents. That being said, to participate in the study the decision makers had to, basically: (1) Be resident/located in Brazil, (2) agree to participate in the study, and 3) have knowledge and experience in the topic analyzed in the study. Hence, the criteria of accessibility and representativeness were chosen to select the sample. These criteria were adapted from the study conducted by Bouzon et al. [18]. The sampling of participants from each category of decision makers was by convenience and detailed as follows:

i) Government. The participants should be staff from the Ministry of Environment (MMA) and have technical knowledge on e-waste reverse logistics and on BPSW. All employees from MMA with those characteristics were invited to answer the questionnaire by email and we had the return of three respondents. While small, this sample was adequate because the respondents act directly in a work group related to the creation of a Brazilian RoHS (Restriction of Certain Hazardous Substances) aiming to control dangerous substances in electronic equipment (EEE). This group is formed by several organizations related to EEE, such as MMA (Brazilian Ministry of Environment), ABNT (Brazilian association of Technical Standard), Abinee (Brazilian Association of Industry of Electro and Electronic Equipment), Eletros (Brazilian Association of Manufacturers of Electro-Electronic Products), and others. Three people from the Brazilian Ministry of Environment (MMA) participating in this work group were related to EEE and also acted in the discussions related to sectoral agreement of reverse logistics of e-waste related to BPSW. Thus, the three selected to participate in our research were chosen due to their knowledge and experience related to the topic analyzed. So, representativeness and accessibility were the main criteria to filter in the participants.

ii) Technical Assistance companies (authorized by the industry of EEE or not). The participants must have a micro or small size and work in a reverse logistics program of e-waste already implemented or under implementation. It is important to point out that retailers or resellers of EEE were not considered for this study; the technical assistance companies (micro and small size) were chosen considering the scarcity of studies approaching this type of organizations and the role they perform in Brazil, considering that the purchase of EEE in a secondary market is a reality [5,17,47]. To access the base of technical assistances, we checked the website of the Brazilian Association of Manufacturers of Electro-Electronic Products (ELETROS), and we estimated the amount of existing technical assistance in Brazil as approximately 11,000 cases. As a result, we obtained 30 responses, with 25 valid cases that represented 11 of the 27 Brazilian states. The states of São Paulo and Rio Grande do Sul have the highest number of respondents. Most of the technical assistance analyzed has more than eight years of existence (68%).

iii) Final consumers. The respondents were individuals who had bought electronic equipment sold by technical assistance at least once, who buy used EEE or who bought repaired or reconditioned electronic equipment. For this group, we obtained 74 responses to the questionnaire. However, according to the criteria presented, 21 responses were excluded and three were duplicated responses. Thus, the final sample was of 49 responses. Among consumers, 27 (55.10%) had already consumed at least one piece of electrical and electronic equipment resulting from technical assistance and another refurbished one; 17 had already purchased reconditioned equipment; and just five consumed technical assistance only.

The main criteria considered for this group was accessibility because consumers should agree to participate. The questionnaire was disclosed by e-mail, listservs, and social networks. Within the sample of final consumers, 28 were female (57.15%) and, regarding the level of education, most of them had completed post-graduate studies (42.86%). With regard to the distribution in Brazilian territory, consumers belong mostly to the Federal District (N=20), which means 40.82% of the sample.

3.3. Research Instrument

To identify the barriers found in the implementation of the reverse logistics of e-waste, considering three different categories of decision makers, we prepared three adapted questionnaires. While each questionnaire has proper questions, it has the same core questions, considering that the aim is to prioritize the barriers to be solved.

The questionnaires were divided into six parts: (1) Term consent, free clarification, and which target audience the respondent is part of, (2) knowledge about reverse logistics activities adapted for each target audience, (3) group of statements referring to internal barriers, (4) group of statements referring to external barriers, (5) open questions, and (6) sociodemographic questions. The statements were developed based on systematic literature reviews and were answered according to a Likert scale of agreement—1 being strongly disagree and 5 totally agree.

As presented by Santa’Anna [56], to facilitate the understanding of the results, simple variables were used to represent the criteria. As some criteria did not have direct measures [56], the Likert scale of agreement could be used in this context. Thus, the higher the result of the criterion, the more evaluations with 5 (I totally agree) obtained and, therefore, it should be prioritized.

Validation by Experts

Before the questionnaires were applied, we applied a procedure of validation by experts, considering the inexistence of validated scales in the literature in this specific context. This is a well-known and widely adopted procedure to validate research instruments, both in qualitative and quantitative approaches. This step aimed to conduct a kind of pre-test of the questionnaire in order to ensure adherence to the research objectives and the clarity and pertinence of the statements elaborated to evaluate the criteria. The choice of experts was based on the study by Bouzon et al. (2016) who used specialists (decision makers) to elicit the importance of the barriers identified in the literature review. Then, an email invitation was sent to 13 professionals and academics, experts in reverse logistics covering logistics professionals, academics, and representatives from the EEE industry. Considering the criteria of accessibility, nine judges agreed to participate in the validation process. The experts assessed the clarity of the questions, the adherence, and the relevance of the items.

3.4. Application of Composition of Probabilistic Preferences—CPP Method

With the collected data from the three categories of decision makers, we conducted the application of the CPP method to prioritize the barriers. As presented by Sant’Anna [58], the CPP is developed in four stages:

First stage: The criteria and/or decision makers to be used are defined [58]. In this way, the decision makers are the respondents of each study (government—Ministry of Environment, technical assistance companies, and consumers of second-hand electronics) and the criteria are the syntheses of each barrier identified in the SLR of foreign and Brazilian studies.

Second stage: Evaluation of alternative options according to each criterion [56]. Due to the large number of barriers presented in the questionnaires, the recommendation given by Sant’Anna [56] was followed and, as the statements/answers were already on the Likert scale, no pairwise comparison needed to be performed. We chose to separate the results of each category, one for internal barriers and another for external barriers. In this way, we could have assessments according to the criteria of each group [56].

Third stage: Choice of the form of composition of the criteria [56]. The joint probability for each alternative was calculated to present its maximum value (M_ij), using Formula 3 [73]_:

$$Mij=\int D x_i\lceil \prod F{X}_j\left(x_j\right)\rceil f{X}_j\left(x_i\right)d{x}_i$$

(3)

In Formula 3 x_i, for i = 1 up to the number of criteria, F_Xj is the cumulative distribution of the variable x_j, f_xj is the density function and D_xi represents the support of the random variable x_i [73]_. The calculations were made for each alternative (barrier) referring to each criterion [23].

With the joint probability of the evaluations carried out, the points of view that will guide the decision and the scenario of the decision maker were defined. As presented in the theoretical framework, two aspects should be analyzed for choosing the scenario: (1) The view adopted by the decision maker in the construction of joint probabilities, and (2) the joint modeling of the disturbances that affect the assessments, being divided into two axes (pessimistic and optimistic) [59]. Sant’Anna [59], presents that there are four positions in which the decision maker can be. Thus, it was defined that the decision maker’s view would be pessimistic since it is intended to maximize all the criteria and not to maximize just one, as is the optimistic point of view. The existence of a barrier is considered negative since it prevents the implementation of reverse logistics. To have a pessimistic point of view in this context means that the preferences of a decision maker should be more rigorous/strict. As a model, we opted for a progressive profile, as we sought to obtain the highest risk probability of each criterion, complementing the idea of pessimism, when considering the chances to maximize preferences.

Thus, the vision for the research was Pessimistic and Progressive (PP). As explained by Sant’Anna [59], when applying the PP view, the global preference score, the estimated joint probability, is the probability of having common points across all event criteria corresponding to the alternative that was best evaluated. Therefore, this research sought to prioritize the barriers, which can all be identified as more important and needing urgent action.

Finally, the preference probabilities are calculated for each criterion. You must define a distribution for the calculation. As presented by Casado [57], the distributions used in the traditional CPP are: Beta, empirical, normal, pareto, triangular, and uniform. As the answers were based on a Likert scale and because it was not a normal distribution, a beta-PERT distribution and a triangular distribution were performed.

Without the knowledge about the interrelationships of the evaluation of the criteria, it is understood that the criteria are independent. However, as Sant’Anna [59] points out, if significant differences are identified between global scores and maximum dependence, a search is made to understand the effect of intermediate levels of dependence. If no differences are identified, it can be assumed that they are independent [59]. Thus, it was decided to use the formula for calculating the compositions of probabilities from the progressive-pessimistic point of view due to the independence hypothesis [73].

$$PP{ }_i=\prod M_{ij}$$

(4)

Thus, the CPP package [74], version 0.1.0, Software R [75], was used, applying the PMax.Beta function. First, the probabilities of assessing the internal barriers for each group of decision makers were calculated and then also for external barriers. Thus, for each group, two rankings were generated: Internal barriers and external barriers.

4. Results

This section presents the results obtained by topic: (1) The RL REE barriers in the three analyzed perceptions (government, micro and small companies, and consumers), (2) the validation of the barriers identified in the SLR by the respondents, and (3) prioritizing internal and external barriers. Each topic is subdivided by group and discussed.

4.1. Qualitative Analysis of Barriers in the Implementation of Reverse Logistics of E-Waste Based on Comments from Decision Makers

In the questionnaire after the evaluation of each barrier, as presented in the method, there was an open space for the respondent to report something that he/she considered relevant. In this way, this topic presents the reports regarding the perceptions of decision makers in relation to barriers. For this reason, first we will present the qualitative analysis of these comments and then the application of the CPP.

During the assessments of barriers, both external and internal, respondents from the government group did not present any comment regarding the barriers. Thus, no new barriers were identified from the government’s point of view.

Regarding the respondents of the micro and small companies (technical assistances) group, only eight of the respondents added some comment when evaluating each category of barrier. In reference to financial barriers, for one of the decision makers this is an environmental issue; for another, it is a matter of different tax laws by state.

Regarding the barriers categorized as management or organizational, one respondent from technical assistance commented on the Ente nazionale per l’energia elettrica (ENEL) program—an energy distributor in the state of São Paulo; for the respondent it is an incentive for him to try to recycle the waste as much as possible, as he receives a bonus on his electricity bill. Another respondent from technical assistance pointed out that there is a lack of interest between the factories and the accredited technical assistance, which makes the process difficult, thus “leaving the minor responsible for the correct disposal” (Decision maker_MSE-23).

Thus, from the point of view of technical assistances, the barriers are related to: i) The lack of commitment with both the technical and legal aspects; ii) costs and inventories; iii) difference in tax legislation in each Brazilian state; iv) absence of educational campaigns and recycling; and v) absence of a material storage structure.

Regarding the perception of respondents from the consumer group, only nine of the 49 presented any comments after the evaluation of each barrier. The first barrier presented was the financial one, which obtained five comments. Two consumers pointed out that it is difficult to judge this type of barrier, since as consumers they have no experience. Considering the comment from Consumer-32: “Many of the issues are difficult to assess, from the user/consumer point of view”.

For other consumers it is the reality, but organizations need to be encouraged in order to have an effective application of reverse logistics and they should disclose more to the community, so they could get their products back. In addition, for another respondent, organizations should foresee the costs involved in reverse logistics and taxes should be reduced to promote a cost dilution without overloading the final consumer.

Other comments are related to the implementation collection points of e-waste. A respondent stated that these barriers exist and are known, but there is little practice to overcome them. In addition, another respondent pointed out that reverse logistics will not be fully implemented unless managers understand the importance and mandatory nature of the law (Brazilian Policy of Solid Waste).

Related to cultural barriers, a respondent from the consumer category demonstrated knowledge about the concept of shared responsibility established by BPSW as inferred from his comment: “If the consumer/company/government is not aware that the destination of their waste is their responsibility there is no way to implement the reverse logistics” (Decision maker_Consumer-36).

Regarding the perceptions of legal or political barriers, one respondent related them to the cultural barriers presented. For this respondent, when the consumer is aware of the need to recycle, he/she looks for information, but if the manufacturer presented this information, such as a return policy sent with the product, informing where to deliver it, it would make the process easier.

Summing up, the perceptions of the respondents in the consumer category show the barriers are mainly related to costs—consumer overload and uncertainties; the need to disclose information and incentives; the absence of RL practices; the recognition of the importance and the mandatory nature of the BPSW in relation to RL; behavior; and people’s resistance to changing habits. The next section presents the application of the CPP.

4.2. Analysis and Prioritization of the Barriers with the Application of CPP

The CPP method was applied, opting for the pessimistic-progressive scenario, and as a result six rankings were generated, two per study, using the beta-PERT distribution after comparisons between the rankings of each distribution. The results are presented by group.

4.2.1. Category of Government (Ministry of Environment)

The results obtained presenting the government’s view by the respondents of the Ministry of Environment are presented in this section. After data triangulation, a ranking of internal and external barriers was generated, shown in

Table 4. Prioritization of internal barriers by group of decision makers.

Category	Barriers	PP Vision
		Government	MSE	Consumers
Financial	IB_1	2,60E-05	3,1022E-198	2,20E-184
	IB_2	1,60E-19	1,96E-190	6,60E-259
	IB_3	1,60E-19	1,29E-209	1,70E-185
	IB_4	1,71E-08	4,57E-139	5,98E-191
	IB_5	2,85E-25	9,24E-46	9,40E-138
	IB_6	1,80E-11	1,00E-180	1,20E-183
	IB_7	1,80E-11	3,01E-128	0
	IB_8	4,34E-22	5,15E-132	2,40E-228
Management or organizational	IB_9	2,37E-37	1,33E-178	9,50E-293
	IB_10	3,02E-28	5,34E-173	4,10E-221
	IB_11	2,83E-15	2,99E-178	1,30E-156
	IB_12	1,34E-17	2,02E-151	1,30E-147
	IB_13	1,02E-15	1,33E-178	3,30E-135
	IB_14	1,34E-17	1,33E-178	7,50E-179
	IB_15	1,34E-17	6,07E-123	2,00E-145
	IB_16	1,02E-15	1,73E-115	1,90E-127
	IB_17	0,004092961	1,58E-126	6,30E-155
	IB_18	8,01E-25	4,05E-150	1,50E-135
	IB_19	2,74E-08	2,91E-162	4,80E-141
Infrastructure	IB_20	1,71E-08	8,25E-145	6,00E-144
	IB_21	2,74E-08	3,68E-188	2,30E-166
	IB_22	2,51E-06	2,36E-121	2,30E-187
	IB_23	2,85E-25	7,60E-101	1,10E-178
	IB_24	1,02E-15	5,48E-118	4,50E-191
	IB_25	1,02E-15	1,62E-116	3,50E-190
	IB_26	2,74E-08	1,62E-137	5,50E-256
	IB_27	1,71E-08	3,27E-173	1,20E-302
Technological	IB_28	6,42E-20	1,33E-151	2,50E-272
	IB_29	2,68E-12	2,36E-121	1,10E-230
	IB_30	8,47E-28	5,68E-156	2,20E-208
	IB_31	1,34E-17	1,60E-133	2,80E-252

Note: The notation 1,11e-01 represents 1,11*10^-01.

and

Table 5. Prioritization of external barriers by group of decision makers

Category	Barriers	PP Vision
		Government	MSE	Consumers
Cultural	EB_1	1,31E-05	2,61E-74	2,95E-87
	EB_2	1,61E-07	5,58E-83	1,10E-158
	EB_3	8,66E-10	2,58E-83	1,62E-133
	EB_4	2,46E-07	9,34E-91	5,90E-144
	EB_5	8,66E-10	5,60E-114	2,40E-197
Political or legal	EB_6	3,74E-13	7,35E-81	6,22E-269
	EB_7	5,69E-10	4,10E-105	3,37E-161
	EB_8	2,46E-07	1,11E-62	2,88E-266
	EB_9	4,91E-18	2,81E-74	6,64E-197
	EB_10	4,91E-18	1,30E-103	4,35E-227
	EB_11	1,61E-07	2,26E-97	1,23E-204
	EB_12	3,93E-20	1,80E-104	0
	EB_13	1,75E-18	3,70E-123	0
Related to the market and competitors	EB_14	1,06E-10	8,40E-112	3,44E-219
	EB_15	1,09E-31	6,40E-160	0
	EB_16	8,34E-20	2,60E-122	0
	EB_17	8,66E-10	1,10E-99	1,47E-260
	EB_18	5,87E-17	2,81E-74	3,57E-245
	EB_19	3,74E-13	1,61E-85	4,32E-269
	EB_20	1,36E-13	1,60E-108	7,63E-307
Related to the supply chain process and product	EB_21	1,11E-14	3,50E-183	2,57E-286
	EB_22	1,06E-10	6,20E-123	2,15E-262
	EB_23	8,66E-10	7,20E-141	3,08E-254
	EB_24	1,06E-10	1,00E-135	5,29E-240
	EB_25	3,74E-13	2,20E-131
	EB_26	1,06E-10	1,50E-118

Note: The notation 1,11e-01 represents 1,11*10^-01.

. The result of the CPP is presented in the “PP Vision” column, as it represents the probability value of that criterion in the pessimistic-progressive scenario.

The ranking presented in Table 4 is sorted in an increasing order, from the highest priority to the lowest priority. Thus, when analyzing the first five positions of the ranking regarding internal barriers—Table 3—it can be understood that, from the government’s point of view, the main internal barriers are of a financial, management or organizational, and infrastructure nature.

This is because the IB_17 barrier was ranked first in the ranking, the importance of RL in relation to other issues (cultural and economic, among others) being a management or organizational barrier. Second, a probability of 0.0000259703 or, as shown in Table 4, 2.59703e-05 to be chosen, was the financial barrier IB_1, which refers to the high initial and operational cost for implementing the RL.

The barriers IB_5 (lack of economic incentives and investments), IB_23 (inconvenience or lack of routine service collection), BI_30 (lack of technical knowledge to support RL practices), IB_10 (difficulty in extended producer responsibility between countries ) and IB_9 (when implementing the RL there is no sharing of costs and responsibilities) are the last five barriers in the ranking and, therefore, it is understood that these barriers may or may not hinder the implementation of the RL. Thus, it appears that barriers were the ones that received low evaluations.

According to respondents from the Ministry of Environment, the five main external barriers that hinder the implementation of reverse logistics are: (i) Lack of education of the population in relation to reverse logistics; (ii) lack of knowledge and awareness of the environmental impacts generated by the lack of RL; (iii) gaps in the regulations (responsibilities, costs); (iv) resistance to the change of the current models to include the processes of reverse logistics; and (v) lack of government oversight in relation to reverse logistics practices, as shown in Table 5.Thus, like the ranking of internal barriers, the ranking presented in Table 5 is also organized in an increasing way; those of lower priority for respondents in study 1 are in the last positions. In this way, the last barriers of the ranking (22 to 26), according to the government’s view, are the legal issues (EB_13); the excess of norms (EB_12); the difficulty of entering as a supplier of recycled material to the industries (EB_16); the lack of support from supply chain partners (EB_23); and the customer’s perception of reverse logistics being negative (EB_15); they may or may not be obstacles external to RL. For, as shown in the “PP Vision” column of Table 4, these five barriers had the least chance of being chosen, proving to be less important than the others.

4.2.2. Category of Micro and Small Companies (Technical Assistances)

The results obtained representing the vision of technical assistances are increasingly organized in the ranking of Table 4; so the most important internal barriers, which should be prioritized are in the first positions. Therefore, the main internal barriers are: (i) Lack of economic incentives and investments; (ii) the inconvenience or lack of routine service collection; (iii) limited forecasting and planning; (iv) lack of waste management practices; and (v) organization of reverse channels that allow reuse, recycling, reconditioning, and remanufacturing.

Thus, it can be noted that, in the first place, there was a financial barrier (lack of economic incentives and investment), with a probability of 9.24423e-46, much higher than the other four main barriers, as shown in Table 4. These four are classified as infrastructure and management or organizational. Table 5 shows the prioritization of external barriers.

Regarding external barriers, the five main external barriers, which are obstacles to the implementation of reverse logistics, are: (i) lack of sectoral agreement for reverse logistics; (ii) little recognition of competitive advantage; (iii) lack of inter-ministry communication (reciprocal relations between ministries and among ministers); (iv) lack of education of the population in relation to reverse logistics; and (v) existence of more informal waste management practices than formal ones, as shown in Table 5.

4.2.3. Category of Consumers

When analyzing the ranking present in Table 4, it can be observed that for consumers the main internal barrier is the absence of waste management practices, which presented a probability of 1.9e-127 and occupied the first position. Following the ranking, the other main barriers are: (i) The policy of organizations; (ii) lack of interest of companies in implementing RL; (iii) lack of economic incentives and investments; and (iv) need to adapt the system of organizations, as shown in Table 4. Of the five main barriers, the first three are related to the management or organizational category, the fourth is financial and the fifth is infrastructure.

According to the respondents, who represent the point of view of consumers, the main barriers are: (i) Underdevelopment of recycling technologies—investments are high and the return on investment is low; (ii) lack of specific and latest equipment; (iii) when implementing RL there is no sharing of costs and responsibilities; (iv) low security of data and information within the supply chain; and (v) lack of funding to implement the RL—this may or may not hinder the implementation of the RL, since they were the last five barriers classified in the ranking.

The internal barrier IB_7 (lack of funding to implement the RL) was in the last position and had a probability equal to zero. Therefore, according the perceptions of consumers, this barrier does not hamper the implementation of reverse logistics of second-hand electronics in Brazil. Table 5 shows the prioritization of external barriers from the point of view of consumers.

Regarding external barriers, according to respondents from the consumer category, the five main barriers, reflected in the first five positions of the ranking, are: (i) The population’s lack of education in relation to reverse logistics; (ii) lack of knowledge on taxation of returned products; (iii) resistance to changing current models to include reverse logistics processes; (iv) lack of knowledge and awareness of the environmental impacts generated by the lack of RL; and v) lack of government oversight regarding reverse logistics practices. The first four barriers are categorized as cultural and the fifth as legal or political, as shown in Table 5.

5. Discussion of Results

This section discusses the results considering the literature. The section is divided into three topics: Knowledge about reverse logistics, internal barriers, and external barriers.

5.1. Knowledge About Reverse Logistics

During the process of disseminating and collecting data through questionnaires, via telephone, regarding the respondents from micro and small companies (technical assistances), a lack of knowledge about the concept of reverse logistics and about the related activities was verified, given that some respondents from this category asked the meaning of reverse logistics or expressed they could not answer because they did not fit the profile or have enough knowledge. When explaining the concept or practices involved in reverse logistics the respondents stated that they perform some activity and agreed to participate in the research, through the link sent electronically. In addition, some conflicting information from respondents was observed, because when asked if they perform any reverse logistics activity, some answered no. However, when asked to list what activities, they point out some activities. At this moment, we corroborated the barrier, highlighted by Bouzon et al. [17], for this category of respondents, related to the lack of knowledge about reverse logistics practices. We can infer from the analysis of the results in this category of decision maker that some respondents learned from the questionnaire that some activities performed in their company are part of reverse logistics.

This contradiction of answers demonstrates the respondents may think that actions of separation and repair are informal practices and are not part of reverse logistics. Technical assistance may simply try to make money from the resale of products, as they do not have the technology or skilled labor to make the most of the disassembly activities and thus resell the material for scrap iron. This fact strengthens another issue raised in the studies analyzed, namely, the predominance of informal practices related to RL [20].

In contrast, when analyzing the question of knowledge about reverse logistics of consumers, they demonstrated, through their reports, knowledge of RL and the legislation related to BPSW itself—which makes mandatory the reverse logistics of e-waste, among other categories of products, besides the shared responsibility. This fact can be explained by the level of education of the respondents, mainly with higher education or postgraduate studies, or by something intrinsic to the respondents, making them seek information related to the topic and buy second-hand products.

5.2. Internal Barriers

For the government, the main barrier is related to the fact that reverse logistics is neglected in relation to other issues, such as financial ones, corroborating the barriers identified and presented in the study of Abdulrahman et al. [16], who exposed the importance of RL in relation to other issues and Bouzon et al. [17]. Some studies emphasized the need of reverse logistics to be supported in terms of financial resources, considering that it should be maintained and should be feasible in terms of cost and revenue management. While reverse logistics of post-consumption allows, clearly, long-term environmental and social gains, it may not be attractive to companies due to the financial issues.

The government recognizes that financial issues must also be prioritized during the implementation of the RL, by presenting in the second place the initial operational cost barriers, which encompassed the barriers involving high costs, high cost of legal disposal, lack of initial capital and financial resources. Thus, this result confirms the barriers presented by Prakash and Barua [20], Bouzon et al. [17] and Gardas et al. [21].

Additionally, the respondents from the government highlighted other important barriers that can interfere in the full implementation of reverse logistics in Brazil: (i) Need for adaptation in organizations; (ii) complexity of this adaptation; and (iii) low development of technologies aimed at recycling, which are barriers related to infrastructure.

Respondents are also aware of the subject when they note that these three barriers may be related, since organizations need to adapt to follow the recommendations of the legislation that determines how the new model should be—including RL practices—and that this adaptation has a level of complexity, which can be higher or lower.

When recognizing that there is a low development of technologies, it is possible to refer to the main internal barrier identified in the category of decision makers from government: The recognition of the importance of RL. If there is no such recognition, there may not be an investment or the search for technologies that assist RL practices. The absence of demand for these technologies may not encourage their development; thus it can create a vicious circle in organizations.

From the point of view of micro and small enterprises (technical assistances), the most relevant internal barrier, ranked in first position, was the lack of economic incentives and investments. So, analyzing only the prioritization of barriers with the application of CPP, this financial barrier showed up among the main positions of the ranking.

However, when analyzing the comments of the respondents of this category, it is observed that they reinforced and corroborated the main barrier presented in the ranking, as they presented others related to cost, incentives, directed public policy, and meeting what was identified by Bouzon et al. [18] that everything is related to cost.

The lack of economic and investment incentives may represent the absence of policies in this area, the existence of preferential tax policies and financial restrictions are present in the process of implementing reverse logistics, and were barriers indicated in the literature. Thus, they confirm the studies by Abdulrahman et al. [16], Prakash and Barua [20], Bouzon et al. [17], Demajorovic et al. [44], Souza et al. [65] who identified these barriers. For micro and small companies there is coherence between the other four main barriers, as they are related to each other and involve management, structure, and technologies of organizations.

In addition, the lack of routine of collection service represents, in this study, the absence of a structure to collect e-waste, which is complemented by the lack of waste management practices. Indeed, with limited forecasting and planning of waste management and the lack of organization of reverse channels—in the context of the research as mentioned earlier—it represents the absence of organization of the reverse channels that would enable the reuse, recycling, and remanufacturing of products.

From the point of view of consumers, the lack of waste management practices would be the main obstacle, reinforcing the barriers identified in the studies by studies by Abdulrahman et al. [16], Prakash and Barua [20], and Bouzon et al. [17]. It appears that this is the main obstacle for consumers, because most factors are internal to organizations; thus the absence of practices is more tangible. Moreover, if the consumer notices this absence, he may feel unmotivated to separate and deliver his waste, generating in consequence another barrier for RL.

Consumers indicate that the organization’s policy and the organization’s lack of interest in implementing RL can hinder the process or reverse logistics, respectively, in the second and third positions in the ranking of priorities. When indicating that the organization’s policy is a relevant barrier, consumers ratify the barrier presented by Abdulrahman et al. [16], Demajorovic et al. [71], Prakash and Barua [20] and, Deus et al. [66].

In addition, they confirm the other barriers that related to the previous one, such as the lack of policies for reverse logistics practices and the policies of organizations that are against RL, which were presented by Sirisawat and Kiatcharoenpol [19] and, Bouzon et al. [17], respectively.

Because of these internal factors, consumers are only aware of what organizations explicitly demonstrate. There is a connection in the most important barriers for consumers: The consumer realizes that there are no RL practices whether it is not disclosed by organizations or because such practices are not found in an easy way. In this way, the consumer can infer that this absence is due to internal politics, which the consumer is unaware of, or the absence of interest in changing. Depending on the profile of the consumers (if environmentally responsible), it can result in a negative perception of the corporation and can, in the long-term, damage the corporate image.

Consumers agree with the respondents from technical assistance that the absence of economic incentives and investments is a barrier and also agree with the respondents from government that there is a need to adapt the system of organizations when implementing RL processes.

Thus, in the context analyzed, considering the perceptions of decision makers of this study, the management or organizational and the infrastructure barriers are the most difficult to overcome in the implementation of reverse logistics of e-waste in Brazil.

The results, in general, differ between the three profiles of decision makers, which can be considered as normal considering that they represent different points of views and, consequently, have different interests and priorities as found by Demajorovic et al. [44], when analyzing conflicting points of views among manufacturers, distributors, and commerce. This difference is expected, since the levels of knowledge and focus of decision makers are different [46]. In addition, it is common to obtain different results involving groups of decision makers [59].

Table 6. Comparison of internal barriers.

	Government		MSE		Consumers
Position	Barrier	Categories	Barrier	Categories	Barrier	Categories
1	IB_17	Management or Organizational	IB_5	Finance	IB_16	Management or Organizational
2	IB_1	Finance	IB_23	Infrastructure	IB_18
3	IB_19	Management or Organizational	IB_25		IB_13
4	IB_21	Infrastructure	IB_16	Management or Organizational	IB_5	Finance
5	IB_26		IB_24		IB_19	Management or Organizational

compares the five main internal barriers identified in the three studies and their respective categories.

As shown in Table 6, in each category of decision makers, only one financial barrier was identified. For the government, the financial barrier was in the second position (there is a high initial and operational cost for implementing the RL). For consumers, it appears in the fourth position (lack of economic incentives and investments). In turn, for technical assistance, the lack of economic incentives and investments is the first barrier, of a financial nature.

It can be inferred that the government has a long-term view and that legal issues, which it proposes and regulates, would assist implementation. In the view of the government, the legal aspects are not seen as obstacles or barriers; instead, they are a first step to implementing some practice, which denotes a perception that values the reactive posture from companies and society. On the other hand, the micro and small size companies (technical assistances) are more interested in operational issues (structure and management) and concerned with the related costs of the entire process, which corroborate the results found by Souza and Vieira [47]. However, in the case of reverse logistics, due to the irregularity and unpredictability of the volume and demand, it is difficult to estimate the total amount of costs involved. As previously presented, there is a low development of technology (recycling/recovery), which can make the available technology more expensive and not attractive to organizations, especially for the micro and small size companies.

As presented by some consumers, internal issues of companies are difficult to judge, because they are not visible and perceptive by consumers, and also because some of them have no experience with issues related to logistics and handle, for example, only the outside point of view. These comments may also indicate why financial barriers have not been indicated as relevant to consumers.

As identified, according to the ranking of each category of the three perspectives analyzed, financial barriers are not the main obstacle to the implementation of reverse logistics, nor the most important, since only one of the first five barriers pointed out in each category of decision makers had a financial nature. This result differs from the results of studies identified in the systematic literature review in which financial barriers were identified as the most influential in the implementation of reverse logistics, using MCDA methods [17,19,20,21].

Moreover, we diverge from the study by Bouzon et al. [18], which also analyzed three perspectives of decision makers (government, large organization, and consumers), and which identified that the financial barrier was the most relevant. The difference can be explained by the fact that, in our study, the decision makers are dealing with second-hand EEE, besides the consideration of the perspective of respondents from micro and small size companies. Another difference is related to the method chosen for this study, which is non-compensatory (without trade-offs) by nature, considering that that is not desirable in this context and does not require the assignment of weights for the criteria.

As presented by Guarnieri [54], the decision maker’s rationality can lead him to accept or not the trade-offs present in the decision. When opting for the progressive and pessimistic scenario, the rationality of the decision makers became non-additive, the trade-offs are not accepted, since the scenario aimed at maximizing all the criteria, which represented the barriers. Non-compensatory methods, such as those from the outranking approach, present a more balanced result, considering it requires that all alternatives present an adequate performance in all criteria considered in the decision-making process.

Thus, the choice of rationality, the scenario, and the method applied can justify the difference in the results, since Bouzon et al. [17] used AHP, which is compensatory. Other authors also used additive methods in the same context, such as Sirisawat and Kiatcharoenpol [19], and Prakash and Barua [20], which combined AHP with TOPSIS in their studies, both methods belong to the multi-attribute utility approach, which accepts trade-offs between criteria.

5.3. External Barriers

In relation to prioritized external barriers, the government group recognizes and corroborates the literature, stating that there are gaps in legislation and that these hinder the implementation of the LR. As identified by Demajorovic et al. [44], companies take advantage of gaps in legislation to avoid being held responsible, which goes against the concept of shared responsibility for the product’s life cycle, one of the principles of the legislation related to BPSW, according to which everyone involved in production and consumption is responsible for its proper destination. However, that the BPSW does not indicate clearly the limit of responsibility for each stakeholder involved and this gap is used by them to not be held responsible [44]. Related to the perception of respondents from government, it seems that organizations are aware of the BPSW and other legislation demands. According to them, companies can seek knowledge and try to understand all the nuances of legislation to identify its gaps. However, instead of this, according to authors such as Guarnieri et al. [8], the gaps on legislation can damage the decision making process related to the implementation of e-waste, because the stakeholders can implement only the minimum established by law, and when it is not clear how to proceed some stakeholders simply do anything because they cannot by restrained legally. This issue motivates a reactive behavior from some stakeholders, which expect to implement some practices only by law.

Consumers pointed out that, in addition to the lack of environmental education, the lack of knowledge about taxation of returned products is an important barrier from their point of view. While it is a cultural barrier, it is related to the financial barrier IB 9 (when implementing the RL there is no sharing of costs and responsibilities) and, in a way, converges with the government’s vision.

However, the view of micro and small size companies differs from the other two stakeholders but corroborates with literature that the absence of a sectoral agreement for the reverse logistics of electronics is the main obstacle in the implementation. However, this barrier may have been in the process of being overcome, considering that, on October 31, 2019, after our data collection, the sectoral agreement was signed by the Ministry of the Environment and by entities representing the electronics sector [76]. However, it is still in the early stages of implementation and does not present any measurable outcome.

The mentioned agreement can minimize some of the barriers identified by the stakeholders, such as the absence of information on the taxation of returned products, since it proposes that the cost of the reverse logistics system will be included in the total cost of the product and can be informed in the invoice as an observation. Another barrier that can in the long-term be overcome is the absence of environmental education programs, since the sectoral agreement proposes the communication plan and non-formal environmental education [77].

Table 7. Comparison of external barriers.

	Government		MSE		Consumers
Position	Barrier	Categories	Barrier	Categories	Barrier	Categories
1	EB_1	Cultural	EB_8	Legal and Policy	EB_1	Cultural
2	EB_2		EB_18	Market and competitors	EB_3
3	EB_11	Legal and Policy	EB_9	Legal and Policy	EB_4
4	EB_4	Cultural	EB_1	Cultural	EB_2
5	EB_7	Legal and Policy	EB_6		EB_7	Legal and Policy

summarizes the main external barriers and their categories

Analyzing Table 7 with the five main barriers identified in each study, it is possible to perceive that the government and consumers have similar points of view. Both pointed out three barriers—all cultural in nature—as external barriers to be prioritized, but with different importance.

The results of the perceptions of the three categories of decision makers converge by recognizing that cultural barriers are external obstacles, since the five barriers inserted in this category are at the beginning or in the middle of the three rankings. As presented in the studies analyzed in the systematic literature review, there was no category of cultural barriers, so we propose this barrier to be considered in further studies, considering that regional differences should be taken into account to implement private and public policies related to reverse logistics, such as environmental education, socio-productive inclusion of waste pickers, and the culture to donate and save the EEE at the end-of-life, among others.

In addition, all respondents agree, by giving low values to the statement “the consumers’ perception of reverse logistics is negative”, that consumers do not have a negative perception of reverse logistics, which demonstrates that, in fact, they may be willing to buy EEE from companies that come from reverse logistics processes, such as the refurbished and reconditioned ones.

6. Conclusions

This section presents the main considerations arising from the results of the research, also the limitations and indication of future studies that were identified throughout the research.

6.1. Main Considerations

It is perceptible that there are particularities related to the implementation of reverse logistics in developed and developing countries. In developed countries, such as Japan, Canada, and those from the European Community, the physical and logistics infrastructure have better conditions, they also have a more consolidated legislation and more financial resources available, while in developing countries these issues are still considered as a barrier to effective implementation of reverse logistics of e-waste [5]. This is the case of Brazil, in which reverse logistics is still considered a challenge to be faced.

Thus, the main purpose of this study was to analyze, under the multicriteria decision approach, the barriers related to the implementation of reverse logistics of e-waste in Brazil. For this purpose, two procedures were carried out: (i) The identification of barriers through systematic literature reviews, considering Brazilian and international bases; (ii) the elicitation process through the application of questionnaires with three categories of decision makers: Government, micro and small size companies, and consumers. The method chosen to be applied in this decision context was the CPP, which has a non-compensatory rationality.

As the main results, we can highlight that, comparing the prioritization rankings for each of the categories of decision makers, we found that the barriers diverge in relation to external factors, which may reflect different and conflicting interests. This is a noteworthy situation considering the nature of the problem, which involves multiple objectives and it is preferably based on the characteristics favorable to the uncertainty arising from the judgment of each expert. Besides, it enhances the mutual understanding among the different parties. The difference in priorities between the three groups of respondents was expected considering the access of information and different levels of decision making. For example, the respondents from the government emphasized the organizational and infrastructure factor as a priority related to internal barriers, maybe because, in general, the government is recognized as the main entity responsible for overcoming barriers, allowing the companies to work. On the other hand, related to external barriers, the views of the government and consumers were very similar. In addition, the micro and small size companies (technical assistances) pointed out the legislation as a priority in terms of external barrier, disagreeing with the other two categories. The government considered that the legislation is the first step to implement reverse logistics, valuing reactive behavior from companies; consumers’ perceptions instead highlight that it is important for companies to know and respect the related legislation.

Thus, the study achieved its objective by presenting the prioritization of external and internal barriers in the view of stakeholders (government, small and medium-sized companies, and consumers) in the implementation of reverse logistics of e-waste in Brazil. We demonstrated that there are similarities between the points of view analyzed mainly related to external barriers, and that, related to internal barriers, there are also differences mainly due to the different roles they perform and the information they possess in the implementation of the RL.

The results of this study cannot be generalized considering that it reflects the inherent subjectivity from perceptions of the decision makers considered and because it is from a soft approach of operational research. However, it can be adapted for different groups of decision makers, criteria, and alternatives (barriers) in the process of prioritization. The study innovates because it focuses on micro and small size companies, which were not approached in previous studies, and by applying a non-compensatory method. The solutions indicated by means of barriers to be overcome could be useful to practitioners, such as managers and owners of micro and small size companies, and organizations that support these kinds of companies, such as Brazilian Service of Support to Micro and Small Size Companies—Sebrae. Besides that, it can be useful to policy makers from the private and public sectors and other organizations involved in the process of reverse logistics of e-waste.

This study, when analyzing the three perspectives, presents a scenario in which micro and small size companies—represented by technical assistances—were included in research related to reverse logistics as indicated in a systematic review on publication addressing LR [78]. In addition, it presents a managerial contribution showing possible ways to overcome some barriers that are common for several stakeholders involved in the implementation of LR, as indicated by Demajorovic et al. [44].

6.2. Limitations and Future Studies

One of the main limitations of the study is related to the non-identification of which type of EEE bought by consumers in second-hand. This kind of identification could provide some information to compare if the electronical equipment bought are the same received by technical assistance to repair, and to resell. In this context, further studies can deal with the traceability of the used EEE and identify the amount reconditioned, refurbished, and resold. Other studies can analyze the perception and acceptability of the consumers related to buying or not buying the second-hand and reconditioned EEE.

In addition, the questionnaire option to collect data enabled a greater number of respondents, even though not all Brazilian states were covered. However, it was not possible to deepen some issues because most of the questions in the questionnaire were closed. Thus, studies including more open questions can be useful. Related to the distribution of the respondents in the Brazilian territory, further studies can apply the same method in the states not reached to compare them to the results of this study.

New studies could include consumers who agreed to respond to the study but were excluded for not having purchased second-hand EEE. This would help to understand the reason why these consumers do not buy this kind of equipment, given the barrier that consists, as presented by this work, in the negative perception of consumers in relation to EEE handled by activities from reverse logistics. In addition, further studies can conduct a sensitivity analysis for the two applied distributions, beta-PERT and triangular, and a correlation analysis of the results obtained, checking if there is any correlation and which ones.

Finally, new studies, mainly in the Brazilian context, could verify the perception of other stakeholders related to the prioritization of barriers through the application of the CPP method from the MCDA approach, aiming to provide a complete overview of the decision context, since during the reviews a scarcity of studies was identified.