Exploring Local Explanation of Practical Industrial AI Applications: A Systematic Literature Review

Abstract

In recent years, numerous explainable artificial intelligence (XAI) use cases have been developed, to solve numerous real problems in industrial applications while maintaining the explainability level of the used artificial intelligence (AI) models to judge their quality and potentially hold the models accountable if they become corrupted. Therefore, understanding the state-of-the-art methods, pointing out recent issues, and deriving future directions are important to drive XAI research efficiently. This paper presents a systematic literature review of local explanation techniques and their practical applications in various industrial sectors. We first establish the need for XAI in response to opaque AI models and survey different local explanation methods for industrial AI applications. The number of studies is then examined with several factors, including industry sectors, AI models, data types, and XAI-based usage and purpose. We also look at the advantages and disadvantages of local explanation methods and how well they work in practical settings. The difficulties of using local explanation techniques are also covered, including computing complexity and the trade-off between precision and interpretability. Our findings demonstrate that local explanation techniques can boost industrial AI models’ transparency and interpretability and give insightful information about them. The efficiency of these procedures must be improved, and ethical concerns about their application must be resolved. This paper contributes to the increasing knowledge of local explanation strategies and offers guidance to academics and industry professionals who want to use these methods in practical settings.

1. Introduction

Machine learning (ML) and deep learning (DL) models have achieved remarkable success in a variety of domains, including healthcare [1,2,3], financial systems [4,5], criminal justice [6,7], and cybersecurity [8,9]. While accuracy is critical, the emphasis on accuracy has frequently resulted in developers sacrificing interpretability in favor of better accuracy by making models more complex and difficult to comprehend [10]. When the learning model has the authority to make critical decisions that influence people’s well-being, this lack of interpretability becomes a major concern.

To overcome this issue, explainable artificial intelligence (XAI) approaches must provide end-users with coherent explanations of these models’ decision-making processes. Indeed, because these models are black boxes, it is difficult to understand how they arrive at their conclusions. XAI technologies provide visualization techniques to comprehend the decision-making processes of these models, making forecast explanations easier to understand and communicate. Although the concept of XAI has recently received significant attention in studies [11,12], it has only recently received significant attention in academia, with an increasing number of research papers being published on the topic [13].

The increasing use of ML and DL models in various applications has highlighted the need to explain the decision-making processes to gain end-users’ trust in industrial applications. However, it is essential to investigate the effectiveness and limitations of local explanation techniques in industrial settings. To address this research gap, we conducted a literature review that focused on XAI studies within various industrial sectors, AI models, data types, and the usage and purpose of XAI. Our study stands out from the existing literature survey papers by covering a comprehensive set of criteria, including data types and industrial sectors, as illustrated in

Table 1. Comparison of existing surveys with our work.

Survey Paper	Year	Data Type				Industrial Sector
		T	I	Ta	Te	ASR	EBM	E	F	H	IE	C	SA
[14]	2020	✘	✘	✘	✘	✘	✘	✘	✘	✔	✘	✘	✘
[15]	2021	✘	✘	✘	✘	✔	✘	✘	✘	✘	✘	✘	✘
[16]	2021	✘	✘	✘	✘	✘	✘	✘	✘	✘	✘	✔	✘
[17]	2021	✘	✘	✘	✔	✔	✘	✘	✘	✘	✘	✘	✘
[18]	2022	✘	✘	✘	✘	✘	✘	✘	✘	✘	✘	✔	✘
[19]	2022	✘	✘	✘	✘	✔	✘	✘	✘	✘	✘	✘	✘
[20]	2022	✘	✘	✘	✘	✔	✘	✘	✘	✘	✘	✘	✘
[21]	2022	✘	✘	✘	✘	✘	✘	✘	✘	✔	✘	✘	✘
[22]	2022	✘	✘	✘	✘	✘	✔	✘	✘	✘	✘	✘	✘
[23]	2022	✘	✘	✘	✔	✘	✘	✘	✘	✘	✘	✔	✘
[24]	2022	✘	✔	✘	✘	✘	✘	✘	✘	✘	✘	✔	✘
[25]	2022	✘	✘	✘	✘	✘	✘	✘	✘	✔	✘	✘	✘
[26]	2022	✘	✘	✘	✘	✘	✘	✘	✔	✘	✘	✘	✘
[27]	2022	✘	✘	✘	✘	✘	✘	✘	✘	✘	✔	✘	✘
[28]	2022	✔	✘	✔	✘	✘	✘	✘	✘	✔	✘	✘	✘
[29]	2023	✘	✘	✘	✘	✘	✘	✘	✔	✘	✘	✘	✘
[30]	2023	✘	✔	✘	✘	✘	✘	✘	✘	✔	✘	✘	✘
[31]	2023	✘	✔	✘	✘	✘	✘	✘	✘	✔	✘	✘	✘
Ours	2023	✔	✔	✔	✔	✔	✔	✔	✔	✔	✔	✔	✔

Legend: ✔ means included, ✘ means not included. T: time series; I: image; Ta: table; Te: text; ASR: autonomous systems and robotics; EBM: energy building management; E: environment; F: finance; H: healthcare; IE: industrial engineering; C: cybersecurity; SA: smart agriculture.

. As shown in Table 1, our survey is more comprehensive and distinctive than previously published surveys in two main criteria: data types and industrial sectors. While many existing surveys only cover one industrial sector or one or two data types, our analysis aims to better understand the local explanation in industrial settings and identify future research directions.

This paper’s main goal is to assess the state of local explanation strategies today thoroughly and explore how they are used in the marketplace. To accomplish this, we present a methodical literature analysis that identifies the gaps and limits in the existing research and analyzes and examines the efficacy and limitations of various local explanation strategies in deriving insights from complex models in the industrial setting. Furthermore, we provide recommendations for future research, including areas where further investigation is needed to improve the effectiveness and applicability of local explanation techniques in the industry. The research motivation emphasizes the precision of the study and the void in the research and highlights the research objectives and questions that need to be addressed throughout the paper. The main contributions of the paper are as follows:

• Identifying and categorizing local explanation techniques for industrial AI applications based on different criteria categories, including XAI usage and purpose, industrial applications and AI models, and data types.
• Analyzing the advantages and disadvantages of different local explanation techniques in explaining complex models in the industrial context.
• Identifying the current challenges of local explanation for industrial AI applications and suggesting recommendations for addressing these challenges, such as developing more effective and efficient local explanation techniques and exploring the ethical implications of using these techniques.
• Summarizing the study’s main findings, identifying gaps and limitations in the current literature, and suggesting future research directions, which can guide future research efforts and lead to more reliable and trustworthy AI systems in industry.

The paper is structured as follows: In Section 2, we describe the methodology we used for the literature review. Section 3 presents the survey results on local explanation techniques and their practical applications in industry. We first provide an overview of local explanation techniques and then analyze their distribution based on different criteria categories, including XAI usage and purpose, industrial applications and AI models, and data types. We then highlight the advantages and disadvantages of these techniques in explaining complex models in the industrial context and identify several challenges. Furthermore, we provide recommendations for addressing these challenges. In Section 4, we summarize the main findings of our study, identify gaps and limitations in the current literature, and suggest future research directions. Finally, in Section 5, we conclude the paper and recommend future research on local explanation techniques and their practical applications in industry.

2. Literature Review Methodology

This section outlines our methodology for conducting a comprehensive literature review on local explanation techniques and their practical applications in industry. Our discussion of local explanation techniques and their practical applications in industry is built upon a thorough and objective literature study, made possible by a rigorous search strategy, inclusion and exclusion criteria, data extraction and analysis, and quality assessment. The workflow of the survey methodology is shown in Figure 1. We used a clear process that included the following phases to attain this goal:

• Identifying the research question: Our research question focused on understanding the different types of local explanation techniques used in industry, their benefits and limitations, and their effectiveness in explaining complex machine learning models. Our review aims to answer the following research questions:
  • (Q1) What local explanation techniques are used in industrial applications?
  • (Q2) How widespread are practical industrial applications of local explanation techniques?
  • (Q3) What are the benefits and limitations of local explanation techniques for industrial applications?
  • (Q4) How to build effective local explanation techniques in practical settings?
  By answering these questions, we hope to provide a comprehensive overview of the current state of the art in local explanation techniques for practical industrial applications.
• Identifying relevant literature: We conducted a comprehensive search of relevant academic and industry sources, including peer-reviewed journals, conference proceedings, technical reports, and gray literature. The search included studies published from 2020 to Mar 2023. We used a combination of keywords and controlled vocabulary terms related to machine learning, interpretability, local explanations, and industrial applications to identify relevant articles. We specifically examined academic resources such as ACM Digital Library, IEEE Xplore, ScienceDirect, Google Scholar, MDPI, and others. The search terms used include “local explanation techniques”, “model interpretability”, “explainable artificial intelligence”, “XAI”, “machine learning”, and “industrial applications”. We also manually searched relevant journals and conference proceedings to ensure comprehensive coverage of the literature.
• Screening and selection of studies: We used a two-stage screening process to identify articles for inclusion in our review. In the first stage, we screened titles and abstracts to identify potentially relevant articles. In the second stage, we screened the full text of articles to determine their eligibility based on our inclusion and exclusion criteria. We included studies focusing on local explanation techniques and their applications in industrial settings. Studies investigating the effectiveness and limitations of different explanation techniques and their comparative analysis were also considered. We excluded studies focusing on global explanations, theoretical aspects of model interpretability, or applications in non-industrial settings.
• Data extraction and quality assessment: Data from the selected papers were extracted, including study design, sample size, research question, methods, results, and limitations. We extracted data from the selected studies, including the authors, publication year, research question, methodology, datasets used, and main findings. The data were analyzed thematically to identify patterns, trends, and research gaps in the literature. We also performed a qualitative synthesis of the studies, highlighting the benefits and limitations of local explanation techniques and their effectiveness in industrial applications.
• Data synthesis and analysis: We synthesized the data from the selected articles using a thematic analysis approach. We identified common themes and patterns across the articles and summarized the findings in a narrative synthesis.
• Interpretation and reporting of results: We interpreted the results of our review in light of our research question and the existing literature. We reported our findings in a structured manner, highlighting the key themes and patterns that emerged from our analysis.

Overall, our literature review methodology was designed to ensure our review’s comprehensiveness, validity, and rigor. We set out to present a thorough and trustworthy evaluation of the available literature on local explanation strategies in industrial applications, using a well-defined methodology. One hundred and one papers were chosen using the aforementioned six procedures. The literature review was completed by 31 March 2023. These papers were categorized according to the industrial sectors, data types, applied AI model, used dataset, and practical local explanation techniques in terms of usage and purpose. After that, we analyzed the growth and distribution of local explanations for industrial AI applications following several important factors, such as the AI model, industrial sector, data types, usage, and purpose. Finally, several representative AI industrial application methods were chosen to analyze the pros and cons in depth, as represented in Section 3.

3. Survey Results on Local Explanation in Industrial AI Applications

The taxonomy of XAI techniques, as mentioned in [32,33,34,35], follows four aspects: (1) scope or level explanation; (2) purpose explanation; (3) implement or usage explanation; (4) data types. First, both global and local explanations are included at the explanation level. The models’ operational and decision-making processes and their ability to be articulated are generally emphasized by each explanation level. For many purposes, selecting the explanation level depends on specific cases or subgroups. The survey scope with the local explanation level is the main priority of this paper. Second, there are post hoc and intrinsic (ante hoc) approaches regarding XAI’s purpose. The characteristics, decision-making procedures, and rules of the models, such as linear models, decision trees, Bayesian networks, etc., provide internal justifications for the model type. Meanwhile, post hoc reasoning holds that black-box models’ innermost workings and decision-making procedures are revealed after training. Numerous post hoc XAI solutions have been created, because post hoc explainers include anything from black-box models to interpretable models. Third, usage or implementation includes both agnostic and specific models. Model-specific approaches use the model’s characteristics and properties to make it [36] interpretable. Such approaches’ strengths come at a cost, because applying them to different models is challenging. What gives them their power is their access to model internals such as weights or structures. Meanwhile, post hoc explanations can be provided after the model has been trained using model-agnostic techniques, which can be used for any ML model. Since these methods can only examine input–output pairings, they have the drawback of being unable to benefit from model internals. Fourth, from an XAI-based data perspective, a data type can be tabular, text, image, time series, or any other type.

Most proposed explainable AI systems from industrial applications have been tested, including autonomous systems and robotics, energy and building management, security and privacy, environment, finance, industrial engineering, healthcare, and smart agriculture. The year-by-year distribution of publications on the local explanation of industrial AI applications from 2020 to March 2023 is depicted in Figure 2. According to the literature review, researchers increasingly use local XAI approaches in industrial applications. In particular, the links between XAI approaches and different industrial application areas are shown in

Table 2. Local explanation on AI industrial applications.

Industrial Sector	Data Type	Ref.	Year	AI Model and Dataset		Practical Local Explanation
				AI Model	Dataset	XAI Method	Usage	Purpose
Autonomous systems and robotics	Time series	[37]	2020	Deep Q-Networks	Voltage load	SHAP	A	P
		[38]	2022	FL	QoE forecasting	FED-XAI	S	An
	Image	[39]	2020	Q-learning	Agent data	Interaction data	A	P
		[40]	2021	DRL	Depth image, UAV states	SHAP-CAM	A	P
		[41]	2022	Faster R-CNN, ResNet-50	BDD-OID, PSI	t-SNE	S	An
	Tabular	[42]	2022	RF	Processed dataset (D*)	SHAP	A	P
	Text	[43]	2021	DT, RF	Lyft Level 5	SHAP	A	P
Energy and building management	Time series	[44]	2020	Seq2seq	EnergyPlus	AM	A	P
		[45]	2020	MLRi, GBT	CBECS	SHAP	A	P
		[46]	2020	XGBoost	PLUTO	SHAP	A	P
		[47]	2020	RF	GEFCOM	SHAP, LIME, ELI5	A	P
		[48]	2021	XGBoost	Historical climate	SHAP	A	P
		[49]	2021	DNN	IPCC’s SRES A2 climate	SHAP	A	P
		[50]	2021	DNN	Energy data	IG, DeepLIFT	A	P
		[51]	2021	LSTM	Energy consumption	AM	S	An
		[52]	2021	LSTM	Heat demand by FMEDH	LIME	A	P
		[53]	2022	XGBoost	UNICON dataset	SHAP	A	P
		[54]	2022	CatBoost	Energy disclosure	LIME	A	P
		[55]	2022	DNN	Enerate synthetic	LIME	A	P
		[56]	2022	LSTM	Electricity load	LIME	A	P
		[57]	2022	GS-XGBoost	Electricity consumption	SHAP	A	P
		[58]	2022	Cubist regression	Electricity two buildings	FI	S	An
		[59]	2022	QLattice	Residential building	FI	S	An
		[60]	2022	Bi-LSTM, CNN-LSTM	SCADA	LIME	A	P
	Tabular	[61]	2021	XGBoost	Singapore’s building	LIME	A	P
		[62]	2022	RF, CNN, IT	CER	LIME, SHAP	A	P
		[63]	2021	XGBoost, SVR, LightGBM, LSTM	Energy data in Seoul	SHAP	A	P
Environment	Time series	[64]	2020	XGBoost, LSTM-RNN	Escherichia coli	SHAP	A	P
		[65]	2022	DFNN	Building-level damage	LIME	A	P
	Image	[66]	2020	RF	Occurrence data	LIME	A	P
		[67]	2021	LSTM, CNN-BiLSTM	Monthly rainfall data	SHAP	A	P
		[68]	2023	XGBoost	Land-cover, topographic	LIME	A	P
Finance	Tabular	[69]	2020	XGBoost	Sports and travel	SHAP	A	P
		[70]	2021	GBDT	Daily observations	SHAP	A	P
		[71]	2021	LSTM	OHLC	LIME	A	P
		[72]	2021	XGBoost	Financial indicators	SHAP	A	P
		[73]	2022	LightGBM	Proprietary	SHAP	A	P
	Time series	[74]	2020	XGBoost	Credit risk	SHAP	A	P
		[75]	2022	DQN	SENSEX, DJIA	SHAP	A	P
Healthcare	Tabular	[76]	2020	DT	Cervical cancer risk	SHAP	A	P
		[77]	2020	Stacked LSTM-CNN-MLP	PhysioNet	SHAP, occlusion maps	A	P
		[78]	2020	DT	MIMIC-III	Doctor XAI	S	An
		[79]	2021	XGBoost	Diabetes	PDP, ICE, ALE, LIME, SHAP, Anchors	A	P
		[80]	2021	XGBoost	ROSMAP	SHAP	A	P
		[81]	2021	LightGBM	K-attention	SHAP	A	P
		[82]	2021	RF, GBDT, XGBoost	UCI CKD	SHAP	A	P
		[83]	2021	RF	Retrospective study	SHAP	A	P
	Time series	[84]	2020	Bi-LSTM ensemble	PhysioNet 2017	Attention	S	An
		[85]	2021	CNN	PhysioNet 2017	LIME, guided saliency	A	P
		[86]	2021	WCPH-RNN	Retrospective study	Saliency	A	P
		[87]	2021	CNN	Gait dataset	LRP	A	P
		[88]	2022	CNN-CRF	Sleep-EDF	Grad-CAM	A	P
	Image	[89]	2020	VGG-16	Chest X-ray	Grad-CAM	S	An
		[90]	2021	ResNet-50	CT	LIME, SHAP	A	P
		[91]	2021	Inception-v3	Diagnosis of retinal images	GBP, SHAP	A	P
		[92]	2021	CNN	IVCM	Grad-CAM, guided Grad-CAM	S	An
		[93]	2021	EfficientNet	Chest X-ray images	Grad-CAM	S	An
		[94]	2021	CNN, LSTM	ISIC 2017 and 2018	Grad-CAM	S	An
		[95]	2021	VGG, ResNet, DenseNet	COVID-19 chest X-ray	Grad-CAM	S	An
		[96]	2021	VGG-16	Chest CT, X-ray image	Grad-CAM, GradCAM++, LRP	S	An
		[97]	2022	VGG-19	Oral images	Grad-CAM	S	An
		[98]	2022	COMiT-Net	ChestXray-14, CheXpert	Grad-CAM	S	An
		[99]	2022	CNN	LC25000, NSCLC	Grad-CAM, OS	A	P
		[100]	2022	DNN	CheXpert, MIMIC, NIH	CAM	S	An
		[101]	2023	DNN	COVID-QU, QaTa-Cov19	Uncertain-CAM	S	An
		[102]	2023	VGG-16	CTs	Grad-CAM	A	P
		[103]	2023	EfficientNet, DenseNet, ResNet	Tooth areas	Grad-CAM	A	P
		[104]	2023	ResNet-50	COVIDNet	LIME	A	P
		[105]	2023	ResNet152	KVASIR	Grad-CAM	S	An
Industrial engineering	Time series	[106]	2020	CNN	Machinery fault	LRP	A	P
		[107]	2020	RF	Bushings testbed	LIME	A	P
		[108]	2020	CNN, VAE	Ford Motor	CAM	A	P
		[109]	2020	Deep-SincNet	Motor currents	t-SNE, SincNet filters	A	P
		[110]	2020	CNN, LSTM, Bi-LSTM	C-MAPSS	SHAP	A	P
		[111]	2021	1D-CNN	Normal and fault conditions	FG-CAM	S	An
		[112]	2021	DNN	Prismatic cantilever steel beam	LIME, SHAP	A	P
		[113]	2021	TScatNet	CWRU, DDS	t-SNE	S	An
		[114]	2022	kNN, OCSVM, etc.	Bearing, Gearbox Fault	SHAP, Local-DIFFI	A	P
		[115]	2022	WaveletKernelNet	Bearing, Gearbox Fault	CWConv layer	S	An
		[116]	2023	SVM, kNN	Bearings	SHAP	A	P
	Image	[117]	2020	CNN	Bearings	GradCAM	A	P
		[118]	2020	CNN	Image fault diagnosis	CAM	A	P
Cybersecurity	Tabular	[119]	2020	SVM	NSL-KDD	SHAP	A	P
		[120]	2021	RF, XGBoost, Sequence Model	ISCX-URL2016, CICMalDroid 2020	SHAP, LIME	A	P
		[121]	2021	Autoencoder	CID-IDS2017	SHAP	A	P
		[122]	2021	DT, ANN	Private dataset	SHAP, LIME	A	P
		[123]	2022	DT	NF-BoT-IoT-v2, NF-ToN-IoT-v2	SHAP	A	P
		[124]	2022	DNN	UNSW-NB15	SHAP	A	P
		[125]	2023	CNN	ToN_IoT	SHAP	A	P
		[126]	2023	ANN	WUSTL-IIoT, NSL-KDD	TRUST, LIME	A	P
Smart agriculture	Tabular	[127]	2021	RF	Wheat, maize, olive groves	LIME	A	P
		[128]	2022	DT, RF	Maize crop yield	LIME	A	P
		[129]	2022	LSTM, Bi-LSTM, Bi-GRU-LSTM-CNN	ProductReview	SHAP, LIME	A	P
		[130]	2022	XGB, MLP, SVM	Crop Recommendation	SHAP, LIME	A	P
	Time series	[131]	2020	CNN	Meteorological, wheat yield data	RAM	A	P
		[132]	2022	LightGBM	Diverse physical agricultural	SHAP, LIME	A	P
		[133]	2023	GRU	Plant SSPs	ISM	S	An
	Image	[134]	2021	ResNet-V2, VGG-19, VGG-16, Inception-V3	Diseased leaves of pearl millet	Grad-CAM	A	P
		[135]	2022	LightGBM	Agri-worker motion	ELI5, PDPbox, Skater	A	P
		[136]	2022	CNN	Fire and smoke	LIME, GradCAM++	A	P
	Text	[137]	2022	OAK4XAI	Graph database	AgriComO	S	An

A: agnostic, P: post hoc, S: specific, An: ante hoc.

.

3.1. Quantitative Analysis

3.1.1. Analysis Based on Usage and Purpose

Figure 3 presents the distribution of publications based on local explanation methods’ usage and purpose for industrial AI applications, as derived from Table 2. The survey shows that the SHAP (Shapley additive explanations), LIME (local interpretable model-agnostic explanations), and various Grad-CAM (gradient-weighted class activation mapping) methods are the most commonly used local explanation techniques, accounting for 39.47%, 22.81%, and 18.42% of the studies, respectively. Other local explanation methods have been used less frequently. Regarding usage and purpose, most of the SHAP and LIME applications in the surveyed literature are agnostic and post hoc. At the same time, the specific and ante hoc of local explanations are primarily used in various Grad-CAM methods for industrial AI applications.

3.1.2. Analysis Based on Industrial Applications and AI Model Implementation

In this section, we present statistics on the distribution of publications on local explanation in the context of industrial AI applications and the AI models employed, based on the findings in Table 2. Figure 4 illustrates the classification of the research papers we reviewed according to their industrial application domain. As depicted in Figure 4a, healthcare and energy and building management are the most frequently studied sectors, accounting for 29% and 20% of the surveyed XAI techniques, respectively. Conversely, cybersecurity, autonomous systems and robotics, finance, and environment constituted less than 10% of the surveyed techniques, suggesting that these domains have yet to see the widespread application of XAI methods.

Additionally, we analyzed the distribution of AI models used in the surveyed works. We found that most local explanation techniques were applied to various CNN models, accounting for 25% of the publications, followed by gradient boosting techniques, particularly XGBoost, at 20%. Decision tree, DQN (deep Q-network), and VGG (visual geometry group) models, among others, were used in less than 10% of the surveyed papers, indicating a lack of widespread use of these models with XAI for other industrial applications. These results are presented in Figure 4b.

3.1.3. Analysis Based on Data Types

Based on the information presented in Table 2, we analyzed the distribution of publications on local explanation techniques used for different types of data in industrial applications. Figure 5 breaks down the studies based on data types, including tabular, time series, text, and image. The results show an increasing interest in interpretable ML for healthcare with image data (16.83%) and energy and building management with time-series data (16.83%). However, fewer applications of local explanation techniques were found for text data (1.98%), consistent with the findings for the entire industrial sector. It is worth noting that since this review covers publications up to 31 March 2023, the number of publications for 2023 is expected to exceed those of 2022.

3.2. Qualitative Analysis

This section discusses the effectiveness and limitations of local explanation techniques in industrial applications from the results of Table 2.

3.2.1. Autonomous Systems and Robotics

According to the survey results in Table 2, several publications have focused on XAI local explanations for autonomous systems and robotics.

Zhang et al., 2020 [37] employed the SHAP method and the probability of SHAP values to explain emergency control based on DRL in power systems, in the context of time-series data. Their study combined a post hoc XAI technique with agnostic usage. As an alternative, Renda et al. [38] proposed the FED-XAI idea, which suggests federated learning of XAI models for AI-pervasive 6G networks while proposing an ante hoc XAI strategy. The concept is anticipated to enhance the performance, intelligence, and trustworthiness of automated vehicle networking and improve user experience, fostering end-users’ protection in network AI processes.

In the context of image data, He et al., 2021 [40] proposed a new DRL scheme for model explainability using an agnostic and post hoc XAI approach. They developed a saliency map generation method that merges CAM and SHAP values to create visual and textual action explanations for non-expert users. The model can be refined and enhanced based on the explanation generated. On the other hand, Zhang et al., 2022 [41] employed a distinctive and ante hoc XAI strategy to improve the performance and transparency of autonomous driving systems’ decision making by offering multi-modal explanations, particularly when interacting with pedestrians.

Cui et al., 2022 [42] used SHAP and random forest (RF) approaches regarding tabular data to encourage transparent DRL-based decision making. They adopted an agnostic and post hoc XAI approach. Regarding text datam and employing an agnostic and post hoc approach, Nahata et al., 2021 [43] developed interpretable machine learning models to evaluate and forecast collision risk based on various sensor data features.

3.2.2. Energy and Building Management

As presented in Table 2, our survey results show that XAI local explanation research has concentrated on time-series and tabular data with energy and building management applications.

Much research has used an agnostic and post hoc design for time-series data and two main approaches, SHAP and LIME. For example, Arjunan et al. [45] proposed an approach that improves the current Energy Star calculation method and employs extra model output processing to illuminate how a building can reach a particular score. Li et al. [57] used SHAP to analyze and interpret the XGBoost-based grid search power load forecasting model. While Zdravkovi et al. [52] presented an AI-assisted control of district heating systems (DHS) for effective heat distribution. Kim et al.’s explanation of the energy demand forecast model employing feature importance and attention methods was published in [44]. To support and elucidate the forecasts, Grzeszczyk et al. [56] suggested a strategy based on the LIME method. Both Wenninger et al. [59] and Moon et al. [58] used distinct and ante hoc XAI techniques with feature-important analysis. While Wenninger et al. proposed the QLattice for estimating the final energy performance of residential buildings, Moon et al. evaluated the most crucial aspects of electrical load forecasting. However, several of these studies’ weaknesses must be addressed in future research. For example, the generalizability of the procedures and outcomes only relates to the input data tested. The explanations may also be difficult for average consumers to comprehend [45]. Future studies could concentrate on developing technical research and apps to provide more intuitive explanations to general consumers [57]. Furthermore, robust model interpretability can sometimes be difficult, and future studies should investigate alternate brain models and ways of explaining black-box models [59].

The application of various XAI techniques to tabular data, with an agnostic and post hoc approach, has been explored. Specifically, LIME and SHAP have been investigated for generating local explanations. Examples include Srinivasan’s SHAP-based XAI-FDD [61], Sim’s SHAP-based analysis of input variables for energy consumption forecasting [63], and Wastensteiner’s LIME and SHAP-based visualizations for personalized feedback on electricity consumption time-series data [62]. Srinivasan et al. [61] proposed XAI-FDD (explainable artificial intelligence-fault detection and diagnosis), which uses explanations generated for each data instance to detect incipient faults. By combining human expertise with the explanations provided by the XAI system, this approach could potentially improve accuracy in classification. The XAI-FDD may examine air-handling systems, renewable energy sources, and other building energy components. Sim et al. [63] used SHAP-based XAI to examine how input variables affect energy use forecasting. Their research divided the input variables into three categories—strong, ambiguous, and weak—providing insights into which variables had the most significant impact. However, they only analyzed certain targets, input factors, and predictive models.

Future studies may need to consider a more comprehensive range of socioeconomic variables for different types of buildings. Wastensteiner et al. [62] developed five visualizations using ML and XAI methods, to provide personalized feedback on electricity consumption time-series data, incorporating domain-specific knowledge. However, their approach only considered SHAP-based XAI visualizations, and future research could explore other XAI methods, such as LIME, for additional visualizations.

3.2.3. Environment

Researchers in the field of environmental science have focused on XAI techniques applied to time-series and image datasets, as summarized in Table 2.

Researchers have seen the application of SHAP and LIME approaches with agnostic and post hoc designs in time-series data, as shown in Table 2. Graham et al. [64] employed deep neural networks (DNN) and XAI algorithms on the “Dynomics” platform to explore patterns in transcriptional data on a genome-scale and identify the genes contributing to these patterns. Gao et al. [65] used LIME to interpret the DFNN (dynamic fuzzy neural networks) models to assess the danger levels of a building in Cameron County, Louisiana, in response to a fictitious impending hurricane, with updated weather predictions. Despite these studies’ achievements, there are still limitations, and future research can expand the analytical targets and models employed in these investigations.

The use of SHAP and LIME techniques with agnostic and post hoc designs on image data has been explored by various research groups, as described in [66,67,68]. Masahiro et al. [66] discuss the importance of XAI in ecological modeling and list tools that can be employed to understand complex model behavior at different scales. Integrating XAI with ecological and biogeographical knowledge can enhance the accuracy of machine learning models. Kim et al. [68] used the XGBoost model and SHAP to analyze urban expansion, where land-cover characteristics were identified as the primary factor, followed by topographic attributes. However, the XGBoost-SHAP model’s accuracy, such as using AutoML algorithms, should be assessed compared to other XAI methods. Dikshit et al. [67] built and compared an XAI model to physical-based models using an explainable deep learning technique. Their study examined how predictors interacted locally for distinct drought conditions and timeframes, providing insight into how the model produced specific findings at different spatiotemporal intervals. Future research should look at SHAP plots for long-term forecasting and other additive SHAP properties.

3.2.4. Finance

In the finance sector, the majority of researchers have focused on using LIME and SHAP methods with an agnostic and post hoc approach for tabular and time-series datasets (as shown in Table 2). Some researchers, including Gramegna et al. [69], Benhamou et al. [70], Babaei et al. [72], and de Lange et al. [73], have applied the SHAP method on tabular datasets, while Kumar et al. [75] and Bussmann et al. [74] have developed their SHAP methods on time-series datasets. Additionally, Gite et al. [71] have utilized the LIME method for a tabular dataset.

Several researchers have focused on applying explainable AI models to tabular datasets in finance. For example, Gramegna et al. [69] proposed a model that uses Shapley values and similarity clustering to explain why customers either buy or abandon non-life insurance coverage. In contrast, Benhamou et al. [70] utilized SHAP to identify important variables in stock market crises and provide local explanations of the probability of a crisis at each date. Babaei et al. [72] developed a Shapley-based model to predict the expected return of small and medium-sized enterprises based on their credit risk and expected profitability. Similarly, de Lange et al. [73] combined SHAP with a LightGBM (light gradient-boosting machine) model to interpret explanatory variables affecting credit scoring, which outperformed the bank’s logistic regression model. Additionally, Gite et al. [71] proposed a model that uses long short-term memory (LSTM) and efficient machine learning techniques to accurately predict stock prices based on user sentiments derived from news headlines. They suggest further research directions to include automated prediction of financial news headlines and adding emotion-based GIFs to enhance the model’s appeal. This model can be used as a decision maker for algorithmic trading. Future research may explore applying these methods to other situations in the finance sector, such as underwriting and claims management.

Kumar et al. [75] have suggested using SHAP with a popular deep reinforcement learning architecture, DQN, to explain an agent’s actions in financial stock trading, in time-series data. Additionally, they advised expanding the method to continuous action space and using DRL models, such as deep deterministic policy gradient (DDPG) and advantage actor–critic (A2C), to improve the explanation by adding more technical indicators as features. In another study, Bussmann et al. [74] created an explainable AI model for fintech risk management, particularly for assessing the risks associated with peer-to-peer lending platforms’ credit borrowing. They used Shapley values to interpret AI predictions based on underlying explanatory variables. Future research could be conducted on enhancing prediction understanding by clustering the Shapley values.

3.2.5. Healthcare

Table 2 shows that in the healthcare sector, most research studies have focused on applying Grad-CAM, SHAP, and LIME methods.

Numerous studies have utilized the SHAP method to interpret tabular data. For example, while Kim et al. [81] developed an interpretable machine learning model for predicting early neurological deterioration related to stroke, Beebe et al. [80] developed an approach that captures non-linear feature effects in personalized risk predictions. Rashed et al. [82] developed models for diagnosing chronic kidney disease (CKD) that identified important features consistent with clinical knowledge. Zhang et al. [83] also used SHAP to develop machine learning models for CKD diagnosis that provided physicians with richer information on feature importance to improve decision making. These studies highlight the usefulness of SHAP in developing interpretable models for medical applications and the potential to improve clinical decision making.

Three XAI approaches were used for time-series data: attention, layer-wise relevance propagation (LRP), and Grad-CAM. Mousavi et al. [84] introduced HAN-ECG. This bidirectional-recurrent-neural network-based technique employs three attention mechanism levels to detect atrial fibrillation (AF) patterns in an ECG (electrocardiogram). However, the success of the strategy is dependent on the preprocessing stage. This method should be applied to different ECG leads and arrhythmias in the future to extract novel patterns that may be beneficial in detecting arrhythmias. An interpretability method was proposed by Filtjens et al. in their paper [87] to explain DNN decisions for recognizing the movement preceding the freezing of gait (FOG) in Parkinson’s disease (PD). The recommended pipeline can help physicians to explain DNN conclusions and let ML experts check the generalizability of their models. This pipeline could be used to start FOG treatment pipelines. In these circumstances, the interpreters can promote the provision of external stimuli and evaluate the effectiveness of the intervention by picturing diminished significance for FOG. Dutt et al.’s SleepXAI [88], an explainable unified method for multi-class sleep stage categorization utilizing modified Grad-CAM, was first proposed in 2012. This technique explains multi-class categorization of univariate EEG (electroencephalogram) signals while significantly increasing overall sleep stage classification accuracy. During the anticipated sleep stage, SleepXAI generates a heatmap depicting relevant features learned from univariate EEG data. It enables sleep specialists to link observed characteristics to traditional manual sleep grading techniques, boosting confidence in opaque systems with justifications.

Various studies have applied the Grad-CAM method for interpreting image data, focusing on medical imaging. For example, Figueroa et al. [97] proposed a two-stage training process to enhance classification performance and guide the CNN’s attention to lesion areas. To detect COVID-19 on chest X-ray images, Chetoui et al. [93] developed DeepCCXR (deep COVID-19 CXR detection), while Barata et al. proposed a hierarchical neural network with spatial attention modules for skin cancer diagnosis. To help COVID-19 patients be triaged more quickly, Singh et al. [95] proposed an AI-based solution; Malhotra et al. [98] presented COMiT-Net; and Oztekin et al. [103] proposed an explainable deep learning model. These investigations show that the model’s decision-making process is now more accurate and comprehensible and can be directed to focus on particular areas of interest. However, limitations include the need for more accurate labeled datasets, limited data availability, and extending the models to process other types of diseases with radiography images.

3.2.6. Industrial Engineering

In industrial engineering, time-series and image datasets are commonly used for local explanation methods such as Grad-CAM and SHAP, as seen in Table 2.

Brusa et al. [116] and Hong et al. [110] have both employed Grad-CAM in their ML models to interpret their predictions for time-series data. Hong et al. employed XAI algorithms to find the most important sensors in predicting the remaining useful life of a turbofan engine, and Brusa et al. used SHAP values to diagnose flaws in industrial bearings. Other interpretability approaches, such as LRP and LIME, were used by Grezmak et al. [106] and by Serradilla et al. [107]. The effectiveness of a CNN trained on pictures of the time-frequency spectra of vibration signals measured on an induction motor was evaluated by Grezmak et al. using LRP. Serradilla et al., in contrast, employed XAI methods to direct the development, improvement, and interpretation of a model for estimating the remaining life during fatigue testing, based on condition and setting variables.

For image data, Grad-CAM and CAM (class activation mapping) are common agnostic and post hoc explanation methods used in the industrial engineering sector. Chen et al. [117] used Grad-CAM to interpret the predictions of their CNN (convolutional neural network) model for bearing fault classification using time-frequency spectra. Similarly, Sun et al. [118] employed CAM to diagnose faults and recognize images in the cantilever beam case. Both studies demonstrated the feasibility of using explainable deep learning to diagnose faulty components from images. Future research directions may include testing different equipment settings to determine the minimum requirements for successfully implementing these techniques.

3.2.7. Cybersecurity

Local explanations based on XAI techniques in cybersecurity are mainly applied to tabular data. This is evident from the industrial sector analysis presented in Table 2, where most methods employed are based on SHAP.

An ML-based intrusion detection system (IDS) employing an ensemble trees technique was proposed in work by Le et al. [123]. The method employs decision trees and random forest classifiers, and it does not require much computational power to train the models. The SHAP approach was utilized to explain and interpret the models’ classification conclusions, allowing cybersecurity specialists to optimize and evaluate the validity of their judgments swiftly. Karn et al. [122] developed an ML-based detection strategy for anomalous pods in a Kubernetes cluster in a different study. To identify and explain crypto-mining applications, the system uses auto-encoding-based techniques for LSTM models, SHAP, LIME, and LIME. The system’s explainability is critical for system administrators to grasp system-level rationales for supporting disruptive administrative decisions. Wang et al. [119] also used SHAP to improve the interpretation of IDSs by integrating local and global explanations. Their proposed architecture can improve IDS transparency, allowing cybersecurity experts to make better decisions and optimize IDS structure. Finally, Alenezi et al. [120] employed RFC (random forest classifier), XGBoost, and the sequential algorithm to study two big cybersecurity datasets and used three SHAP approaches to explain the feature contributions. The study emphasizes the need to understand the value of data in order to increase the explanatory capabilities of cybersecurity threats data using SHAP methodologies, which can lead to future data collection operations in cybersecurity or other fields.

The benefits of employing SHAP for XAI in cybersecurity include improved model transparency and interpretability, the ability for cybersecurity specialists to make better decisions, and the optimization of model structures. However, drawbacks include the method’s intricacy and the possibility of misleading explanations if the model is not thoroughly understood.

3.2.8. Smart Agriculture

The smart agriculture sector is explained locally using various datasets, including tabular, time-series, and text data, as illustrated in Table 2.

Using publicly available tabular data, Ryo et al. [128] used XAI and interpretable ML to study the influence of no-tillage on agricultural yield compared to conventional tillage. The authors assessed the importance of factors for prediction, variable interactions, and the relationship between relevant variables and the response variable. Adak et al. [129] used sentiment analysis to assess customer evaluations in the food delivery services (FDSs) domain, and they justified their predictions using SHAP and LIME. Viana et al. [127] proposed a machine learning model to discover the factors influencing agricultural land usage at the regional level for wheat, maize, and olive grove plantings. Using a model-agnostic methodology, they presented global and local interpretations of the significant elements. XAI technologies were used by Cartolano et al. [130] on the ’Crop Recommendation’ dataset to make ML models clear and trustworthy. Their research focused on sensitivity analysis, comparing what the models discovered to what farmers and agronomists already knew. Future research could look into other XAI methodologies and visualization techniques in sectors as diverse as computational creativity and emotion recognition.

Several studies have looked into the usage of deep neural networks in time-series data research. For example, Wolanin et al. [131] used a deep neural network to predict wheat yield in the Indian wheat belt. They used regression activation maps (RAMs) to improve interpretability to show the model’s learned features and yield drivers. Similarly, Kawakura et al. [132] created an XAI-based technique for agri-directors to train agri-workers by analyzing varied data and merging agri-informatics, statistics, and human dynamics. Li et al. [133] created the ExamPle model, which employed a Siamese network and multi-view representation to forecast plants’ small secreted peptides (SSPs) and revealed the SSPs’ sequential pattern. Additionally, Kundu et al. [134] presented the AIDCC (automatic and intelligent data collector and classifier) framework for automating the collection of imaging and parametric datasets from farms producing pearl millet, disease prediction, and feature visualization using deep learning and IoT.

Finally, Apostolopoulos et al. [136] showed that the Xception network outperforms other CNNs in recognizing suspicious situations in various photos. It improved its post hoc explainability by using the Grad-CAM++ and LIME algorithms. They recommended that future studies look at various methodologies, such as fuzzy logic and fuzzy cognitive maps (FCM), to examine timely fire and smoke incident detection.

Ngo et al. [137] have presented OAK4XAI (model towards out-of-box explainable artificial intelligence), an XAI system for text data analysis that combines domain knowledge semantics via an ontology and knowledge map model. To describe the knowledge mined in agriculture, they developed the agriculture computer ontology (AgriComO), a well-structured framework suitable for agriculture and computer domains. In future research, the authors intend to create an explanation interface as a service for user engagement and to expand the model by integrating multiple ML algorithms for prediction utilizing explainable methodologies.

3.3. Current Challenges and Recommendations of Local Explanation for Industrial Applications

Based on our study and analysis of the surveyed papers, we identify several challenges and derive possible recommendations to solve those issues.

3.3.1. Challenges

Local explanations are essential to industrial AI applications but face two significant challenges. The first challenge is that local explanations require computational resources and only provide a narrow view of the model’s behavior. The second issue is the trade-off between interoperability and performance accuracy, as a human-centric strategy is still required.

Local explanation techniques aim to provide insight into the decision-making processes of complex ML models by highlighting the contribution of individual features to the final prediction for a specific input instance. Some of the popular local explanation techniques used in industrial applications include LIME [138], SHAP [139], and Anchors [140]. These techniques have been applied to a wide range of applications such as credit risk assessment [73], image classification [141], and text classification [142]. These techniques provide transparency, increase trustworthiness, and can improve model performance. However, the effectiveness of these techniques may vary depending on the type of model, dataset, and task [138,139]. Several studies have demonstrated the effectiveness of local explanation techniques in industrial applications. For example, Selvaraju et al. [141] used LIME to explain the predictions of a deep neural network for image classification and showed that the technique improved the model’s performance and helped to detect biases in the dataset. De Lange et al. [73] used SHAP to interpret the predictions of a credit risk assessment model and found that the technique increased the transparency and trustworthiness of the model. Different local explanation techniques have strengths and weaknesses in explaining complex ML and DL models in industrial applications. Several studies have compared the performance of different techniques, such as LIME, SHAP, and Anchors, on various tasks and datasets [73,139,140]. These studies have shown that the effectiveness of these techniques may depend on the type of model, dataset, and task and that no single technique is universally superior.

XAI has been proposed as a way to address the lack of transparency and limitations of AI’s usage in critical fields [14,143]. The goal of XAI, according to [144], is to provide techniques that let end users comprehend, believe in, and control the rising number of AI systems. For typical learning models with high-dimensional datasets, interpretability and performance accuracy frequently trade off. Models with high architectural complexity, such as DL and random forests, often provide better performance accuracy, but are called "black-box models" because they lack transparency and explainability. In contrast, white-box or glass-box models, such as linear, graphical, and decision trees, provide transparent and understandable results. Still, their performance may be subpar [145]. However, recent research by [146] challenges the assumption that relatively simple ML models are easy to interpret, as interpretability depends on the context of use, users involved, and complexity. Therefore, Schoonderwoerd et al. [147] argue for a more human-centric approach to XAI, which prioritizes user needs over the explainability of the ML model. As a result, there may be situations where a high-performance model is preferred over an explainable one, or vice versa [148].

3.3.2. Recommendations

Section 3.3.1 discusses the challenges of interpreting complex models and the limitations of local explanations in capturing global model behavior, which can hinder the adoption and trust in AI models. To address these challenges and improve the interpretability and transparency of industrial AI applications, we suggest the following remedial measures based on our analysis and knowledge:

• Data quality assurance: Ensuring high-quality datasets is essential to minimize the impact of challenges associated with local explanation techniques. By performing data cleaning, normalization, and preprocessing, we can ensure that the datasets are high quality and minimize the risks of producing unreliable and inaccurate explanations [149].
• Model validation: Thorough model validation and testing should be carried out to ensure that the model is accurate and reliable [145,146]. Our study suggests involving end-users in the interpretability process, prioritizing the evaluation of the model’s interpretability according to the context of use, and providing clear explanations of the model’s limitations and assumptions to enhance transparency and trust in the model [150].
• Appropriate choice of explanation techniques: The efficacy of local explanation strategies varies according to model type, dataset, and task. Thus, choosing the most appropriate technique for the application is crucial. Researchers can also develop hybrid models that combine black-box and white-box models’ strengths to achieve high performance and interpretability [148].
• Human-in-the-loop: A human-in-the-loop approach can improve the quality of local explanations and enhance trust in the model. By including human experts in the decision-making process, we can ensure that the local explanations are relevant and accurate for the intended use case [151].

Different local explanation techniques have strengths and weaknesses in explaining complex ML and DL models in industrial applications. It is, therefore, essential to select the appropriate technique, ensure data quality, validate models, and include human experts in the decision-making process to minimize the impact of the challenges associated with local explanation techniques. To support our arguments, we draw on the literature, which has shown that the effectiveness of local explanation techniques may vary depending on the model type, dataset, and task [152]. Furthermore, recent research has challenged the assumption that relatively simple ML models are easy to interpret, as interpretability depends on the context of use, the users involved, and complexity [153]. Therefore, our study emphasizes the need for a more human-centric approach to XAI, prioritizing user needs over the explainability of the ML model.

In conclusion, our paper provides a roadmap for using local explanation techniques in industrial AI applications, addressing the challenges associated with their use and enhancing the models’ transparency, interpretability, and trust. However, we acknowledge that our framework is not exhaustive and that further research is needed to develop more robust and effective approaches to XAI in industrial settings.

4. Discussion of Gaps and Limitations in the Current Literature

Local explanation techniques have gained significant attention due to their potential to increase transparency and interpretability in complex ML and DL models used in industrial applications. While these techniques have shown promise, several gaps and limitations must be addressed.

4.1. Discussion of Survey Results and Gaps

Section 3 presents our survey results, allowing us to answer the four main research questions outlined in Section 2. Analyzing the data collected gave us insights into using local explanation techniques in industrial applications and their distribution, benefits, limitations, and effectiveness.

Q1: What local explanation techniques are used in industrial applications? SHAP, LIME, and Grad-CAM variations are among the most extensively employed local explanation approaches in industrial applications, according to the survey data presented in Section 3.1.1. SHAP assigns an importance score to each feature based on the Shapley value from cooperative game theory. LIME is a model-agnostic explanation method that learns an interpretable model locally around the prediction to explain the predictions of any black-box model. Grad-CAM variations in deep neural networks use gradient information to depict and explain the features contributing to a certain prediction. In numerous industrial areas, these strategies have proven beneficial in delivering local explanations and boosting transparency and trust in AI models.

Q2: How widespread are practical industrial applications of local explanation techniques? The practical industrial applications of local explanation techniques have gained significant traction in recent years. According to the survey results in Section 3.1.2, local explanation techniques are widely used in various fields, including healthcare, energy and building management, and industrial engineering. Among these fields, healthcare is one of the most prominent sectors in adopting local explanation techniques. These techniques are used extensively for clinical decision making, disease prediction, and patient monitoring. In healthcare, local explanation techniques play a vital role in helping medical professionals to understand the reasoning behind the predictions of medical diagnosis models, improving the accuracy of diagnosis, and building trust in the models. Energy and building management is another sector that has embraced local explanation techniques to optimize energy consumption and improve building performance. These techniques are used to interpret the predictions of energy consumption models, identify the factors affecting energy consumption, and recommend energy-efficient practices. Similarly, industrial engineering uses local explanation techniques for predictive maintenance and quality control. These techniques can help to identify potential machine failures, predict maintenance needs, and improve product quality. The survey results demonstrate that local explanation techniques are increasingly prevalent in various industrial sectors. Healthcare, energy and building management, and industrial engineering are among the leading fields in adopting these techniques.

Q3: What are the benefits and limitations of local explanation techniques for industrial applications? The ability to explain specific predictions of complex models, identify the essential components that lead to the prediction, find biases and errors, and establish trust in the models are all advantages of local explanation techniques. Local explanations are also useful in increasing AI systems’ transparency, accountability, and fairness. On the other hand, as outlined in Section 3.3.1, one of the major limitations of local explanation techniques is the lack of standardization in evaluating and comparing them. Although numerous studies have proposed various techniques, there is currently no agreed-upon standard for evaluating their performance, which makes it difficult to compare different methods. This limitation poses a challenge in determining the most effective technique for specific applications. To address this issue, there is a need for standard datasets, metrics, and benchmarks, to enable a fair and accurate comparison of different techniques in different applications [14]. Another important gap is the limited understanding of the robustness and generalizability of local explanation techniques across different datasets and applications. While many studies have evaluated the effectiveness of these techniques on specific tasks and datasets, it is unclear whether the findings can be generalized to other tasks and datasets. The effectiveness of these techniques may vary depending on the type of data and application, making it challenging to determine the most appropriate technique to use in different scenarios. Therefore, there is a need for a more comprehensive and diverse evaluation of these techniques across different applications and datasets [154]. Additionally, local explanation techniques may have ethical and social implications that must be addressed. For example, these techniques may reveal sensitive information about individuals or groups, and there is a risk of unintended consequences if these explanations are not carefully designed and implemented. Therefore, it is important to consider these techniques’ ethical and social implications and to develop guidelines for their responsible use [155]. Finally, after analyzing our survey results and statistical distributions, it is evident that the development of local explanation techniques is imbalanced across different industrial application fields. In summary, local explanation techniques can become more useful and effective in real-world settings by addressing these limitations and gaps in the current literature.

Q4: How to build effective local explanation techniques in practical settings? The effectiveness of local explanation techniques in practical settings can vary depending on the specific use case and the data quality and models, as mentioned in Section 3.3.2. While these techniques can provide valuable insights into the behavior of complex models, their effectiveness may be limited in some cases. For example, if the model is too complex or the dataset is too noisy, the local explanations may not be reliable or accurate. However, studies have shown that when combined with other interpretability methods, such as global explanations or feature importance analysis, local explanations can provide a more comprehensive understanding of the model behavior and improve the trustworthiness of the models. Additionally, the human-centered approach is crucial in building effective local explanations in industrial applications. By involving domain experts and end-users in the explanation process, the explanations can be tailored to meet the stakeholders’ specific needs and preferences, increasing their effectiveness and acceptance.

4.2. Analysis of Potential Biases and Limitations of the Study

One of the potential biases and limitations of this study is the focus on a limited number of local explanation techniques and industrial applications. While we have discussed some popular techniques and applications, this study has not covered many other techniques and applications. Moreover, the effectiveness of these techniques may vary depending on the specific task, dataset, and model, and our findings may not be generalizable to other contexts.

Another limitation is the reliance on the existing literature and case studies, which may be subject to publication bias and other forms of bias. Moreover, many of the studies that we have reviewed have used different evaluation metrics and benchmarks, which may make it difficult to compare their findings. Future studies should consider using standard metrics and benchmarks to enable better comparison and evaluation of different local explanation techniques.

5. Conclusions and Future Directions

In conclusion, this literature review has provided insights into the industrial applications of local explanation techniques for ML models. We have discussed the benefits and limitations of these techniques and highlighted the challenges in their practical use.

Our findings imply that local explanation strategies can help to improve the interpretability and transparency of black-box models in various industrial settings, including banking, healthcare, and manufacturing. However, limitations and roadblocks remain, such as computational costs and trade-offs between model accuracy and interpretability. Consistent evaluation measures, datasets, and benchmarks are required to address these issues to enable fair and reliable comparisons of diverse methodologies. Furthermore, the human-centered design of local explanations should be considered, as the ultimate goal is to enable human users to understand and trust AI systems.

To advance the field, we recommend that future research focuses on developing more effective and efficient local explanation techniques that can be applied to large-scale datasets and complex models. Additionally, ethical considerations must be addressed to ensure the responsible use of these techniques and minimize potential harm to stakeholders.

It is crucial to highlight limitations to this literature evaluation, such as the study scope and inherent biases in the selection criteria and methods. Nonetheless, this review contributes significantly to the field and can be a resource for researchers and practitioners interested in XAI and industrial applications.

To summarize, local explanation techniques can potentially increase the transparency and interpretability of complex ML and DL models in industrial settings. Further research, however, is required to address the challenges and limitations of these techniques and ensure their responsible use.