Reliable Multi-View Deep Patent Classification

Abstract

Patent classification has long been regarded as a crucial task in patent information management and patent knowledge mining. In recent years, studies combining deep learning automatic patent classification methods with deep neural networks have significantly increased. Although great efforts have been made in the patent deep classification task, they mainly focus on information extraction from a single view (e.g., title or abstract view), but few studies concern multi-view deep patent classification, which aims to improve patent classification performance by integrating information from different views. To that end, we propose a reliable multi-view deep patent classification method. Within this method, we fuse multi-view patent information at the evidence level from the perspective of evidence theory, which not only effectively improves classification performance but also provides a reliable uncertainty estimation to solve the unreliability of classification results caused by property differences and inconsistencies in the different patent information sources. In addition, we theoretically prove that our approach can reduce the uncertainty of classification results through the fusion of multiple patent views, thus facilitating the performance and reliability of the classification results. The experimental results on 759,809 real-world multi-view patent data in Shanghai, China, demonstrate the effectiveness, reliability, and robustness of our approach.

1. Introduction

Patent documents collected from various domains are important intellectual resources for information and knowledge management, which can help people to understand detailed concepts and the underlying technologies of components [1,2]. With the rapid development of technology, the number of patents has significantly increased in recent years. Automatic patent classification, which divides each patent into corresponding categories by information extraction in terms of machine learning methods, has become an essential task in patent information management and patent knowledge mining [3].

Early patent classification methods were generally designed using traditional natural language processing algorithms, for example, the bag-of-words (BOW) model is used to represent patent information and then the K-nearest neighbors (KNN) model is utilized to classify patent representations [4]. Recently, combining deep learning with deep patent classification methods that classify patent information in terms of deep neural networks (DNNs) has become an important direction as it can effectively improve patent classification performance.

However, although great efforts have been made in patent deep classification, they mainly focus on information extraction from a single view (e.g., title or abstract view), and few studies concern multi-view deep patent classification, which ignores the complementary information in the latent patent feature space. Taking the real-world patent information collected from Shanghai, China, shown in

Table 1. The limitation of existing single-view-based patent classification methods and multi-view-based patent classification methods.

Case	Patent Title View	Patent Abstract View	Category
1	Eco-board	Steel plate with a layer of an inorganic material fireproof composite board glued on each side.	B32B3/18
2	Eco-board	A protective layer of an inorganic nano waterproof fireproof material.	E04F13/075
3	@&$%$\hat{7}$@%	A new environmentally friendly aluminum alloy material.	Inconsistency sources
4	@&$%$\hat{7}$@%	(*@$&*)(*$%%$\widehat{\mathrm{@}}$@!$&%	Unknown sources

as an example, the designed single-title-view-based patent classification model will generate incorrect results since the same title-view patent information has different categories (shown as Case 1 and Case 2 in Table 1). Moreover, due to the inconsistencies and unknowns of patent information sources (shown as Case 3 and Case 4 in Table 1; the title view and abstract view contain abnormal information), the multi-view deep classification results may be uncertain and unreliable [5] because the traditional deep neural networks focus on the accuracy of the classifications but ignore the credibility of the results, which is a great limitation of patent classification applications. Thus, it is vital to devise a multi-view deep patent classification model that not only improves patent classification performance by integrating information from different views but also provides a reliable uncertainty estimation to measure the reliability of the classification results, which can express “I do not know” for uncertain predictions. To achieve this goal, we revisit multi-view deep patent classification from the perspective of evidence theory to generate accurate and reliable patent classification results in this paper.

Evidence theory, also referred to as Dempster–Shafer Theory (DST) [6,7], is a generalization of Bayesian theory to subjective probabilities [8] for reasoning with partial, unreliable, incomplete, deceptive, or conflicting evidence. In contrast to existing uncertainty-based Bayesian methods, such as the Laplacian approximation [9], Markov Chain Monte Carlo (MCMC) [10], and variational techniques [11,12,13], which are computationally expensive, evidence theory directly quantifies different dimensions of uncertainty such as vacuity (i.e., lack of evidence), which is beneficial for providing reliable uncertainty estimations for classification results [14].

From the perspective of evidence theory, we devise a reliable multi-view deep patent classification (RMDPC) method. Within our method, we fuse multi-view patent information at the evidence level instead of the feature or output levels as done previously. Concretely, we use a BERT neural network to extract the information from each patent view and then adopt evidence theory to represent information collected from each patent view as a subjective opinion that contains beliefs about the truth of propositions under degrees of uncertainty. By a simple fusion strategy, we effectively integrate these opinions to potentially improve both classification performance and reliability. Overall, the RMDPC method provided reliable multi-view classifications, which were validated by sufficient empirical results. In summary, the contributions of this paper are:

(1)

We first propose a reliable multi-view deep patent classification (RMDPC) method that integrates the complementary information from each view in the latent patent feature space to improve classification performance and guarantee the reliability of the patent classification results.

(2)

By constructing a simple and effective fusion strategy using evidence theory, RMDPC can provide a theoretical reliability guarantee that reduces the overall uncertainty of the patent classification results while integrating each patent view effectively.

(3)

The experimental results on a large-scale real-world patent multi-view dataset including 759,809 data validate the superiority of our method in terms of accuracy, reliability, and robustness.

The rest of this article is organized as follows. In Section 2, the related works on deep patent classification, multi-view deep learning, uncertainty of deep learning, and evidence theory are briefly reviewed. Then, the proposed reliable multi-view deep patent classification (RMDPC) method is illustrated in detail in Section 3. Then, the effectiveness and reliability of the RMDPC are demonstrated on a large-scale real-world patent multi-view dataset in Section 4. Finally, Section 5 concludes this manuscript.

2. Related Works

2.1. Deep Patent Classification

Recently, with the development of deep learning, deep patent classification methods that combine patent classification with DNNs has become an important direction for automatic patent classification [15,16,17,18]. Due to the inherent properties of the patent, textual information is often the most widely exploited factor that distinguishes the different types of patents. Therefore, the automatic deep patent classification method always builds a deep neural network, such as a convolutional neural network (CNN) [19,20], residual network (ResNet) [21], BERT [22], graph neural network (GNN) [23,24], and so on, to extract and classify the textual information. For example, PatentBERT [22] utilizes the pre-training language model BERT to extract the features and classify them into the corresponding classes. DeepPatent [2] develops a convolutional neural network to recognize the type of patent document. However, few works consider the patent classification task from the perspective of multi-view learning. Patent2vec [24] devises a novel framework for patent classification from a multi-view graph-based perspective, which is only suitable for the graph’s patent data and lacks a reliability guarantee. To that end, we propose a reliable multi-view deep patent classification method that fuses multiple views to potentially improve both classification performance and reliability.

2.2. Multi-View Deep Learning

In recent years, many multi-view deep learning methods have been proposed to extract the correlations among multiple views [25,26,27,28,29,30]. The deep CCA (DCCA) method [25] captures the nonlinear relationships between different views. The deep canonically correlated autoencoder (DCCAE) [27] trains autoencoders to obtain common representations. The enhanced trusted multi-view classification (ETMC) network [28] focuses on the trusted multi-view classification results. The MvNNcor method [29] seamlessly embeds and fuses various view-specific information and deep interaction information to promote multi-view performance.

2.3. Uncertainty in Deep Learning

Aleatoric and epistemic uncertainties are always considered in deep learning. In general, uncertainty-based models can be divided into two main categories, i.e., Bayesian and non-Bayesian models. Bayesian neural networks (BNNs) replace the deterministic weight parameters with distributions to estimate the uncertainty. For example, MC-dropout [31,32] performs dropout sampling from the weights during training and testing. Bayesian methods are always computationally expensive. To avoid the computational cost, a number of non-Bayesian methods have been proposed. The deep ensemble method [33] trains and fuses multiple deep neural networks to improve classification performance. Evidential neural networks [14,34,35] model uncertainty by introducing subjective logic.

2.4. Uncertainty in Evidence Theory

Evidence theory, also referred to as Dempster–Shafer Theory (DST), directly models inherent uncertainty in information resulting from unreliable, incomplete, deceptive, and/or conflicting evidence and combines evidence from different sources with various fusion operators to produce new representations [36]. Typically, the considered uncertainty in evidence theory is vacuity (caused by a lack of evidence), which is used in out-of-distribution detection in deep learning [14]. Recently, more dimensions of uncertainty have been studied such as dissonance (due to conflicting evidence) and consonance (due to evidence about the composite subsets of state values) [37].

3. Method

3.1. Overview

In this section, we describe how to construct the RMDPC model in detail. Figure 1 presents the framework of our reliable multi-view deep patent classification method, which includes three major components: (1) the backbone of the feature extractor f that maps the input patent view to the embedded representations, (2) the evidential neural network head g that collects evidence from the patent embeddings, and (3) a multi-view aggregation strategy that fuses evidence from multiple views into a unified patent evidence representation, and the aggregated evidence is then transformed into a patent opinion that contains the probability $\mathit{P}$ and uncertainty degree u of the input patent based on evidence theory. Within this framework, we fuse multi-view patent information at the evidence level from the perspective of evidence theory, which can not only effectively improve the classification performance but also provide a reliable uncertainty estimation to solve the unreliability of the classification results caused by the property differences and inconsistencies of the different patent information sources. The details of our RMDPC model are described in the following subsections.

3.2. Reliable Multi-View Deep Patent Classification

Given a number of patent multi-view inputs $\mathit{X}={\left\{{\left\{{\mathit{x}}_i^v\right\}}_{i=1}^n\right\}}_{v=1}^m$, where n is the number of samples and m is the number of patent views. The RMDPC firstly utilizes the backbone ${\left\{f^v\left(\theta \right)\right\}}_{v=1}^m$ parameterized by $\theta$ to extract the features from each view of the input so we have the patent feature embedding $f^v\left({\mathit{x}}_i^v,\theta \right)$. Note that the backbone can be various neural networks without a softmax layer. In this paper, considering the superiority of the processing of text, we use a BERT neural network as the backbone to extract the features from multi-view patent information. Then, the patent feature embedding $f^v\left({\mathit{x}}_i^v,\theta \right)$ is fed into the evidential head.

In contrast to existing deep learning-based models that typically use a softmax layer on top of deep neural networks (DNNs) for classification, which produces over-confident outputs in false prediction [38], the evidential head introduces the evidence framework of the Dempster–Shafer theory (DST) and subjective logic (SL) [36] to overcome the limitations of softmax-based DNNs. It provides a simple and efficient way to jointly formulate multi-class classification and uncertainty modeling through a minor change that replaces the softmax layer with an activation layer (i.e., ReLu layer) to produce a non-negative output, termed evidence [14].

Formally, for the K-classification task, the feature embedding $f^v\left({\mathit{x}}_i^v,\theta \right)$ from the backbone can be transformed into evidence ${\mathit{e}}_i^v$ by the evidential head from each patent view in terms of the following equation:

$${\mathit{e}}_i^v=g^v\left(f^v\left({\mathit{x}}_i^v,\theta \right)\right),$$

(1)

where ${\mathit{e}}_i^v=\left\{e_{i1}^v,e_{i2}^v,\dots ,e_{iK}^v\right\}$ and $g^v\left(·\right)$ is the v-th evidential function to keep evidence ${\mathit{e}}_i^v$ non-negative. In particular, we assume that the class probability follows a prior Dirichlet distribution $\mathrm{Dir}\left(\mathit{P}\left|\mathbf{\alpha }\right.\right)$, which is parameterized by $\mathbf{\alpha }\in {\mathbb{R}}^K$ and given by

$$Dir\left(\mathit{P}\left|\mathbf{\alpha }\right.\right)=\left\{\begin{array}{cc}\frac{1}{B\left(\mathbf{\alpha }\right)}{\prod }_{i=1}^K{p}_i^{{\alpha }_i-1}& \mathrm{for} p\in S_K,\\ 0& \mathrm{otherwise},\end{array}\right.$$

(2)

where $S_K$ is the K-dimensional unit simplex,

$$S_K=\left\{\mathit{P}\left|{\sum }_{i=1}^K{p}_i=1 \mathrm{and} 0\le p_1,\dots ,p_K\le 1\right.\right\}$$

(3)

and $B\left(\mathbf{\alpha }\right)$ is the K-dimensional multinomial beta function.

Based on the DST and SL theories, we have the ${\mathbf{\alpha }}_i^v$ linked to the learned evidence ${\mathit{e}}_i^v$ by the equality ${\mathbf{\alpha }}_i^v={\mathit{e}}_i^v+1$, where ${\mathbf{\alpha }}_i^v=\left\{{\alpha }_{i1}^v,{\alpha }_{i2}^v,\dots ,{\alpha }_{iK}^v\right\}$. In the inference, the predicted probability ${\mathit{P}}^v=\left\{p_{i1}^v,p_{i2}^v,\dots ,p_{iK}^v\right\}$ of the $k$-th class is calculated as $p_{ik}^v={\alpha }_{ik}^v/\phantom{{\alpha }_{ik}^v{S}_i^v}{S}_i^v$, where $S_i^v={\sum }_{k=1}^K{\alpha }_{ik}^v$ is the Dirichlet strength. Then, we have the predictive uncertainty $u_i^v=K/\phantom{K{S}_i^v}{S}_i^v$ to represent the vacuity of evidence for each patent view. The evidential head uses the Dirichlet distribution parameterized over evidence to represent the density of such a probability assignment and the predictions of the learner as a distribution over the possible softmax outputs, which models the second-order probabilities to indicate the uncertainty of the neural network results to enable the model to become “know unknown”.

The evidence ${\mathit{e}}_i^v$ from a single patent view is collected by each evidential head. Now, we should focus on the patent classification with multiple patent views. Given a set of m evidence ${\left\{{\mathit{e}}_i^v\right\}}_{v=1}^m$ for the i-th sample, we devise a simple and efficient aggregation strategy for multi-view deep patent classification with evidence theory, which is shown in Definition 1:

Definition 1

(Evidence-theory-based aggregation strategy for multi-view deep patent classification).The aggregation strategy for multi-view deep patent classification with evidence theory simply consists of evidence parameter addition. Given the i-th sample with m multiple views for the K-classification task, we can obtain a set of evidence ${\left\{{\mathit{e}}_i^v\right\}}_{v=1}^m$, which is collected from m evidential heads. For $k=1,\dots ,K$, ${\mathit{e}}_i^v=\left\{e_{i1}^v,\dots ,e_{ik}^v\right\}$, we have $e_{ik}={\sum }_{v=1}^m{e}_{ik}^v$, which represents the process of multi-view aggregation, and $S_i={\displaystyle \sum _{k=1}^K}\left(e_{ik}+1\right)$, which is the aggregated Dirichlet strength.

Following Definition 1, we combine the evidence from multiple patent views into the aggregated representation ${\mathbf{\alpha }}_i=\left\{{\alpha }_{i1},\dots ,{\alpha }_{ik}\right\}$, where ${\alpha }_{ik}=e_{ik}+1$. Then, we have $\mathit{P}\left({\mathbf{\alpha }}_i\right)=\left\{p_{i1},\dots ,p_{ik}\right\}$ to produce the final probability of each patent class, where $p_{ik}={\alpha }_{ik}/\phantom{{\alpha }_{ik}{S}_i}{S}_i$. After we obtain the aggregated representation ${\mathbf{\alpha }}_i$, we discuss how to train our reliable multi-view deep patent classification model.

For the classification task, the loss function aims to minimize the generalization risk $R=\mathbb{E}\left[L\left(\mathit{P}\left(\mathbf{\alpha }\right),\mathit{Y}\right)\right]$, where $\mathit{P}\left(\mathbf{\alpha }\right)$ indicates the predictive class probability, $\mathit{Y}$ represents the ground truth and $L\left(·\right)$ refers to the certain loss function, e.g., mean squared error or cross-entropy loss. However, the generalization risk is hard to compute as the data distribution is unknown. The most common approach is to approximate the generalization risk by minimizing the empirical risk on the labeled data, i.e., $min\widehat{R}=\sum _{i=1}^{n}L\left(\mathit{P}\left({\mathbf{\alpha }}_i\right),{\mathit{y}}_i\right)$. In this work, we adopt this method with a cross-entropy loss function.

For conventional neural networks, the cross-entropy loss is usually employed as

$$L_{ce}=-\sum _{j=1}^K{y}_{ij}log\left(p_{ij}\right),$$

(4)

where $y_{ij}$ is the true label and $p_{ij}$ is the predicted probability of the i-th sample for class j. Within our model, given the evidence ${\left\{{\mathit{e}}_i^v\right\}}_{v=1}^m$ of the i-th sample with m views obtained from the evidential head, we can obtain the overall evidence ${\mathit{e}}_i$ after the fusion of evidence from multiple patent data views. Then, we can obtain the parameter ${\mathbf{\alpha }}_i$ (e.g. ${\mathbf{\alpha }}_i={\mathit{e}}_i+1$) and the corresponding class probability ${\mathit{P}}_i$ (i.e., ${\mathit{P}}_i={\mathbf{\alpha }}_i/\phantom{{\mathbf{\alpha }}_i{S}_i}{S}_i$) of the Dirichlet distribution $\mathrm{Dir}\left({\mathit{P}}_i\left|{\mathbf{\alpha }}_i\right.\right)$. Considering the $\mathrm{Dir}\left({\mathit{P}}_i\left|{\mathbf{\alpha }}_i\right.\right)$ as a prior on the likelihood, we modify Equation (4) into the following form:

$$\begin{array}{cc}\hfill L_{ce}\left({\mathbf{\alpha }}_i\right)& =\int \left[\sum _{j=1}^K-y_{ij}log\left(p_{ij}\right)\right]\frac{1}{B\left({\mathbf{\alpha }}_i\right)}\prod _{j=1}^K{p}_{ij}^{{\alpha }_{ij}-1}d{\mathbf{p}}_i\hfill \\ \hfill & =\sum _j^K{y}_{ij}\left(\psi \left(S_i\right)-\psi \left({\alpha }_{ij}\right)\right),\hfill \end{array}$$

(5)

where $\psi \left(·\right)$ is the digamma function and $B\left({\mathbf{\alpha }}_i\right)$ is the K-dimensional multinomial beta function. The modified cross-entropy loss $L_{ce}\left({\mathbf{\alpha }}_i\right)$ aims to encourage the evidence of the ground-truth category to reach a large value; however, it cannot guarantee that less evidence will be generated for incorrect categories. To address this limitation, we introduce a Kullback–Leibler (KL) divergence term into our loss function that shrinks the evidence for the incorrect categories to zero, which can regularize the predictive distribution by penalizing those divergences from the “I do not know” state that do not contribute to data fitting. The KL divergence term is as follows:

$$L_{KL}\left({\mathbf{\alpha }}_i\right)=KL\left[Dir\left({\mathit{P}}_i|{\tilde{\mathbf{\alpha }}}_i\right)\left|\right|Dir\left({\mathit{P}}_i|〈1,\dots ,1〉\right)\right],$$

(6)

where

$${\tilde{\mathbf{\alpha }}}_i={\mathit{y}}_i+\left(1-{\mathit{y}}_i\right)\odot {\mathbf{\alpha }}_i,$$

(7)

and $KL\left[Dir\left({\mathit{P}}_i|{\tilde{\mathbf{\alpha }}}_i\right)\left|\right|Dir\left({\mathit{P}}_i|〈1,\dots ,1〉\right)\right]$ is calculated as

$$\begin{array}{c}\hfill log\left(\frac{\mathsf{\Gamma }\left({\sum }_{k=1}^K{\tilde{\alpha }}_{ik}\right)}{\mathsf{\Gamma }\left(K\right){\prod }_{k=1}^K\mathsf{\Gamma }\left({\tilde{\alpha }}_{ik}\right)}\right)+\sum _{k=1}^K\left({\tilde{\alpha }}_{ik}-1\right)\left[\psi \left({\tilde{\alpha }}_{ik}\right)-\psi \left(\sum _{j=1}^K{\tilde{\alpha }}_{ij}\right)\right],\end{array}$$

(8)

where $\mathsf{\Gamma }\left(·\right)$ is a gamma function, $\psi \left(·\right)$ is a digamma function, and $Dir\left({\mathit{P}}_i|〈1,\dots ,1〉\right)$ means the uniform Dirichlet distribution.

By synthesizing the cross-entropy loss $L_{ce}\left({\mathbf{\alpha }}_i\right)$ and the KL divergence regularization term $L_{KL}\left({\mathbf{\alpha }}_i\right)$, the overall optimization problem of our RMDPC model is formulated as

$$\underset{\mathbf{\alpha }}{min}L\left(\mathbf{\alpha }\right)=\sum _{i=1}^n{L}_{ce}\left({\mathbf{\alpha }}_i\right)+{\lambda }_t{L}_{KL}\left({\mathbf{\alpha }}_i\right),$$

(9)

where ${\lambda }_t$ is the balance factor. In general, we consider ${\lambda }_t$ as a warm-up function, which gradually increases the value of ${\lambda }_t$ to prevent the model from paying too much attention to the KL divergence term $L_{KL}\left({\mathbf{\alpha }}_i\right)$ in the initial stage of training and avoid the premature convergence to the uniform distribution for the misclassified samples, which may be correctly classified in future epochs.

Here, we also give the overall algorithm framework of the RMDPC method to show the optimization process of our method in Algorithm 1.

Algorithm 1 Algorithm for Reliable Multi-view Deep Patent Classification (RMDPC)

1: /*Training*/   2: Input: Multi-view patent dataset: $D={\left\{{\left\{x_i^v\right\}}_{v=1}^m,y_i\right\}}_{i=1}^n$.   3: Initialize: Initialize the parameters of the neural network $\theta$.   4: while not converged do   5:      for $v=1:m$   do   6:          $f^v\left({\mathbf{x}}_i^v,\theta \right)\leftarrow$ feature embedding of BERT neural network;   7:          ${{\mathit{e}}_{\mathbf{i}}}^v\leftarrow$ evidence from the evidential head according to Equation (1);   8:      end for   9:      ${\mathit{e}}_i\leftarrow$ aggregation in terms of Definition 1;  10:     ${\mathbf{\alpha }}_i\leftarrow {\mathit{e}}_i+1$;  11:     Obtain the overall loss by updating $\mathbf{\alpha }$ with Equation (9);  12:     Update the neural networks with a gradient descent according to Equation (9);  13: end while  14: Output: parameters of neural networks.  15: /*Test*/  16: Obtain the patent class probability and corresponding uncertainty degree.

3.3. Theoretical Study

Existing deep patent classification methods only focus on a single view and cannot provide a confidence degree to express “I don’t know” if we feed in abnormal patent information after training the network on a set of normal patent information. To this end, we devise a reliable multi-view deep patent classification model that not only effectively improves classification performance but also generates reliable multi-view patent classification results by reducing overall uncertainty after the fusion of multiple patent data views. In this section, we theoretically prove that our model can achieve reliable classification results, whose overall uncertainty decreases after fusing multiple data views.

In terms of the proposed evidence-theory-based aggregation strategy shown in Definition 1 in this paper, we can easily obtain the following theorem.

Theorem 1

(Overall Uncertainty Minimization).The overall uncertainty u of the multi-view patent classification results is smaller than the uncertainty $u^v$ of any single patent view $x^v$ after the fusion of multiple patent data views.

Proof. 

Assume that the ${\mathit{e}}_i^v$ is the evidence obtained from the v-th evidential head for the i-th sample. Let ${\mathit{e}}_i$ be the aggregated evidence after the arrival of the v-th patent view in terms of the evidence-theory-based aggregation strategy. Then, the overall uncertainty $u_i$ is updated as

$$\begin{array}{cc}\hfill u_i& =1-\sum _{j=1}^K\frac{e_{ij}}{S_i}\hfill \\ \hfill & =\frac{K}{S_i}\hfill \\ \hfill & =\frac{K}{S_i^v+{\displaystyle \sum _{v^{\prime }\ne v}}{\sum }_{j=1}^K{e}_{ij}^{v^{\prime }}}.\hfill \end{array}$$

(10)

which is smaller than the uncertainty of the single-patent-view results $u_i^v=\frac{K}{S_i^v}$ since

$$S_i^v+\sum _{v^{\prime }\ne v}{\sum }_{j=1}^K{e}_{ij}^{v^{\prime }}>S_i^v.$$

(11)

□

Theorem 1 theoretically guarantees that our RMDPC model can reduce overall uncertainty after the aggregation of multiple patent data views, which can produce reliable multi-view patent classification results.

4. Experiments

In this section, we extensively evaluate the proposed method on a real-world multi-view patent dataset and compare it with existing deep patent classification methods, uncertainty-aware patent classification methods, and multi-view classification methods. Furthermore, we also provide an analysis of the reliability estimation on noisy patent data. The experimental results show that our algorithm achieves state-of-the-art performance.

4.1. Datasets

In this part, we introduce the real-world multi-view patent dataset in our work, which contains 759,809 Chinese patent samples with two views consisting of a patent title view and a patent abstract view. This real-world multi-view patent dataset was collected from Shanghai, China. According to the International Patent Classification (IPC) rules established by the Strasbourg Agreement 1971, a multi-view patent dataset is divided into eight categories involving (A) human necessities; (B) performing operations; transporting; (C) chemistry; metallurgy; (D) textiles; paper; (E) fixed construction; (F) mechanical engineering; lighting; heating; weapons; blasting; (G) physics; and (H) electricity.

4.2. Implementations

For our algorithm, we use a BERT as the backbone to extract the information from multiple patent data views and then use the fully connected networks with Batch normalization as the evidential heads to collect the evidence from all datasets. The Adam optimizer [39] is used to train the network, where the $l_2$-norm regularization is set to 1$e^{-5}$. We then use 5-fold cross-validation to select the learning rate from $\left\{1e^{-4},3e^{-4},1e^{-3},3e^{-3}\right\}$. For all datasets, 20% of the samples are used for the test sets, and there are no data augmentation or preprocessing methods used in our experiments, except that we set the biggest padding size for each view to 32 and 128, respectively. Furthermore, we run each method five times and report the average values in the figures or the mean values and standard deviations in the tables. The model is implemented by PyTorch on an NVIDIA A100 with a GPU with 40 GB memory.

4.3. Comparative Methods

We compare our method with existing patent deep classification methods, uncertainty-based classification methods, and state-of-the-art multi-view classification methods.

The three patent deep classification methods are:

• DCPC: The CNN-based deep patent classification method extracts the information using a traditional conventional neural network.
• DRPC: The ResNet-based deep patent classification method uses a ResNet neural network to obtain the information from the patent data to produce the classification results.
• DBPC: The BERT-based deep patent classification method utilizes a BERT neural network as the information extractor to classify the patent data.

The three uncertainty-based classification methods are:

• MCDO: The Monte Carlo dropout method [32] considers the dropout network training as the approximate inference in a Bayesian neural network.
• DE: The deep ensemble method [33] generates multiple deep sub-models and combines the predictions of these sub-models.
• EDL: The evidential deep learning method [14] forms a subjective opinion in terms of evidence theory to obtain predictions.

The four multi-view classification methods are:

• DCCA: A deep canonically correlated analysis [25] obtains the correlations through deep neural networks, which maximizes the correlations among the views.
• DCCAE: Deep canonically correlated autoencoders [27] employ autoencoders to search for a common representation.
• MVTCAE: A multi-view total correlation autoencoder [28] obtains the correlations among multiple views using an autoencoder to learn a complete representation.
• ETMC: Enhanced trusted multi-view classification [29] focuses on the uncertainty estimation problem and produces reliable classification results.

4.4. Ablation Study

In this part, we first conduct a detailed ablation study to demonstrate the effectiveness of our main technical components, which consist of the multi-view fusion strategy and KL divergence loss term. Except for the results shown in the first and second rows in

Table 2. Ablation study on Chinese multi-view patent data, “✔” means RMDPC with the corresponding component, “-” means “not applied”.

Main Components		Metric
Fusion	KL Divergence	ccuracy (%)
-		82.93 ± 0.60
-	✔	83.12 ± 0.01
✔		84.31 ± 0.20
✔	✔	85.21 ± 0.50

, which indicate the best accuracy among each single patent view, we evaluate these two components on a multi-view patent dataset with all the views. There are four combinations of these major components. As shown in Table 2, our RMDPC outperformed all other combinations over five runs, which validates the effectiveness of each of the components of our method.

4.5. Comparison with the Patent Deep Classification Methods Using the Best View

In this part, we first compare our RMDPC model with three existing patent deep classification methods. The detailed experimental results are shown in

Table 3. Comparison with other patent deep classification methods.

Method	Accuracy (%)
DCPC	76.75 ± 0.02
DRPC	78.50 ± 0.12
DBPC	79.55 ± 0.32
RMDPC	85.21 ± 0.50

. Since most current patent deep classification methods only focus on a single patent view, we report the experimental results of each method with the best-performing patent view in terms of accuracy. As shown in Table 3, our model outperformed other patent deep classification methods and accuracy was improved by about 6% compared to the second-best model (DBPC), which verifies the improved performance of the proposed multi-view patent deep classification method.

4.6. Comparison with the Uncertainty-Based Classification Methods Using the Best View

Then, in this subsection, we also compare our RMDPC model with three uncertainty-based deep classification methods. For all of these uncertainty-based deep classification methods, we use a BERT as the backbone, which is similar to our method. The detailed experimental results are shown in

Table 4. Comparison with other uncertainty-based deep classification methods.

Method	Accuracy (%)
MCDO	80.95 ± 0.12
DE	82.00 ± 0.01
EDL	82.93 ± 0.60
RMDPC	85.21 ± 0.50

. Since most current patent deep classification methods only focus on a single patent view, we report the experimental results of each method with the best-performing patent view in terms of accuracy. As shown in Table 4, our model had better performance compared with the other uncertainty-based deep classification methods and increased accuracy of nearly 3% compared to the second-best model (EDL), which indicates that the proposed RMDPC is more effective than the other single-view uncertainty-based deep classification methods.

4.7. Comparison with the State-of-the-Art Multi-View Deep Classification Methods

Now, we focus on a performance comparison of the multi-view deep classification methods. In this section, we compare our RMDPC model with four state-of-the-art multi-view deep classification methods. For all of these multi-view deep classification methods, we use a BERT as the backbone, which is similar to the RMDPC method. The detailed experimental results are shown in

Table 5. Comparison with other multi-view deep classification methods.

Method	Accuracy (%)
DCCA	78.65 ± 0.01
DCCAE	79.00 ± 0.00
MVTCAE	81.16 ± 0.21
ETMC	83.79 ± 0.43
RMDPC	85.21 ± 0.50

. We report the experimental results of each method with the fusion of multiple patent views in terms of accuracy. As shown in Table 5, our model outperformed other multi-view deep classification methods and showed an improvement of about 2% compared to the second-best multi-view deep classification method (ETMC), which shows that the fusion of our method is more effective than other multi-view methods.

4.8. Reliability Estimation

Our RMDPC method is based on evidence theory, which can model the uncertainty of a multi-view patent classification. In this part, we conduct qualitative experiments to provide some insights into the estimated uncertainty, which can evaluate the uncertainty estimation performance of our method.

We first randomly sampled 1000 instances from the multi-view testing patent data with five runs, and then added noise to half of these testing patent samples for two views. Similar to the work of [40], the noise vectors (denoted by ${ϵ}^v$) were sampled from the Gaussian distribution $\mathcal{N}\left(0,\mathrm{I}\right)$. Then, we added the noise vectors, i.e., the i-th sample for the v-th view ${\tilde{\mathit{x}}}_{\mathit{i}}^{\mathit{v}}={\mathit{x}}_i^v+{\mathbf{ϵ}}_i^v$. Figure 2 shows the average uncertainty histogram of the unpolluted testing patent data (green) and noisy patent data (red) with five runs. We can observe that by adding noise vectors to the testing data, the overall uncertainty of the noisy data significantly increased. This means that the uncertainty is associated with the sample quality and our method can capture the uncertainty of patent data, which verifies the reliability of our RMDPC method.

5. Conclusions

Existing patent deep classification methods only consider a single view, which ignores the complementary information in the latent patent feature space. Moreover, softmax-based patent deep classification always produces over-confident outputs in false predictions, which makes the patent classification results unreliable. To that end, we propose a reliable multi-view deep patent classification (RMDPC) method, which can not only improve performance by fusing multiple patent data views but also generate reliable patent classification results. Furthermore, the theoretical study shows that the RMDPC method can reduce overall uncertainty after the fusion of multiple data views, which theoretically guarantees the reliability of the method. The experimental results on large-scale real-world multi-view patent data validate the effectiveness, reliability, and robustness of the proposed multi-view deep patent classification method.

Although our method has the advantage of a supervised patent classification task, it is difficult to process the patent data with insufficient labels. In future work, we will propose a novel reliable semi-supervised multi-view deep patent classification method to address the patent classification problem with limited labels.