Next Article in Journal
Processing Small-Sized Trees at Landing by a Double-Grip Machine: A Case Study on Productivity, Cardiovascular Workload and Exposure to Noise
Previous Article in Journal
Effects of Aspen and Spruce Density on Size and Number of Lower Branches 20 Years after Thinning of Two Boreal Mixedwood Stands
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Transfer Residual Neural Network Based on ResNet-34 for Detection of Wood Knot Defects

College of Science, Northeast Forestry University, Harbin 150040, China
*
Authors to whom correspondence should be addressed.
Submission received: 18 January 2021 / Revised: 30 January 2021 / Accepted: 8 February 2021 / Published: 11 February 2021

Abstract

:
In recent years, due to the shortage of timber resources, it has become necessary to reduce the excessive consumption of forest resources. Non-destructive testing technology can quickly find wood defects and effectively improve wood utilization. Deep learning has achieved significant results as one of the most commonly used methods in the detection of wood knots. However, compared with convolutional neural networks in other fields, the depth of deep learning models for the detection of wood knots is still very shallow. This is because the number of samples marked in the wood detection is too small, which limits the accuracy of the final prediction of the results. In this paper, ResNet-34 is combined with transfer learning, and a new TL-ResNet34 deep learning model with 35 convolution depths is proposed to detect wood knot defects. Among them, ResNet-34 is used as a feature extractor for wood knot defects. At the same time, a new method TL-ResNet34 is proposed, which combines ResNet-34 with transfer learning. After that, the wood knot defect dataset was applied to TL-ResNet34 for testing. The results show that the detection accuracy of the dataset trained by TL-ResNet34 is significantly higher than that of other methods. This shows that the final prediction accuracy of the detection of wood knot defects can be improved by TL-ResNet34.

1. Introduction

In recent years, due to the shortage of wood resources, delaying the consumption of forest resources has become a research focus. If wood knot defects are found quickly on the surface of wood, then the utilization rate of wood can be improved effectively and the excessive consumption of woods can be reduced [1,2,3,4]. The combination of digital image processing and artificial intelligence algorithms is a common method of wood knots defect detection and classification [5]. Among them, fixed feature extraction and classification recognition technology have been widely used [6,7]. This technology is mainly composed of computer vision technology, spectral analysis technology, and other digital image processing methods [8]. The effective feature parameters are extracted from the samples by changing the operating mode through fixed mapping, and multiple statistical or machine learning methods are compared to determine the effective feature parameters. Deep learning (DL) has become a new method with great potential in the field of artificial intelligence [9]. The DL method enables the feature values of the original data to be learned automatically, thereby the impact of manual operations on feature extraction can be reduced [10]. However, due to the small sample size in wood knot defect detection, it is easy to generate overfitting or underfitting, which limits the final prediction accuracy of the model [11]. In order to rectify this problem, the deep convolution neural network (CNN) model was trained on ImageNet [12] by researchers and used as a feature extractor in small data sets in different fields after being combined with transfer learning [13]. A large amount of literature proves that good results have been achieved after these improvements. For example, Thenmozhi et al. (2019) used the AlexNet pre-trained model for classification of insects [14]. The results showed that the accuracy of the AlexNet model based on transfer learning was 7.65% higher than that of AlexNet. In the same year, Gao et al. proposed a tree species identification method based on transfer learning [15]. The experimental results showed that the highest accuracy of trunks and leaves based on transfer learning was improved by 51.38% and 51.69%, respectively, compared with the accuracy of ordinary deep learning. In 2020, Kentsch et al. conducted research on classification identification of a winter orthomosaic dataset by ResNet-50 based on transfer learning [16]. The results showed that from no transfer learning to transfer learning from a general-purpose dataset, the accuracy with a 9.78% improvement. Meanwhile, they also observed a further 2.7% improvement when transfer learning was performed from a dataset that was closer to our type of images.
The feature extraction of wood knot defects is the prior condition for the realization of the above method. For example, a Hu invariant moments feature extraction method combined with a BP (back propagation) neural network to classify wood knot defects was proposed by Qi et al. [17]. The accuracy of this method for wood knot defect recognition is over 86%. Another method of sub-region variable scale Gaussian fitting model combined with a BP neural network to classify wood defects was proposed by Yuan et al. [18]. The accuracy of this method to identify wood knots reaches 92.80%. In 2020, Beshaier and Yossra et al. conducted a comparative study on two different image retrieval methods of gray level co-occurrence matrix (GLCM) and Hu invariant moments [19]. The results showed that the average accuracy of GLCM and Hu invariant moments is 92.8% and 84.4%, respectively. However, due to the unique shape of each wood knot defect, the process of feature extraction is difficult and complicated. Therefore, a method combining near infrared spectroscopy (NIR) with machine vision was proposed by Yu et al. [20]. Compounds with different spectral absorption rates are used to identify special types of wood knot defects, and the recognition accuracy rate reaches 92.0%. It should be emphasized that luminescent equipment is required to collect data before processing the acquired data, so the recognition speed of this method is not fast enough. These network and learning-based recognition methods depend on a priori wood feature extraction. However, the process of artificial feature extraction is difficult and complex for wood products, whether it is extracting the morphological features (Hu invariant moments, etc.) or physical features (NIR, etc.) of wood. Since each knot defect has its own unique appearance, it is difficult to detect it accurately. At the same time, it also requires a lot of time to extract the knot defect features by manual operation equipment. Therefore, a convolution neural network that can learn wood knot defect features automatically, instead of complex artificial extraction for defect detection, is needed. A fully convolutional neural network (Mix-FCN) was proposed by He et al. in 2019 to identify and locate wood defects [21], but the depth of the network is too deep, resulting in too much computation. In 2020, an improved SSD algorithm was proposed by Ding et al. [22]. Although the amount of computation is reduced, the average precision of knot defect detection is not high enough. To solve the above problems, a high accuracy detection algorithm for wood knot defects based on a convolution neural network with automatic feature extraction may be required.
In this article, a new transfer residual neural network based on ResNet-34 is proposed, named TL-ResNet34, as a classifier for the detection of wood knots. The depth of TL-ResNet34 is 35 layers. The high-quality features of images can be extracted on ImageNet, due to the good performance of ResNet-34 in image classification. First, it is assumed that the feature extraction layer of ResNet-34 also performs well in the detection of wood knot defects. After that, we use deeper network layers and better feature extraction layers to train and test the knot defect data set. Finally, we compare the test results with other DL and traditional models. The results show that the prediction accuracy of TL-ResNet34 is as high as 98.69%, indicating that the model has a good recognition effect. This provides more possibilities for non-destructive testing of wood knot defects.

2. Methods

2.1. Dataset

The experimental dataset was provided by the Computer Laboratory of the Department of Electrical Engineering, University of Oulu [23,24,25] and includes the image information for 448 spruce knots.
Appropriate datasets are required to train or evaluate the performance of the algorithm. A total of 448 images of wood knots were prepared as the experimental samples, so 450 original images were obtained. The wood knots dataset includes seven classes of 448 images, which were divided into training set, validation set, and test set at a ratio of 6:2:2, including 268 training, 90 validation, and 90 testing images, respectively (Table 1). Because of the particularity of wood, the color of wood knots is usually darker than wood. However, sometimes the heartwood has a darker color, which may cause the neural network to evaluate the image of the wood knot defects incorrectly, which will affect the recognition of the wood knot defects in the heartwood during the network training. In order to solve this problem, the images of the wood knot defect in the darker heartwood were pre-processed and their contrast was enhanced to make the wood knot defects more obvious. To maximize the collected data, the original images were processed by different methods. Additional data were used to expand the dataset through data augmentation. Finally, we obtained 1885, 636, and 615 images for the training, validation, and test datasets, respectively.
Overfitting can usually be reduced by training the network with additional data [26,27,28], and the generalization ability of machine learning models can be improved. Massive data is required for deep learning; expanding the data through multiple methods was necessary due to the insufficiency of the data collected. The problem of limited data can be solved easily by enlarging the dataset artificially and adding it to the training set. Six distinct forms of data augmentation were used for color images (Figure 1). After data initialization, new images were added by vertical mirroring and horizontal mirroring, rotating by 180°, adding Gaussian noise, adding salt-and-pepper noise to images, and increasing the hue by 10. After data augmentation, the training dataset, validation, and testing datasets were expanded to seven times their original size.

2.2. ResNet-34 and Transfer Learning

2.2.1. ResNet-34

With the rapid development of computer technology and the improvement of computer hardware performance, deep learning has made great progress [29]. Artificial neural networks have been widely used in different fields due to their excellent performance in image classification and recognition [30]. CNN is a multilayer feedforward neural network with a convoluted structure, which has good fault tolerance and self-learning capabilities. It can deal with problems in complex environments and unclear backgrounds. Its generalization ability is significantly better than that of other methods. CNN generally consists of an input layer, several convolution layers, pooling layers, a fully connected layer, and an output layer. It can carry out supervised learning and unsupervised learning and is applied to computer vision, natural language processing, and other fields.
Residual building block is the infrastructure of the ResNet-34 network, and the whole network is mainly composed of it. A shortcut-connection [9] was used by the residual building block to skip the convolutional layers, which effectively alleviates the problem of gradient disappearance or gradient explosion caused by increasing depth in the neural networks, and it helped us to construct CNN structures more flexibly and improve the recognition rate of wood knot defects.
The structure of the basic-block is shown in Figure 2. It is used for the 34 layers of ResNet. The residual building block is composed of several convolutional layers (Conv), batch normalizations (BN), a rectified linear unit (ReLU) activation function, and a shortcut. The output of the residual building block can be formulated as follows:
y = F ( x ) + x
where F is the residual function and x and y are the input and output of the residual function, respectively. The entire residual network is composed of the first convolutional layer and several basic-blocks.
The ResNet-34 contains 33 convolutional layers, a max-pooling layer of size 3 × 3, an average pool layer, followed by a fully connected layer. A classical ResNet-34 model involves 63.5 million parameters, where rectification nonlinearity (ReLU) activation and batch normalization (BN) is applied to the back of all convolution layers in the “BasicBlock” block and the softmax function is applied in the final layer. The architecture of ResNet-34 is shown in Table 2.

2.2.2. Transfer Learning

A large number of annotated datasets are needed for CNN training to achieve high prediction performance. However, it is difficult to obtain such a large quantity of data in general, and there are high costs for image labeling [31]. Therefore, transfer learning is often applied in a relatively small number of datasets to train the neural network, and it has been proven to be a very effective method.
Due to their fixed network structure, some mature CNNs, such as AlexNet and GoogLeNet, etc. [32], have certain feature extraction capabilities that can be obtained by pre-training on large-scale mature datasets (such as ImageNet) before training on their own datasets.
It is easy to produce overfitting problems because the amount of data in this experiment is relatively small [26,27,28], and the model also requires more epochs during the training phase, resulting in poor model recognition capability. Therefore, the idea of transfer learning can be used to pre-train the model on ImageNet to optimize the classification of wood knot defects. ResNet-34 was fine-tuned to fit the data in this article, and a lot of training time was saved.

2.3. Improved CNNs Based on ResNet-34

2.3.1. ReLU Nonlinearity

The rectified linear unit (ReLU) is a commonly used activation function in artificial neural networks; we denoted the regularized ReLU function as follows:
f ( x ) = max ( 0 , x )
where f is ReLU and x is input.
Equation (2) zeroes out the output of some neurons when using the ReLU function, resulting in sparsity of the network, which reduces the interdependence of parameters and alleviates the occurrence of overfitting f ( x ) = max ( 0 , x ) g [33]. Compared with the larger number of calculations needed when using a sigmoid function and other functions, using ReLU can save a lot of time.
For deep networks, when sigmoid and tanh functions are back propagating, their gradients are close to zero in the saturation region. It is easy for the gradients to disappear, resulting in slower convergence and information loss. In most cases, the gradient of ReLU is a constant, which helps to solve the convergence problem of deep networks. Meanwhile, as a unilateral function, ReLU is more in line with the characteristics of biological neurons. CNNs with ReLU can train several times faster than their equivalents using sigmoid units (Figure 3) or hyperbolic tangents [34].

2.3.2. Adaptive Moment Estimation

The Adam algorithm is a one-step optimization algorithm for random objective function. Based on adaptive low-order moment estimation, the neural network weights can be updated iteratively according to training data. This method is easy to implement and has high computational efficiency and low memory requirements [35]. The diagonal scaling of the gradient of the Adam algorithm is invariant, so it is suitable for solving problems with large-scale data or parameters. The super parameters of the Adam algorithm are well explained and usually need little or no adjustment. Different adaptive learning rates were also designed for different parameters, and the weights of a neural network can be iteratively updated according to the training data [29,36]. When there are back propagation and updated parameters, the Adam algorithm can better adjust the learning rate. Adam is also suitable for the problem of unstable objective functions and gradient sparsity. Therefore, the Adam algorithm has faster convergence speed and effective learning.

2.3.3. Cross Entropy

In the training process, cross entropy was used as a loss function to update ω and b . The cross-entropy function is defined as follows:
H ( p , q ) = i = 1 n p i ( x ) log 2 q i ( x )
where H is cross entropy, x is input, p is the probability, and q is the probability of the predicted value. Compared with the variance loss function, the problem of updating weights and bias too slowly was overcome by this method. The updating of weights and deviations is affected by errors [37,38]. For this reason, when the error is large enough, the updating speed of weights is very fast. Similarly, when the error is small, the updating speed of weights and deviations is slow.

2.3.4. Overall Architecture

The structure of the proposed TL-ResNet34 is shown in Figure 4. There are 35 trainable layers in this architecture, H ( p , q ) = i = 1 n p i ( x ) log 2 q i ( x ) including a convolutional layer, 16 identity blocks (there are 16 × 2 = 32 Conv layers in these identity blocks), and two fully-connected layers. The TL-ResNet34 consists of 5 convolution groups; each convolution group is composed of one or more basic convolution calculation processes (Conv->BN->ReLU). The first convolution group contains only one convolution operation, its kernel is 7 × 7 and the stride is 2. The second to fifth convolution groups contain multiple, identical residual units, which were named Conv2_x, Conv3_x, Conv4_x, Conv5_x, respectively (Figure 4).
The first 33 layers of TL-ResNet34 were transferred with the ImageNet dataset (there are 1 + 16 × 2 = 33 Conv layers in Figure 4). Then, a fully connected (FC) layer and the softmax classifier were added to the TL-ResNet34 to fit the category label of the wood knots defect dataset. At the same time, the final prediction accuracy of the neural network in the recognition of wood knots defect dataset was improved.

2.4. Training

The proposed TL-ResNet34 was used and trained on one GPU (GTX 960M 2G). The experimental environment is presented in Table 3. The model using the Adam optimization algorithm and the cross-entropy loss function was trained for 300 iterations, with a batch size of 128 and learning rate of 1 × 10−4. The parameter configuration is shown in Table 4.

3. Results and Analysis

Figure 5 shows the process of training the model using the training and validation datasets. The best accuracy was 99.22%, and the best loss was about 2.83%. At the same time, the overall accuracy in the test phase was about 98.69%.

3.1. Comparison of Model Performance

To evaluate the performance of the TL-ResNet34, the precision (P), recall (R), f1-score (F1), and false alarm rate (FAR) were determined. All the evaluation indices are defined as follows:
p i = T i i T i i + j = 1 , j i k T i j
R i = T i i T i i + j = 1 , j i k T j i
F L i = 2 × P × R P + R
F A R i = T i j T i j + j = 1 , j 1 k T j j
where P i is the class i precision, R i is the class i recall, F l i is the class i weighted harmonic average ( α = 1 ), F A R i is the class i false alarm rate, k is the number of knots, T i i is the class i predicted to be class i, T i j is the class i predicted to be class j , T j i is the class j predicted to be class i , and T j j is the class j predicted to be class j .
Table 5 shows the precision, recall, f1-score, false accept rate, and accuracy of TL-ResNet34 and three other comparison methods for seven types of wood knot defect images. It can be seen that the five evaluation indicators of TL-ResNet34 for decayed knots, dry knots, edge knots, encased knots, and sound knots are the best among the four models. In the recognition of horn knots, the R, F1, and accuracy of TL-ResNet34 are slightly lower than those of GoogLeNet, but these five evaluation indicators of TL-ResNet34 are still better than those of AlexNet and VGGNet-16, and the P and FAR of TL-ResNet34 are the best among the four models. In the recognition of leaf knots, the performance of TL-ResNet34 is slightly inferior to that of GoogLeNet and AlexNet, but all indicators of TL-ResNet34 are better than those of VGGNet-16. Meanwhile, the P and FAR of TL-ResNet34 are better than those of AlexNet. For the seven types of knots, all networks have the highest precision for horn knots, the lowest precision for encased knots, the highest accuracy for horn knots, and the lowest accuracy for sound knots. Compared with the other three methods, the TL-ResNet34 performs well in these five evaluation indicators.
The comprehensive indicators of the five models are shown in Figure 6. The results show that the accuracy rates of AlexNet, VGGNet-16, GoogLeNet, and TL-ResNet34 are 96.89%, 95.58%, 95.42%, and 98.69%, respectively. It can be seen that TL-ResNet34 reached the highest value in terms of P, R, F1, FAR, and accuracy. This shows that the TL-ResNet34 model has the best recognition effect.

3.2. Convergence Rate Analysis

Figure 7 shows the accuracy and loss values of TL-ResNet34, AlexNet, VGGNet-16, and GoogLeNet during the training phase. The experimental results show that these four models can converge within 300 training epochs, but their convergence rates are different. The convergence rate of GoogLeNet is the slowest. The training process of AlexNet is similar to that of VGGNet-16, it converges and tends to be stable after 150 training epochs. TL-ResNet34 converges within 50 training epochs, which is faster than the other three CNN models. Meanwhile, within 300 training epochs, the loss value of TL-ResNet34 is always lower than that of the other three models, and the accuracy value is always higher than the that of other three models. The results show that, compared with the other three models, TL-ResNet34 has the highest accuracy and the fastest convergence rate.

3.3. Transfer Learning Analysis

In order to reduce the overfitting and enhance the feature extraction ability of the network, transfer learning was applied in TL-ResNet34. Table 6 shows the test results of the wood knot classification of TL-ResNet34 and the comparison network. During the test phase of TL-ResNet34, a total of 19 decayed knots, 96 dry knots, 91 edge knots, and 49 horn knots were correctly recognized, and no misidentification occurred. Among the 40 encased knots, 37 knots were correctly recognized, and 3 knots were incorrectly recognized as a dry knot, an edge knot, and a sound knot. Among 66 leaf knots, 64 knots were correctly recognized, and 2 knots were incorrectly recognized as a horn knot and a sound knot. Among 250 sound knots, 247 knots were correctly recognized, and 3 knots were incorrectly recognized as leaf knots. It can be seen that TL-ResNet34 is better than ResNet-34 in the recognition ability of seven kinds of wood defect knots.
Figure 8 shows the impact of transfer learning on the accuracy and loss of the model. In the training phase, TL-ResNet34 combined with transfer learning has a faster convergence rate. It can be found that the combination of transfer learning and ResNet-34 can increase the recognition accuracy to 98.69%, which is higher than that of the original ResNet-34 (the recognition accuracy of ResNet-34 is 97.05%). It can be seen that the combination with transfer learning is effective at improving the performance of the model.

3.4. Comparison of Optimal Algorithms

An optimization algorithm is very important for a model’s performance. In this paper, SGD (stochastic gradient descent), Adagrad, and Adam optimization algorithms were used to train the TL-ResNet34 network, and their convergence rates were compared. Figure 9 shows the training process of these three optimization algorithms, and their learning rate is 1 × 10−4. The result shows that the model with the Adam algorithm has the fastest convergence rate. It can be seen from the loss curve in Figure 9 that the Adam algorithm can converge quickly and it is more stable than are SGD and Adagrad.

3.5. Comparison of Detection Methods for Wood Surface Defects

The comparison between the method proposed in this paper and the other methods is shown in Table 7. It can be seen that compared with other detection methods, the classification accuracy of this method is relatively high, about 98.69%. Second, this method can detect more kinds of defects: decayed knots, dry knots, edge knots, encased knots, horn knots, leaf knots, and sound knots. In addition, although the method in [39] has the highest accuracy, about 99%, this method needs to use NIR to extract features manually, which leads to the decline of the overall recognition efficiency. The authors of [39] were committed to reducing the artificial extraction process of image features and reducing the time required for recognition; however, although their method can identify a variety of wood defects and has a high recognition rate, it had not been trained for the types of wood knot defects specifically, so their method was not suitable for this study. All the feature extraction processes in this method are completed by the convolutional neural network automatically, which overcomes the limitations of manually extracting image features.

4. Conclusions

In summary, a transfer residual neural network TL-ResNet-34 was proposed to identify wood knot defects quickly and accurately. The accuracy of the network was improved by more than 0.78% after extracting structural defect features, training parameters, and optimization of datasets and images. At the same time, transfer learning was added to build a pre-training model. The experimental results show that TL-ResNet34 achieved a high recognition rate of 99.22% in the training dataset and a low training loss of 2.83% in the validation dataset during the process of identifying seven wood knot defects. The overall accuracy reached 98.69%, and the fluctuation ranges of the loss curve and the accuracy curve were small when TL-ResNet34 was applied to the test dataset. Moreover, this method does not require a large amount of image preprocessing and feature extraction when detecting various types of wood defects, and has high efficiency and recognition accuracy in both training and testing stages. This means that the collected wood knot defects can be identified accurately and quickly by the proposed TL-ResNet34. Based on the above analysis, the proposed TL-ResNet-34 has potential applications in wood non-destructive testing and wood defect identification.

Author Contributions

Conceptualization, M.G.; methodology, M.G.; software, M.G.; validation, M.G.; formal analysis, M.G.; investigation, M.G.; resources, M.G.; data curation, M.G.; writing—original draft preparation, M.G.; writing—review and editing, M.G.; visualization, M.G.; supervision, D.Q., H.M., and J.C.; project administration, D.Q.; funding acquisition, D.Q. and H.M. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the National Natural Science Foundation of China under grant number 31570712 and the Fundamental Research Funds for the Central Universities under 2572020BC07. The APC was funded under grant number 31570712.

Acknowledgments

We are highly grateful to the anonymous reviewers and handling editor for their insightful comments, which greatly improved an earlier version of this manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Norlander, R.; Grahn, J.; Maki, A. Wooden Knot Detection Using ConvNet Transfer Learning. In Lecture Notes in Computer Science, Image Analysis; Paulsen, R., Pedersen, K., Eds.; SCIA: Cham, Switzerland, 2015; Volume 9127. [Google Scholar]
  2. Pölzleitner, W.; Schwingshakl, G. Real-time surface grading of profiled wooden boards. Ind. Metrol. 1992, 2, 283–298. [Google Scholar] [CrossRef]
  3. Qiu, Z.F. A Simple Machine Vision System for Improving the Edging and Trimming Operations Performed in Hardwood Sawmills. Master’s Thesis, Virginia Polytechnic Institute and State University, Blacksburg, VA, USA, 1996. [Google Scholar]
  4. Schmoldt, D.L.; Li, P.; Abbott, A.L. Machine vision using artificial neural networks with local 3D neighborhoods. Comput. Electron. Agric. 1997, 16, 255–271. [Google Scholar] [CrossRef]
  5. Qi, D.W.; Zhang, P.; Jin, X.; Zhang, X. Study on wood image edge detection based on Hopfield neural network. In Proceedings of the 2010 IEEE International Conference on Information and Automation, Harbin, China, 20–23 June 2010; pp. 1942–1946. [Google Scholar]
  6. Ji, X.Y.; Guo, H.; Hu, M.H. Features Extraction and Classification of Wood Defect Based on Hu Invariant Moment and Wavelet Moment and BP Neural Network. In Proceedings of the 12th International Symposium on Visual Information Communication and Interaction (VINCI’2019), Shanghai, China, 20–22 September 2019; Association for Computing Machinery: New York, NY, USA, 2019. Article 37. pp. 1–5. [Google Scholar]
  7. Mu, H.; Qi, D.W. Pattern Recognition of Wood Defects Types Based on Hu Invariant Moments. In Proceedings of the 2009 2nd International Congress on Image and Signal Processing, Tianjin, China, 17–19 October 2009; pp. 1–5. [Google Scholar]
  8. Hermanson, J.C.; Wiedenhoeft, A.C. A brief review of machine vision in the context of automated wood identification systems. IAWA J. 2011, 32, 233–250. [Google Scholar] [CrossRef] [Green Version]
  9. Wen, L.; Li, X.Y.; Gao, L. A transfer convolutional neural network for fault diagnosis based on ResNet-50. Neural Comput. Appl. 2020, 32, 6111–6124. [Google Scholar] [CrossRef]
  10. LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef]
  11. Zhao, R.; Yan, R.Q.; Chen, Z.H.; Mao, K.Z.; Wang, P.; Gao, R.X. Deep learning and its applications to machine health monitoring. Mech. Syst. Signal Prosess. 2019, 115, 213–237. [Google Scholar] [CrossRef]
  12. Donahue, J.; Jia, Y.Q.; Vinyals, O.; Hoffman, J.; Zhang, N.; Tzeng, E.; Darrell, T. Decaf: A deep convolutional activation feature for generic visual recognition. In Proceedings of the International Conference on Machine Learning, Beijing, China, 21–26 June 2014; pp. 647–655. [Google Scholar]
  13. Yosinski, J.; Clune, J.; Bengio, Y.; Lipson, H. How transferable are features in deep neural networks? In Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada, 8–13 December 2014; pp. 3320–3328. [Google Scholar]
  14. Thenmozhi, K.; Reddy, U.S. Crop pest classification based on deep convolutional neural network and transfer learning. Comput. Electron. Agric. 2019, 164, 104906. [Google Scholar] [CrossRef]
  15. Gao, X.; Zhao, Y.F.; Xiong, Q.; Chen, Z. Identification of Tree Species Based on Transfer Learning. For. Eng. 2019, 35, 68–75. [Google Scholar]
  16. Kentsch, S.; Lopez Caceres, M.L.; Serrano, D.; Roure, F.; Diez, Y. Computer Vision and Deep Learning Techniques for the Analysis of Drone-Acquired Forest Images, a Transfer Learning Study. Remote Sens. 2020, 12, 1287. [Google Scholar] [CrossRef] [Green Version]
  17. Qi, D.W.; Mu, H.B. Detection of wood defects types based on Hu invariant moments and BP neural network. J. Southeast Univ. (Nat. Sci. Ed.) 2013, 43, 63–66. [Google Scholar]
  18. Yuan, W.Q.; Li, S.L.; Li, D.J. Wood surface defect recognition based on sub-region zoom Gaussian fitting. Chin. J. Sci. Instrum. 2016, 37, 879–886. [Google Scholar]
  19. Abdulla, B.A.; Ali, Y.H.; Ibrahim, N.J. Extract the Similar Images Using the Grey Level Co-Occurrence Matrix and the Hu Invariants Moments. J. Eng. Technol. 2020, 38, 719–727. [Google Scholar] [CrossRef]
  20. Yu, H.; Liang, Y.; Liang, H.; Zhang, Y. Recognition of wood surface defects with near infrared spectroscopy and machine vision. J. For. Res. 2019, 30, 2379–2386. [Google Scholar] [CrossRef]
  21. He, T.; Liu, Y.; Xu, C.; Zhou, X.; Hu, Z.; Fan, J. A fully convolutional neural network for wood defect location and identification. IEEE Access 2019, 7, 123453–123462. [Google Scholar] [CrossRef]
  22. Ding, F.; Zhuang, Z.; Liu, Y.; Jiang, D.; Yan, X.; Wang, Z. Detecting Defects on Solid Wood Panels Based on an Improved SSD Algorithm. Sensors 2020, 20, 5315. [Google Scholar] [CrossRef] [PubMed]
  23. Kauppinen, H.; Silven, O. A Color Vision Approach for Grading Lumber. In Theory & Applications of Image Processing II—Selected Papers from the 9th Scandinavian Conference on Image Analysis; Borgefors, G., Ed.; World Scientific: Singapore, 1995; pp. 367–379. [Google Scholar]
  24. Silven, O.; Kauppinen, H. Recent Developments in Wood Inspection. Int. J. Pattern Recognit. Artif. Intell. IJPRAI 1996, 10, 83–95. [Google Scholar] [CrossRef]
  25. Kauppinen, H.; Silven, O. The effect of illumination variations on color-based wood defect classification. In Proceedings of the 13th International Conference on Pattern Recognition (13th ICPR), Vienna, Austria, 25–29 August 1996; pp. 828–832. [Google Scholar]
  26. Zhou, Y.; Dong, F.; Liu, Y.; Li, Z.; Du, J.; Zhang, L. Forecasting emerging technologies using data augmentation and deep learning. Scientometrics 2020, 123, 1–29. [Google Scholar] [CrossRef] [Green Version]
  27. Liu, Z.; Cao, Y.K.; Li, Y.J.; Xiao, X.Y.; Qiu, Q.C.; Yang, M.J.; Zhao, Y.F.; Cui, L.Z. Automatic diagnosis of fungal keratitis using data augmentation and image fusion with deep convolutional neural network. Comput. Methods Programs Biomed. 2020, 187, 105019. [Google Scholar] [CrossRef]
  28. Mushtaq, Z.; Su, S.F.; Tran, Q.V. Spectral images based environmental sound classification using CNN with meaningful data augmentation. Appl. Acoust. 2021, 172, 107581. [Google Scholar] [CrossRef]
  29. Yan, Q.; Yang, B.; Wang, W.; Wang, B.; Chen, P.; Zhang, J. Apple Leaf Diseases Recognition Based on An Improved Convolutional Neural Network. Sensors 2020, 20, 3535. [Google Scholar] [CrossRef]
  30. Hinton, G.E.; Osindero, S.; Teh, Y.W. A fast learning algorithm for deep belief nets. Neural Comput. 2006, 18, 1527–1554. [Google Scholar] [CrossRef] [PubMed]
  31. Gao, Y.; Mosalam, K.M. Deep transfer learning for image-based structural damage recognition. Comput. Aided Civ. Infrastruct. Eng. 2018, 33, 748–768. [Google Scholar] [CrossRef]
  32. Dawei, W.; Limiao, D.; Jiangong, N.; Jiyue, G.; Hongfei, Z.; Zhongzhi, H. Recognition pest by image-based transfer learning. J. Sci. Food Agric. 2019, 99, 4524–4531. [Google Scholar] [CrossRef] [PubMed]
  33. Zhang, Y.D.; Pan, C.; Chen, X.; Wang, F. Abnormal breast identification by nine-layer convolutional neural network with parametric rectified linear unit and rank-based stochastic pooling. J. Comput. Sci. 2018, 27, 57–68. [Google Scholar] [CrossRef]
  34. He, T.; Liu, Y.; Yu, Y.B.; Zhao, Q.; Hu, Z.K. Application of deep convolutional neural network on feature extraction and detection of wood defects. Measurement 2020, 152, 107357. [Google Scholar] [CrossRef]
  35. Kingma, D.; Ba, J. Adam: A Method for Stochastic Optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
  36. Isola, P.; Zhu, J.Y.; Zhou, T.; Efros, A.A. Image-to-Image Translation with Conditional Adversarial Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 1125–1134. [Google Scholar]
  37. Jamin, A.; Humeau-Heurtier, A. (Multiscale) Cross-Entropy Methods: A Review. Entropy 2020, 22, 45. [Google Scholar] [CrossRef] [Green Version]
  38. Boudiaf, M.; Rony, J.; Ziko, I.M.; Granger, E.; Pedersoli, M.; Piantanida, P.; Ayed, I.B. A Unifying Mutual Information View of Metric Learning: Cross-Entropy vs. In Pairwise Losses. In Proceedings of the Lecture Notes in Computer Science, Computer Vision—ECCV 2020, Glasgow, UK, 23–28 August 2020; Vedaldi, A., Bischof, H., Brox, T., Frahm, J.M., Eds.; Springer: Cham, Switzerland, 2020; Volume 12351. [Google Scholar]
  39. Sandak, J.; Sandak, A.; Zitek, A.; Hintestoisser, B.; Picchi, G. Development of Low-Cost Portable Spectrometers for Detection of Wood Defects. Sensors 2020, 20, 545. [Google Scholar] [CrossRef] [Green Version]
  40. Ren, R.; Hung, T.; Tan, K.C. A Generic Deep-Learning-Based Approach for Automated Surface Inspection. IEEE Trans. Cybern. 2018, 48, 929–940. [Google Scholar] [CrossRef]
  41. Zhang, Y.; Xu, C.; Li, C.; Yu, H.; Cao, J. Wood defect detection method with PCA feature fusion and compressed sensing. J. For. Res. 2015, 26, 745–751. [Google Scholar] [CrossRef]
  42. Zhang, Y.X.; Zhao, Y.Q.; Liu, Y.; Jiang, L.Q.; Chen, Z.W. Identification of wood defects based on LBP features. In Proceedings of the 35th Chinese Control Conference (CCC), Chengdu, China; 2016; pp. 4202–4205. [Google Scholar]
  43. Li, C.; Zhang, Y.; Tu, W.; Jun, C.; Liang, H.; Yu, H. Soft measurement of wood defects based on LDA feature fusion and compressed sensor images. J. For. Res. 2017, 28, 1285–1292. [Google Scholar] [CrossRef]
  44. Li, S.; Li, D.; Yuan, W. Wood Defect Classification Based on Two-Dimensional Histogram Constituted by LBP and Local Binary Differential Excitation Pattern. IEEE Access 2019, 7, 145829–145842. [Google Scholar] [CrossRef]
  45. Urbonas, A.; Raudonis, V.; Maskeliūnas, R.; Damaševičius, R. Automated Identification of Wood Veneer Surface Defects Using Faster Region-Based Convolutional Neural Network with Data Augmentation and Transfer Learning. Appl. Sci. 2019, 9, 4898. [Google Scholar] [CrossRef] [Green Version]
  46. Yang, Y.; Zhou, X.; Liu, Y.; Hu, Z.; Ding, F. Wood Defect Detection Based on Depth Extreme Learning Machine. Appl. Sci. 2020, 10, 7488. [Google Scholar] [CrossRef]
Figure 1. Seven common wood knots and data augmentation of dataset. 1. a decayed knot, 2. a dry knot, 3. an edge knot, 4. an encased knot, 5. a horn knot, 6. a leaf knot, 7. a sound knot. Original images and those created through data augmentation: (A) original image, (B) vertical mirror (C) rotated by 180°, (D) horizontal mirror, (E) added Gaussian noise to image, (F) added salt-and-pepper noise to image, and (G) increased the hue by 10.
Figure 1. Seven common wood knots and data augmentation of dataset. 1. a decayed knot, 2. a dry knot, 3. an edge knot, 4. an encased knot, 5. a horn knot, 6. a leaf knot, 7. a sound knot. Original images and those created through data augmentation: (A) original image, (B) vertical mirror (C) rotated by 180°, (D) horizontal mirror, (E) added Gaussian noise to image, (F) added salt-and-pepper noise to image, and (G) increased the hue by 10.
Forests 12 00212 g001
Figure 2. A “Basic-Block” building block for ResNet-34.
Figure 2. A “Basic-Block” building block for ResNet-34.
Forests 12 00212 g002
Figure 3. Three kinds of activation functions.
Figure 3. Three kinds of activation functions.
Forests 12 00212 g003
Figure 4. Structure and flowchart of wood knots recognition on the Pytorch framework.
Figure 4. Structure and flowchart of wood knots recognition on the Pytorch framework.
Forests 12 00212 g004
Figure 5. The model was trained with a training dataset and validating datasets: (a) Loss value; (b) Accuracy value.
Figure 5. The model was trained with a training dataset and validating datasets: (a) Loss value; (b) Accuracy value.
Forests 12 00212 g005
Figure 6. Recognition accuracy of four different models: (a) Accuracy; (b) Precision; (c) Recall; (d) F1-Score; (e) False Accept Rate.
Figure 6. Recognition accuracy of four different models: (a) Accuracy; (b) Precision; (c) Recall; (d) F1-Score; (e) False Accept Rate.
Forests 12 00212 g006
Figure 7. Convergence comparison: (a) Loss value; (b) Accuracy value.
Figure 7. Convergence comparison: (a) Loss value; (b) Accuracy value.
Forests 12 00212 g007
Figure 8. Comparison of the accuracy and loss of the networks with and without transfer learning: (a) Loss value; (b) Accuracy value.
Figure 8. Comparison of the accuracy and loss of the networks with and without transfer learning: (a) Loss value; (b) Accuracy value.
Forests 12 00212 g008
Figure 9. Comparison of the optimal algorithms: (a) Loss values; (b) Accuracy values.
Figure 9. Comparison of the optimal algorithms: (a) Loss values; (b) Accuracy values.
Forests 12 00212 g009
Table 1. Number of datasets.
Table 1. Number of datasets.
Wood KnotBefore Data AugmentationAfter Data Augmentation
Training
Dataset
Validation
Dataset
Testing
Dataset
Training
Dataset
Validation
Dataset
Testing
Dataset
Decayed knot1033682519
Dry knot4114142919696
Edge knot3913132739191
Encased knot20661364444
Horn knot21771474949
Leaf knot2710101986566
Sound knot1103737772266250
Table 2. The structure of ResNet-34.
Table 2. The structure of ResNet-34.
Layer NameOutput Size34-Layer
Conv1112 × 1127 × 7, 64, stride 2
Conv2_x56 × 563 × 3 max pool, stride 2
3 × 3 , 64 3 × 3 , 64 × 3
Conv3_x28 × 28 3 × 3 , 128 3 × 3 , 128 × 4
Conv4_x14 × 14 3 × 3 , 256 3 × 3 , 256 × 6
Conv5_x7 × 7 3 × 3 , 512 3 × 3 , 512 × 3
1 × 1average pool, 1000-d fc, softmax
Table 3. Experimental environment.
Table 3. Experimental environment.
Hardware EnvironmentSoftware Environment
Memory16.00 GBSystemWindows 10
CPUIntel Core i5-4210H 2.90 GHz
(2 core)
Environment
configuration
Pytorch-gpu 1.6.0
+ Python 3.7.3 + cuda 10.1
+ cudnn 7.6.5
Graphics cardNVIDIA GeForce GTX 960M (2G)
Table 4. Training parameters.
Table 4. Training parameters.
Related ParameterValueMeaning
Batch size128Number of pictures per training
Learning rate1 × 10−4Initial learning rate
Epoch300Training iteration times
CUDAEnableComputer unified device architecture
Table 5. The evaluation index values of the network.
Table 5. The evaluation index values of the network.
ClassesModelPRF1FARAccuracy
Decayed knotAlexNet84.21%94.12%88.89%0.51%99.35%
VGGNet-1678.95%100%88.24%0.67%99.35%
GoogLeNet84.21%57.14%68.08%0.51%97.55%
TL-ResNet34100%100%100%0.00%100%
Dry knotAlexNet97.92%97.92%97.92%0.39%99.35%
VGGNet-1698.96%95.00%96.94%0.20%99.02%
GoogLeNet94.79%90.10%92.39%0.98%97.55%
TL-ResNet34100%98.97%99.48%0.00%99.84%
Edge knotAlexNet97.80%98.89%98.34%0.38%99.51%
VGGNet-16100%97.85%98.91%0.00%99.67%
GoogLeNet100%97.85%98.91%0.00%99.67%
TL-ResNet34100%98.91%99.45%0.00%99.84%
Encased knotAlexNet90%100%94.74%0.70%99.35%
VGGNet-1677.50%100%87.32%1.55%98.52%
GoogLeNet62.50%100%76.92%2.56%97.55%
TL-ResNet3492.5%100%96.10%0.52%99.51%
Horn knotAlexNet100%92.45%96.08%0.00%99.35%
VGGNet-1697.96%96.00%96.97%0.18%99.51%
GoogLeNet100%100%100%0.00%100%
TL-ResNet34100%98%98.99%0.00%99.84%
Leaf knotAlexNet95.45%100%97.67%0.55%99.51%
VGGNet-1695.45%90.00%92.64%0.55%98.36%
GoogLeNet100%97.06%98.51%0.00%99.67%
TL-ResNet3496.97%95.52%96.24%0.37%99.18%
Sound knotAlexNet98%97.61%97.80%1.39%98.20%
VGGNet-1696.40%99.18%97.78%2.45%98.20%
GoogLeNet98.00%99.19%98.59%1.37%98.85%
TL-ResNet3498.8%99.20%99.00%0.83%99.18%
Note: our method is bold.
Table 6. Wood knots classification results of TL-ResNet34 and the contrast network. The green background color represents the number of correctly identified wood knot defects, and the orange back-ground color represents the total number of wood knot defects in each category.
Table 6. Wood knots classification results of TL-ResNet34 and the contrast network. The green background color represents the number of correctly identified wood knot defects, and the orange back-ground color represents the total number of wood knot defects in each category.
MethodActual CategoryPrediction Category
Decayed KnotDry KnotEdge KnotEncased KnotHorn KnotLeaf KnotSound KnotTotal
TL-ResNet34Decayed knot1900000019
Dry knot0960000096
Edge knot0091000091
Encased knot0113700140
Horn knot0000490049
Leaf knot0000164166
Sound knot000003247250
ResNet-34Decayed knot1900000019
Dry knot0941000196
Edge knot0190000091
Encased knot1303400240
Horn knot0000490049
Leaf knot0000460266
Sound knot030000247250
Table 7. Comparison of the proposed method with previous detection methods for wood surface defects. Our method is bold.
Table 7. Comparison of the proposed method with previous detection methods for wood surface defects. Our method is bold.
Wood Defect Detection MethodAccuracyWood Defects
1Decaf convolutional neural network and feature transferring [40]91.55%Encased knot, leaf knot, edge knot, and sound knot
2Principal component analysis (PCA) and compressed sensing [41]92.00%Live knot, dead knot, and crack
3Near infrared spectroscopy and BP neural network [20]92.00%Live knots, dead knots, pinholes, and cracks
4LBP texture feature extraction and BP neural network [42]93.00%Live knot, dead knot, and leaf knot
5Linear discriminant analysis (LDA) and compress sensor images [43]94.00%Live knot, dead knot, and crack
6LBP and local binary differential excitation pattern [44]94.30%Crack and the mineral line
7The single-shot multibox detector (SSD), a target detection algorithm, and the DenseNet network [22]96.10%Live knots, dead knots, and checking
8A faster region-based CNN with data augmentation and transfer learning [45]96.10%Branch, core, split, and stain defects
9The deep learning feature extraction method combined to extreme learning machine (ELM) classification method [46]96.72%Dead knot, live knot, worm hole, decay
10TL-ResNet3498.69%Decayed knot, dry knot, edge knot, encased knot, horn knot, leaf knot, sound knot
11NIR, PLS-DA, and SVMC [39]99%Knot, decay, resin pocket, bark, reaction wood
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Gao, M.; Qi, D.; Mu, H.; Chen, J. A Transfer Residual Neural Network Based on ResNet-34 for Detection of Wood Knot Defects. Forests 2021, 12, 212. https://0-doi-org.brum.beds.ac.uk/10.3390/f12020212

AMA Style

Gao M, Qi D, Mu H, Chen J. A Transfer Residual Neural Network Based on ResNet-34 for Detection of Wood Knot Defects. Forests. 2021; 12(2):212. https://0-doi-org.brum.beds.ac.uk/10.3390/f12020212

Chicago/Turabian Style

Gao, Mingyu, Dawei Qi, Hongbo Mu, and Jianfeng Chen. 2021. "A Transfer Residual Neural Network Based on ResNet-34 for Detection of Wood Knot Defects" Forests 12, no. 2: 212. https://0-doi-org.brum.beds.ac.uk/10.3390/f12020212

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop