Detection and Monitoring of Pitting Progression on Gear Tooth Flank Using Deep Learning

Abstract

Gears are essential machine elements that are exposed to heavy loads. In some cases, gearboxes are critical elements since they serve as machine drivers that must operate almost every day for a more extended period, such as years or even tens of years. Any interruption due to gear failures can cause significant losses, and therefore it is necessary to have a monitoring system that will ensure proper operation. Tooth surface damage is a common occurrence in operating gears. One of the most common types of damage to teeth surfaces is pitting. It is necessary for normal gear operations to regularly determine the occurrence and span of a damaged tooth surface caused by pitting. In this paper, we propose a machine vision system as part of the inspection process for detecting pitting and monitoring its progression. The implemented inspection system uses a faster R-CNN network to identify and position pitting on a specific tooth, which enables monitoring. Prediction confidence values of pitting damage detection are between 99.5–99.9%, while prediction confidence values for teeth recognized as crucial for monitoring are between 97–99%.

1. Introduction

Gears, as essential mechanical elements of transmissions, are a subject of intensive research that ranges from quality control of their manufacturing [1], via optimization [2,3,4], estimation of load capacity [5,6], and application of various types of materials [7], to thermal effects in their operation [8].

The operation of gearboxes is based on the contact of gear teeth. This contact involves high pressures that can lead to surface damage of gear teeth. Calculations of gear load carrying capacities are made so that a gear set can work appropriately for the required operation time. High contact pressures can lead to the removal of the material surface during the stroke of two gear wheels. One of the most common types of gear surface damage is pitting. Pitting is followed by a remarkable rise in noise, which can cause a gear to become unworkable. The integration of fault diagnostics systems that can detect and monitor pitting progression can help prevent catastrophic failures in advance and reduce maintenance costs.

There are many studies on the detection of pitting by using vibration or acoustic analysis. Elasha [9] presented pitting detection in the case of a worm gearbox with vibration analysis. He used the spectral kurtosis and enveloping technique to identify the presence of defects. Sarvestani [10] used vibration analysis for early detection and progression monitoring of pitting on gears with large helical transmission used in a ball-mill operation, analyzed using basic visual inspection. Boyu [11] proposed using digital holographic surface imaging to observe the electrode surface during a pitting corrosion process. This method provides visual evidence of localized dissolution on the electrode surface and offers an approach to monitoring pitting progression in real time. Li [12,13] proposed a method for early gear pitting fault diagnosis based on the raw vibration signal as a direct input. The method was developed by stacking a sparse autoencoder and a Gauss–Binary restricted Boltzmann machine.

Qu [14] proposed an unsupervised feature extraction method to learn fault levels directly from the frequency spectra of the data. Test data were collected on seeded pitting faults. The results show that the proposed method can be used to identify faults and for real-time fault diagnosis and prognosis. Liang [15] investigated the tooth pitting propagation of a spur gear using model-based analysis. He used his model to estimate pitting growth by investigating effects on vibration properties. Grzeszkowski [16,17] carried out an experimental study with spur gears and detection of pitting with acoustic and vibration sensors. Through the early detection of pitting damage, he proved that it is possible to predict severe pitting a few hours before it actually occurs.

In recent times, researchers have started using machine vision and deep learning to detect pitting on the tooth flank. Kopiláková [18] suggested a method for evaluating gear damage by pitting. She proposed and tested her setup by taking photos from the gear tooth surface and employing the macrophoto method to evaluate pitting. Alam [19] implemented an inspection system that uses a faster R-CNN network to identify teeth defects after the manufacturing process and combines domain knowledge to reduce the manual inspection of non-defective gears by 66%. Wang [20] proposed a method for measuring the area ratio of tooth pitting that is based on a deep convolutional generative adversarial network (DCGAN) and a fully convolutional segmentation network (U-Net). DCGAN is applied to expand pitting samples. Li [21] proposed a real-time detection of gear tooth pitting based on machine vision. He used a principle of gear meshing and the shooting principle of a line-scan camera in order to obtain the optimal centrifugal shooting distance for further analysis.

Xi [22,23,24] enlarged training samples with multilevel pitting. He used a multipath fusion Mask R-CNN with double attention (DAMF Mask R-CNN) to implement the simultaneous segmentation of tooth surface and gear pitting. He designed a platform for online collection of gear pitting images that are measured with deep Mask R-CNN. After pitting was detected, he considered the size of the defect and calculated the pitting area ratio. He also proposed an integrated object detection Yolov5-Deeplabv3 + real-time segmentation network (YDRSNet) for gear pitting measurement, which can be used to measure the pitting area ratio and detect the degree of gear failure online.

1.1. Gear Failure

There are many different types of gear teeth damage such as pitting, wear, scoring, and scuffing. Reasons for gear damage and gear failure can be inadequate lubrication, lubrication contamination, installation errors, overload, handling errors, etc. One of the most common types of damage of gear teeth surfaces is pitting. Pitting occurs due to repeated load and contact stress that exceeds surface fatigue strength of the gear material. Pitting can occur as soon as gears start to operate.

Pitting is a result of rolling and sliding contact fatigue damage, which can occur at the scale of the nominal areas subject to this type of contact or at the scale of roughness. Pitting can be micropitting and macropitting. Micropitting manifests itself as roughness and it can be observed as changed color on some part of the gear tooth flank. Macropitting is the teeth surface damage that includes the appearance of pin-holes and extended flank spalling [25].

In this paper, we will focus on macropitting. Pitting is divided into initial and destructive pitting. Initial pitting is caused by high local stress due to uneven surfaces on the gear teeth and it can occur without any further progression. In some cases, after the initial pitting occurs, wear may occur in further operation, which could decrease the size of pits or even lead to the disappearance of pitting traces on the teeth surface. Destructive or progressive pitting, as a result of surface overload, increases progressively in size and number of pits. The existence of pits that increase in their number and size decreases the operation effectiveness of the drive and increases noise. Therefore, detection and monitoring of pitting progression are crucial for understanding operating conditions and detecting damage causes.

1.2. Contributions

In this paper, we present a novel method for monitoring pitting damage that occurs on the gear flank. This is achieved by detecting pitting damage on a single gear tooth and calculating damage using a neural network. The calculation of the actual size of pitting damage is performed by detecting the pixel size of a single tooth and the pixel size of pitting damage.

The aim of this paper is to test the above-mentioned idea by using a deep learning framework [26] combined with the images that simulate the progression of pitting damage. It is expected that this method could be used for automated monitoring of pitting damage without human inspection. The practical application of the method shown in this paper is important in the case of gears for which any interruption due to gear failures can cause significant losses.

2. Method Description

To detect and monitor pitting on gear teeth, we used a method that is based on integrating domain knowledge with the faster-RCNN deep learning model [27] trained using bounding box annotations of the defects and teeth. The output of the faster R-CNN is an image with the bounding box of the area that has the shape of the defect and teeth, and the corresponding prediction confidence. This approach was used in [17] on a similar problem, where it gave satisfying results from a practical point of view, which is why the authors decided to use this neural network, as it is a tool that can lead to the practical contribution of this research.

Faster-RCNN [27] is a CNN-based model whose input is an image. During processing, it proposes a bounding box localization of target objects and corresponding class probabilities, and results in an image of a desired region of interest as its output. Faster R-CNN (Figure 1) consists of three network components: a Feature Pyramid Network (FPN), a Regional Proposal Network (RPN) and a Fast R-CNN.

The Feature Pyramid Network is a backbone network for feature extraction. The network input is an RGB image from which the Feature Pyramid Network extracts features that can be used for object localization and classification. These features are taken in the form of width × height × depth, where the depth is always 256 channels and it is constant across the extracted layers, which are generated in different scales depending on the layer in the network.

The Regional Proposal Network identifies interesting areas on an image. It takes the feature maps from the backbone network through a 3 × 3 convolutional filter to generate bounding boxes over the image along with their prediction confidence. Anchors are generated at each position with different scales and aspect ratios so that they can be considered as the foreground or background class. For the balanced dataset, the network reduces the number of labels to the level that they are equal to the foreground class by randomly selecting labels from the background class.

The Fast R-CNN is used for object classification and bounding box regression by taking both the output feature map from the FPN and the prediction objects from the RPN. The network classifies the predicted object as being either an exact tooth on the gear or pitting on a tooth or a background class, and also outputs the corner coordinates of the bounding box that contains the object.

3. Data Collection

In this paper, a spur gear (Figure 2) is used with the following characteristics: normal module m = 4, number of teeth z = 15, pressure angle at normal section α = 20°, reference diameter d = 60 mm, tip diameter d_a = 68 mm, root diameter d_f = 50 mm, profile shift coefficient x = 0, and width b = 40 mm. For the experiment, one ring was printed with the outer shape of the gear that was positioned next to the gearing so that the whole gear width was increased to 50 mm. On this gear ring, every tooth had its own number. For the creation of the dataset, four identical gears were used, and pitting damage was seeded on them. The smallest defect that was seeded on the gear teeth had the size of 4 mm and the biggest was approximately 12 mm.

The spatial resolution of the camera that was used in photographing gear teeth was 2048 × 1536 pixels. Bounding boxes around the teeth and pitting damage were generated with a Hasty annotator [28]. A total of 823 images were used for the whole training and testing process. Out of these images, 349 were images with defects, while all the images had two or sometimes three recognized teeth—the size and the angle from which these teeth were photographed was relevant for further processing. For the recognition of teeth, a total of 1392 teeth were labeled. The dataset was created using a regular Nikon COOLPIX L320 camera from the distance of approx. half a meter in room lighting conditions without the use of a flash. Since the monitoring of gear pitting development is new, the authors chose to take images that had an almost perpendicular view to a single tooth, since then the images would show one tooth with potential damage. The intention was that the whole tooth with potential damage was in just one plane, which would simplify the process of labeling and detection with a bounding box, and finally lead to the possibility of calculating the size of pitting. The dataset was created by constant rotation of the gear and burst mode shooting.

The Faster-RCNN model was built using Facebook’s Detectron2 python library [26]. In training the network, 16 foreground classes were considered, from which 15 were for each gear tooth and the sixteenth was for pitting damage. We used a learning rate of 0.005 and trained the network for 12,000 iterations.

The selection of crucial parameters of the neural network is very important for effective training in terms of convergence speed and avoidance of local minima. The selection of these parameters is a quite complex task given the differences in modeling problems and available datasets, and, moreover, there are no rules that would systematically “fine” adjust these parameters. Therefore, some time-consuming empirical experimental trials are often needed, in the form of the trial-and-error method [29], in order to determine the most adequate set of neural network weights and biases, values which would ultimately ensure model of the high generalization capability of the model.

Nguyen and Lee [30] also stated that: “The hyper-parameters for the training process affect results of the training, that is, the weight factors. Unfortunately, since a set of parameters for optimization depends on the network model and the dataset, it is generally chosen arbitrarily or based on prior experiences and is adjusted by trial and error”.

In our case, there are two different class types for the neural network: (1) pitting as damage and (2) single tooth. In the training process of the neural network, was much easier to detect pitting damage than a single tooth. For pitting, satisfying results (prediction confidence higher than 95%) start to occur with 5000 iterations. The detection of a single tooth was harder, and the final parameters were connected to the recognition of teeth. In the end, we achieved high prediction confidence for pitting and lower confidence for single tooth detection.

For the evaluation of the general ability of this model to detect teeth and pitting damage, we trained the model using a dataset of 349 defect images. Approximately 90% of the images were used for training, while the remaining 10% were used for testing.

The performance of neural network models may be changed through different splitting ratios of training and testing data. Since there are no strict rules regarding the dataset splitting methods, the applications of R-CNNs in the field of machine vision usually consider the following ratios: 80% training and 20% testing [31], 90% training and 10% testing [32,33,34,35]. Additionally, good results in machine vision detection in the similar case [19] were obtained by using the 90% training and 10% testing approach. Therefore, in the present study, the available dataset was randomly divided into training (90%) and testing (10%).

4. Validation

For the validation of the model and the whole approach to detecting and monitoring pitting damage, three images of one tooth (tooth no. 7) that has three-level pitting damage were used. The first level (Figure 3) is the damage with “smaller pitting”, which is seeded in two places—“Pitting 1” on the left and “Pitting 2” on the right side of tooth 7.

The second level (Figure 4) of the damage is “middle-size damage”, in which pitting damage (Pitting 1 and 2) is extended on the left side. The third level (Figure 5) of the damage is “bigger-size damage”, in which pitting damage is further developed (Pitting 1 and 2) on the left side.

The prediction confidence for Pitting 1 and 2, on the one hand, and for tooth 7, on the other hand, shows that the prediction confidence for the tooth is a little bit smaller. Figure 3, Figure 4 and Figure 5 show the prediction confidence for other teeth, such as tooth 6 or 8, and it can be seen that the prediction confidence for these teeth is between 97 and 99%, while pitting damage prediction confidence is between 99.5 and 99.9%.

Table 1. Results of output images for Pitting 1 and 2, and Tooth 7.

		Coordinates of Boxes (Start) (Pixel)	Coordinates of Boxes (End) (Pixel)	Prediction Confidence (%)
Pitting 1	1st level	3637.1069; 1157.3285	3948.2188; 1461.5745	99.86
	2nd level	2357.3508; 1171.1965	2741.0522; 1418.6853	99.94
	3rd level	2561.0249; 959.416	3165.9492; 1308.5024	99.91
Pitting 2	1st level	2555.4255; 1114.2948	2839.6267; 1399.51	99.8
	2nd level	3289.771; 1180.4796	3832.9392; 1448,9137	99.91
	3rd level	3396.7253; 1029.2837	4248.0825; 1308.5024	99.91
Tooth 7	1st level	1803.7611; 1096.854	4927.0244; 1528.479	97.5
	2nd level	1698.1724; 1136.9203	4797.167; 1581.1718	97.78
	3rd level	2155.6003; 978.2216	5201.3003; 1474.3358	98.47

provides the results concerning the output images with start and end coordinates of the bounding boxes. Rectangular bounding boxes are defined with two diagonally distant points. Table 1 also lists diagonal coordinates for Pitting 1 and Pitting 2 for all three levels of pitting damage. Additionally, Table 1 gives the output data for tooth 7 as they are important for further analysis. The size of pitting or tooth in pixels was obtained by subtraction of end and start coordinates.

The sizes of pitting damage in the x and y direction were calculated according to the data from Table 1. For further calculation the gear width was set to 50 mm, which corresponds to the y direction on the image, and through a simple ratio of pixels for the pitting area and the tooth, the x and y direction sizes are obtained in millimeters.

Table 2. Difference between the measured and the calculated size of pitting damage for Pitting 1 and 2.

		Physically Measured [mm]	Calculated [mm]	Difference [%]
Pitting 1 x direction	1st level	4.3	4.905	11.7
	2nd level	4.3	4.054	−3.37
	3rd level	4.95	5.586	13.61
Pitting 2 x direction	1st level	4.1	4.599	10.09
	2nd level	4.3	4.399	0.46
	3rd level	4.4	4.465	3.93
Pitting 1 y direction	1st level	4.3	4.977	13.31
	2nd level	5.9	6.195	4.84
	3rd level	9.2	9.914	7.16
Pitting 2 y direction	1st level	4.1	4.545	9.89
	2nd level	8.4	8.777	4.33
	3rd level	12.8	13.985	8.51

lists the physically measured and calculated results with a difference that is given in percentages. The difference between the measured and the calculated results show that the calculated results are almost always bigger than the measured results. The average difference for all the cases is 7.04%, while it is 6.07% for the cases in the x direction and 8.01% for the y direction.

Figure 6 shows the results for Pitting 1 and 2 in the y direction. It can be observed that pitting is stagnating or has a very small increase, which corresponds to the progression of pitting given in Figure 3, Figure 4 and Figure 5.

Figure 7 shows the results for Pitting 1 and 2 in the x direction. It can be observed that pitting increases at all levels, which corresponds to the progression of pitting given in Figure 3, Figure 4 and Figure 5.

5. Conclusions

In this paper, we focused on training a single model for detecting and monitoring a single defect. This was achieved by detecting a defect—pitting—and detecting each tooth on the gear. Monitoring pitting progression was achieved by detecting the defect and the tooth containing the defect on the same image. This provided the opportunity to follow pitting progression by estimating the pitting size through comparing it with the size of the tooth.

The verification of this approach was achieved by seeding pitting damage on the single gear tooth. Prediction confidence values were greater for pitting damage (99.5–99.9%) than for the recognized tooth (97–99%). A comparison of the results for the pitting size based on machine vision calculations, on the one hand, and physical measurements, on the other hand, showed a difference of 7.04%. The calculated pitting was almost always bigger than the measured one.

With all these conclusions, in future research, we will try to capture other gear teeth defects that can occur on the gear, and investigate the possibility of using semantic segmentation in obtaining more accurate results.