RCE-GAN: A Rebar Clutter Elimination Network to Improve Tunnel Lining Void Detection from GPR Images

Abstract

Ground penetrating radar (GPR) is one of the most recommended tools for routine inspection of tunnel linings. However, the rebars in the reinforced concrete produce a strong shielding effect on the electromagnetic waves, which may hinder the interpretation of GPR data. In this work, we proposed a method to improve the identification of tunnel lining voids by designing a generative adversarial network-based rebar clutter elimination network (RCE-GAN). The designed network has two sets of generators and discriminators, and by introducing the cycle-consistency loss, the network is capable of learning high-level features between unpaired GPR images. In addition, an attention module and a dilation center part were designed in the network to improve the network performance. Validation of the proposed method was conducted on both synthetic and real-world GPR images, collected from the implementation of finite-difference time-domain (FDTD) simulations and a controlled physical model experiment, respectively. The results demonstrate that the proposed method is promising for its lower demand on the training dataset and the improvement in the identification of tunnel lining voids.

1. Introduction

Void is a common defect in tunnel lining structure due to aging, environmental factors, inadequate or poor construction and maintenance. Void in tunnel linings may pose a threat to the security, durability, and serviceability of tunnel structures. Therefore, it is necessary to inspect void presence in tunnel lining during tunnel construction or operation stage. Compared with traditional methods such as core sampling and impact-echo, the ground penetrating radar (GPR) has gained widespread use in tunnel lining inspection due to its non-destructiveness and high-efficiency [1,2,3,4,5,6]. However, the interpretation of GPR data is a challenging task. The obtained GPR data are reflected electromagnetic waveforms of the subsurface rather than direct imaging of the tunnel lining structure [7,8]. Further interpretation of GPR data is required to characterize the internal structure of tunnel linings. At present, the interpretation of GPR data relies heavily on manual work, which involves technicians’ expertise and the efficiency and accuracy cannot be guaranteed when the amount of data are huge. Worse still, rebars in the near surface of concrete linings act as scatters that generate clutters in GPR profiles and mask the target signatures, making defect echoes beneath the rebars difficult to distinguish. Therefore, an effective method to recognize defects beneath rebars from GPR images is recommended to improve GPR detection accuracy.

Automatic GPR image interpretation using deep learning (DL) methods has been the current trend. With the advent of convolutional neural networks (CNNs) [9,10], the DL methods have made promising advances in the field of image recognition. Pham et al. [11] used a pre-trained Faster R-CNN framework to achieve GPR image identification. The detection results confirmed the superiority of the DL methods over conventional machine learning methods in terms of accuracy. It also proved the effectiveness of the pretraining strategy when the training samples are not sufficient. In order to identify hyperbolic signatures in GPR images, Lei et al. [12] developed a double cluster seeking estimate and column-based transverse filter points algorithm to fit curves in the hyperbola region detected by the Faster R-CNN. The proposed scheme is able to extract the hyperbola signature automatically and efficiently, but it cannot characterize other features in GPR images. Xu et al. [13] improved the Faster R-CNN using feature cascade, adversarial spatial dropout network, and soft-nonmaximum suppression. The low-, middle-, and high-level representations are combined through the feature cascade to form a multi-sized feature, which is proved to be beneficial for detecting small objects in GPR images. Qin et al. [14] introduced an automatic recognition method based on the Mask R-CNN to identify steel ribs, voids, and initial linings from GPR images for tunnel detection. The DL method attracts more and more attention for its outstanding abilities in the field of object detection. The combination of DL and GPR data is promising for its high accuracy and efficiency.

However, applications of DL method on the interpretation of GPR data have encountered challenges. The signatures in GPR images are reflected electromagnetic waves rather than their real shapes, which makes commonly used object detection networks invalid for GPR image identification. Moreover, rebar clutters in GPR images interfere with target reflections, leading to more difficulties in applying DL methods to GPR image recognition. In order to mitigate the adverse effects of rebar clutters induced by the rebar layer in the near-surface of reinforced concrete, many researchers have proposed signal processing and image enhancement methods. Dinh et al. [15] used the migration method to intensify the vertex of clutters caused by rebars and picked them by thresholding the GPR images. The proposed scheme is in good agreement with manual rebar picking, but object detection beneath rebars was not considered. The hyper-curvelet transform was applied in [16] to suppress the rebar echo in GPR data, but validation on real GPR data was not conducted. Xiao et al. [17] proposed an algorithm using the multi-bandpass filter technique to suppress the clutters in GPR profiles caused by periodic elements in the near-surface like rebars in reinforced concrete. This method needs less data than those with migration and data processing being simple and fast. However, this method is only adequate for the suppression of periodic clutters. In general, most traditional rebar clutter suppression techniques rely heavily on parameters and thresholds that need to be tuned by hand and are not fit for automatic processing.

DL methods based on autoencoder [18] has shown great potential in terms of denoising and data reconstruction in signal and image processing tasks [19]. Autoencoder is a type of generative network that learns a mapping between inputs and outputs. Huang et al. [20] proposed an autoencoder-based method to improve the quality of portable ultrasonic B-mode images. The simulation results revealed that the autoencoder-based networks outperform the CNNs and DNNs. Feng et al. [21] attenuated random GPR noise for enhancing signal-to-noise ratio by applying autoencoders. The experiments verified that the proposed method effectively reduces noise and causes less damage to the original GPR data. In [22], a convolutional autoencoder for landmine detection using GPR data was presented and reported state-of-art and robust results of a wide variety of targets. There are a lot of variants of autoencoders that have been reported until recently, such as sparse autoencoder [23], denoise autoencoder [24], and contractive autoencoder [25]. A wide range of applications of autoencoders demonstrates their great potential in GPR data denoising and reconstruction.

In a recent study [26], an encoder–decoder structure with residual-inception blocks and spatial attention modules was proposed. The network was designed for rebar clutters removal and defect echoes enhancement in GPR B-scan images. However, these frameworks demand spatial-paired source data. Pixels that belong to the target echoes from two sets of GPR images, in other words, a set of GPR images containing rebar clutters and a set of GPR images only having defect echoes should represent exactly the same spatial location. This is a strict condition to meet, especially for the rebar clutter elimination task. At present, there is no standard GPR dataset similar to ImageNet [27] or COCO [28] based on real photos, which contains a large amount of data. In addition, it is almost impossible to gather two sets of GPR data that have a spatial-paired relationship in the real world to train the supervised learning network. Although it is possible to obtain spatial-paired synthetic GPR data using the finite-difference time-domain (FDTD) method, generalization problems are likely to occur if the network is trained on the FDTD synthetic data and tested on real-world GPR data. Pixel-level mapping in synthetic GPR data may hard to be transferred to real GPR data because real and synthetic GPR data have different features in terms of noise, waveform, etc.

Motivated by the aforementioned issues, the main goal of this study is to minimize the impact of rebar interference by designing a deep-learning network to eliminate rebar clutters in GPR images automatically. At the same time, in order to make the network easier to implement, it is crucial to reduce the requirement for training data. In this study, a generative adversarial network (GAN) based rebar clutter elimination network (RCE-GAN) is proposed. Unlike the supervised learning networks like pix2pix [29] and the network used in [26,30], our network is an unsupervised learning model and does not need paired GPR images as source data. Our network is improved from the CycleGAN [31], which has two sets of generators and discriminators. By introducing the cycle-consistency loss, the network is capable of learning high-level features between GPR data with and without rebar clutters and establishing a mapping between two kinds of images. Hence, our network can generate GPR images without rebar clutters according to the inputs. Considering that defect echoes might be different in shape or size in a GPR image, an attention module and dilation center part are introduced to the generator of our network to improve rebar clutter elimination performance.

The remainder of this paper is organized as follows: we begin with the introduction of the network configuration. An ablation study is carried out to examine the function of modules in the network. Then, we use synthetic data to investigate the performance of the network by comparing cases with different rebar spacings and rebar-void distances. Finally, an experiment on real-world GPR data is performed to validate the proposed method.

2. Methodology

The workflow of the proposed method is illustrated in Figure 1. Firstly, the original GPR data are processed by our rebar clutter elimination network. In this stage, the GPR images which contain rebar clutters are taken as input to a trained encoder–decoder network. The encoder–decoder network generates new GPR images in which the rebar clutters are eliminated. Next, the processed GPR data are sent to an object detection network, such as YOLOv4 [32]. The signatures of the targets are marked out, and herein, the exact positions of defects beneath rebars are obtained accordingly.

The novel contributions of the proposed method are as follows: (1) Designing a deep learning network (RCE-GAN) to eliminate rebar clutters in GPR images. (2) Two sets of generators and discriminators are designed in the network to learn how to interpret the obtained GPR data with an unsupervised learning mode. (3) Combining the existing object detection network and RCE-GAN to improve the identification of the void beneath the rebar. In this section, the principle of RCE-GAN is introduced, and a detailed configuration of the generator and discriminator of the network is discussed.

2.1. Principles of Proposed Networks

The overall structure of the RCE-GAN is illustrated in Figure 2. The whole network is established for learning a mapping function G, which can hereafter be used to transform GPR images with rebar clutters to GPR images without rebar clutters. The network contains two generators and discriminators. The two generators learn two mappings, G: A to B and F: B to A, where A and B represent two domains of GPR data, domain A and domain B. Domain A comprises GPR data that have rebar clutters while domain B contains GPR data which only have defect echoes. There are two adversarial discriminators, ${\mathrm{D}}_{\mathrm{A}}$ and ${\mathrm{D}}_{\mathrm{B}}$, where ${\mathrm{D}}_{\mathrm{B}}$ encourages the generator G to generate images that are indistinguishable from domain B, and vice versa for ${\mathrm{D}}_{\mathrm{A}}$, F, and domain A. The two adversarial learning processes are expressed with the loss functions

$$L_{\mathrm{GAN}}\left(\mathrm{G},{\mathrm{D}}_{\mathrm{B}},\mathrm{B},\mathrm{A}\right) =E_{\mathit{b}\sim \mathit{p}\left(\mathit{b}\right)}\left[log{\mathrm{D}}_{\mathrm{B}\left(\mathit{b}\right)}\right]+E_{\mathit{a}\sim \mathit{p}\left(\mathit{a}\right)}\left[log(1-{\mathrm{D}}_{\mathrm{B}}\left(\mathrm{G}\left(\mathit{a}\right)\right)\right]$$

(1)

$$L_{\mathrm{GAN}}\left(\mathrm{F},{\mathrm{D}}_{\mathrm{A}},\mathrm{A},\mathrm{B}\right) =E_{\mathit{a}\sim \mathit{p}\left(\mathit{a}\right)}\left[log{\mathrm{D}}_{\mathrm{A}\left(\mathit{a}\right)}\right]+E_{\mathit{b}\sim \mathit{p}\left(\mathit{b}\right)}\left[log(1-{\mathrm{D}}_{\mathrm{A}}\left(\mathrm{F}\left(\mathit{b}\right)\right)\right]$$

(2)

where a and b represent the GPR images in domain A and domain B, respectively. We denote $a\sim p\left(a\right)$ and $b\sim p\left(b\right)$ as data distribution and E as the expectation of the loss function based on x and y. Equation (1) is the adversarial loss for mapping function G: A to B, in which G generates images G(a) that look similar to images from domain B, while ${\mathrm{D}}_{\mathrm{B}}$ distinguishes between generated sample G(a) and real sample b. G aims to minimize this objective against an adversary ${\mathrm{D}}_{\mathrm{B}}$ that tries to maximize it. The principle of the adversarial loss for the mapping function F: B to A is the same as G, which is expressed as Equation (2).

With the adversarial loss, the networks can be trained to learn mappings G and F that produce outputs identically distributed as those of the target domains B and A, respectively. However, if we only apply the adversarial loss, the network would randomly generate GPR images without rebar clutters that have no relationship with the inputs at all. Thus, the learned mapping function should be cycle-consistent. As shown in Figure 2, the reconstructed images $\widehat{a}$ should be identical to the original inputs b. The cycle consistency loss is expressed as

$$L_{\mathrm{cycle}}\left(\mathrm{G},\mathrm{F}\right)=\Vert \widehat{a}-a{\Vert }_{\mathrm{L}1}$$

(3)

As shown in Equation (3), L1-distance loss is applied to make the reconstructed images close to the original input in an L1 sense. By introducing the cycle-consistency loss, the network can learn high-level features between the two domains and establish a mapping from domain A to domain B. Finally, our full objective function is defined as

$$L=L_{\mathrm{GAN}}\left(\mathrm{G},{\mathrm{D}}_{\mathrm{B}},\mathrm{B},\mathrm{A}\right)+L_{\mathrm{GAN}}\left(\mathrm{F},{\mathrm{D}}_{\mathrm{A}},\mathrm{A},\mathrm{B}\right)+\lambda L_{\mathrm{cycle}}\left(\mathrm{G}\right)$$

(4)

where $\lambda$ is a balancing parameter that controls the relative importance of the two objectives.

2.2. Improved Generators and Discriminators

2.2.1. Encoder–Decoder Generator

As shown in Figure 3a, our network uses the encoder–decoder structure as the basis of the generator. The input GPR images are first cropped and resized to $512\times 512$, and then passed through a series of convolutional layers and down-sampling layers, until a bottleneck layer, at which point the process is reversed. The encoder structure obtains the deeper features by continuously compressing shallow features. The entire encoder consists of five convolutional blocks with kernel size of 3 and four maxpooling layers. After a series of processing, the encoder finally obtains deep features. After the bottleneck layer, a decoder is used to amplify the compressed features to the size of original images by the deconvolutional blocks and up-sampling layers. A total of four convolutional blocks with kernel size of 3 and four deconvolutional blocks are employed in the decoder network.

Considering the connectivity, complexity, and various shapes of the defect echoes, it is important to increase the receptive field of feature points in the center part of the network as well as keep detailed information. Hence, we introduce the dilation center part to the bottleneck of our network [33]. Different from the general encoder–decoder structure, several dilated convolution layers with skip connections are placed in the center part.

The structure of dilation center part is presented in Figure 3b. The dilated convolutional layers with dilation rate of 1, 2, 4, and 8 are assembled to the network. The output of the encoder is taken as the input to the five paths of the dilation center part. The receptive field of each path, from top to bottom, is 31, 15, 7, 3, and 1, respectively. With multi-scale receptive fields, the dilation center part produces multi-resolution features.

For the rebar clutter elimination task, there is a great deal of low-level information shared between the input and output. For example, the input and output share the locations of defect echoes. Thus, it would be desirable to shuttle this information directly across the net. Because the echoes of defects are weak and have various positions, it is advisable to make the areas that contain reflection waves attract more attention from the network before connecting to the deeper layers. Therefore, an attention module is performed right before the concatenation operation to merge only relevant activation. As shown in Figure 3a, the features of the coding and decoding structures are taken as the input of the attention module. This operation facilitates the network to integrate the features of the encoder and decoder and learn the weight information.

The detailed information of the attention module is shown in Figure 3c. The attention module accepts two inputs, which are the vectors x and g. Vectors x and g are passed through a convolutional block with kernel size of 1 × 1, respectively, and added together. This process imposes more weights on the relevant area and gates the irrelevant background. In this way, the network learns features that require more attention. Therefore, by adding the attention module, the network distributes more attention on defect echoes.

2.2.2. Discriminator of the Network

The generator with L1-distance loss captures the high-level content in terms of objects and their arrangement. However, the L1-distance loss fails to encourage high-frequency features such as texture and outline and produces blurry results. It is advisable to restrict the GAN discriminator to only model high-frequency structure, relying on L1 loss to force low-frequency correctness. We utilize PatchGAN [29] as the discriminator of RCE-GAN. The architecture of the PatchGAN discriminator is shown in Figure 4. The discriminator first divides the whole GPR images into several $N\times N$ patches, in which N is taken as 70. The patches are put into a series of convolutional layers, behind which are the batch normalization layers and LeakyReLU layers, except for the last convolutional layer. The discriminator encourages high-frequency features by penalizing only at the scale of patches. We run this discriminator convolutionally across the image, averaging all responses to provide the ultimate output of D. The sizes of the patches are much smaller than that of the whole image. Hence, the network has fewer parameters, runs faster as well as produces high-quality results.

3. Ablation Study

Our method combines the RCE-GAN and an object detection network to improve the identification of the void beneath the rebar. We improve RCE-GAN from CycleGAN by adding attention modules and a dilation center to the original encoder–decoder generator. We altered the RCE-GAN to produce three variants (encoder–decoder, attention net, and dilation net) to validate the effectiveness of added modules. Here, the original CycleGAN is denoted as encoder–decoder. The encoder–decoder with the attention module is denoted as attention net, while the encoder–decoder with the dilation center part is denoted as dilation net. RCE-GAN that has both the attention module and dilation center part is denoted as attention-dilation net in this section. The comparison and validation process is carried out in three aspects: (1) the quality of generated images is evaluated manually, concerning the correctness of defect echoes of generated GPR images; (2) the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are used to quantitatively evaluate the similarity between the generated and ground truth GPR images; and (3) the average processing time of the four networks is compared. In this section, we also compare the detection method in our method with two mainstream target detection algorithms, SSD [34] and Faster-RCNN [35].

3.1. Quality of Generated Images

We use two numerical cases with different rebar spacings to evaluate the GPR images processed by the aforementioned four networks. Figure 5a depicts the permittivity map representing the tunnel lining concrete, in which two voids are buried beneath a dense rebar mesh of 10 cm spacing. In this situation, as shown in Figure 5b, the echoes of the voids are weak, and the reflection waves of rebars overlap and form layers of pseudo-hyperbola, which further affect recognition of defects echoes.

Compared with the ground truth GPR image without rebar reflections incorporated (Figure 5c), the GPR image generated by the encoder–decoder (Figure 5d) fails to suppress the pseudo-hyperbolas formed by the closely spaced rebars. The generated image is noisy and target reflection waveforms are not smooth. The GPR image generated by the dilation-net (Figure 5e) suppresses the pseudo-hyperbolas but fails to reconstruct the target reflections correctly. The vertexes of the reconstructed hyperbolas are shallower than their actual positions in the ground truth GPR image. The image generated by the attention-net (Figure 5f) eliminates the pseudo-hyperbolas largely and focuses on the reconstruction of the target waveform. However, it still fails to fully eliminate the pseudo-hyperbolas, which may cause false identification. The GPR image generated by the attention-dilation net (Figure 5g) balances the suppression of the pseudo-hyperbolas and the reconstruction of the target waveform, therefore having the best quality by visual inspection.

Figure 6a depicts the second numerical case, in which the rebar spacing is 30 cm. In this situation, as shown in Figure 6b, the rebar reflection waves are sparse and the target reflections are clearer. However, the rebar clutters are similar to the defect echoes, which may confuse the network and lead to false GPR images. The encoder–decoder fails to eliminate the pseudo-hyperbolas (Figure 6d), whereas the attention net does not do any useful modifications to the original GPR image (Figure 6e). The dilation net and attention-dilation net, both of which are equipped with a dilation center part, demonstrate superior performance in this situation (Figure 6f,g).

From the cases illustrated above, our model (the attention-dilation net) outperforms the other network structures in the comparison. It eliminates the rebar clutters completely in both dense and sparse rebar spacing situations. The attention module helps the network to focus on the features of target echoes and improves the immunity to interference. The dilation center part helps the network take advantage of multi-resolution features and reconstruct GPR images from the global perspective. Hence, the network with dilation center part can distinguish between rebar clutters and defect echoes. Our network combines the advantages of two modules, and is proved eligible for the rebar clutter elimination task.

3.2. Similarity Assessment

We use PSNR and SSIM to quantitatively evaluate the performance of each network. PSNR is an indicator to measure the quality of the generated GPR images. SSIM is used to evaluate the similarity between generated and the ground truth images. In the evaluation process, we find that, if we take the whole GPR images as input to evaluate PSNR and SSIM, the generated GPR images that fail to reconstruct defect echoes but have less noise have higher PSNR and SSIM scores. On the contrary, the generated GPR images which have defect echoes with more noise may get lower PSNR and SSIM scores. The aim of our method is to make the existing object detection network perform better on the GPR images processed by the RCE-GAN. If we use the PSNR and SSIM values as the network performance indicators, the situations described above are obviously unreasonable. This phenomenon occurs because the background accounts for the most proportion of area in a GPR image. Hence, a noisy image has lower PSNR and SSIM scores no matter whether the defect echoes are generated correctly or not.

We therefore present a local binarization and evaluation method to tackle the aforementioned problems and provide more reasonable evaluations for the quality of the generated GPR images. The workflow of the local binarization and evaluation method is shown in Figure 7a. Firstly, the ground truth GPR image is proposed by the object detection network, for example, a trained YOLOv4 network in this study. The target echoes are detected as shown in Figure 7b. This step is used to locate the target echoes and zoom the scope that is subject to similarity evaluation. Secondly, all GPR images that need to be evaluated, which is shown in Figure 7c, are cropped at the same positions as those of the detection boxes of the ground truth images. Then, the cropped images are binarized to highlight the target echoes. In the binarization process, the threshold value is a key parameter. Because a constant threshold value cannot work for all generated GPR images with various qualities, in this step, we use the iterative thresholding algorithm for binarization of the cropped GPR images. The binarized cropped GPR images are shown in Figure 7d. After binarization, all the generated GPR images are compared with the ground truth image and their corresponding PSNR and SSIM scores are computed. The values are summarized in

Table 1. Performance of each variants of RCE-GAN.

Networks	PSNR	SSIM	Processing Time (s)
Encoder–decoder	2.677	0.955	0.585
Attention net	2.873	0.960	0.671
Dilation net	2.453	0.951	0.575
Attention-dilation net	3.139	0.963	0.733

. It can be seen that the attention-dilation-net, which is adopted by our method, has the best performance in both PSNR and SSIM evaluations. The results further confirm that our network performs better in the rebar reflection wave elimination task than the other network structures.

3.3. Processing Time

In actual projects, there are large quantities of GPR images that need to be processed. Thus, the processing time of a single image is one of the factors which should be taken into consideration when we design the network. The processing time of the four networks is summarized in Table 1. Although the processing speed of the attention-dilation net is $9\%$∼$27\%$ slower than that of the other three networks, it is still acceptable in real-world applications. It is worth noting that the GPR images processed by our network have higher quality than the other networks, which is more suitable for rebar clutter elimination task due to its high accuracy and robustness.

3.4. Detection Networks

Different detection networks including SSD, YOLOv4, and Faster-RCNN are compared to evaluate the target detection performance. The three models are trained for 100 epochs on the same dataset which contains 450 synthetic GPR images with void echoes. Four metrics including Recall, Precision, F1-score, and mAP are used to evaluate the detection performance, which are defined as follows:

$$\mathrm{Precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$$

(5)

$$\mathrm{Recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$$

(6)

$$F1-\mathrm{score}=\frac{2\times \mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}$$

(7)

$$\mathrm{AP}={\int }_0^1\mathrm{P}\left(r\right)dr$$

(8)

where true positive (TP) and false positive (FP) denote the number of targets that are correctly and incorrectly identified as positive samples; false negative (FN) signifies the number of targets that are falsely identified as negative samples; precision is the rate of TP in all samples that are detected by the network as positive samples (including TP and FP), representing the network’s ability to correctly identify the positive samples; recall is the rate of TP in all samples that are actually positive samples (including the TP and FN), reflecting the capability to find all the actual positive samples; F1-score is also called balanced score, which can be considered as a weighted average of Precision and Recall of the model, with a maximum value of 1 and minimum value of 0. Greater F1-score means that the network achieves higher precision and recall values simultaneously; average precision (AP) indicates the area enclosed by the P–R (precision–recall) curve and the abscissa R (recall) axis. Mean average precision (mAP) is the mean value of AP for all classes.

The performances of the three models are summarized in

Table 2. Performance of each detection network.

Networks	Precision	Recall	F1-Score	mAP	Training Time (s)	Processing Time (s)
SSD	97.78	100.00	0.99	0.99	2040	0.181
YOLOv4	98.87	100.00	0.99	1.00	2880	0.213
Faster-RCNN	100.00	100.00	100.00	1.00	4620	0.234

. All the detection networks reach high precisions, indicating that they can satisfy our defect detection task. Taking into consideration the training and processing time of the models, we choose the YOLOv4 as our detection network.

4. Synthetic Example

4.1. Synthetic Data Preparation

We create a series of numerical models to simulate void defect in tunnel linings using the gprMax program, a Python based FDTD solver of Maxwell’s equations [36]. The synthetic models mainly contain two layers, which are air and the tunnel lining, inside which a row of rebars are placed 5 to 10 cm beneath the concrete surface. Different rebar spacings of 10, 20, 30, and 40 cm are simulated, and arbitrarily shaped voids that are filled with water or air are distributed beneath the rebars. We use the built-in materials free_space for air and air-filled voids, and pec for rebars. We also define the tunnel lining concrete with ${\epsilon }_r=9$, $\sigma =1\times {10}^{-5}$ s/m and water-filled voids with ${\epsilon }_r=81$, $\sigma =0.01$ s/m. The dimensions of the models are $2.0\times 1.0$ m. The antenna center frequency is 800 MHz, and the time window is set to 20 ns. After FDTD simulations, we obtain 400 GPR images containing both defect echoes and rebar clutters (Domain A), and 400 GPR images that contain only defect echoes (Domain B) as the training dataset. Note that the GPR images in Domains A and B are not paired data, which means that the pixel-wise mapping between two images from the two domains is not required.

4.2. Network Training

There are over 10 million parameters that need to be determined in the RCE-GAN, yet the amount of data are not sufficient to train all the parameters. Transfer learning is proved to be an effective approach to improve network performance in most situations, especially when the amount of training data are inadequate. Instead of training the network with a set of randomly initialized parameters, we use pre-trained weights that are trained on the Cityscape dataset [29].

As the size of the training dataset is comparatively small, mode collapse of the RCE-GAN occurs occasionally. To cope with this problem, we apply data augmentation methods including mirror flip and saturation adjustment to increase the number of training samples. Additionally, we monitor the network training process by analyzing the output images of the generator in every training epoch. If almost identical GPR images are produced within several consecutive epochs, we deem that mode collapse has occurred. Subsequently, we stop this training session and launch a new one.

The RCE-GAN uses Adam solver [37] as the optimizer. The learning rate and decay rate of first-order momentum are set as 0.0002 and 0.5, respectively. We keep the learning rate unchanged for the first 100 epochs and linearly decayed to zero after 100 epochs. The batch size is set as 1 during the training. The RCE-GAN is implemented on the Pytorch framework and trained on a system equipped with three NVIDIA RTX 2080Ti GPUs.

4.3. Detection Results

The detection results are demonstrated and compared using two testing datasets. Dataset I contains synthetic GPR images with different rebar spacings. For the cases with rebar spacings of 10, 20, 30, and 40 cm, 100 GPR images are simulated for each case. A total of 400 GPR images are created for dataset I. Dataset II consists of 300 GPR images with different rebar-void distances. Void defects are randomly distributed with different burial depths ranging from 0 to 80 cm. The rebars are located 5 cm beneath the concrete surface with 20 cm spacing.

The test results of the RCE-GAN on the dataset I are displayed in Figure 8. The first column (Figure 8a,d,g,j) depicts the FDTD permittivity models with different rebar spacings. The second column (Figure 8b,e,h,k) presents the FDTD simulated GPR images and void detection results using the YOLOv4. When the spacing between rebars is 10 cm, there are layers of pseudo-hyperbolas and the void echoes are masked. In this situation, the targets are almost impossible to be distinguished by either the detection network or experienced technicians. The target echoes are clearer as the spacing between rebars is bigger. When the rebar spacing is greater than 30 cm, the target echoes are no longer affected by the rebar clutters. However, the detection network still fails to identify all the voids in these situations. The GPR images processed by the RCE-GAN are shown in the third column (Figure 8c,f,i,l). After being processed by our RCE-GAN, the rebar clutters are removed largely for the four cases with different rebar spacings, and the void echoes become clearer to identify. We implement the YOLOv4 for defect detection and find that all the void echoes are successfully recognized on the RCE-GAN processed images. We also notice that the RCE-GAN performs better when the rebar spacing is larger. In the example with rebar spacing of 10 cm, there are still some miscellaneous waves in the GPR images processed by the RCE-GAN. When the rebar spacing is greater than 30 cm, the RCE-GAN generated image has much higher quality.

Figure 9 illustrates the detection results with respect to varying rebar-void distances that are roughly divided into three zones, which are 0 to 30 cm, 30 to 60 cm, and greater than 60 cm. The first column (Figure 9a,d,g) plots three examples of the FDTD permittivity models with different rebar-void distances. As displayed in the second column, when the rebar-void distance is smaller than 30 cm, the defect echoes are masked greatly by the rebar reflections. When the rebar-void distance is larger than 60 cm, the defect echoes are affected by the attenuation of GPR signals and layers of pseudo-hyperbolas caused by the rebar reflections. Hence, the defect echoes are hard to identify if the voids are too close to the rebars or the void burial depth is too large. As expected, the detection network does not recognize the void echoes in the first case when the rebar clutters are not removed. The third column (Figure 9c,f,i) shows the RCE-GAN processed GPR images with different rebar-void distances. The rebar reflections and pseudo-hyperbolas are eliminated completely and hence the target waveforms are successfully detected by the object detection network. It is also noted that the RCE-GAN processed GPR image has the best quality when the rebar-void distance is in the range of 30 to 60 cm.

The four metrics signified in Equations (5)–(8) on the two testing datasets are computed and summarized in Figure 10. The first row compares the detection performance on the original images and images after rebar clutter elimination when the spacing between rebar is different; the second row concerns the performance on the two types of images with different rebar-void distances. The four columns represent the Recall, Precision, F1-score, and mAP indicators, respectively.

As revealed in the first row of Figure 10, closely spaced rebars affect the detection performance greatly. When the spacing between rebars is 10 cm, the recall rate is 0.17 on the original image, which is a particularly low value. This is a quite intuitive result since the closely placed rebars form layers of pseudo-hyperbolas and make the target echoes almost invisible. Therefore, the detection network ignores most of the targets. The recall rate increases when the rebar spacing becomes greater. The other three evaluation metrics show a similar trend as the recall rate. After applying the RCE-GAN, the detection performance is improved, especially in the first case that the rebar spacing is 10 cm. The improving effect of the RCE-GAN is not that evident when the spacing between rebar is bigger than 30 cm because the target echoes are already discernible even if the RCE-GAN is not used. The precision rate decreases slightly after processing in each condition because the RCE-GAN network might do some false modification, such as treating pseudo-hyperbolas as multiple-waves or falsely eliminating the multiple-waves of targets. Although the precision rate declines, the increment of F1-score and mAP values encourages that the detection performance reaches a balance where both the Precision and Recall values are high.

The second row of Figure 10 summarizes the detection performance on testing dataset II with different rebar-void distances. When the voids are close to the rebars, the detection performance on the original images is worse because the target echoes are masked by the pseudo-hyperbolas. After applying the RCE-GAN, the detection performance improves obviously, for the recall rate, F1-score, and mAP values increase in each condition. The precision rate drops slightly after rebar clutter elimination processing for the same reason explained previously.

From the results of the synthetic examples, we can conclude that the RCE-GAN is capable of improving the detection performance effectively. Compared to detecting void echoes on original GPR images, the proposed method improves F1-score by an average of 16.9% and mAP value of 11.9%. For the situations of dense rebars, in which it is almost impossible for the detection network to recognize the targets on the original images, the F1 value is improved from 0.289 to 0.793, and the mAP value from 0.321 to 0.775. After being processed by our RCE-GAN, the rebar clutters are eliminated and the target echoes are highlighted, which improves the detection performance tremendously.

5. Real-World Application

In the previous section, we have demonstrated the ability of our method to eliminate rebar clutters and improve void identification using FDTD synthesized GPR data. However, the real-world GPR data, which are subject to noise, are different from synthetic data. The target echoes are more complex and difficult to recognize. Thus, the verification of our method on real-world data is necessary. In this section, we demonstrate the effectiveness of our method using the GPR data acquired from a physical model experiment. Firstly, we describe the acquisition of the GPR data. Then, the GPR data are processed and put into our network for rebar clutter removal and void detection. Finally, an analysis of the experimental results is delivered.

5.1. Data Acquisition

5.1.1. Physical Model Setup

According to the general design of the road tunnel, we build a tunnel lining model. As shown in Figure 11a, there are three layers to simulate tunnel lining structures. From bottom to top, the first layer is sand to simulate the surrounding rock behind the tunnel lining; the second layer is plain concrete to simulate the initial lining made of shotcrete, the thickness of which is 0.25 m; the third layer is made of 0.6 m thick reinforced concrete to simulate the secondary lining. Inside the secondary lining, there are two rows of equally spaced rebars with a diameter of 22 mm and spacing of 33 cm. Inside this layer, there are four voids which are denoted as V1, V2, V3, and V4. As depicted in Figure 11a, V1 and V2 are cubic voids with side lengths of 0.1 and 0.2 m, respectively, placed inside the secondary lining, while V3 and V4 are cubic voids with side lengths of 0.1 and 0.2 m, respectively, located in the secondary lining with their rear surfaces next to the interface between the secondary and initial linings. The burial depth of the four voids is 0.3, 0.36, 0.47, and 0.37 m, respectively. Figure 11b shows the GPR survey line along the lining surface.

5.1.2. Data Acquisition and Processing

We use the pulse EKKO PRO GPR system equipped with a 500 MHz center frequency antenna to obtain GPR data. Figure 12a presents the original GPR image, in which the defect echoes are hard to discern. Prior to being put into the RCE-GAN for rebar clutter elimination, the obtained GPR image is put through data processing procedures to suppress noise and highlight target echoes. We perform a series of data processing operations, including time-zero correction, direct-current(DC)-shift removal, time-varying gain, 1D frequency filtering, and 2D spatial filtering. First, the starting time of the GPR data is adjusted to the air–concrete surface. Then, the DC component is removed by subtracting the mean value of each trace from the GPR data. Next, the time-varying gain is performed with the exponential function to compensate for energy attenuation. After that, a 1D band-pass frequency filtering with lower and upper cut-off frequencies of 200 and 650 MHz is applied to suppress noises. Finally, a $3\times 3$ 2D spatial filtering is carried out to smooth the GPR image. Figure 12b displays the processed GPR image. In order to get ideal results for the actual GPR data, it is advisable to match the center frequency between the training dataset of RCE-GAN and the actual GPR data. We obtain 400 GPR images containing both defect echoes and rebar clutters (Domain A), and 400 GPR images that contain only defect echoes (Domain B) as the training dataset for RCE-GAN. The center frequency for all radar images is 500 MHz.

5.2. Experimental Results

In Figure 12b, it can be seen that the target echoes are almost invisible due to the shielding effect of the rebars. There are four voids in total but only one hyperbola can be visually detected in the processed GPR image. Figure 12c shows the recognition result on the processed GPR image using the YOLOv4, in which two out of four voids are detected. As depicted in Figure 12d, after being processed by the RCE-GAN, the rebar reflections are suppressed largely, and the void echoes are clearly visible. Now, as shown in Figure 12e, the YOLOv4 detects all the voids in the rebar clutter eliminated GPR image. The processing time for a single GPR image is around 0.7 to 0.9 s, which could fulfill the time demand in most cases although real-time processing is not reached.

It can be seen in the image generated by our network, the rebar reflections are eliminated and target echoes are clear enough even their reflections are almost invisible in the original GPR image. The detection results reveal that after processing the GPR image using our model, the detection performance is greatly improved. It is rather remarkable that, during the training process of our network, all of the training data are FDTD simulated data, which means that our network can be generalized to real GPR data. The physical model experiment confirms the feasibility of our method on real-world GPR detection. It should also be noted that this experiment is a preliminary verification of the effectiveness of our method for a streamlined scenario. In-depth investigations of this method using different voids shapes and burial depths will be reported in due course.

6. Conclusions

In this research, we have developed a GAN-based rebar clutter elimination network coined the RCE-GAN. The novel contribution of the proposed method is that the RCE-GAN improves tunnel lining void detection by eliminating rebar clutters in GPR images. At the same time, our network does not need a paired GPR data as a training set, which enhances the feasibility of applying the RCE-GAN network to real-world applications. The detection results confirmed that the proposed method improves the void detection performance from the GPR images that are disrupted by rebar interference. Compared to detecting void echoes on original GPR images, the proposed method improves F1-score by an average of 16.9% and mAP value of 11.9%. In some extreme cases, such as closely spaced rebar, the F1-score surges from 28.9% to 78.9% and mAP value from 0.321 to 0.775. The validation in a real-world experiment demonstrated that our method performs well on actual GPR data obtained in a controlled physical experiment.

Future directions of this research include: (1) to expand the ability of our network for detecting more types of defects; (2) to create more reliable training dataset using labelled real-world data or non-homogeneous FDTD models to improve the network performance; and (3) to apply our method to real-world tunnel applications and realize real-time processing of GPR images.