Efficient Training Data Acquisition Technique for Deep Learning Networks in Radar Applications

Article information

J. Electromagn. Eng. Sci. 2024;24(5):451-457
Publication date (electronic) : 2024 September 30
doi : https://doi.org/10.26866/jees.2024.5.r.246
Radar R&D Center, Hanwha Systems Co. Ltd., Yongin, Korea
*Corresponding Author: Young-Jae Choi (e-mail: radar.yjchoi@hanwha.com)
Received 2023 April 11; Revised 2023 September 16; Accepted 2024 February 2.

Abstract

In the field of radar, deep learning techniques have shown considerably superior performance over traditional classifiers in detecting and classifying targets. However, acquiring sufficient training data for deep learning applications is often challenging and time consuming. In this study, we propose a technique for acquiring training data efficiently using a combination of synthesized data and measured background data. We utilized graphics processing unit (GPU)-based physical optics methods to obtain the backscattered field of moving targets. We then generated a virtual dataset by mixing the synthesized target signal with the background signal real data. Subsequently, we trained a convolutional neural network using the virtual dataset to identify three different classes—Bird, Drone, and Background—from a range-Doppler map. When tested using the measurement data, the trained model achieved an accuracy of over 90%, demonstrating the effectiveness of the proposed method in acquiring training data for radar-based deep learning applications.

I. Introduction

Deep learning techniques are utilized for several applications in the field of radar. For instance, it can help improve the detection range of targets and recognize human or hand gestures from micro-Doppler signature images [1, 2]. While well-trained deep learning networks achieve dramatically better performance than traditional classifiers, they require a significantly larger amount of data contains the label [3]. Acquiring training data for deep learning applications typically involves measuring input data and precisely labeling them, which is both costly and time consuming. This process becomes even more complicated when considering military targets, such as fighter planes or missiles, as it might be difficult or impractical to obtain accurate training data through measurement in such cases. To overcome this challenge, training data can be acquired by obtaining a backscattered field for moving targets. Numerical methods, such as physical optics (PO) and finite-difference time domain, can be obtain the backscattered field from the 3D shapes of targets [4, 5]. However, these methods require long computation times and are mainly used for stationary targets. In the last decade, several methods have been proposed to accelerate the calculation speed of PO through the implementation of parallel processing techniques based on the general purpose computing of graphics processing units (GPUs) [5, 6]. A method for calculating the backscattered field of moving 3D targets using GPU-based PO has also been proposed [7]. The method proposed in [7] can calculate the backscattered field, including the radar cross-section, range, velocity, Doppler, and micro-Doppler components of a target in motion and micro-motion in a 3D space.

In this study, we propose a method for acquiring training data for deep learning networks. We first acquired a virtual dataset based on the range-Doppler map (RDMAP) by mixing the target signal with the background signal measured by the radar using the method proposed in [7]. Subsequently, we trained a convolutional neural network (CNN) to classify the target and the background from the RDMAP using the virtual dataset, and then validated it using measurement data. The validation results showed that the CNN trained using virtual signals achieved an accuracy of over 90% when performing target classification on the measurement data.

II. Proposed Method

1. Overview

The geometry pertaining to the radar and the target can be represented using a 3D coordinate system, as shown in Fig. 1.

Fig. 1

Geometric relationship between the radar and the target.

In Fig. 1, the radar is assumed to be located at origin O⃗ and directed toward direction while using a stepped-frequency waveform. The target is located at position R⃗ in a 3D coordinate system that satisfies the conditions y > 0 and z > 0. It exhibits both bulk motion, which refers to the movement of the entire target over time, and micro motion, which involves the movement of specific structures within the target, such as propellers. Fig. 2 provides an overview of the proposed method using a block diagram.

Fig. 2

Block diagram of the proposed method.

In accordance with Fig. 2, the “Generate 3D Scene” block first creates a 3D mesh that captures the micro motions of the specific structures within the target. Next, the “Generate Field” block calculates the backscattered field of the 3D mesh and stores it in Target Storage. The “Synthesize” block generates a dataset similar to the measurements by blending the target and background data obtained from the storage. Following this, the backscattered field for a single radar pulse is obtained by simultaneously conducting the Generate 3D Scene and Generate Field steps. This process is repeated for a desired number of pulses, after which the results are stored in Target Storage. During this process, the time interval of the Generate 3D Scene is set to be the same as that of the pulse repetition interval (PRI) of the radar pulse.

2. Generate 3D Scene Block

The Generate 3D Scene block generates a 3D mesh representing the shape and posture of the target, with the target considered the sum of its N triangle patches in the mesh.

(1) Vobject=[Pm]3×3M,

where M denotes the total number of triangular patches in the 3D mesh, while P1,m3,P2,m3, and P3,m3 are the position vectors representing the vertices of the triangular patches. Therefore, if Vobject is a rigid body, all its positions and orientations can be expressed by repeatedly applying Eqs. (2) and (3) to its vertices.

(2) Vobject=RrotateVobject,
(3) Pn,m=Pn,m+Ptranslate,n=1,2,3,

where Rrotate ∈ ℝ3×3 is the Euler rotation matrix and Ptranslate3 is the position vector used to translate Vobject. However, targets with complex movements, such as drones, cannot be represented using simple rotational and translational transformations because the entire shape changes as the propellers rotate. Therefore, assuming that the target can be represented as a combination of rigid bodies, all its possible poses can be expressed by applying rotational and translational transformations to the 3D mesh of each component that constitutes the target and then assembling them. Taking this into account, Fig. 3 shows an internal block diagram of the Generate 3D Scene block.

Fig. 3

Block diagram of the Generate 3D Scene block.

According to Fig. 3, each element of the target is loaded into a separate coordinate system, after which different rotational and translational transformations are applied. Following this, the elements are assembled into a single coordinate system. Subsequently, rotational transformations are applied to the assembled 3D target to determine the pose angles. Figs. 4 and 5 show examples of the 3D meshes obtained using the proposed method.

Fig. 4

Examples of 3D mesh (Drone).

Fig. 5

Examples of 3D mesh (Bird).

In Figs. 4 and 5, the targets represented in the 3D coordinate system not only show the orientation angles but also exhibit movements, such as blade rotation or wing flapping. Therefore, by applying the above method, the state of the target at the moment of radar pulse emission can be represented in a 3D mesh by varying the time of the target according to the PRI.

3. Generate Field Block

Fig. 6 shows a block diagram of the Generate Field block.

Fig. 6

Block diagram of the Generate Field block.

In Fig. 6, the Generate Field block uses the GPU-based PO method proposed in [7] to calculate the backscattered field of the target from the 3D mesh.

Fig. 7 compares the computation results obtained using FEKO—a commercial electromagnetic numerical analysis software—and those acquired on implementing the PO characterized by GPU and interpolation. The simulation was performed using the following hardware: CPU, Intel Core i7-10750H CPU @2.60 GHz, 64 GB RAM, and NVIDIA GeForce RTX 2070 Super with Max-Q.

Fig. 7

Comparison of RCS calculation results: (a) frequency sweep and (b) angle sweep.

We measured the computation time of the GPU-based PO in the same environment. It was found that the computation time required by the GPU-based PO to calculate 2,048 frequency samples for a target containing 10,782 triangular patches was approximately 1.54 seconds. Although this is substantially faster than the conventional PO method, it is too slow to acquire data on moving targets. For example, it takes approximately 197 seconds to simulate one RDMAP using 128 samples along the Doppler axis and 2,048 samples along the range axis. Assuming that a radar updates 4 RDMAPs per second, it would take about 6 days (132 hours) to generate 2,400 frames of RDMAPs for a duration of 10 minutes.

We addressed this problem by employing range compensation and linear interpolation. In the range profile, targets situated at a closer range are represented by lower-frequency components in the frequency domain, whereas targets situated at a farther range are represented by higher-frequency components. Therefore, by aligning the target’s position at the origin and conducting scattered field analysis using PO, we can fully reconstruct the target’s range profile using fewer calculated samples. In other words, the number of samples Q′ used for an analysis would be significantly larger (by approximately 10 times or more) than the number of cells that represent the target’s length in the range profile.

Therefore, the following methods were implemented:

  • Step 1. The starting point of the target’s 3D mesh at origin was aligned and the Q′ scattered fields were calculated.

  • Step 2. Linear interpolation was applied to the scattering fields to generate the same number of samples Q as that of the radar received signals.

  • Step 3. Range compensation was applied to shift the target’s position in the range profile back to its original range.

This methodology can also be expressed in mathematical terms. For instance, the backscattered field E can be formulated as Eq. (4).

(4) E=[sq]Q,

where Q′ is the number of frequency samples calculated by the PO, q′ refers to the index for Q′, and sq is the q′-th element of vector E.

In Step 1, since the target location is aligned at the origin, an approximate estimation of frequency samples can be obtained by acquiring fewer frequency samples and then interpolating them. In this context, Q′ should satisfy the following condition:

(5) Q>α2Bcrmax,

where B is the bandwidth of the radar, c refers to the velocity of light in vacuum, and rmax indicates the maximum range that can be represented in the range profile calculated by the PO. Furthermore, the coefficient for oversampling is denoted as α, and the value α = 10 is used.

In Step 2, the interpolated field E can be obtained using the following equation:

(6) E=[sq]=interp(E)Q,

where Q is the number of frequency samples acquired from the radar, q refers to the index for Q, sq signifies the q-th element of vector E, and interp(·) is an interpolation operation. Notably, a linear interpolation method was employed in this study.

In Step 3, the backscattered field of the moving target is obtained by substituting the position vector of the target into Eq. (7).

(7) E=[sq]=[sqexp{-j4πfqR-O2c}]CQ,

where O⃗ and &Rrarr; are the position vectors of the radar and the target, respectively, and fq indicates the frequency value of the q-th frequency sample.

Table 1 compares the computation time required to calculate 2,048 frequency samples for a target containing 10,782 triangular patches using the two methods explained above.

Comparison of calculation time (Q’ = 20, Q = 2,048)

The results show a significant improvement in the calculation speed of the GPU when using interpolation, exhibiting approximately 100 times faster performance compared to the GPU-only case.

Fig. 8 demonstrates the results obtained by simulating the Drone and Bird targets, each flying at 20 m/s, using the proposed method to acquire spectrograms displaying the velocity of the targets. For the Drone target, the presence of micro-Doppler signatures generated by the blade is revealed.

Fig. 8

Examples of range-Doppler maps for (a) the Drone and (b) Bird targets.

4. Synthesize Block

The signal obtained from radar measurements can be considered the sum of the target and background signals. Notably, the background signal, which includes clutter, is challenging to obtain using numerical methods, but can be readily acquired through measurement. To obtain a signal similar to that resulting from measuring the target in the presence of background clutter, a strategy that involves mixing the background and target signals obtained through measurement and numerical methods, respectively, was employed.

In accordance with the steps in Fig. 9, the backscattered field of the target was obtained using a numerical method, whereas the backscattered field of the background was acquired through measurement. Blending was performed by adding the two signals. Fig. 10 presents a comparison of the synthesized data and the measured data for the Drone target. In this context, it should be noted that the classification performance of a deep learning model may vary based on the method used to normalize the training and test data. In this paper, the RDMAP data included in the presented dataset were normalized to pixel values ranging from 0 to 255, with the maximum value set to 0 dB and values extending down to −15 dB.

Fig. 9

Block diagram of the synthesize block.

Fig. 10

Comparison of the synthetic and measured range-Doppler maps (the horizontal axis represents the Doppler and the vertical axis represents the range).

As shown in Fig. 10, the RDMAPs of the synthesized and measured data are highly similar to the extent that identifying any discernible differences with the naked eye is challenging.

III. Experimental Result

The goal of the proposed method is to generate training data for deep learning networks in radar applications. To verify the method’s performance, two items must be checked: the computation time required to generate the training data and the accuracy of the output provided by the deep learning network trained using the generated data. The structure of the CNN employed in this experiment is illustrated in Fig. 11.

Fig. 11

Structure of the CNN used in the experiment.

Fig. 11 considers a 21 × 21 cropped segment of the RDMAP, showing the process of determining whether a given image represents the background or a target. Fig. 12 illustrates an instance of the RDMAP data employed for training.

Fig. 12

Examples of the data employed for (a) training and (b) testing.

In Fig. 12, the training dataset comprises 10,000 frames of the background measurement data and 10,000 frames of the synthesized target data, whereas the test dataset consists of 10,000 frames of background and target measurement data that differ from the training data. Notably, the measurement data were obtained using an X-band radar. Table 2 presents the specifications of the X-band radar used in the experiment.

Specifications of the X-band radar used in the experiments

The test accuracy of the trained model was 95.57%. Therefore, the first experiment confirmed that a deep learning model trained using virtual data can successfully classify measurement data. In the second experiment, a CNN was trained using the proposed method to identify three classes—Bird, Drone, and Background—from the RDMAP. Fig. 13 shows an example of the RDMAP data used for training.

Fig. 13

Examples of the data used in the second experiment: (a) Bird, (b) Drone, and (c) Background.

As shown in Fig. 13, we used 5,000 frames per class for both the training and testing data. Measured data were used for the background and synthesized data were considered for the Bird and Drone classes. The trained model achieved a test accuracy of 95.13%. Table 3 presents the confusion matrix of the second experiment.

Confusion matrix of the second experiment (unit: %)

The deep learning network correctly identified the background, achieving an accuracy of 100%. Meanwhile, the accuracy in distinguishing between Drones and Birds was approximately 89.06% and 96.34%, respectively, indicating a slightly higher confusion rate. To compare these results with those of the previous experiment, we trained a deep learning model using a dataset without synthesizing the background based on measurements. We then performed tests using the same test data. The results showed that the model classified all test data as background, resulting in a test accuracy of 33%. This indicates that the classifier trained on unsynthesized background data failed to classify the test data correctly.

IV. Application and Conclusion

In conclusion, this study proposes an effective method for generating datasets necessary for training deep learning networks used in military radar applications. The advantages offered by the proposed method are as follows:

  • It helps obtain radar received data for targets with limited measurement opportunities (e.g., fighter planes, missiles, etc.) through simulation.

  • It can simulate the micro-Doppler signatures of targets.

  • It allows for the flexible simulation of target positions, velocities, and micro-movements.

  • It enables the calculation of the scattering field when applying electromagnetic numerical analysis methods to the 3D model of the target, achieving a high degree of similarity with the real measurement data.

The proposed method offers a means to obtain complex and raw radar signal data while also allowing for the acquisition of various forms of data, including 2D images, through radar signal processing. Fig. 14 provides examples of the 2D images of Drone and Bird targets presented in Fig. 5, obtained using the proposed method.

Fig. 14

Examples of 2D images of (a) Drone and (b) Bird targets obtained.

Furthermore, by combining the received signals from moving targets with signals for the background acquired through measurements using GPU-based PO, the proposed approach enables the training of deep learning networks for classifying radar targets characterized by different micro-motions. The potential benefits of this method include the ability to obtain training data for difficult-to-operate targets and the continuous acquisition of background data during radar operation, allowing for continuous updating of deep learning networks. Overall, the proposed method offers a promising solution for training deep learning networks used in radar applications.

Acknowledgments

This research was supported by the Challengeable Future Defense Technology Research and Development Program (No. 912770601) of Agency for Defense Development in 2022.

References

1. Gusland D., Rolfsjord S., Torvik B.. Deep temporal detection: a machine learning approach to multiple-dwell target detection. In : Proceedings of 2020 IEEE International Radar Conference (RADAR). Washington, DC, USA. p. 203–207. 2020; https://doi.org/10.1109/RADAR42522.2020.9114828.
2. Kim Y., Toomajian B.. Hand gesture recognition using micro-Doppler signatures with convolutional neural network. IEEE Access 4:7125–7130. 2016; https://doi.org/10.1109/ACCESS.2016.2617282.
3. Park S., Lee S., Kwak N.. Range-Doppler map augmentation by generative adversarial network for deep UAV classification. In : Proceedings of 2022 IEEE Radar Conference (RadarConf22). New York City, NY, USA; 2022; p. 1–7. https://doi.org/10.1109/RadarConf2248738.2022.9764177.
4. Zhang B., Xue Z., Ren W., Li W., Sheng X.. Research on GPU Cluster acceleration of FDTD algorithm. In : Proceedings of 2012 International Conference on Microwave and Millimeter Wave Technology (ICMMT). Shenzhen, China; 2012; p. 1–4. https://doi.org/10.1109/ICMMT.2012.6229926.
5. Kee C. Y., Wang C. F.. Efficient GPU implementation of the high-frequency SBR-PO method. IEEE Antennas and Wireless Propagation Letters 12:941–944. 2013; https://doi.org/10.1109/LAWP.2013.2274802.
6. Kee C. Y., Wang C. F.. High-frequency PO-SBR code on GPU. In : Proceedings of 2013 IEEE Antennas and Propagation Society International Symposium (APSURSI). Orlando, FL, USA; 2013; p. 1890–1891. https://doi.org/10.1109/APS.2013.6711603.
7. Choi Y. J., Choi I. S.. Dynamic RCS calculation of wind turbine blade using GPU-based TSM-RT. The Journal of Korean Institute of Electromagnetic Engineering and Science 31(3):245–252. 2020; https://doi.org/10.5515/KJKIEES.2020.31.3.245.

Biography

Young-Jae Choi, https://orcid.org/0000-0002-8632-5265 received his B.S. and M.S. degrees in Electronics Engineering from Hannam University, Daejeon, Korea, in 2013 and 2018, respectively. He completed his Ph.D. degree in Electronics Engineering at Hannam University in 2021. Since 2021, he has been working as a Senior Engineer at the Advanced R&D Center of Hanwha Systems, Yongin-City, South Korea. His current research interests in the development of radar signal processing software. His research interests include radar signal processing, AI algorithms for radar, and RCS simulation and analysis.

Woojin Cho, https://orcid.org/0009-0006-6035-2907 received as a B.S. degree from the Department of Computer Information Engineering from Kwangwoon University, Seoul, Republic of Korea, in 2022. Since 2022, he has been working as a Junior Engineer at the Advanced R&D Center of Hanwha Systems, Yongin-City, South Korea. His current research interests include AI radar and AI model compression.

Seungeui Lee, https://orcid.org/0000-0003-3560-785X received a B.S. degree in Electrical Engineering from the Korea University, Seoul, Republic of Korea, in 2006, and a M.S. degree in Electrical Engineering from the Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea, in 2009. He is a Chief Engineer at the Advanced R&D Center, Hanwha Systems, Yongin-City, South Korea. His current research interests include radar system design and radar signal processing with deep learning.

Article information Continued

Fig. 1

Geometric relationship between the radar and the target.

Fig. 2

Block diagram of the proposed method.

Fig. 3

Block diagram of the Generate 3D Scene block.

Fig. 4

Examples of 3D mesh (Drone).

Fig. 5

Examples of 3D mesh (Bird).

Fig. 6

Block diagram of the Generate Field block.

Fig. 7

Comparison of RCS calculation results: (a) frequency sweep and (b) angle sweep.

Fig. 8

Examples of range-Doppler maps for (a) the Drone and (b) Bird targets.

Fig. 9

Block diagram of the synthesize block.

Fig. 10

Comparison of the synthetic and measured range-Doppler maps (the horizontal axis represents the Doppler and the vertical axis represents the range).

Fig. 11

Structure of the CNN used in the experiment.

Fig. 12

Examples of the data employed for (a) training and (b) testing.

Fig. 13

Examples of the data used in the second experiment: (a) Bird, (b) Drone, and (c) Background.

Fig. 14

Examples of 2D images of (a) Drone and (b) Bird targets obtained.

Table 1

Comparison of calculation time (Q’ = 20, Q = 2,048)

Time (s)
GPU-only 1.54
GPU with interpolation 0.015

Table 2

Specifications of the X-band radar used in the experiments

Parameter Specification
Operating frequency X-band
RF bandwidth 86 MHz for 3.6 km instrumented range
Electronic scan rate 4 Hz
Number of range samples 2,048
Number of Doppler samples 128

Table 3

Confusion matrix of the second experiment (unit: %)

Bird Drone Background
Bird 89.06 2.84 0
Drone 10.24 96.34 0
Background 0.70 0.82 100