I. Introduction
Inverse synthetic aperture radar (ISAR) plays an important role in space target observation and recognition by providing two-dimensional images of targets in all kinds of weather, as well as in day and night conditions [1]. ISAR image registration enables the analysis of target performance and behavior. By comparing the registered images from different views, researchers can monitor changes in the target’s structure, track its movements, and study its dynamics. This information is valuable for understanding the target’s capabilities, mission objectives, and operational patterns.
There are many methods for optical image registration and synthetic aperture radar image registration, such as normalized cross-correlation (NCC) [2], scale invariant feature transform (SIFT) [3], speeded up robust feature (SURF) [4], and a plethora of emerging deep learning methods [5]. However, it is difficult to migrate these methods directly into the ISAR field due to the different imaging mechanisms and imaging scenarios involved. The main differences can be summarized as follows:
· Limited information in ISAR images: Unlike optical images characterized by rich texture and gradient information and SAR images with richly detailed scenes, ISAR images usually involve several scatterers on a manmade target. In ISAR images, the most significant information is typically related to the brightness or reflectivity of the target, which is represented by the amplitude of the scatterers in an image.
· Low signal-to-noise ratio (SNR) in ISAR images: The complex electromagnetic scattering characteristics of the target, radar system errors, and translation compensation errors involved in practical imaging degrade image quality and reduce image SNR.
Research on ISAR image registration has mainly focused on the processing of three domains: signal, frequency, and image. Kang et al. [6] proposed a method for ISAR image registration in the signal domain, using raw echo data from different receive antennas to estimate the time-varying angular motion and compensate for the jointly translational motion to ultimately align the range and cross-range directions. Tang et al. [7] used the Fourier-Merlin transform to carry out phase conjugate multiplication in the frequency domain based on the Fourier theorem to achieve ISAR image registration. Furthermore, Gao et al. [8] utilized the correlation coefficient as a similarity measure to estimate alignment parameters. Meanwhile, Adel et al. [9] combined SIFT [10] and SURF [11], based on point features, to achieve image registration. However, in these methods, some meaningless noise points are often extracted as feature points, resulting in unstable performance. Since most space targets feature linear structures (e.g., solar panels and rectangular cavities), line features can be used to solve this problem, since they are more reliable and are less affected by the glinting or defocusing caused by scatterers.
There are many ways in which lines can be detected. The Hough transform (HT) [12] is a popular line detection technique that converts image space into parameter space, where lines are represented as points. However, it is computationally expensive and sensitive to parameter settings. Almazan et al. [13] proposed a probabilistic algorithm to detect lines and thereby improve the sensitivity of parameters, but it required large labeled datasets. Line segment detection (LSD) [14] is another commonly used method for line detection that is based on gradient information that does not rely on HT. Line segment merging (LSM) [15] is an upgraded version of the LSD that applies additional post-processing steps to merge overlapping line segments or remove redundant ones. Notably, line detection is often preceded by contour extraction, which helps segment foreground objects from the background, allowing for more accurate and focused line detection. However, the discontinuity and defocusing caused by scatterers in ISAR images may lead to the extraction of only a coarse contour, which can in turn degrade the performance of the line detection algorithm and lead to large errors in the registration results.
This study proposes a line feature-based ISAR image registration method that exhibits enhanced robustness against image degradation caused by noise, secondary reflections, and system errors. The proposed registration process involves line feature detection, feature matching, and parameter estimation. In particular, feature matching is accomplished by utilizing the spatial relations of line features between the reference image and the sensed image. Furthermore, the registration parameters are estimated by minimizing the sum of the Euclidean distances between the line features. Additionally, the accuracy of line detection in the LSM is improved by employing relevant techniques, such as image enhancement, contour extraction, and scatterer fitting. Moreover, the efficiency of parameter estimation is optimized by implementing the Snake Optimization (SO) [16] algorithm, which dynamically adjusts its search strategy and parameters, demonstrating solid capabilities in global exploration, robustness, and adaptability.
The remainder of this paper is organized as follows: Section II analyses the transform model for ISAR images from different perspectives, Section III introduces the proposed line feature detection and matching method, Section IV demonstrates the parameter estimation process of the ISAR image registration method, Section V provides the different experimental results to validate the efficacy of the proposed method, and Section VI presents the conclusions of this study.
II. Registration Model of ISAR Image Sequences
This section of the study uses a monostatic scenario as a reference point to analyze the relationship between the different viewpoints of ISAR images.
The ISAR imaging geometry is depicted in Fig. 1, where TUVW represents the inertial coordinate system and TXkYkZk is the imaging coordinate system at the k-th moment. Furthermore, Xk and Yk represent the Doppler and range axes of imaging, respectively. Zk is the normal vector of the imaging plane and elos stands for the unit vector from the radar to the target. Notably, the relative motion of the targets with respect to the radar can be divided into translation and rotation. Translation must be compensated for before azimuth compression [17], while rotation is usually retained for ISAR imaging. Assuming ωlos represents the line of sight (LOS) rotation and ωtar refers to the target self-rotation, the effective rotation vector can be formulated as:
Furthermore, assuming translation compensation [18] is complete, the target motion can be considered equivalent to an ideal turntable model at a coherent processing time, as shown in Fig. 2.
On considering the coordinates of scatterer Q on the target at the initial time as (x0, y0) and the range of the target center as ROk, the instantaneous distance of Q from the radar can be approximated as:
Here, c denotes the speed of light, B is the bandwidth, ω refers to the effective target speed, and the coherent processing time is tobs. Furthermore, the range resolution is ηr = c/2B and the azimuth resolution is ηa = λ/2ωtobs. According to the range-Doppler image algorithm, the position of Q relative to the center of the ISAR image can be obtained as follows:
Substituting Eq. (2) into Eqs. (3) and (4), the coordinates of Q in the image can be rewritten in the form of the following matrix:
Furthermore, on considering the image I1 corresponding to t1 as the sensed image, and the image I2 corresponding to t2 as the reference image, the position relationship of scatterer Q in these two images can be expressed as:
where γ = ω(t2 − t1).
However, offsets between images might still remain because of the errors introduced during the delay and the compensation process [19]. Therefore, supposing that the relative offset vector of the centers is p = [a, b], the transformation matrix can be expressed as:
The relationship between the same lines in the two images can be obtained in the same way. In this context, it is worth noting that, due to target scattering properties, noise, and low image quality, it is difficult to accurately determine the length of the lines. To address this, the direction angle and the vertical distance from the origin of the image to the line were utilized to describe a line, namely L = [ρ, θ], as shown in Fig. 2. Therefore, the position vector, which indicates the vertical distance vector from Ok to line L, can be formulated as:
Notably, the direction vector of line L is:
Therefore, the relationship between line L in the two images can be expressed as follows:
where r1 and n1 represent the position vector and direction vector in image I1, respectively, while r2 and n2 denote the position and direction vectors in image I2.
III. Line Feature Detection and Matching
ISAR images are collections of points, with the edge of the images being discontinuous. As a result, traditional automatic feature extraction methods present considerable challenges for ISAR images. In this section, the LSM algorithm is improved upon to extract line features from ISAR images, thus aiding in the subsequent image registration process.
1. Image Preprocessing
Preprocessing, such as image enhancement and edge detection, is often necessary to improve the robustness of line detection.
Considering that ISAR images are characterized by the presence of several strong scatterers, the weak scatterers in the images may become less distinguishable. To address this issue, the contrast of the ISAR images used in this study was adjusted to enhance the visibility of weak scatterers, thereby preserving more target details. In addition, morphological image processing methods [20, 21] were implemented to connect adjacent points and obtain a complete contour. Finally, the outer contour [22] was extracted utilizing a typical edge detection operator, Canny [23].
Although the outer contour exhibited gaps or deviations from the natural target boundary, possibly resulting from the alterations caused in the target’s area during the dilatation process, it was still able to offer valuable indications regarding the spatial position of the lines.
2. Line Detection
LSM is well known for its efficiency and accuracy. The LSM process primarily comprises scale-space analysis, gradient calculation, line support region estimation, line validation, line pruning, and line mergence. Notably, the possibility of LSM will be further investigated in the experimental section of this study. In this paper, the outer contour image was used as the input for the LSM to extract coarse line features L = [ρ, θ].
In this context, it should be noted that during the course of this process, the outer contour errors were transmitted into the line errors. For example, the LSM result presented in Fig. 3 exhibits a gap between the LSM results and the natural edge of the target. To address this problem, the CLEAN technique [24] was employed to pick up dominant scatterers from the entire image and filter the scatterers located close to L = [ρ, θ]. Following this, the discrete scatterers were fitted to improve the accuracy of the LSM results. The following is a detailed description of the process adopted in this study for filtering scatterers and calculating line parameters.
Assuming Pj(xj,yj) is the j-th dominant scatterer of the target and Lh = [ρh, θh] is the h-th line feature, the distance between Pj and Lh can be estimated using the following equation:
To calculate the distance between scatterer Pj and the other lines, and obtain an H-dimensional vector, dj = [dj,1, dj,2, … , dj,H], , where H denotes the number of lines. Subsequently, the association of Pj with a specific line can be determined based on an appropriate threshold ɛd = σ|η1 − η2|, where η1 and η2 are the dilation parameters involved in image preprocessing. Notably, σ can adjust the threshold value to affect the sensitivity of the association. If min (dj) ≤ ɛd, Pj is considered to belong to the associated line, or it is excluded otherwise. Notably, the index of min(dj) represents the line located closest to Pj.
After classification, the dominant scatterers in the same group can be linearly fitted. Assuming there are W points in the h-th group, with each point e represented as (xw, yw), the corresponding coefficient matrix after fitting will be:
Subsequently, the fitted line Lh = [ρ̃h, θ̃h] can be obtained. Furthermore, the parameters ρ̃h and θ̃h can be formulated as:
3. Line Feature Matching
In image registration, establishing a corresponding relationship between line features is a primary task. For this purpose, the current study implemented a novel feature matching strategy to identify corresponding lines between two images based on geometric constraints. The initial stage of this feature matching process involved grouping based on angle information, while subsequent stages focused on matching the lines within the same group by ensuring consistency in the distance between them. The details of this process are as follows:
Step I: Categorize the lines with similar orientations into groups based on their angle parameters θ. For instance, assume that I1 (the sensed image) and I2 (the reference image) are divided into M and N groups.
Step II: Calculate the average angle of each group to attain θ̄m, where m ∈ [1,2, … , M].
Step III: Calculate the angle difference Δθ̄m,n = θ̄m − θ̄m| between each group of I1 and I2. Store Δθ̄m,n in a M×N matrix Gθ.
Step IV: Analyze the statistical properties of Gθ and identify the correct match with the highest vote rate by applying a clustering algorithm. Then identify the approximate rotation parameter γ0.
Step V: Rotate I1 using γ0 as Ĩ1.
Step VI: Determine the intersection point of any two lines in the same image. Assuming qi,j is the intersection point of Li and Lj, the coordinates (Xqi,j, Yi,j) of qi,j can be calculated as follows:
Remove any intersection points falling beyond the range of image I1 and store the remaining intersection points in the set Q1. Similarly, generate Q2 for image I2.
Step VII: Choose one point from each point set (Q1 and Q2) to calculate the distance vector formed by the two points. Record the distance vectors to create a matrix Gq while iterating through all the intersection points.
Step VIII: Perform cluster analysis on the distance vectors in Gq to identify the most frequent occurrences, ultimately completing the line matching.
After correctly matching the intersection points, the average distance vector of the intersection point pairs can be approximated as the translation vector p0 = [a0, b0] between two images.
The key steps of the feature matching process are presented in the flowchart depicted in Fig. 4.
IV. Parameter Estimation based on Line Features
This section presents the results achieved using the SO algorithm to estimate the image registration parameters. Taking γ0 as the initial rotation angle and p0 = [a0, b0] as the initial translation vector, the angle, range, and cross-range offset can be defined as Δγ, Δa;, and Δb. Therefore, the refined values can be considered as follows:
Furthermore, by substituting Eq. (17) into Eq. (10), the transformed line feature of Lh can be formulated as:
where θ̂h,1 represent the angle of the h-th line in I1 after transformation and ρ̂h,1 denotes the distance from the origin to the h-th line. Consequently, the new position vector r̂h,1 of h-th line in I1 can be obtained.
The cost function is crucial for an optimization algorithm that guides optimization processes. In this study, the sum of the Euclidean distance between the position vectors of the lines was utilized as the cost function to measure the degree of image alignment. Therefore, assuming there are Hm pairwise line features, the cost function can be expressed as follows:
Notably, a minimum value of the cost function indicates that the two concerned images are well aligned. To improve efficiency, this study used SO to calculate the optimal solution for γ, a;, and b.
V. Experiments
To validate the superiority of the proposed method, different experiments were performed in three parts. The first part is concerned with proving the effectiveness of the proposed ISAR image registration method for simulated ISAR images, while the second part presents comparisons of the accuracy of different feature detection, optimization, and registration methods for ISAR applications using real measured images. In the last part, the robustness of the proposed method is analyzed through error analysis.
1. Effectiveness Validation
To validate the effectiveness of the proposed method, a number of experiments were carried out on simulated ISAR images, with the space target being RADARSAT-2, whose three-dimensional model is available in [25], as depicted in Fig. 5. The radar was set at Beijing (39.9 N, 116.4 E, 0 m), and the satellite two-line orbital element parameters were based on a set of public data available in the public satellite database [26]. The simulated images were generated using the range-Doppler algorithm. The main parameters of the ISAR imaging radar are listed in Table 1.
Two specific imaging times were selected from the observation views, as shown in Fig. 6. The ISAR images corresponding to the two views are presented in Fig. 7. In this context, it should be emphasized that the rectangular solar panels in both images are projected as lines, displaying limited information. Additionally, the reference image depicts an occurrence in which a solar panel obstructs the SAR antenna.
The line results detected by the proposed extraction method are depicted in Fig. 8, while the specific parameters of each line are listed in Table 2.
After feature matching, 12 association pairs were obtained—L1,1L2,1, L1,2L2,2, L1,3L2,9, L1,4L2,11, L1,5L2,12, L1,6L2,13, L1,7L2,10, L1,8L2,5, L1,9L2,6, L1,10L2,7, L1,11L2,8, and L1,12L2,3. Despite some obvious occlusions, the line features could be correctly matched. Most importantly, feature matching yielded the following rough register parameters: a0 = 0.6438, b0 = 0.3985, and γ0 = −11.5629°.
In subsequent optimization algorithms, the search range of Δa and of Δb were set to [−100,100], while of Δγ was set to [−3,3]. Table 3 lists the estimated register parameters, while Fig. 9 depicts the registration results obtained by overlaying the two images—the one before and the one after registration. It is observed that although the main body and the solar panel are well aligned, the SAR antenna presents an alignment error caused by a projection difference between the two imaging planes.
2. Comparison Experiments
Comparison experiments were conducted using real measured ISAR images obtained from the laboratory's official website [27] (@ Fraunhofer FHR). The experiments were carried out in a Windows 10 Professional environment using Intel Core i7-1165G7 processors with 2.80 GHz speed and 16.0 GB RAM, while the codes were implemented in MATLAB R2021a.
2.1 Comparison of line feature extraction methods
The line detection results had a significant effect on the accuracy of the image registration. In particular, the results of three different line detection methods were compared based on the outer contour of the real measured ISAR images shown in Fig. 10(a). The line features extracted by conducting HT and LSM are presented in Fig. 10(c) and 10(b), respectively, in which the parameters of the distance merge while the shortest length limit of a line is the same. Notably, the HT results show more than two lines near certain areas. In both results, the lines do not fit the actual edges of the target, indicating failed ISAR image registration. In contrast, the result obtained using the proposed line detection method, presented in Fig. 10(d), shows that the line features align well with the target boundary, highlighting that it is more suitable and effective than both HT and LSM.
2.2 Comparison of optimization algorithms
An experiment was conducted to compare the efficiency of particle swarm optimization (PSO) [28] and SO under the same configuration. The iterate time was set to 200, and the parameters related to the learning rate in the SO and PSO were reduced to 0.6 times.
Fig. 11(a) shows the cost function value for each iteration. It can be observed that the PSO quickly converges to reach a local optimum. Additionally, the variation curves of the absolute register errors in terms of the number of iterations during the iterative process are shown in Fig. 11(b), 11(c), and 11(d). It is evident that the parameter estimation accuracy of the SO is better than that of the PSO. Furthermore, the time consumption of PSO and SO was 0.0488 seconds and 0.0324 seconds, respectively.
2.3 Comparison of different registration methods
To verify the superiority of the proposed method, real measured ISAR images featuring the same target were first obtained. Subsequently, the results of the proposed method were compared with those obtained using SIFT and artificial bee colony by carrying out normalized cross correlation (ABC-NCC) [29], with mutual information (MI) [30], normalized image correlation (NIC), and algorithm runtime as the performance indicators. The results are listed in Table 4, while the two overlapped images generated by the experiment are depicted in Fig. 12. It is evident that the proposed method is both fast and accurate. Fig. 12(d) shows that the main body in the image obtained using the proposed method almost overlaps with the real measured one, although the tail is displaced due to secondary reflections.
Moreover, a different space object was also employed for image registration. The estimated register parameters are listed in Table 5, and the two overlapped images are presented in Fig. 13. It can be observed that the image produced by the proposed registration process successfully aligns with the target solar panel and accurately captures the change in the position of the SAR antenna.
3. Robustness Analysis
To further analyze the robustness of the proposed method, 1,000 Monte Carlo simulations were conducted using different SNRs. Under the same scatterer model and mapping parameters, the SNR range was set as 10–30 dB, while the step size was considered 2 dB. Fig. 14(a), 14(c), and 14(e) present the mean error (ME) of the range offset, cross-range offset, and rotation of the images at different SNRs, respectively. Furthermore, Fig. 14(b), 14(d), and 14(f) present the root mean square error (RMSE) of the range offset, cross-range offset, and rotation, respectively. The results show that the proposed method is more robust than SIFT and ABC-NCC, especially with regard to the rotation angle estimation, which reached the order of 0.001. In fact, the main reason for the large errors observed in SIFT or ABC-NCC was that they were based on point or area features, while the ISAR images were noisy and sparse.
VI. Conclusion
In this paper, a line features-based ISAR image registry method is proposed. First, a transformation model that incorporated both translational and rotational components was built, after which the traditional LSM was initially applied to detect the rough line features of the target contour. Subsequently, scatterers located in close proximity to the rough line features were fitted to refine and adjust the line features. The pairwise correspondence between the lines was examined using their spatial relations. Finally, considering the sum of the Euclidean distances between the lines as the cost function, the SO algorithm was employed to obtain precise parameters for ISAR image registration. The experimental results confirmed the effectiveness of the proposed algorithm in the ISAR image registration of different targets. Compared to existing methods, the proposed line features exhibited stronger robustness against noise and changes in image, as well as higher uniqueness and stability in matching, thereby improving the accuracy of the registration algorithms.
PDF Links
PubReader
ePub Link
Full text via DOI
Download Citation
Print
