On the Detection Performance Verification of Airborne AESA Radar Systems
Article information
Abstract
The detection performance of an airborne radar for air targets is generally evaluated by the cumulative probability of detection. To verify that the radar meets the required detection performance, a number of flight runs are repeated to obtain the detection samples. The number of samples is usually limited due to the constraint on the number of flight runs. Consequently, the estimated cumulative probability of detection can highly deviate from the ideal value (i.e., the probability estimated by infinite number of samples). In this paper, the statistical characteristics of the detection performance depending on a limited number of samples are analyzed for the two representative flight test methods: HM and FD. Based on the analysis, a guideline for radar power budget design to determine the margin is proposed to ensure that the estimated detection performance through the flight test with the limited number of runs meets the required performance.
I. Introduction
The detection of targets through scanning search area is one of the most fundamental functions of radar. The detection performance of a radar is generally evaluated by the probability of detection [1]. In particular, the detection performance of an airborne radar for air targets is commonly evaluated by the cumulative probability of detection [2–5], which is defined as the probability that the target is detected at least once until it reaches a given range. To verify that the detection performance of the developed radar meets the requirement on the cumulative probability of detection at a specified range, a number of flight tests (or runs) with a pre-defined scenario are repeated [4, 5]. In this scenario, the radar platform and target fly over a fixed flight path at a constant velocity. It is desired that the initial range between the radar and target should be large enough so that the single probability of detection is close to zero [6, 7]. The radar must scan with a fixed search area with a constant search frame time since the cumulative probability of detection is highly dependent on the search frame time [8]. In a traditional mechanically scanned array (MSA) radar with track-while-scan (TWS) mode, the same target can be detected continuously at a number of successive scans as it approaches the radar. In TWS mode, the radar initiates and updates a track for the detected target through search dwells which are allocated to detect new targets simultaneously. Thus, the single (blip-scan) probability of detection can be estimated through the statistics of hit (detection) and miss (no-detection) in each scan of each run. The cumulative probability of detection, then, can be easily calculated from the single probability of detection. Since this method depends on the statistics of “hit and miss,” it is denoted by HM in the rest of this paper.
However, a modern radar with active electronically scanned array (AESA) adopts advanced multi-target tracking policies using search-while-track (SWT) or active tracking [9–13] to improve tracking accuracy. In SWT mode, the radar initiates and updates a track with individual dwells for track-only. Unlike TWS mode of conventional MSA radar, detections from search dwells that are associated with existing tracks are rejected in SWT mode. Thus, once a target is detected by a search dwell and a track is formed for the target, no additional detection of the target from search dwells is reported even if the target is detected in the subsequent scans, and as a result, the statistics for the single probability of detection cannot be obtained. Therefore, instead of HM, an alternative method which depends on the “first detection” ranges for each run (denoted by FD in the rest of this paper) is applied. In FD, the first detection ranges for each run are collected and ordered by range, then the first detection range of the order corresponding to the required cumulative probability of detection is chosen [5]. For example, suppose that it is required that the range corresponding to the cumulative probability of detection Pc = 90% be longer than Rreq = 50 nmi. If 12 flight runs are conducted, then the first detection ranges Ri (i = 1, 2, …, 12) for each run are sorted in descending order. The requirement is satisfied if the 11th Ri (or the range corresponding to 10.8 (= 12×0.9) between 10th and 11th ranges by interpolation) is the same or longer than Rreq.
While HM requires each flight run to be conducted until the target reaches a range less than Rreq, FD saves time and resource since each flight run can be aborted as soon as the first detection occurs. Thus, FD is more cost-effective than HM, regardless of the type of the radar under test (whether MSA or AESA), and it is advisory to choose FD over HM, considering high costs of flight tests [14]. On the other hand, HM provides more information about the radar detection characteristics than FD by continuing the observation of hit and miss after the first detection. With the additional data of hit and miss, for instance, the tracking performance can be analyzed simultaneously in TWS mode. Thus, it should be emphasized that the choice between HM and FD is not a matter of inherent superiority, but rather of practical considerations such as the available resources and the objectives of the flight test.
In this context, this paper addresses the following two issues. First, the equivalence between HM and FD is investigated. To the best of our knowledge, the equivalence between HM and FD has not been clearly investigated, either theoretically or experimentally. This paper shows that the two methods are ideally equivalent (i.e., when an infinite number of samples are available), but that such equivalence does not hold in practice when the number of flight runs is limited. Second, a guideline for a radar power budget design is presented by accounting for the discrepancy between the limited-sample case and the infinite-sample case for each method. Most previous studies on the topic of detection performance verification through a limited number of flight runs focused on effectively estimating the cumulative probability of detection from limited samples in post-test phase [5, 15, 16]. On the other hand, the work in this paper focuses on determining the radar power budget that can cover the discrepancy between the cumulative probability of detection directly obtained from flight test and the theoretical predictions in pre-test phase (i.e., in design phase).
This paper is organized as follows. In Section II, the statistics of HM and FD are analyzed and it is shown that the equivalence between the two methods does not hold in the limited-sample case. In Section III, based on the statistical analysis from the previous section, a design guideline for radar power budget in the limited-sample case is proposed to ensure that the required cumulative probability of detection is satisfied for both HM and FD. Section IV concludes this paper.
II. Statistical Analysis of Detection Performance with a Limited Number of Samples
Assuming that the radar is scanning the search area at a constant time intervals (or a search frame time) and that the target and radar are approaching at a constant velocity, then the cumulative probability of detection at the nth scan, Pc (n), is expressed as follows [5, 17, 18].
where Pd(i) is the single probability of detection at the ith scan, which is obtained from the statistics of hit and miss in HM as follows. For M trials (or flight runs), the results of detection are stored in an array h(n, m). The element of h(n, m) is set 1 or 0 if the target is detected or missed at the nth scan of the mth trial [17]. Then, Pd (n) is estimated as
and Pc (n) is estimated by Eq. (1), which is the result that can be obtained from HM. On the other hand, the cumulative probability of detection is estimated by M first detections from each trial in FD, as stated in the Introduction with an example.
Ideally (or when M → ∞), the cumulative probabilities of detection from HM and FD are equivalent, as proved in the Appendix A. As M approaches infinity, the obtained single probability of detection in HM converges to the theoretical single probability of detection, which is determined by the SNR, detection threshold, and the fluctuation of target radar cross section (RCS). However, the number of available runs is usually limited due to time and resource constraint [5]. In the following, through the analysis of the statistics from Monte Carlo simulation results, it is shown that the detection performance can be over- or underestimated in the limited-sample case for both HM and FD, and that the detection performances obtained using HM and FD, respectively, exhibit different statistical behaviors as the number of sample varies, even though they are ideally equivalent.
The stochastic nature of the radar detection is primarily due to the fluctuation of the target RCS [5]. The instantaneous SNR at each scan or pulse also fluctuates due to the fluctuating RCS [19]. The average SNR for the average RCS of the fluctuating target, ρ̄, at the nth scan can be modelled simply as
where VT is the detection threshold of the burst. Eq. (3) assumes that ρ̄ at the 1st scan is VT/(1 + n0Δr)4. The larger the n0, it means that the opportunity to detect the target is given at the farther range. The normalized range decrement Δr is defined as
where ΔR is the decrement of the radial distance between the radar and target within a search frame time, and R0 is the range where ρ̄ = VT. The instantaneous SNR, ρ, at the nth scan is then given as
where γn is the instantaneous RCS of the target at the nth scan normalized by its average RCS. In most of the previous works on the analysis of the detection performance, the range decrement was normalized to the range for unity SNR to generalize the analysis [2, 3, 7, 17, 19]. Instead, in this work, the range decrement is normalized to the range where ρ̄ = VT. In this way, it is possible to generalize the analysis to the variation of VT, as shown in the next section.
The details of the Monte Carlo simulation performed for the analysis of the statistics of the detection performance are as follows. It was assumed that a target with its RCS fluctuation modeled as Swerling I [20] is approaching the radar. The radar scans to detect the target with Δr = 0.01 and the probability of false alarm pfa = 10−6 (thus, VT = − ln(pfa) = 13.82 = 11.4 dB [21]). A single dwell consists of 8 bursts and a detection is declared by 3- out-of-8 binary integration, which is a widely used detection strategy for surveillance mode [22]. Noise-only situation was assumed to generalize the simulation result. The simulated results obtained from M = 100,000 Monte Carlo trials are shown in Figs. 1–3. At every nth scan of mth trial, h(n, m) was updated to obtain the single probability of detection. In Fig. 1, both of the single and cumulative probabilities of detection against ρ̄ from Eqs. (2) and (1), respectively, are presented. Thus, Fig. 1 shows the result obtained by HM. In Fig. 2, the values of ρ at the first detections from M trials are shown. The black dotted symbol presents ρ of the first detections in the order of occurrence, while the red dotted symbol presents the same ρ sorted in ascending order. Thus, the cumulative probability of detection from FD can be obtained by dividing the sorted order by M. The results in Figs. 1 and 2 are compared in Fig. 3. The difference between the cumulative probabilities of detection obtained from HM and FD is less than 0.0017, which are originated from the finite number of trials.
Single and cumulative probabilities of detection against the average SNR ρ̄ when Δr = 0.01, VT = 11.4 dB, and M = 100,000.
Instantaneous SNR ρ corresponding to the 1st detections when Δr = 0.01, VT = 11.4 dB, and M = 100,000.
Cumulative probabilities of detection from HM and FD, respectively, and the difference between them when Δr = 0.01, VT = 11.4 dB, and M = 100,000.
Suppose that the required detection performance is defined as the range corresponding to the cumulative probability of detection 0.9. From Fig. 3, ρ̄ corresponding to Pc = 0.9 is 7.64 dB when M = 100,000 or larger. However, it is almost impossible to conduct 100,000 flight runs in development phase or test and development (T&E) phase for a single radar system. Practically, the number of runs may be limited to 20, 10, or even less, depending on the available time and resources. The statistics of the normalized ρ̄ when Pc = 0.9, ρ̄n,90, which is given as
are presented as a function of the number of samples in Fig. 4 for both HM and FD. The red circle and blue square symbols show μ90, the population mean of ρ̄n,90, for both HM and FD, respectively. The error bars indicate σ90, the standard deviation of ρ̄n,90. While the results from the two methods converge to the same value as the number of sample increases, a noticeable discrepancy is observed in the limited-sample case. The statistical characteristics of HM and FD in Fig. 4 are analyzed in detail as follows.
Mean μ90 of ρ̄n,90 against M when Δr = 0.01 and VT = 11.4 dB. The error bars indicate the standard deviation σ90 of ρ̄n,90.
In Fig. 4, it is observed that μ90 decreases as M decreases. In other words, the required SNR to achieve desired detection performance decreases as M decreases, i.e., the radar performance is over-estimated on average. For the random variable X~B(M, Pc ) and μ = PcM, the probability of under-estimation P(X < μ) and the probability of over-estimation P(X > μ) are calculated from the probability mass function as follows [5, 15]:
Note that in (7), Pc and M are chosen so that μ is always a positive integer. The probabilities calculated by (7) for M = 10, 100, and 1,000 are shown in Fig. 5. It is observed that P(X > μ) > P(X < μ) for Pc > 0.5, while P(X > μ) < P(X < μ) for Pc < 0.5. The gap between P(X > μ) and P(X < μ) decreases as M increases, and eventually, as M → ∞, both P(X > μ) and P(X < μ) converge to 0.5. Thus, for the case of Pc = 0.9 (>0.5) in Fig. 4, for small M, the detection performance is highly over-estimated and the level of the overestimation gradually decreases as M increases.
Probabilities of under- and over-estimation for various values of Pc when M = 10, 100, and 1,000, respectively.
It should be also noted that the level of the over-estimation is more severe in FD than in HM for the same value of M. To investigate this phenomenon, suppose that ten flight runs with Δr = 0.01 have been conducted while the radar is scanning in TWS mode (so that both HM and FD can be applied) with VT = 11.4 dB. The values of ρ̄ corresponding to the range of each detection of each run are shown in Fig. 6, where the red circular symbol indicates the first detection of each run. By HM, the calculated Pd and Pc from Eqs. (2) and (1), respectively, are shown in Fig. 7. Also, Pc obtained from the first detections from each run by FD is presented. (The staircase nature of Pc observed in Fig. 7 was described in [16].)
Average SNR ρ̄ corresponding to the range of the detections from 10 flight runs when Δr = 0.01 and VT = 11.4 dB: (a) ordered in occurrence and (b) sorted by the order of the first detections. The red circular symbol indicates the first detection of each run.
Single and cumulative probabilities of detection against the average SNR ρ̄ when Δr = 0.01, VT = 11.4 dB, and M = 10.
It is observed that Pc of FD is higher than that of HM for the most ranges of the SNR. For instance, until the target approaches to the range where ρ̄ = 4.83 dB, the first detections occurred at three flight runs (when ρ̄ = 1.58 dB (in the 10th run), 3.24 dB (in the 2nd run), and 4.83 dB (in the 7th run), respectively), and no other detections occurred. (To avoid any confusion, it should be emphasized that the instantaneous SNR ρ was higher than VT at each detection even if the average SNR ρ̄ corresponding to the range of the detection is lower than VT, due to the RCS fluctuation). Therefore, in FD, Pc = 0.3 at ρ̄ = 4.83 dB. However, in HM, Pc = 1 − (1 − 0.1)3 = 0.27 at the same SNR. Occasionally, Pc of FD is lower than that of HM. For instance, until the target approaches to the range where ρ̄ = 6.06 dB, two more first detections occurred (when ρ̄ = 5.19 dB (in the 5th run) and 5.43 dB (in the 8th run), respectively). Therefore, in FD, Pc = 0.5 at ρ̄ = 6.06 dB. However, the 2nd and 3rd detections occurred at the 8th run at ρ̄ = 5.68 and 6.06 dB, respectively, and the corresponding Pc = 1 − (1 − 0.1)7 = 0.52 in HM.
From these observations, it can be concluded that Pc in HM can be higher than that in FD if a number of detections occur in certain flight runs before the first detections of the other runs. However, on average, the first detections occur at the ranges longer than the ranges of the subsequent detections, and therefore, Pc in FD is higher than that in HM. Denoting the normalized number of occurrences when Pc in FD is higher, lower than, or the same as that in HM by η, the calculated η for 1,000 iterations of M = 10 trials are shown in Fig. 8. The average of η when Pc in FD is higher than that of HM is 0.28. Meanwhile the average of η when Pc in FD is lower than that of HM is only 0.06.
Normalized number of occurrence η when Pc in FD is higher (blue), lower (red) than, or the same (black) as that of HM for 1,000 iterations of M trials.
Although the detection performance can be over-estimated on average with the limited number of samples, the stochastic nature of radar detection causes the deviation of ρ̄n,90 from its mean, as shown by the error bars in Fig. 4. Clearly, σ90 increases as M decreases, implying that, if the number of flight run is reduced, a greater margin should be included in the radar power budget to ensure the pass of the flight test for detection performance. The guideline for the power budget margin is introduced in the next section.
III. Power Budget Margin for a Limited Number of Samples
To ensure that the required detection performance is satisfied through the flight test with the limited number of runs, it is desired to have enough power budget margin that can cover the deviation of ρ̄n,90. To provide a generalized guideline for the power budget design with sufficient margin that can be applied for various ranges of Δr, VT, and M, the upper bound of ρ̄n,90, which is denoted by β90, is defined as the summation of μ90 and σ90. The value of β90 can be estimated through the following approximation.
which is derived by heuristic model fitting to capture the dominant statistical behavior depending on the number of samples. Specifically, the combined term 1 − (C1(M + C2)) 53 represents μ90, while the term (C4(M + C4)) 56 models the behavior of σ90 depending on M. The constants in Eq. (8) are actually the functions of Δr which are given as follows for 3-out-of-8 binary integration with Swerling I target.
Note that Eq. (9a) is for HM and Eq. (9b) is for FD. To show how β̂ 90 represents β90 accurately, the error ɛ90, which is defined as ɛ90 = (β90 −β̂ 90)/β90, is presented in Fig. 9(a)–9(c) for Δr = 0.01, 0.006, and 0.002, respectively. For wide ranges of Δr, the maximum of the absolute error is only less than 1.5%. Thus, β̂ 90 can applied to radar power budget design for various values of Δr. Also, it can be applied for various values of VT. In Fig. 10(a) and (b), μ90 and σ90 when M = 10 are presented for both HM and FD, respectively. The solid lines show the values for the case when ΔR is normalized to the range where ρ̄ = VT, while the dashed lines show those when ΔR is normalized to the range for unity SNR (thus, ρ̄ = 0). Although μ90 is almost constant even if pfa (or equivalently, VT ) changes for both cases, σ90 monotonically increases with pfa for the case of ρ̄ = 0 while it is almost constant for the case of ρ̄ = VT.
Note that β̂ 90 in Eq. (8) can also be applied to other Swerling types, as long as its constants are adjusted as follows. For instance, for Swerling II,
where Eq. (10a) is for HM and Eq. (10b) is for FD. The RCS fluctuation of fighter is generally modeled as Swerling I or II. Ground or sea targets are conventionally modeled as Swerling III or IV, and the detection performance is defined with the single probability of detection, not the cumulative probability of detection [23, 24]. Thus, Swerling III and IV are outside the scope of this work.
The procedure to determine the power budget margin using Eq. (8) is provided in Fig. 11. When M and Δr are given (where M is usually limited due to test resources, and Δr is determined based on the test scenario), the designer can decide the constants of β̂90 from either Eq. (9) or Eq. (10) depending on the expected target RCS fluctuation type and the test methods (whether HM or FD). To outline the procedure more clearly, the following example is considered (Although the proposed procedure was validated through real flight test, the test information such as radar parameters, scenario and detection data cannot be disclosed due to security restriction. Only partial information from the flight test that can be disclosed is provided in Appendix B). Suppose that it is required to detect a target (fighter) at Rreq = 50 nmi under the following conditions:
• Target RCS: 5 m2
• Cumulative probability of detection: 0.9
• Closing velocity of the radar platform and target: 500 m/s
To meet this requirement, suppose that the preliminary design parameters are determined as follows:
• Operating frequency: 10 GHz
• Transmitter peak power: 10 kW
• Antenna gain: 33 dBi
• Receiver noise figure: 3.3 dB
• System loss: 12 dB
• Pulse repetition frequency: 20 kHz
• Duty cycle: 10%
• Coherent processing interval: 6.4 ms
• Detection threshold VT: 11.4 dB (pfa = 10−6)
• Detection criteria: 3-out-of-8
• Search frame time: 1.5 sec.
Applying the design parameters to the radar range equation, the range R0 where ρ̄ = VT is calculated as 40.2 nmi for the preliminary design. The theoretically estimated cumulative probability of detection at Rreq is 0.9 where ρ̄ = 7.6 dB.
Since ΔR = 750 m (= 500 m/s × 1.5 sec), from Eq. (4), Δr = 0.0097. If 10 flight runs are available and the detection performance is supposed to be evaluated by FD, the constants of Eq. (8) are calculated as follows from Eq. (9b): C1 = 0.4154, C2 = 6.2893, C3 = −1.3455, C4 = 2.29, C5 = 2.34, and C6 = −0.5333. Finally, β̂90 when M = 10 is given as 1.09 (≈ 0.4 dB). It means that the detection performance can be under-estimated so that Pc = 0.9 is satisfied at ρ̄ = 8.0 dB (= 7.6 dB + 0.4 dB) where the range is 48.8 nmi. Thus, it is recommended to increase the power budget with a margin of 0.4 dB to ensure that the requirement is satisfied through the flight test with 10 runs.
IV. Conclusion
In this paper, the statistical characteristics of the radar detection performance depending on a limited number of samples have been analyzed. Based on the analysis, the guideline of the radar power budget design considering the margin to ensure that the detection performance requirement is satisfied through the flight test with the limited number of runs.
The detection performance, which is evaluated by the cumulative probability of detection, is generally verified by the repetition of flight tests. Unfortunately, the available number of flight runs are limited due to time and resource constraints. In this paper, the detection performance can deviate from the ideal value as the number of runs is limited in both HM and FD. It is expected that the proposed guideline for the radar power budget design in this paper reduces the failure of the flight test for detection performance verification by providing sufficient margin to overcome the under-estimation originated from the limited number of samples.
Notes
This work was supported by the Agency for Defense Development - Grant funded by the Korean Government (Defense Acquisition Program Administration).
References
Appendices
The equivalence between the cumulative probabilities of detection from HM and FD are shown as follows. At a single trial, the probability P1(n) that the target is detected for the first time at the nth scan, after being missed in all previous scans, is given as
Note that Eq. (11) is valid only when n > 1. It is obvious that P1(1) = Pd(1). In FD, the first detections are collected for a number of flight runs, and sorted in ascending order to obtain the cumulative probability of detection, as shown in Figs. 2 and 3. Thus, the result from FD is actually the probability P1st (n) that the target is detected for the first time at any of the scans until the nth scan, which is given as the summation of P1(i) for i from 1 to n:
Expanding Eq. (13) from n to 2 gives
Then, the summation of the left- and right-hand sides of Eq. (14) is simply given as
Since Pc (1) = Pd (1) from Eq. (1), and P1(1) = Pd(1), from Eqs. (1) and (15),
The proposed procedure in Fig. 11 has been validated through a number of real flight tests. Due to security restrictions, only partial information from the test is provided as follows. From a pre-defined test scenario, Δr was calculated as 0.0068. It was expected that 10 flight runs would be available, but considering unexpected reduction of the number of runs due to unforeseen factors such as adverse weather conditions or aircraft malfunction (either the radar platform or the target), M = 6 was applied. Since the detection performance had been decided to be verified through FD, an additional power budget margin that can cover the deviation of SNR was calculated by using Eq. (9b). The trajectories of the radar platform and the target from a single representative run are shown in Fig. 12, and the obtained first detections at each run are provided in Fig. 13. The dashed line indicates the required cumulative probability of detection. Note that the range corresponding to this probability from the sorted first detections is longer than Rreq, which shows that the required detection performance is satisfied.
Biography
![]()
Ji Hwan Yoon, https://orcid.org/0000-0002-1661-4395 received the Ph.D. degree in Electrical and Electronic Engineering from Yonsei University, South Korea, in 2016. Since March 2016, he has been with the Agency for Defense Development, South Korea, where he is currently a senior researcher. His research interests include airborne radar systems, phased array radars, radar waveform design, radar signal/data processing, and reflectarrays.
![]()
Jin Ju Won, https://orcid.org/0000-0001-7733-2474 received the M.S. degree in Electronic Engineering from Yeungnam University, South Korea, in 2017. Since March 2017, she has been with the Agency for Defense Development, South Korea, where she is currently a senior researcher. Her research interests include airborne AESA radar systems, radar resource management, and radar data processing.
![]()
Won Jin Lee, https://orcid.org/0009-0003-5537-0340 received the M.S. degree in Electrical and Computer Engineering from Sungkyunkwan University, South Korea, in 2017. Since March 2017, he has been with the Agency for Defense Development, South Korea, where he is currently a senior researcher. His research interests include airborne AESA radar systems and terrain following radars.
![]()
Ji Eun Roh, https://orcid.org/0000-0001-5156-9860 received the Ph.D. degree in Computer Science and Engineering from POSTECH, South Korea, in 2006. Since March 2006, she has been working for the Agency for Defense Development, South Korea, where she is currently a principal researcher. Her research interests include airborne AESA radar systems, radar resource management, and radar data processing.
