Statistical Analysis of Parameter Estimation for 2-D Harmonics in Multiplicative and Additive Noise

This article considers the problem of parameter estimation for two dimensional (2-D) multi-component harmonics in non zero-mean multiplicative and additive noise. The least squares estimators (LSEs) are proposed to estimate the coherent model parameters, and some statistical results of the LSEs are obtained, including strong consistency, strong convergence rate, and asymptotic normality. Furthermore, the LSEs-based estimators are proposed to estimate the noncoherent model parameters, and the strong consistency and the asymptotic normality are also proved. Finally, some numerical experiments are performed to see how the asymptotic results work for finite sample sizes.


Introduction
In this article, we consider the following model of two dimensional (2-D) multi-component harmonic signals in nonzero-mean multiplicative and additive noise: where m = 1, 2, . . . , M, n = 1, 2, . . . , N, i = √ −1, p, (λ 0 k , μ 0 k )'s, φ 0 k 's, s k (m, n) 's, and e(m, n) are the number of signals, frequency pairs, phases, multiplicative noise, and additive noise, respectively. In addition, it is assumed that the following assumption holds Assumption 1. (i) The frequency pairs (λ 0 k , μ 0 k )'s are distinct in (0, π) × (0, π) and the phases φ 0 k 's are deterministic constants in [0, 2π ). (ii) The multiplicative noise s k (m, n) is a sequence of i.i.d. real-valued random variables with mean μ 0 sk > 0 and finite variance σ 2 sk . (iii) The additive noise e(m, n) is a sequence of i.i.d. complex valued random variables with mean zero and both the real and imaginary parts have finite variance σ 2 0 /2 and they are independent. (iv) The noises s k (m, n)'s and e(m, n) are mutually independent.
The rest of the article is organized as follows. In Sec. 2, LSEs are proposed to estimate coherent model parameters and related theoretical results are provided. In Sec. 3, LSEs-based estimators are proposed to estimate non-coherent model parameters and related theoretical results are also provided. In Sec. 4, numerical experiments are presented. Finally, the article is concluded in a conclusion in Sec. 5. All proofs are provided in the Appendix.

LSEs and Their Statistical Analysis
In this section, we propose the LSEs to estimate the coherent model parameters consisting of frequencies, phases, and multiplicative noise means, then focus on their statistical analysis. We denote the coherent model parameter vector θ = (θ 1 , θ 2 , . . . , θ p ), where θ k = (μ sk , φ k , λ k , μ k ), and the corresponding true value vector . Now the LSEs of θ 0 , i.e.,θ , can be obtained by minimizing the following formula with respect to θ (2) Now we state the results of strong consistency and strong convergence rate of LSEs as follows.
Theorem 2.2. Under Assumption 1, M(λ k − λ 0 k ) → 0 a.s. and N (μ k − μ 0 k ) → 0 a.s. for k = 1, 2, . . . , p. Now we address the asymptotic distribution of the LSEs that helps us to form the asymptotic confidence band. Let D = diag{D 1 , D 2 , . . . , D p }, which is a p-dimensional block diagonal matrix whose block element is given by We state the result of asymptotic distribution of LSEs as follows.
Corollary 2.1 follows from Theorem 2.3 directly. Theorems 2.1-2.3 imply that it is possible to accurately estimate coherent model parameters when sample size is large enough. From expression of asymptotic covariance matrix (ACM) in Theorem 2.3, it is observed that corresponding LSEs are asymptotically uncorrelated with each other for distinct harmonic components, LSE of multiplicative noise mean is asymptotically uncorrelated with LSEs of both frequency and phase for same harmonic component, LSEs of frequencies are asymptotically uncorrelated with each other for same harmonic component, ACM only depends on noise, and asymptotic variances of LSEs of both frequency and phase are inversely proportional to square of corresponding multiplicative noise mean. Note that Theorem 2.3 also gives us an idea of weak convergence rate of LSEs, such as O P (M −3/2 N −1/2 ) and O P (M −1/2 N −3/2 ) for the first and second frequency, respectively, but O P ((MN) −1/2 ) for both phase and multiplicative noise mean, which indicates that the estimation accuracy for non-linear parameters is better than the one for linear parameters.

LSEs-based Estimators and their Statistical Analysis
In this section, we propose the LSEs-based estimators to estimate the non-coherent model parameters consisting of multiplicative noise variances, total noise variance and additive noise variance, then focus on their statistical analysis. The estimator of multiplicative noise variance σ 2 sk , i.e.,σ 2 sk , is given bŷ where Re[.] denotes the real part operator,λ k ,μ k , andφ k are the LSEs. The estimator of total noise variance σ 2 , i.e.,σ 2 , is given bŷ where Q(θ) is defined in (2) andθ is the LSEs of θ 0 . The estimator of additive noise variance σ 2 0 , i.e.,σ 2 0 , is given byσ To state the statistical results of the LSEs-based estimators, we need the following additional assumption Assumption 2. We assume that all the noise have finite fourth-order moments, i.e., where e r (m, n) and e c (m, n) denote the real and imaginary parts of e(m, n), respectively. Now we state the statistical results of the LSEs-based estimators, including strong consistency and asymptotic normality, as follows Theorem 3.1. Under Assumptions 1-2,σ 2 sk 's are strongly consistent estimators of σ 2 sk 's, and it also can be obtained that √ MN(σ 2 sk − σ 2 sk ) converges in distribution to a normal distribution with mean zero and variance σ * sk given by Under Assumptions 1-2,σ 2 is a strongly consistent estimator of σ 2 , and it also can be obtained that √ MN(σ 2 −σ 2 ) converges in distribution to a normal distribution with mean zero and variance σ * given by

Theorem 3.3.
Under Assumptions 1-2,σ 2 0 is a strongly consistent estimator of σ 2 0 , and it also can be obtained that √ MN(σ 2 0 −σ 2 0 ) converges in distribution to a normal distribution with mean zero and variance σ * 0 given by Corollaries 3.1 and 3.2 follow from Theorems 3.1-3.3 directly. Theorems 3.1-3.3 imply that it is possible to accurately estimate the non-coherent model parameters when the sample size is enough large, and also tell us that the weak convergence rate of all non-coherent model parameters is (MN) −1/2 .

Numerical Results
In this section, we present some numerical experiments to see how LSEs and LSEs-based estimators work for finite sample sizes and whether asymptotic results can be used for small    (1.0, 2.0), σ 2 = 4.0 and σ 2 0 = 1.0. we replicate the process 500 times and calculate average estimates (AE) and mean squared errors (MSE) for all parameters. The corresponding asymptotic variances (AVAR) are also reported for purpose of comparison. We also calculate approximate 90% confidence limits for all parameters and obtain expected confidence interval length (length) using true parameter values. The coverage percentages (coverage) are also obtained over five hundred replications. Note that, due to highly nonlinearity of objective function of LSEs, we propose a hybrid stochastic search method to solve this non-linear programming problem. Genetic algorithm is first used as global method to search entire parameter space, then followed by Nelder-Mead Simplex algorithm to search for local minimum in vicinity of output from genetic algorithm. The corresponding optimization procedures are implemented using MATLAB gatool and fminsearch functions, respectively. Multiple settings for the genetic algorithm were tested and the following were selected: Generations = 500; PopulationSize = 160 (20 × number of parameters); EliteCount = 8; CrossoverFraction = 0.5; MutationFcn = {@mutationuniform, 0.5}. All computations are performed in MATLAB 7.5 (R2007b). All results are presented in Tables 1-4 corresponding to frequencies, phases, multiplicative noise means, and noise variances, respectively.
The following observations are very clear from numerical experiments. First, it is observed that the AEs of all parameters are very close to the true parameter values in all considered cases, meanwhile the MSEs of all parameters gradually decrease and approach the AVARs as sample size increases, which verifies the consistency of the proposed estimators and also shows the validity of asymptotic results even for moderate sample sizes. Next, it is clear that the Lengths decrease as sample size increases and the Coverages are nearly 90% in all cases, which means that the asymptotic results can be used to obtain the confidence bounds of unknown parameters even for moderate sample sizes. Then, it can be observed as expected from the presentation of asymptotic behaviors that frequency estimation is more accurate than other parameters for almost all of considered sample sizes. Finally, it can also be seen from Table 4 that the LSEs-based estimators proposed for the estimations of noise variances also work well.

Conclusions
This study focused on parameter estimation of 2-D multi-component harmonics in nonzeromean multiplicative and additive noise. The LSEs and the LSEs-based estimators were proposed to estimate the coherent and non-coherent model parameters respectively. By performance analysis, some excited statistical results for the proposed estimators were theoretically proved. Finally, the numerical results suggested that the asymptotic results can be used even for moderate sample sizes, which practically shows the effectiveness of the proposed estimators. Note that the set S δ,C , defined in Lemma A.1, can be written as where, for k = 1, 2, . . . , p, By some calculations, the following expression can be obtained

Proof of Theorem 2.2. Let
Similarly, let Q (θ) be a p×p block matrix with block element Expanding Q (θ ) around θ 0 by using multivariate Taylor series expansion up to the first order term, it can be obtained that whereθ is a point betweenθ and θ 0 . Since Q (θ) = 0, (A.7) implies that To obtain the asymptotic distribution ofσ 2 sk , we first transform (A.13) into the following form: First, the first term on the right side of (A.14) converges in distribution to a normal distribution with mean zero and variance Var[s 2 k (1, 1)] because of Lindberg-Levy central limit theorem (see Rao, 1973, p. 127). Next, the second term also converges in distribution to a normal distribution with mean zero and variance 2(σ 2 + σ 2 sk )(μ 0 sk ) 2 because of the strong consistency and asymptotic normality ofμ sk and the limit properties of sequence of random variables (see Rao, 1973, p. 122). Finally, the third term converges to zero in distribution by some calculations combining Lemma A.3. Therefore, √ MN(σ 2 sk − σ 2 sk ) converges in distribution to a normal distribution with mean zero and variance σ * sk = Var[s 2 k (1, 1)] + 2(σ 2 + σ 2 sk )(μ 0 sk ) 2 , which proves the result of asymptotic normality ofσ 2 sk . Proof of Theorem 3.2. To obtain the strong consistency ofσ 2 , we first transform it into the following form by referring to the proof of Theorem 2.1 Var[|x(m, n)| 2 ]/(mn) 2 < ∞. Thus, T MN (θ 0 ) → σ 2 a.s., because of Kolmogorov strong law of large numbers (see Rao, 1973, p. 114). Next, f MN (θ , θ 0 ) → 0 a.s., by some calculations combining Cauchy-Schwarz inequality, Taylor series expansion and Theorems 2.1 and 2.2 (see supplementary for details). Finally, g MN (θ, θ 0 ) → 0 a.s., by some calculations combining Lemma A.2. Therefore, the result of strong consistency ofσ 2 follows.
To obtain the asymptotic distribution ofσ 2 , we first transform (A.15) into the following form