西安工业信号检测与估计SDE_05 ch3c CRLB 3.9 3.10 + 应用举例

1、Ch3 CRLBChapter 3 Cramer-Rao Lower Bound (c)应用举例lecture 5第五讲第五讲雷斌雷斌QQ：25016434 Signal Detection and Estimation 信号检测与估值信号检测与估值信息与通信工程专业信息与通信工程专业 研究生学位课程研究生学位课程2021-12-22SDE_05 (C)2对于所有的 定理定理3.1 标量情况下的标量情况下的CRLB Theorem 3.1 CRLB for Scalar Parameter假设 PDF p(X; ) 满足“正则”条件，既0);(lnxpEAssume

2、“regularity” condition is met:则有 Then 222);(ln1varxpE其中whiledxxpxpxpE);();(ln);(ln2222Fisher信息信息 Fisher Information3.4 Cramer-Rao Lower Bound信息越多，估计的方差越信息越多，估计的方差越小，估计越准小，估计越准1.非负的；2.对于独立观测，具有可加性。进一步2021-12-22SDE_05 (C)3Example 3.3 CRLB for DC in AWGNxn = A + wn, n=0,1,2,N-1wn= WGN N(0,2)单独观测的概率密度函数

3、PDF of an individual data sample: 无关的高斯随机变量是统计独立的Uncorrelated G- RVs are Independent 联合概率密度函数是各自PDF相乘Joint PDF= product of the individual PDFs:AxNAnxAApNn21021);(lnxxgA)(xNAApECRLB222);(ln1x)()();(lnxxgIp通过多次观测降低估计噪声2021-12-22SDE_05 (C)4未知参量与信号的依赖程度:Signal in AWGNAWGN信号对参数变化很敏感，则信号对参数变化很敏感，则CRLB较小，估

4、计很准较小，估计很准If signal is very sensitive to parameter change. then CRLB is small . can get very accurate estimate! x x n n = = s s n n; ; + + w w n n, 似然函数 likelihood function:未知参量与信号的依赖程度2021-12-22SDE_05 (C)53.6 参数的变换参数的变换 Transformation of Parameters 已知 的CRLB，但实际关心的 ，它是 的函数Example: Speed of Vehicle F

5、rom Elapsed TimeTest T, want to measure speed V = d/T线性变换线性变换非线性变换非线性变换幅度估计A 功率估计 A2 则有Is this an efficient estimator of ?an efficient estimator破坏了估计的有效性破坏了估计的有效性渐近有效渐近有效 Asymptotic Efficiency22222)var()()(ANAxxExE参数变换估计的有效性参数变换估计的有效性2021-12-22SDE_05 (C)6Example: Speed of Vehicle From Elapsed TimeTe

6、st T, But. really want to measure speed V = d/T2021-12-22SDE_05 (C)73.7 CRLB 矢量情况矢量情况假设 PDF p(X X; ) 满足“正则”条件，既0);(lnXpE其中数学期望是对p( X;X; )求出的。那么，任何无偏估计 的协方差矩阵满足0)(1ICjiijpE);(ln)(2XIFisher信息矩阵半正定矩阵positive semi-definite引申到得到MVU估计Vector Parameter: = 1 2 . p T对于所有的标量参数我们看其方差For a scalar parameter we lo

7、oked at its variance.矢量参数我们看其协方差阵 for a vector parameter we look at its covariance matrix:2021-12-22SDE_05 (C)8Ex.3.6:高斯白噪声中的DC电平 A 2 均未知；参数矢量 = A 2 T10222)(21ln22ln2);(lnNnAnxNNXp42200)(NNINAVar2)(NVar422)(2021-12-22SDE_05 (C)9Ex.3.7:直线拟合的引申BBAANNNNNNNNnBnAnxBnAnxBpAppNnNn22221021026) 12)(1(2) 1(2)

8、 1()(1)(1);(ln);(ln);(lnXXX1010) 1(6) 1() 12(2NnNnnnxNNnxNNNA10222221exp)2(1),(NnNBnAnxpx10210) 1(12) 1(6NnNnnnxNNnxNNB) 1() 12(2)var(2NNNA) 1(12)var(22NNB2021-12-22SDE_05 (C)103.8 Vector Transformations矢量参数变换的CRLB 将前述标量变换的结论推广到矢量，则当)(ga 可以证明：)30. 3 (0)()()(1TggICa 其中prrrppgggggggggg)()()()()()()()(

9、)()(212221212111雅克比矩阵 Jacobian Matrix半正定矩阵positive semi-definite2021-12-22SDE_05 (C)11参数的变换参数的变换Transformation of ParametersBAa)(g其中A是rxp阶矩阵，B是rx1矢量BAaaBAa)(ETTTaggIAAIAACC)()()()(11 已知 的CRLB，但实际关心的 ，它是 的函数 对于非线性变换，有效性只有当N趋近无穷大时保持 因为那时，因为那时， 的估计集中在真值附近的估计集中在真值附近 统计线性统计线性2021-12-22SDE_05 (C)12Example

10、 3.8：DCinWGN信噪比的估计Example 3.8 22Aa42200)(NNITA2NNANAAANNAAggT24422422424221242422002)()()(I2021-12-22SDE_05 (C)13Example2 通过方位和距离估计XY坐标Example2 : Usually can estimate Range (R) and Bearing (?) directly, But might really want emitter (x, y)在基于雷达和光电经纬仪的弹道测试系统中，通过方位和距离估计XY坐标2021-12-22SDE_05 (C)143.9 CR

11、LB for General Gaussian Case一般高斯情况的CRLB推广到以下的情况推广到以下的情况 Now generalize to the case where:数据仍然是高斯分布的 Data is still Gaussian, but均值不一定为0 Parameter-Dependence not restricted to Mean噪声非白 Noise not restricted to White协方差不一定为对角阵 Cov not necessarily diagonalIn Sect. 3.5 we saw the CRLB for “signal + AWGN”确

12、定信号Deterministic Signal w/ 标量确定参数Scalar Deterministic ParameterOne way to get this case: “signal + AGN”Random GaussianSignal w/ VectorDeterministic ParameterNon-WhiteNoise均值和方差均与待估参量有关 X=A+W N(0,2)一般高斯情况X N(A,2I) X=S()+W N(0,2)X N(S(),2I)2021-12-22SDE_05 (C)153.9 CRLB for General Gaussian Case一般高斯情况

13、的CRLB con. 信信号号加噪加噪声声的的统统一描述：一描述： 确定性信号：协方差部分仅为噪声协方差 信号随机： 期望和方差均与待估参量有关 噪噪声声相相关关的情的情况况 噪声电压等于电子随机运动的积累 前一时刻的电压+这一时段的电子迁移 噪声通过带宽受限的信道 低通滤波积分电路2021-12-22SDE_05 (C)16Gen. Gauss. Ex.: Time-Difference-of-Arrival一般高斯情况举例：到达时间差异一般高斯情况举例：到达时间差异考察这两部分2021-12-22SDE_05 (C)17Est. Cov. uses average over only no

14、ise Variability of Cov w.r.t. parametersVariability of Mean w.r.t. parametersFisher信息jiiTiijtruu)()()()(21)()()()(111CCCCCIThis shows the impact of signal model assumptions 信号模式假设的影响deterministic signal + AGN 确定性信号加平稳高斯噪声random Gaussian signal + AGN 随机高斯信号加平稳高斯噪声Est. Cov. uses average over signal &a

15、mp; noise方阵的迹对角线元素之和iNiiiuuuu)()()()(21iNNiNiNiNiiiNiii)()()()()()()()()()(212222111211CCCCCCCCCC附录3C中有证明2021-12-22SDE_05 (C)18协方差矩阵与 无关，即下式0Example3.9 WSN中信号参数中信号参数jNniiTijiiTiijnsnsuutruu,1)(1)()()()()(21)()()()(1022111ICCCCCI Signal Model: x x n n = = s s n n; ; + + w w n n, , n n = 0, 1, 2, . ,

16、= 0, 1, 2, . , N N-1-1 例中，wn独立同分布，IC2与 无关2021-12-22SDE_05 (C)19Example3.10 噪声参数噪声参数4222222111122121)()(21)()()()(21)()()()(NtrtrtruujiiTiICCCCCCCInwnx2待估参量与例3.6的结果一致2021-12-22SDE_05 (C)20待估参量Example3.11 WSN中的随机中的随机DC电平电平2222221112211111121)()()()(21)()()()(ATTAjiiTiANNNtrtruuCCCCCInwAnx2AA为高斯随机变量，与w

17、n独立所以，CRLB是ijAijAjwAiwAEjxixE222)1)(1( 1 1)(C22222)(NVarAA注意即使当N趋近无穷大，CRLB也不会降低到 之下42A2021-12-22SDE_05 (C)21 ACF ACF 自相自相关关函函数数3.10 广义平稳(WSS)高斯随机过程的渐近CRLBknxnxEkrxx如果数据记录长度比过程的如果数据记录长度比过程的相关时间相关时间大很多大很多解析计算很难，且严格说来仅解析计算很难，且严格说来仅N N趋近于无穷大时才有效趋近于无穷大时才有效相关函数不为相关函数不为 0 0 的最大步长的最大步长k k，即相关时间，即相关时间 dffPfP

18、NIjxxixxij2121;ln;ln2是是过过程的程的PSD PSD 功率功率谱谱密度密度; fPxx其中其中 FisherFisher信息信息 的元素（的元素（ N - N - 时时）近似）近似为为2021-12-22SDE_05 (C)223.11 CRLB Examples 1 1：距离估：距离估计计Range EstimationRange Estimation 声纳、雷达、机器人学、发射区定位 sonar, radar, robotics, emitter location 2 2：正弦：正弦参数参数估估计计 Sinusoidal Parameter EstimationSinu

19、soidal Parameter Estimation 幅度幅度 频频率率 相位相位 (Amp., Frequency, Phase) (Amp., Frequency, Phase) 声纳、雷达、计量经济学、光谱测定 sonar, radar, communication receivers (recall DSB Example), etc 3 3：方位估：方位估计计Bearing EstimationBearing Estimation 声纳、雷达、源定位 sonar, radar, emitter location 4 4：自回：自回归参数归参数ARAR估估计计Autoregressi

20、ve Parameter EstimationAutoregressive Parameter Estimation 语音处理、计量经济学 speech processing, econometrics这这里里仅研仅研究究CRLBCRLB，及可能直接，及可能直接产产生的估生的估计计方法方法2021-12-22SDE_05 (C)23Ex. 3.13距离估计 Range Estimation Problem发送脉冲Transmit Pulse: s(t) nonzero over t0,Ts 非0区间接收信号Receive Reflection: s(t .o)测量延时Measure Time

21、Delay: o声纳、雷达、机器人学、发射区定位 sonar, radar, robotics, emitter location连续信号模型连续信号模型C-T Signal Model从发射机到目标再返回的双程延迟o声纳与距离R有关，即o 2R/c带通滤波器带通滤波器放大器放大器非定频信号非定频信号2021-12-22SDE_05 (C)24Range Estimation D-T Signal Model距离估计的离散信号模型Sample Every = 1/2B secwn = w(n)xn = s(n o ) + wn n = 0,1, , N 1sn;o. has M non-zer

22、o samples starting at noDT WhiteGaussian NoiseVar 2 = BNo功率谱密度功率谱密度自相关函数自相关函数2021-12-22SDE_05 (C)25Range Estimation 距离估计 CRLB Now apply standard CRLB result for signal + WGN:=Plug in. and keepnon-zero terms微积分展开Exploit Calculus!Use approximation: o = noThen do change of variables!2021-12-22SDE_05 (C

23、)26Range Estimation CRLB (cont.) Assume sample spacing is small. approx. sum by integral.A type of .SNR.ParsevalDefine a BW带宽 measure:Brms is .RMS BW. (Hz)均方带宽 增加 信号脉冲变窄信号能量2021-12-22SDE_05 (C)27Range Estimation CRLB (cont.)Using these ideas we arrive at the CRLB on the delay:To get the CRLB on the

24、range. use .transf. of parms. result:CRLB is inversely proportional to: . SNR Measure . RMS BW MeasureSo the CRLB tells us. Choose signal with large. Ensure that SNR is large. Better on Nearby/large targets. Which is better?. Double transmitted energy?. Double RMS bandwidth?从CRLB中我们可以获得如下信息 应选择Brms较

25、大的信号 信噪比的提高 没有将Brms提高对精度的影响大2021-12-22SDE_05 (C)28Ex. 2 正弦估计 Sinusoid Estimation CRLB ProblemGiven DT signal samples of a sinusoid in noise.Estimate its amplitude, frequency, and phaseRecall. SNR of sinusoid in noise is:2021-12-22SDE_05 (C)29Sinusoid Estimation CRLB Approach ApproachApproach: . Find

26、 Fisher Info Matrix 找到Fisher信息矩阵 . Invert to get CRLB matrix . Look at diagonal elements to get bounds on parm variances RecallRecall: Result for FIM for general Gaussian case specialized to signal in AWGN case:2021-12-22SDE_05 (C)30正弦信号估计的正弦信号估计的Fisher信息信息Sinusoid Estimation Fisher Info ElementsTak

27、ing the partial derivatives and using approximations given in book (valid when o is not near 0 or ) : = A o T2021-12-22SDE_05 (C)31Sinusoid Estimation Fisher Info Matrix正弦估计的Fisher信息矩阵Fisher Info Matrix then is:Recall. SNR =A2/(22) and closed form results for these sums2021-12-22SDE_05 (C)32Sinusoid

28、 Estimation CRLBsInverting the FIM by hand gives the CRLB matrix. and thenextracting the diagonal elements gives the three bounds:To convert to Hz2multiply by (Fs /2)2.Amp. Accuracy:Amp. Accuracy: Decreases as 1/N, Depends on Noise Variance (not SNR).Freq. Accuracy: Freq. Accuracy: Decreases as 1/N3

29、, Decreases as as 1/SNR.Phase Accuracy: Phase Accuracy: Decreases as 1/N, Decreases as as 1/SNR2021-12-22SDE_05 (C)33Frequency Estimation CRLBs and FsThe CRLB for Freq. Est. referred back to the CT is是否意味着高于是否意味着高于Nyquist采样频率会更糟？采样频率会更糟？Does that mean we do worse if we sample faster than Nyquist?NO!

30、 对于固定采样周期对于固定采样周期For a fixed duration T of signal: N = TFsAlso keep in mind that 谨记谨记 Fs has effect on the noise structure:2021-12-22SDE_05 (C)34Ex. 3 方位估计 Bearing Estimation CRLB ProblemUniformly spaced linear array with M sensors:. Sensor Spacing of d meters. Bearing angle to target radiansPlanarW

31、avefrontsTargetEmits 发送 orreflects 反射signal s(t)简单模型Simple modelPropagation Time to nth Sensor:n = 0,1, , M 12021-12-22SDE_05 (C)35方位估计 Bearing Estimation 传感器信号快照 Snapshot of Sensor SignalsSignal at nth Sensor:Now instead of sampling each sensor at lots of time instants.we just grab one .snapshot. o

32、f all M sensors at a single instant tsSpatial sinusoid w/spatial frequency sFor sinusoidal transmitted signal. BearingEst. reduces to Frequency Est.And. we already know its FIM & CRLB!Spatial Frequencies:. s is in rad/meter. s is in rad/sensor2021-12-22SDE_05 (C)36Bearing Estimation Data and Par

33、ametersEach sample in the snapshot is corrupted by a noise sample.and these M samples make the data vector x = x0 x1 . xM-1 :Each wn is a noise sample that comes from a different sensor so.Model as uncorrelated Gaussian RVs (same as white temporal noise)Assume each sensor has same noise variance 2So

34、. the parameters to consider are:which get transformed to:Parameter of interest!Bearing2021-12-22SDE_05 (C)37Bearing Estimation CRLB ResultUsing the FIM for the sinusoidal parameter problem. Together with the transform. of parms resultDefine: Lr = L/Array Length .inwavelengths.L = Array physical len

35、gth in metersM = Number of array elements = c/fo Wavelength in meters (per cycle)Bearing Accuracy:. Decreases as 1/SNR . Depends on actual bearing . Decreases as 1/M ! Best at = /2 (.Broadside.). Decreases as 1/Lr2 ! Impossible at = 0! (.Endfire.)Low-frequency (i.e., long wavelength) signals needver

36、y large physical lengths to achieve good accuracy2021-12-22SDE_05 (C)38LPC编码：语音合成 在语音处理中，语音产生的一个重要模型是自回归在语音处理中，语音产生的一个重要模型是自回归(AR)(AR)过程。如图过程。如图3.93.9所示，将数据看成一个因果的全极所示，将数据看成一个因果的全极点离散滤波器在点离散滤波器在WGN WGN unun 激励下的偷出。激励噪声激励下的偷出。激励噪声unun 是模型固有的一部分，对于确保是模型固有的一部分，对于确保xnxn 为为WSSWSS随机随机过程则是必须的。全极点滤波器起着模拟声带过

37、程则是必须的。全极点滤波器起着模拟声带( vocal ( vocal tract)tract)的作用，而激励噪声模拟了通过喉咙的收缩所产的作用，而激励噪声模拟了通过喉咙的收缩所产生的空气的作用力，喉咙的收缩对于产生无声的声音生的空气的作用力，喉咙的收缩对于产生无声的声音( (如如s)s)是必要的。滤波器的效果就是将白噪声变成色是必要的。滤波器的效果就是将白噪声变成色噪声，以便模拟具有几个谐振的噪声，以便模拟具有几个谐振的PSDPSD。这个模型也称。这个模型也称为线性预测编码模型为线性预测编码模型( (LPC)MakboulLPC)Makboul 1975 1975。由于妞。由于妞模型具有产生多

38、种模型具有产生多种PSDPSD的能力，因此取决于的能力，因此取决于ARAR滤波器滤波器参数参数a1,a2,apa1,a2,ap2021-12-22SDE_05 (C)39ARMA模型 自回自回归归模型模型AR(pAR(p) ) 移移动动平均模型平均模型MA(qMA(q) ) ARMA(p,qARMA(p,q) )模型模型 qtqtttu11p阶自回归模型记作阶自回归模型记作AR(p)其中：参数其中：参数c 为常数；为常数； 1 , 2 , p 是自回归模型系数；是自回归模型系数；p为自回归模型为自回归模型阶数；阶数； t是均值为是均值为0，方差为，方差为 2的白噪声序列。的白噪声序列。tptp

39、tttuuucu2211qtqttptpttuucu1111q阶移动平均模型记作阶移动平均模型记作MA(q)混合模型，常记作混合模型，常记作ARMA(p，q)2021-12-22SDE_05 (C)40Ex. 4 自回归自回归AR参数估计参数估计 Estimation CRLB ProblemIn speech processing (and other areas) we often model the signal as an AR random process and need to estimate the AR parameters. An AR process has a PSD

40、given byAR Estimation Problem: Given data x0, x1, . , xN-1estimate the AR parameter vectorThis is a hard CRLB to find exactly. but it has been published.The difficulty comes from the fact that there is no easy directrelationship between the parameters and the data.It is not a signal plus noise probl

41、emY和它的若干期滞后之间往往存在数据的高度相关2021-12-22SDE_05 (C)41AR Estimation CRLB Asymptotic ApproachApproach: The asymptotic result we discussed is perfect here:. An AR process is WSS. is required for the Asymp. Result. Gaussian is often a reasonable assumption. needed for Asymp. Result. The Asymp. Result is in ter

42、ms of partial derivatives of the PSD. andthat is exactly the form in which the parameters are clearly displayed!Recall:2021-12-22SDE_05 (C)42AR Estimation CRLB Asymptotic ResultAfter taking these derivatives. you get results that can besimplified using properties of FT and convolution.The final resu

43、lt is:To get a little insight. look at 1st order AR case (p = 1):Complicateddependence onAC Matrix!Both Decreaseas 1/N2021-12-22SDE_05 (C)43Single-Rx Emitter Location via Doppler基于多普勒效应的单接收器 源定位 Then Use Measured Frequencies to Estimate Location Estimate Rx Signal Frequencies: f1, f2, f3, , fNReceiv

44、ed Signal Parameters Depend on Location接收到信号的参数与位置有关接收到信号的参数与位置有关DopplerDoppler效应：听效应：听汽车驶过附近汽车驶过附近飞机飞过附近？飞机飞过附近？声音频率的变化声音频率的变化2021-12-22SDE_05 (C)44Problem BackgroundRadar to be Located Radar to be Located 定位定位: at Unknown Location (: at Unknown Location (X,Y,ZX,Y,Z) )Transmits Radar Signal at Unkn

45、own Carrier Frequency Transmits Radar Signal at Unknown Carrier Frequency fo 雷雷达达信信号号 不知道不知道频频率率Signal is intercepted by airborne receiverSignal is intercepted by airborne receiver信信号号由机由机载载的接收机接收的接收机接收Known (Navigation Data):Known (Navigation Data):已知已知导导航航数数据据 Antenna Positions天线位置: (Xp(t), Yp(t),

46、 Zp(t) Antenna Velocities移动速度: (Vx(t), Vy(t), Vz(t) Goal: Estimate Parameter Vector Goal: Estimate Parameter Vector x = = X Y Z foT2021-12-22SDE_05 (C)45Physics of ProblemRelative motion between相对运动emitter and receiver发射机和接收机causes a Doppler shift of 多普勒频移the carrier frequency:载波频率Because we estimat

47、e the frequency there is an error added:2021-12-22SDE_05 (C)46Estimation Problem StatementRight now only Right now only want to consider want to consider the CRLBthe CRLB2021-12-22SDE_05 (C)47The CRLB Note that this is a “signal” plus noise scenario: The “signal” is the noise-free frequency values T

48、he “noise” is the error made in measuring frequency Assume zero-mean Gaussian noise with covariance matrix C: Can use the “General Gaussian Case” of the CRLB Of course validity of this depends on how closely the errors of the frequency estimator really do follow this Our data vector is distributed a

49、ccording to:Only need the first term in the CRLB equation:Only Mean ShowsDependence onparameter x!I use J for the FIM instead of I to avoid confusion with the identity matrix.2021-12-22SDE_05 (C)48Convenient Form for FIM To put this into an easier form to look at Define a matrix H:Called “The Jacobian” of f(x)whereThen it is east to verify that the FIM becomes:2021-12-22SDE_05 (C)49CRLB Matrix The Cramer-The Cramer-RaoRao bound bound covariance matrix then is:covariance matrix then is:A closed-form expression for the partial deriv