《坏数据处理》PPT课件.ppt

Function: Provide network structure 提供网络结构 Provide system state 提供系统实际运行状态 Identify bad measurement data 识别测量坏数据 Identify erroneous status information 识别错误状态信息,SE,Parameter R,X,B/2,K,Measurements PiQiPijQijPjiQjiVi,Status 0,1,Network structure Y matrix,System state V as well as PiQiPijQijPjiQji,State Estimation 状态估计包含,-Bad data - analog measurement 坏数据 -遥测(power flows, voltage magnitude功率、电压值） -Topology errors-logic one 拓扑错误-遥信 （ status of breakers and switchers 开关量） -Sudden load (state ) change-only appears in DSE 负荷突变-只在动态状态估计中涉及,Estimation algorithm 估计计算 Treatment of anormalies 不正常事件处理,9.3 bad data detection and identification 坏数据的检测与识别 contaminate submerge,The object function of SE should be 状态估计的目标函数测量误差最小 the minimization of the measurement error. error =measured value- true value 误差=测量值-真值,Since the true values can not be known, 由于真值不知道，只能用 the residuals are used instead of them. 残差来代替 residual =measured value-estimated value 残差=测量值-估计值,The precondition of the method assumes that the error is in the form of gauss distribution noise. It seeks an average value and better result can be obtained from redundant measurements.,When there are bad data, v i 3 i ，only after 若测量值是不良数据，即 the bad datas influence is excluded, the reliable v i 3 I 则必须排除坏数据 results can be obtained. 的影响，才能得到可信的结果,前提是误差 v 是高斯分布 的噪声，多次测量寻求某 种平均值，可以得到较好的结果,measurement测量值 1 -2 -2,the measurement 测量值 1 -2 the estimate value 估计值 object function J=(-.5)2+(-.5)2=0.5 is minimum, and both residuals are -0.5,Example 1: two measurements,Delete the measure with lager residual 删除残差大的测量值 -2 -2,Example 2: three measurement 三个测量值,the estimate results are not good. 估计结果偏差较大,estimate value估计值 1.67 -1.67 -1.67,residual残差 -.67 -0.33 -0.33,-2.00 -2.00 The estimate results become much better. 估计结果较好,Observability detectability and identifiability 可观测性、可检测性、可识别性,Detection: the process justifying if there are bad data and selecting which data are suspicious ones .,Identification : the process of verifying true 不良数据识别： bad data among suspicious ones 验证真正不良数据的过程,Measurement error: 量测错误： solid error: just at the SE installation period 稳定错，刚安装状态估计 random error: measuring and transmitting process 随机错，量测、传输受到干扰,Measurement system redundancy 量测系统冗余度 K=m-n -the ability to exclude bad data 排除坏数据的能力 The measurements should be distributed evenly 量测均匀分布 Observability ：the voltage magnitude and phase angle can be calculated , and the inverse matrix of HTR-1H can be found. 可观测性: 能计算出电压幅值和相角， HTR-1H可求逆,不良数据检测:判断是否存在不良数据，并指出可疑量测数据的过程,Example: A measurement system with m(=14) measurements. The number of state variables is n (=7) 1 2 4 3,Since the measurement are not distributed evenly , the v4and 4 can not be calculated by the give measurement system 但由于测量不均匀, 算不出v4和4,v,v -voltage measurement -P+jQ measurement pair,v,9.3.1不良数据检测与辨识基本原理 the basic principles of bad data detection and identification,We want to establish the relationship between residuals and errors, and justify the bad data 建立残差和误差的关系，根据残差判断误差较大的不良数据,x真值, 在真值附近线性化,Substitute it into the residual equation 代入残差方程,Now the true value is selected as starting point,W is called residual -sensitivity matrix of mm 残差灵敏度矩阵,Since the rank of residual sensitivity matrix is m-nm, the equation can not be solved to find v. However, some suspicious measurement can be judged. 残差灵敏度矩阵的秩为m-n, 不能求解出v. 但可以判断出一些可疑量测。 ri =wi Wii vi,The ith residual is the co-action of all the measurement errors 第 i 个残差是全部量测误差的联合作用,残差灵敏度矩阵的性质,The degree of interaction is ascertained by the magnitude of element Wi,k . Only if the diagonal element of matrix W is largest in its row,the residual can correctly reflect corresponding error. 相互影响程度，由灵敏度矩阵Wi,k的大小来确定。只有W矩阵对角占优, 残差才能反映误差的大小。 W is related to network configuration as well as measurement system allocation. At the place of poor measurement allocation, diagonal element may not be lager. In the extreme condition, m=n, and r=0. W和与电网结构和量测配置有关，量测配置薄弱的地方甚至无优势，极限情况m=n, 残差r=0。 The ith error will affect other residuals according to wji 第 i 个误差将会依据wji的大小影响其它残差,ri = wi vi,残差灵敏度矩阵的性质,ri =wi Wii vi,9.3.2 detection method 不良数据的检测方法,The requirement for detection: minimize the number of suspicious data as possible, in condition of not letting any bad data undetected 检测要求:不漏掉不良数据的条件下,尽可能缩小可疑数据范围 primary detection 粗检测: SCADA detection by residual 残差检测 detection by sudden measurement change 量测量突变 the combine detection by residual and sudden measurement change 残差与突变联合检测,9.3.2.1. Primary detection in SCAD 粗 检测 The evident bad data can be removed by limit value detection, as example, the voltage value of 1.4p.u. SCADA中进行极限值检测，去掉明显不良数据,如电压功率明显偏大,9.3.2.2. Detection by residual: 残差检测： Weighted residual 加权残差 rw=R-1/2r |rw,i|w,i w,i is the threshold of weighted residual 加权残差检测的门槛值，Ri=i2 is the measurement variance. The larger the variance, the smaller part the measurement takes in estimation . Therefore larger weighted residual could be treated as bad one. 量测方差，方差大，估计中所占比重小，残差较大才认为是坏数据 Normalized residual 标准化残差 rN=D-1/2r |rN,i|N,i w,i is the threshold for normalized residual 标准化残差检测的门槛值， D=diagWR The detection by rN has better properties for single bad data identification, but it needs more calculation of D. 标准化残差检测单个不良数据的性能好，计算量大,9.3.2.3 Measurement sudden change-overcome the submerge of residuals 量测突变检测-克服残差淹没,While there are two bad data , i and k, 两个坏数据,误差vi ,vk的绝对值较大 vi and vk are large in absolute values. The ith residual will be 第I个残差将是 r i Wiivi+ Wikvk If Wii and Wik are similar in value, 若Wii和Wik数值接近，符号相反 but opposite in sign, we have Wii -Wik ， The residual r i will be small in absolute value, 当vi 和 vk近似相等时，when vi and vk are nearly equal simultaneously 的绝对值将会很小 This case is called residual submerge 叫作残差淹没,0,Example :,When P1 and P12 are bad data (不良数据), their residuals are (其残差) r1=1.3MW and r12 =1.1MW much less than the other residuals 小于其他 残差 r13=-4.4MW r31=-4.2MW r32=-4.9MW,Precondition for detection of measurement sudden change : no topology changes and state changes , and also the data in the previous intervals are reliable,量测量突变检测前提:无拓扑等运行状态变化,前一时刻数据可靠,Take the measurement change between two sampling C i=z i (k) - z i (k-1) |C i|c k is the number of time interval c is the threshold,量测量两次采样的变化量 C i=z i (k) - z i (k-1) |C i|c k 为采样序号， c 为突变检测门槛值,The advantage of this method is that it dose not influenced by multi bad data, but it is necessary that there are not larger operating state changes between two time intervals.,突变检测的优点是不受多个坏数据的影响，但两次采样的运行状态没有大的变化,量测突变检测,9.3.2.4. The combinatorial detection of residual and sudden measurement change 残差与突变联合检测,Combinatorial detecting index 联合检测指标S S i=Krw | rw,i|+K cw |C w,i | k Krw is one combinatorial coefficient concerning weighing residual 为联合检测中加权残差联合系数， Kcw is another combinatorial coefficient concerning sudden measurement change 为联合检测中加权突变联合系数， C w,iweighted sudden changing variable 加权突变量， C w,i = Ri-1/2C i When Krw=1，Kcw=0，it is the weighted residual detection 即为加权残差检测 When Krw=0，Kcw=1，it is the sudden measurement change detection 即为加权突变检测,9.3.3 The identification of bad data 不良数据的辨识方法,The identification method Residual searching method 残差搜索法 Non quadratic criterion 非二次准则法 Zero residual approach 零残差法 Estimate identification on the whole总体型估计辨识法 Recursive estimation identification 逐次型估计辨识法 The difference and characteristic区别和特点 Identification by individual or on the whole 逐个辨识或总体辨识 Change weighing factor, residual or eliminate bad data directly 变权重、变残差或直接删除坏数据 Going to iteration again or updating residual and estimated results by linear approach 迭代或线性修正残差和状态估计结果,9.3.3.1 Residual searching approach 残差搜索法,Arrange the weighted residuals according to their absolute values 加权残差按大小排队 Removal the largest one and run state estimation again 把最大的去掉，再进行状态估计 Characteristic: slow, and it is hard to be used in multi bad data condition 特点： 慢,多不良数据无法实时应用,改变不良数据的权重：在迭代过程中排除坏数据的影响. 残差大，权重小; 再次迭代时，估计结果进一步向权重大的测量倾斜， 坏数据残差更大，权重进一步减小，最终排除其影响,Let the weighing factor for the left be 0.5, and it is equivalent to half a measurement. There are equivalent 5 measurement in all. Average error is 1/5=0.2. Results as follows. 令左侧权重为0.5, 相当于半个测量, 总共相当于5个测量 平均分配1/5=0.2, 估计值为1.8, 残差为-0.8 和-0.2 1.8 est 估计值 -1.8 -.8 r 残差 -0.2 -1.8 -0.2,例 1 meas测量值 -2 1.67 est估计值 -1.67 -.67 r 残差 -0.33 -2 -1.67 -0.33,Changing the weighing factor of bad data, The bad datas influence can be excluded in the process of iteration When the residual is large, the weighing factor becomes less. In the next iteration, the estimate results will incline to the measurement with lager weighing. ， The residual of bad data will be larger, the weighing factor becomes less further. And its influence is vanished at last.,9.3.3.2 non quadratic criterion非二次准则,The original object function is quadratic with respect to residual. 原来目标函数是残差的二次方 While the residual is large, it changes to a constant or linear one for the large residual. Therefore its contribution to the object function will becomes less and less 在残差大时,目标函数改为常数,或一次方 J r / It belongs to a robust estimation against measurement errors, but it requires more iteration and is difficult to converge 抗差估计，迭代量大，不易于收敛,Illustration 说明,9.3.3.3The method of zero residual 零残差法,Not changing weighing factor, but putting the residual violating the threshold to be zero instead 不改变权重，而是把残差超过门槛值的置零, ri =0 When a residual is zero, it will not play any effect to the next iteration, and x will turns to the normal correct value slowly like the non quadratic approach 某一残差ri为零，不会在下次迭代中影响 x, x慢慢回到正常值 Advantage: the matrix H is not changed, and it is suitable for the fast decoupled state estimation. 优点： H不变，适用于快速分解状态估计 Disadvantage: the convergence is not good enough 缺点：收敛性能欠佳,9.3.3.4 Estimate identification on the whole总体型估计辨识法 r =Wv z=h(x)+v find the error v from given r and W according to the residual equation, 根据残差方程，已知r和W,求出v Since the rank of mm residual sensitivity matrix W is k=m-n which is less than expected, it only identifies k bad data. 由于mm阶灵敏度矩阵W的秩小，为k=m-n，只能识别k个坏数据, Moreover, not any k-measurements can be identified, especially when the measurements are not distributed evenly. 而且还不是任意的k个（当测量量不均匀分布时）,n=7 m=16 k=m-n=9 but the two bad data can not be identified. 坏数据不能识别,Bad data,If there are p bad data, and a reliable detecting system has detected S suspicious data. 若有p个坏数据，可靠的检测系统检测出S个可疑数据，即 pS, pSk k=m-n The corresponding error vs can be estimated from their suspicious residual rs and the corresponding part of the residual sensitivity matrix Ws. 由可疑数据对应的残差 rs ，通过残差灵敏度矩阵的对应部分Ws作误差向量vs的估计， where Ws is ms matrix with respect to those suspicious measurements, G-1 is m m weighing diagonal matrix Ws为 ms阶矩阵,和可疑量测对应，G-1为m m阶加权对角阵,In state estimation 状态估计 z-h(x）=v, z-Hx=v,State estimation: the diagonal elements of R are the variance of v. 状态估计 R是取误差v的方差为对角元素 Estimate identification: the diagonal elements of G are the variance of W t vt. 估计辨识法 G取 W t vt的方差为对角元素 It is a very good approach, since it can identify bad data directly by solving the equation. 可以直接识别坏数据，方法很好 However the bad data should be included in the suspicious set S, and it is hard to detect accurately. 不良数据要包含在S集合中，难于检测准确,m (m-s),m s,9.3.3.5 Recursive 逐次型估计辨识法（递归识别法）,The measurements are divided into two groups: good one and bad one . They are arranged in a descending order with respect to the absolute value of residuals. 测量值分为两组，好数据和坏数据, 残差大到小排列 good bad larger residual 残差大 残差大 less residual 残差小 残差小 The linear equations are given, so the verification of residual and state variables can be calculated without iteration , so it is very fast in speed. 主要是推出了公式，可直接修正残差和状态量，速度很快,9.4. Topology error identification 拓扑错误识别,Measurement error - bad data 遥测有误，为不良数据 Status error- topology error 遥信有误，为拓扑错误 Topology error identification 拓扑错误识别 Topology structure determination 拓扑结构识别 - 结线分析 system configuration Classification of topology error 拓扑错误分类 Reported : breaker true status unchanged, but logical data changed 报告拓扑错误 实际开关状态未变，遥信表明变化 Unreported : breaker true status changed, but logical data unchanged 未报告拓扑错误 实际开关状态变化，遥信表明未变 Topology error at branch 支路型拓扑错误 Topology error at bus 节点型拓扑错误,Sample time k Original network 时刻k 初始网络,Sample time k+1 topology is changed 时刻 k+1, 拓扑变化,Unreported topology error. Using the original network results in large number of residual 拓扑错误导致矛盾方程残差大,Measurement value测量值,0.5,Identifying topology error is more difficult than that of bad data, and topology error makes lager influence to system 拓扑错误比不良数据更难于识别，对系统的影响也更大,Identifying topology error is more difficult than that of bad data, and topology error makes lager influence to the system 拓扑错误比不良数据更难于识别，对系统的影响也更大 Searching approach -select a branch with the largest residual, alternate its open or closed state and go on state estimation again . Slow in speed. 搜索法 -挑出一个残差大的支路，改变开断状态，重新进行状态估计，很慢,9.4.1Detect and identify based on the measurements 根据量测量进行检测识别,If the measurement is zero, the branch should be open. 测量值为零-断开 Advantage: simple; disadvantage: it will not work with bad data at the same time 优点：简单，缺点：坏数据，无法进行,If