模式识别课件prch5part3_ding.ppt

1、,Pattern ClassificationAll materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley & Sons, 2000 with the permission of the authors and the publisher,5.8 Minimum Squared Error Procedures,Criterion function involves all of the sam

2、ples, not just misclassified ones Previously we were interested in making all of the inner products positive Now try to make where are some arbitrarily specified positive constants Thus replace the problem of solving a set of linear inequalities with more stringent but better understood problem of f

3、inding a solution to a set of linear equations,2,Minimum Squared Error and Pseudoinverse For all the samples we want a weight vector a so that for some arbitrarily specified positive numbers. The matrix notation : Error vector:,3,Sum-of-squared-error criterion function: The gradient Set it to zero,

4、we get If is nonsingular, The d by n matrix is call the pseudoinverse of Y. Remarks: For an arbitrarily fixed b, MSE solution may not be a separating vector.,4,5,The Widrow-Hoff or LMS Procedure Need not converge to a separating hyperplane even if there exist one,6,5.9 The Ho-Kashyap Procedure,Take

5、the criterion function as a function of two variables a and b: If the training samples are linearly separable, then there should exist an such that: If we knew such beforehand. We would get the separating vector using the MSE procedure,7,8,5.12 Multicategory Generalizations,Multicategory case Linear

6、ly-separable for Multicategory case there exists a set of weight vectors,9,Keslers Construction suppose that , we have,10,Remark: it allows us to convert many muylticategory error-correction procedures to two-category procedures for the purpose of obtaining a convergence proof Fixed-Increment Rule Let denote the kth sample requiring correction, and suppose that , there must be at least one j i for which then the fixed-increment rule for correcting is,11,Remark:Keslers construction can also be used to extend all of results for the Perceptron and relaxation procedure to the multic