Fisher线性分类器.ppt

1、2020/7/23,ICL,1,Fishers Linear Classifier,Machine Learning Seminar,2020/7/23,ICL,2,Content,Classifier, Linear Classifier Fishers Discriminant Function The best project vector Comparison with Linear SVM Conclusion,2020/7/23,ICL,3,Classification Problem,Classification Problem: Mapping M: X-C X is the

2、Instances of the problem C is the known set of class Feature Vector For every xX, x is represented as vector (x1,xn) xi (1 i n) is called feature of x,2020/7/23,ICL,4,Classifier,Classifiers task (narrow sense) Given a classified sample X=x1,xN and C=c1,ck, Mapping? Cluster Given a unclassified sampl

3、e X and the count of classes cluster unconsidered in this report,2020/7/23,ICL,5,Linear Classifier,Find n linear function gi(x) i=1,n M(x)= argmaxgi(x) i,2020/7/23,ICL,6,Project Vector,2020/7/23,ICL,7,Discriminant Function,Different Criterion Fisher Perceptron SVM ,2020/7/23,ICL,8,2-Classes,Simple-C

4、omplex Consider 2-Classes problem 2-Classes problem X-0,1 Concept learning,2020/7/23,ICL,9,Ronald Aylmer Fisher(1890-1962),Brilliant biologists Development of methods suitable for small samples Discovery of the precise distributions of many sample statistics Invention of analysis of variance.,2020/7

5、/23,ICL,10,2020/7/23,ICL,11,Fishers Discriminant,Fishers Discriminant Function (FDF) Fishers Principle Maximize JF(w),2020/7/23,ICL,12,Numerical Characters,Class Mean Vector Inter Divergence Matrix Class Divergence Matrix,2020/7/23,ICL,13,Transformation,y1-y2=(wTx1+w0)-(wTx2+w0) = wT(x1-x2) (m1-m2)2

6、= wTSbw D12+D22=wT(S1+S2)w= wTSw,2020/7/23,ICL,14,Find the best “w”,Optimization problem Mathematic solution Lagrange Lemma of Maximization,2020/7/23,ICL,15,Lagrange method,Let wTSw=c0 L(w,)= wTSbw wTSw = Sbw* Sw*=0 = Sbw* =Sw* = S-1Sbw* =w* 1.The eigenvector of S-1Sb 2.w*= S-1(m1-m2),2020/7/23,ICL,

7、16,The best “w”,Lemma of Maximization Despite of scale, w* is w*=S-1(m1-m2),2020/7/23,ICL,17,Remain Problem,High dimension - 1-D w*-ok, what about w0? w0 is a threshold g(x)= w*x+w0 If g(x)0, xC1; If g(x)0, xC2 If g(x)=0?,2020/7/23,ICL,18,Strategy to determine w0,Several advised method y0=(m1+ m2)/2 y0=(N1m1+ N2m2)/(N1+N2) ,2020/7/23,ICL,19,FDF VS SVM,Fishers Linear Classification Maximize JF(w) Linear SVM Maximize “margin”,2020/7/23,ICL,20,2020/7/23,ICL,21,Conclusion,Provide an app