台大-李宏毅-B站机器学习视频-课件神经网络与深度学习SVM v6

SupportVectorMachine,Outline,BinaryClassification,Step1:Functionset(Model)Step2:Lossfunction:Step3:Trainingbygradientdescentisdifficult,=,Thenumberoftimesggetincorrectresultsontrainingdata.,=+1,1,=,=,=,Step2:Lossfunction,Idealloss,Largervalue,smallerloss,Idealloss:,=,Approximation:,Step2:Lossfunction,Idealloss,Squareloss,Largervalue,smallerloss,=12,SquareLoss:,closeto1,不合理,closeto1,If=1,If=1,12,12,+12,Step2:Lossfunction,Idealloss,Squareloss,Largervalue,smallerloss,=12,Sigmoid+Squareloss,Sigmoid+SquareLoss:,closeto1,closeto0,If=1,If=1,12,12,112,2,Step2:Lossfunction,Idealloss,Squareloss,Largervalue,smallerloss,Sigmoid+crossentropy,=1+,Sigmoid+Squareloss,Sigmoid+crossentropy,(logisticregression),努力可以有回報,沒有回報不想努力,Dividedbyln2here,=+1,=1,1,1.0,GroundTruth,crossentropy,Step2:Lossfunction,Idealloss,Squareloss,Largervalue,smallerloss,=0,1,Sigmoid+Squareloss,Sigmoid+crossentropy,HingeLoss,及格就好,好還要更好,0,1,If=1,If=1,11,0,1+,1+0,1,penalty,Goodenough,LinearSVM,Step1:Function(Model)Step2:LossfunctionStep3:gradientdescent?,Comparedwithlogisticregression,linearSVMhasdifferentlossfunction,=+,=1,Neww,Newx,=,=0,1,=,Recallrelu,maxoutnetwork,YichuanTang,“DeepLearningusingLinearSupportVectorMachines”,ICML2013ChallengesinRepresentationLearningWorkshop,+2,convex,Deepversion:,ShaiShalev-Shwartz,YoramSinger,NathanSrebro,“Pegasos:PrimalEstimatedsub-GrAdientSOlverforSVM“,ICML,2007,LinearSVMgradientdescent,=0,1,=,=,=,0,1=,0,otherwise,If1,=0,LinearSVManotherformulation,=0,1,0,1,1,MinimizinglossfunctionL:,=,:slackvariable,QuadraticProgramming(QP)Problem,Outline,DualRepresentation,=,Linearcombinationofdatapoints,111,=,Hingeloss:usuallyzero,maybesparse,withnon-zeroaresupportvectors,c.f.forlogisticregression,itisalwaysnon-zero,Ifwinitializedas0,DualRepresentation,=,=,1,2,=,=12,Step1:,=,=,=,12,1,=,=,DualRepresentation,Find1,minimizinglossfunctionL,Step2,3:,=,Step1:,=,=,Wedontreallyneedtoknowvectorx,Weonlyneedtoknowtheinnerprojectbetweenapairofvectorsxandz,KernelTrick,KernelTrick,Directlycomputing,canbefasterthan“featuretransformation+innerproduct”sometimes.,=12,=1221222,=1212+21212+2222,=11+222,=12122,=2,Kerneltrickisusefulwhenwetransformallxto,=,=12212221221222,KernelTrick,=1,=122212213223,=1,=11+22+2,=2,Directlycomputing,canbefasterthan“featuretransformation+innerproduct”sometimes.,=1212+2222+22,+21212+21313+,+22323+22424+,=,=122,=?,hasinfdim!,=122122+,=122122,=,=0!,RadialBasisFunctionKernel,=+122,=12,=12,1212,12122121212212,SigmoidKernel,Whenusingsigmoidkernel,wehavea1hiddenlayernetwork.,=,=,Thenumberofsupportvectorsisthenumberofneurons.,Theweightofeachneuronisadatapoint,x,1,2,2,1,=,AudioSegment,AudioSegment,HiroshiShimodaira,Ken-ichiNoma,MitsuruNakai,ShigekiSagayama,“DynamicTime-AlignmentKernelinSupportVectorMachine”,NIPS,2002,MarcoCuturi,Jean-PhilippeVert,OysteinBirkenes,TomokoMatsui,Akernelfortimeseriesbasedonglobalalignments,ICASSP,2007,Youcandirectlydesign,insteadofconsidering,Whenxisstructuredobjectlikesequence,hardtodesign,issomethinglikesimilarity,(Mercerstheorycancheck),MoreaboutkerneldesigninBishopchapter6.2,Evaluatethesimilaritybetweensequencesas,SVMrelatedmethods,SupportVectorRegression(SVR)Bishopchapter7.1.4RankingSVMAlpaydin,Chapter13.11One-classSVMAl