台大-李宏毅-B站机器学习视频-课件神经网络与深度学习tsne

UnsupervisedLearning:NeighborEmbedding,ManifoldLearning,Suitableforclusteringorfollowingsupervisedlearning,LocallyLinearEmbedding(LLE),representstherelationbetweenand,Findasetofminimizing,2,Thenfindthedimensionreductionresultsandbasedon,LLE,Sourceofimage:http:/feetsprint.blogspot.tw/2016/02/blog-post_29.html,LLE,OriginalSpace,New(Low-dim)Space,Findasetofminimizing,Keepunchanged,2,LLE,LawrenceK.Saul,SamT.Roweis,“ThinkGlobally,FitLocally:UnsupervisedLearningofLowDimensionalManifolds”,JMLR,2013,LaplacianEigenmaps,Graph-basedapproach,Constructthedatapointsasagraph,Distancedefinedbygraphapproximatethedistanceonmanifold,LaplacianEigenmaps,Reviewinsemi-supervisedlearning:If1and2arecloseinahighdensityregion,1and2areprobablythesame.,=,+,Asaregularizationterm,=,=12,2,L:(R+U)x(R+U)matrix,GraphLaplacian,=,Sevaluateshowsmoothyourlabelis,LaplacianEigenmaps,DimensionReduction:If1and2arecloseinahighdensityregion,1and2aretheclosetoeachother.,=12,2,Spectralclustering:clusteringonz,Anyproblem?,Howabout=?,Givingsomeconstraintstoz:,IfthedimofzisM,Spanz1,z2,zN=RM,Belkin,M.,Niyogi,P.Laplacianeigenmapsandspectraltechniquesforembeddingandclustering.Advancesinneuralinformationprocessingsystems.2002,T-distributedStochasticNeighborEmbedding(t-SNE),ProblemofthepreviousapproachesSimilardataareclose,butdifferentdatamaycollapse,LLEonMNIST,LLEonCOIL-20,t-SNE,Computesimilaritybetweenallpairsofx:,Computesimilaritybetweenallpairsofz:,|=,|=,Findasetofzmakingthetwodistributionsascloseaspossible,=|,=|,t-SNESimilarityMeasure,=2,=11+2,SNE:,t-SNE:,2,2,=2,2,11+2,Ignoreforsimplicity,t-SNE,Goodatvisualization,t-SNEonMNIST,t-SNEonCOIL-20,Tolearnmore,LocallyLinearEmbedding(LLE):Alpaydin,Chapter6.11LaplacianEigenmaps:Alpaydin,Chapter6.12t-SNEL