台大-李宏毅-B站机器学习视频-课件神经网络与深度学习auto

UnsupervisedLearning:DeepAuto-encoder,Auto-encoder,NNEncoder,NNDecoder,code,Compactrepresentationoftheinputobject,code,Canreconstructtheoriginalobject,Learntogether,28X28=784,Usually784,StartingfromPCA,Inputlayer,outputlayer,hiddenlayer(linear),Ascloseaspossible,Bottlenecklater,Outputofthehiddenlayeristhecode,encode,decode,InitializebyRBMlayer-by-layer,Reference:Hinton,GeoffreyE.,andRuslanR.Salakhutdinov.Reducingthedimensionalityofdatawithneuralnetworks.Science313.5786(2006):504-507,Ofcourse,theauto-encodercanbedeep,DeepAuto-encoder,InputLayer,Layer,Layer,bottle,OutputLayer,Layer,Layer,Layer,Layer,Code,Ascloseaspossible,1,1,2,2,Symmetricisnotnecessary.,DeepAuto-encoder,OriginalImage,PCA,DeepAuto-encoder,784,784,784,1000,500,250,30,250,500,1000,784,30,784,784,784,1000,500,250,2,2,250,500,1000,784,Pokmon,5:2880/tlkagk/pokemon/pca.html5:2880/tlkagk/pokemon/auto.htmlThecodeismodifiedfrom,Auto-encoderTextRetrieval,wordstring:“Thisisanapple”,1,1,0,1,1,0,Bag-of-word,Semanticsarenotconsidered.,VectorSpaceModel,document,query,Auto-encoderTextRetrieval,Bag-of-word,(documentorquery),LSA:projectdocumentsto2latenttopics,2000,500,250,125,2,Thedocumentstalkingaboutthesamethingwillhaveclosecode.,Auto-encoderSimilarImageSearch,RetrievedusingEuclideandistanceinpixelintensityspace,(ImagesfromHintonsslidesonCoursera),Reference:Krizhevsky,Alex,andGeoffreyE.Hinton.Usingverydeepautoencodersforcontent-basedimageretrieval.ESANN.2011.,Auto-encoderSimilarImageSearch,32x32,8192,4096,2048,1024,512,256,code,(crawlmillionsofimagesfromtheInternet),retrievedusing256codes,RetrievedusingEuclideandistanceinpixelintensityspace,Auto-encoderPre-trainingDNN,GreedyLayer-wisePre-trainingagain,Target,784,1000,1000,10,Input,output,Input,784,1000,784,W1,500,W1,Auto-encoderPre-trainingDNN,GreedyLayer-wisePre-trainingagain,Target,784,1000,1000,500,10,Input,output,Input,784,1000,W1,1000,1000,fix,1,1,W2,W2,Auto-encoderPre-trainingDNN,GreedyLayer-wisePre-trainingagain,Target,784,1000,1000,10,Input,output,Input,784,1000,W1,1000,fix,1,2,W2,fix,2,1000,W3,500,500,W3,Auto-encoderPre-trainingDNN,GreedyLayer-wisePre-trainingagain,Target,784,1000,1000,10,Input,output,Input,784,1000,W1,1000,W2,W3,500,500,10,output,W4,Randominit,Find-tunebybackpropagation,Auto-encoder,De-noisingauto-encoder,encode,decode,Addnoise,Ascloseaspossible,Vincent,Pascal,etal.Extractingandcomposingrobustfeatureswithdenoisingautoencoders.ICML,2008.,LearningMore-RestrictedBoltzmannMachine,Neuralnetworks5.1:RestrictedBoltzmannmachinedefinition,LearningMore-DeepBeliefNetwork,Neuralnetworks7.7:Deeplearning-deepbeliefnetwork,Auto-encoderforCNN,Convolution,Pooling,Convolution,Pooling,Deconvolution,Unpooling,Deconvolution,Unpooling,Ascloseaspossible,CNN-Unpooling,14x14,28x28,Sourceofimage:https:/leonardoaraujosantos.gitbooks.io/artificial-inteligence/content/image_segmentation.html,Alternative:simplyrepeattheval