常见的深度学习(deep-learning)开发平台介绍PPT课件

1,IntroductionofCurrentDeepLearningSoftwarePackages,2,ThreePopularones,1.Caffe/2.Theano/pypi/Theano3.TensorFlow/Thesewebsitesprovideinformationabouthowtoinstallandrunrelateddeeplearningsoftware.,3,1.Caffe,1.Overview:Caffe(ConvolutionalArchitectureForFeatureExtraction)CreatedbyYangqingJia(贾扬清),UCBerkeley.WritteninC+,hasPythonandMATLABinterface.2.Githubpage:,4.Installmethod(CUDA+Caffe):Ouxinyu.github.io/Blogs/2014723001.html,4,AnatomyofCaffe,Blob:StoresdataandderivativesLayer:TransformsBottomblobstotopblobsNet:Manylayers;computesgradientsviaforward/backward,Blob,Layer,Net,5,Blob,ABlobisawrapperovertheactualdatabeingprocessedandpassedalongbyCaffe,andalsounderthehoodprovidessynchronizationcapabilitybetweentheCPUandtheGPU.,Theconventionalblobdimensionsforbatchesofimagedataare(numberN)x(channelK)x(heightH)x(widthW).,Foraconvolutionlayerwith96filtersof11x11spatialdimensionand3inputstheblobis96x3x11x11.Foraninnerproduct/fully-connectedlayerwith1000outputchannelsand1024inputchannelstheparameterblobis1000 x1024.,6,Layer,Thelayeristheessenceofamodelandthefundamentalunitofcomputation.Layersconvolvefilters,pool,takeinnerproducts,applynonlinearitieslikerectified-linearandsigmoidandotherelement-wisetransformations,normalize,loaddata,andcomputelosseslikesoftmaxandhinge.,7,Case：ConvolutionLayer,8,Net,Thenetjointlydefinesafunctionanditsgradientbycompositionandauto-differentiation.Thecompositionofeverylayersoutputcomputesthefunctiontodoagiventask,andthecompositionofeverylayersbackwardcomputesthegradientfromthelosstolearnthetask.,name:LogReglayername:mnisttype:Datatop:datatop:labeldata_paramsource:input_leveldbbatch_size:64layername:iptype:InnerProductbottom:datatop:ipinner_product_paramnum_output:2layername:losstype:SoftmaxWithLossbottom:ipbottom:labeltop:loss,9,HowtouseCaffe?,Just4steps!1.Convertdata(runascript)2.Definenet(editprototxt)3.Definesolver(editprototxt)4.Train(withpretrainedweights)(runascript)TakeCifar10imageclassificationforexample.,10,DataLayerreadingfromLMDBistheeasiest,createLMDBusingconvert_imagesetNeedtextfilewhereeachlineis“path/to/image.jpeglabel”(useimageDataLayerread)CreateHDF5fileyourselfusingh5py(useHDF5Layerread),Step1:ConvertDataforCaffe,11,ConvertDataonCIFAR10,12,Step2:DefineNet(cifar10_quick_train_totxt),Layername,Blobsname,Learningrateofweight,Learningrateofbias,Inputimagenumperiteration,Trainingimagedata,Datatype,Blobsname,13,Numberofoutputclass,Outputaccuracyduringtest,Outputlossduringtrain,Ifyoufinetunesomepre-trainmodel,youcansetlr_mul=0,Step2:DefineNet(cifar10_quick_train_totxt),14,VisualizetheDefinedNetwork,http:/ethereon.github.io/netscope/#/editor,15,Step3:DefineSolver(cifar10_quick_totxt),#reducethelearningrateafter8epochs(4000iters)byafactorof10#Thetrain/testnetprotocolbufferdefinitionnet:examples/cifar10/cifar10_quick_train_totxt“#test_iterspecifieshowmanyforwardpassesthetestshouldcarryout.#InthecaseofMNIST,wehavetestbatchsize100and100testiterations,#coveringthefull10,000testingimages.test_iter:100#Carryouttestingevery500trainingiterations.test_interval:500#Thebaselearningrate,momentumandtheweightdecayofthenetwork.base_lr:0.001momentum:0.9weight_decay:0.004#Thelearningratepolicylr_policy:fixed“#Displayevery100iterationsdisplay:100#Themaximumnumberofiterationsmax_iter:4000#snapshotintermediateresultssnapshot:4000snapshot_prefix:examples/cifar10/cifar10_quick“#solvermode:CPUorGPUsolver_mode:GPU,DefinedNetfile,Keyparameters,Importantparameters,16,Step4:Train,Writeashellfile(train_quick.sh):,Thenenjoyacupofcaffe,17,ModelZoo(Pre-trainedModel+Finetune),Wecanfinetunethesemodelsordofeatureextractionbasedonthesemodels,18,Sometricks/skillsabouttrainingCaffe,1NeuralNetworks:tricksofthetrade,1.DataAugmentationtoenlargetrainingsamples2.ImagePre-Processing3.NetworkInitializations4.DuringTraining5.ActivationFunctions6.Regularizationsmoredetailscanreferto1,2,2,19,DataAugmentation,20,DataAugmentation,Veryusefulforfaceandcarrecognition!,21,DataAugmentation,Togetridofocclusionandscalechange,likevisualtracking,22,DataAugmentation,23,DataAugmentation,24,ImagePre-Processing,Step1:subtractthedataset-meanvalueineachchannel,Step2:swapchannelsfromRGBtoBGR,Step3:moveimagechannelstooutermostdimension,Step4:rescalefrom0,1to0,255,25,NetworkInitializations,26,DuringTraining,helpalleviateoverfittingduringtraininginCaffe,1Srivastava,Nitish,etal.Dropout:asimplewaytopreventneuralnetworksfromoverfitting.JournalofMachineLearningResearch15.1(2014):1929-1958.2S.IoffeandC.Szegedy.Batchnormalization:Acceleratingdeepnetworktrainingbyreducinginternalcovariateshift.arXivpreprintarXiv:1502.03167,2015,Overfitting,27,ProsandConsofCaffe,28,ApracticalexampleofCaffe,ObjectdetectionRCNN/Fast-RCNN/Faster-RCNNCaffe+MATLAB,29,lr=0.1xbaselearningrate,lr=baselearningrate,30,31,32,33,2.Theano,1.Overview:APythonlibrarythatallowstodefine,optimizeandevaluatemathematicalexpression.FromYoshuaBengiosgroupatUniversityofMontreal.Embracingcomputationgraphs,symboliccomputation.High-levelwrappers:Keras,Lasagne.2.Github:,34,35,ProsandConsofTheano,36,3.TensorFlow,1.Overview:VerysimilartoTheano-allaboutcomputationgraphs.Easyvisualizations(TensorBoard).Multi-GPUandmulti-nodetraining.2.Tutorial:http:/terryum.io/ml_practice/2016/05/28/TFIntroSlides/,37,LoaddataDefinetheNNstructureSetoptimizationparametersRun!,BasicFlowofTensorFlow,38,1.Loaddata,39,1.Loaddata,40,2.DefinetheNNstructure,3.Setoptimizationparameters,41,4.RUN,42,TheProsandConsofTensorFlow,43,Ove