常见的深度学习deep-learning开发平台介绍

IntroductionofCurrentDeepLearningSoftwarePackages,ThreePopularones,1.Caffe/2.Theano/pypi/Theano3.TensorFlow/Thesewebsitesprovideinformationabouthowtoinstallandrunrelateddeeplearningsoftware.,1.Caffe,1.Overview:Caffe(ConvolutionalArchitectureForFeatureExtraction)CreatedbyYangqingJia(贾扬清),UCBerkeley.WritteninC+,hasPythonandMATLABinterface.2.Githubpage:,4.Installmethod(CUDA+Caffe):Ouxinyu.github.io/Blogs/2014723001.html,AnatomyofCaffe,Blob:StoresdataandderivativesLayer:TransformsBottomblobstotopblobsNet:Manylayers;computesgradientsviaforward/backward,Blob,Layer,Net,Blob,ABlobisawrapperovertheactualdatabeingprocessedandpassedalongbyCaffe,andalsounderthehoodprovidessynchronizationcapabilitybetweentheCPUandtheGPU.,Theconventionalblobdimensionsforbatchesofimagedataare(numberN)x(channelK)x(heightH)x(widthW).,Foraconvolutionlayerwith96filtersof11x11spatialdimensionand3inputstheblobis96x3x11x11.Foraninnerproduct/fully-connectedlayerwith1000outputchannelsand1024inputchannelstheparameterblobis1000 x1024.,Layer,Thelayeristheessenceofamodelandthefundamentalunitofcomputation.Layersconvolvefilters,pool,takeinnerproducts,applynonlinearitieslikerectified-linearandsigmoidandotherelement-wisetransformations,normalize,loaddata,andcomputelosseslikesoftmaxandhinge.,Case：ConvolutionLayer,Net,Thenetjointlydefinesafunctionanditsgradientbycompositionandauto-differentiation.Thecompositionofeverylayersoutputcomputesthefunctiontodoagiventask,andthecompositionofeverylayersbackwardcomputesthegradientfromthelosstolearnthetask.,name:LogReglayername:mnisttype:Datatop:datatop:labeldata_paramsource:input_leveldbbatch_size:64layername:iptype:InnerProductbottom:datatop:ipinner_product_paramnum_output:2layername:losstype:SoftmaxWithLossbottom:ipbottom:labeltop:loss,HowtouseCaffe?,Just4steps!1.Convertdata(runascript)2.Definenet(editprototxt)3.Definesolver(editprototxt)4.Train(withpretrainedweights)(runascript)TakeCifar10imageclassificationforexample.,DataLayerreadingfromLMDBistheeasiest,createLMDBusingconvert_imagesetNeedtextfilewhereeachlineis“path/to/image.jpeglabel”(useimageDataLayerread)CreateHDF5fileyourselfusingh5py(useHDF5Layerread),Step1:ConvertDataforCaffe,ConvertDataonCIFAR10,Step2:DefineNet(cifar10_quick_train_totxt),Layername,Blobsname,Learningrateofweight,Learningrateofbias,Inputimagenumperiteration,Trainingimagedata,Datatype,Blobsname,Numberofoutputclass,Outputaccuracyduringtest,Outputlossduringtrain,Ifyoufinetunesomepre-trainmodel,youcansetlr_mul=0,Step2:DefineNet(cifar10_quick_train_totxt),VisualizetheDefinedNetwork,http:/ethereon.github.io/netscope/#/editor,Step3:DefineSolver(cifar10_quick_totxt),#reducethelearningrateafter8epochs(4000iters)byafactorof10#Thetrain/testnetprotocolbufferdefinitionnet:examples/cifar10/cifar10_quick_train_totxt“#test_iterspecifieshowmanyforwardpassesthetestshouldcarryout.#InthecaseofMNIST,wehavetestbatchsize100and100testiterations,#coveringthefull10,000testingimages.test_iter:100#Carryouttestingevery500trainingiterations.test_interval:500#Thebaselearningrate,momentumandtheweightdecayofthenetwork.base_lr:0.001momentum:0.9weight_decay:0.004#Thelearningratepolicylr_policy:fixed“#Displayevery100iterationsdisplay:100#Themaximumnumberofiterationsmax_iter:4000#snapshotintermediateresultssnapshot:4000snapshot_prefix:examples/cifar10/cifar10_quick“#solvermode:CPUorGPUsolver_mode:GPU,DefinedNetfile,Keyparameters,Importantparameters,Step4:Train,Writeashellfile(train_quick.sh):,Thenenjoyacupofcaffe,ModelZoo(Pre-trainedModel+Finetune),Wecanfinetunethesemodelsordofeatureextractionbasedonthesemodels,Sometricks/skillsabouttrainingCaffe,1NeuralNetworks:tricksofthetrade,1.DataAugmentationtoenlargetrainingsamples2.ImagePre-Processing3.NetworkInitializations4.DuringTraining5.ActivationFunctions6.Regularizationsmoredetailscanreferto1,2,2,DataAugmentation,DataAugmentation,Veryusefulforfaceandcarrecognition!,DataAugmentation,Togetridofocclusionandscalechange,likevisualtracking,DataAugmentation,DataAugmentation,ImagePre-Processing,Step1:subtractthedataset-meanvalueineachchannel,Step2:swapchannelsfromRGBtoBGR,Step3:moveimagechannelstooutermostdimension,Step4:rescalefrom0,1to0,255,NetworkInitializations,DuringTraining,helpalleviateoverfittingduringtraininginCaffe,1Srivastava,Nitish,etal.Dropout:asimplewaytopreventneuralnetworksfromoverfitting.JournalofMachineLearningResearch15.1(2014):1929-1958.2S.IoffeandC.Szegedy.Batchnormalization:Acceleratingdeepnetworktrainingbyreducinginternalcovariateshift.arXivpreprintarXiv:1502.03167,2015,Overfitting,ProsandConsofCaffe,ApracticalexampleofCaffe,ObjectdetectionRCNN/Fast-RCNN/Faster-RCNNCaffe+MATLAB,lr=0.1xbaselearningrate,lr=baselearningrate,2.Theano,1.Overview:APythonlibrarythatallowstodefine,optimizeandevaluatemathematicalexpression.FromYoshuaBengiosgroupatUniversityofMontreal.Embracingcomputationgraphs,symboliccomputation.High-levelwrappers:Keras,Lasagne.2.Github:,ProsandConsofTheano,3.TensorFlow,1.Overview:VerysimilartoTheano-allaboutcomputationgraphs.Easyvisualizations(TensorBoard).Multi-GPUandmulti-nodetraining.2.Tutorial:http:/terryum.io/ml_practice/2016/05/28/TFIntroSlides/,LoaddataDefinetheNNstructureSetoptimizationparametersRun!,BasicFlowofTensorFlow,1.Loaddata,1.Loaddata,2.DefinetheNNstructure,3.Setoptimizationparameters,4.RUN,TheProsandConsofTensorFlow,Ove