cs230ppt深度学习cs230pptc2m2_W

1、deeplearning.aiOptimizationAlgorithmsMini-batch gradient descentBatch vs. mini-batch gradient descentVectorization allows you to efficiently compute on m examples.Andrew NgMini-batch gradient descent deeplearning.aiOptimizationAlgorithmsUnderstanding mini-batch gradient descentTraining with mini bat

2、ch gradient descentcostcostBatch gradient descentMini-batch gradient descent# iterationsmini batch # (t)Andrew NgChoosing your mini-batch sizeChoosing your mini-batch sizeAndrew NgAndrew Ngdeeplearning.aiOptimizationAlgorithmsUnderstanding exponentially weighted averagesdaystemperature! = $!%& + (1

3、$),Exponentially weighted averagesAndrew NgExponentially weighted averages!/ = 0!/1 + (1 0)+/!# = 0.9!( + 0.1+#!( = 0.9!(, + 0.1+(!(, = 0.9!(- + 0.1+(,Implementing exponentially weighted averages! = 0!% = &! + (1 &) -%!/ = &!% + (1 &) -/!0 = &!/ + (1 &) -0deeplearning.aiOptimizationAlgorithmsBias co

4、rrection in exponentiallyweighted averagedaystemperatureBias correction! = $!%& + (1 $),Andrew Ngdeeplearning.aiOptimizationAlgorithmsGradient descent with momentumGradient descent exampleAndrew NgImplementation detailsOn iteration 8:Compute )*, ),on the current mini-batch!#= %!# +1 % ),!+= %!+ +1 %

5、 )*= * -!#, = , -!+Hyperparameters: -, % = 0.9deeplearning.aiOptimizationAlgorithmsRMSpropRMSpropAndrew Ngdeeplearning.aiOptimizationAlgorithmsAdam optimization algorithmAdam optimization algorithm yhat = np.array(.9, 0.2, 0.1, .4, .9)Andrew NgHyperparameters choice:Adam Coatesdeeplearning.aiOptimizationAlgorithmsLearning rate decayLearning rate decayAndrew NgLearning rate decay Other learning rate decay methodsdeeplearning.aiOptimizationAlgorithmsThe problem of local optimaLocal optima in neural networksAndrew NgProb