台大-李宏毅-B站机器学习视频-课件神经网络与深度学习BP

Backpropagation,GradientDescent,StartingParameters,Millionsofparameters,Tocomputethegradientsefficiently,weusebackpropagation.,Networkparameters,ChainRule,Case1,Case2,Backpropagation,=1,=1,1,2,1,2,Backpropagation,1,=?,(Chainrule),2,Forwardpass:,Computeforallparameters,Backwardpass:,Computeforallactivationfunctioninputsz,1,2,=11+22+,BackpropagationForwardpass,1,2,1,2,=11+22+,1=?,1,Computeforallparameters,2=?,2,Thevalueoftheinputconnectedbytheweight,BackpropagationForwardpass,1,-2,1,-1,0.98,0.12,2,-1,-1,-2,3,-1,4,-1,0.86,0.11,1,-1,Computeforallparameters,=1,=0.12,=0.11,BackpropagationBackwardpass,Computeforallactivationfunctioninputsz,1,2,1,2,3,4,=,=,BackpropagationBackwardpass,Computeforallactivationfunctioninputsz,1,2,1,2,3,4,=,=,BackpropagationBackwardpass,Computeforallactivationfunctioninputsz,1,2,1,2,3,4,=,=,=+,(Chainrule),=3+,3,4,?,?,Assumeditsknown,1,2,1,2,3,4,=3+4,BackpropagationBackwardpass,Computeforallactivationfunctioninputsz,3,=3+4,BackpropagationBackwardpass,4,isaconstantbecausezisalreadydeterminedintheforwardpass.,1,2,1,2,3,4,BackpropagationBackwardpass,Computeforallactivationfunctioninputsz,1,2,Case1.OutputLayer,=11,=22,Done!,BackpropagationBackwardpass,Computeforallactivationfunctioninputsz,Case2.NotOutputLayer,BackpropagationBackwardpass,Computeforallactivationfunctioninputsz,Case2.NotOutputLayer,5,6,BackpropagationBackwardpass,Computeforallactivationfunctioninputsz,Case2.NotOutputLayer,5,6,Untilwereachtheoutputlayer,Computerecursively,BackpropagationBackwardPass,Computefromtheoutputlayer,Computeforallactivationfunctioninputsz,1,2,1,2,1,2,3,4,5,6,6,5,4,3,2,1,BackpropagationBackwardPass,Computefromtheoutputlayer,Computeforallactivationfunctioninputsz,1,2,1,2,3,4,5