16-卷积神经网络背景

DeepLearninganditsApplicationConvolutionalneuralnetworksFrankRosenblatt,~1957:

PerceptronTheMarkIPerceptronmachinewasthe

first

implementationoftheperceptron

algorithm.Themachinewasconnectedtoacamerathatused20×20cadmiumsulfidephotocellstoproducea400-pixelimage.recognizedlettersofthealphabetupdate

rule:Abitof

history...2Thisimage

byRockyAcostaislicensedunderCC-BY

3.0Rosenblatt,Frank(1957),ThePerceptron--aperceivingandrecognizingautomaton.Report85-460-1,CornellAeronautical

Laboratory.SJTUDeepLearningLecture.Abitof

history...recognizable

mathIllustrationofRumelhartetal.,1986byLaneMcIntosh,copyrightCS231n

20176Rumelhartetal.,1986:Firsttimeback-propagationbecame

popularRumelhart,DavidE.;Hinton,GeoffreyE.;Williams,RonaldJ.(1986-10-09)."Learningrepresentationsbyback-propagatingerrors".Nature.323(6088):533–536.SJTUDeepLearningLecture.[HintonandSalakhutdinov

2006]Reinvigoratedresearch

inDeep

LearningAbitof

history...IllustrationofHintonandSalakhutdinov2006byLaneMcIntosh,copyrightCS231n

20177GeoffreyEHinton,RuslanRSalakhutdinov(2006)."Reducingthedimensionalityofdatawithneuralnetworks".science.

313(5786)SJTUDeepLearningLecture.Abitof

history:Hubel&

Wiesel,1959RECEPTIVEFIELDSOF

SINGLENEURONES

INTHECAT'SSTRIATE

CORTEX1962RECEPTIVEFIELDS,

BINOCULARINTERACTIONANDFUNCTIONALARCHITECTURE

INTHECAT'SVISUAL

CORTEX1968...9SJTUDeepLearningLecture.Abitof

historyTopographicalmappinginthe

cortex:nearbycellsincortexrepresentnearbyregionsinthevisual

fieldRetinotopyimagescourtesyofJesseGomezintheStanfordVision&PerceptionNeuroscience

Lab.10Human

brainVisualcortexSJTUDeepLearningLecture.Hierarchical

organizationIllustrationofhierarchicalorganizationinearlyvisualpathwaysbyLaneMcIntosh,copyrightCS231n

201711SJTUDeepLearningLecture.Abitof

history:Neocognitron[KunihikoFukushima福岛邦彦

1980]“sandwich”architecture(SCSCSC…)simplecells:modifiableparameterscomplexcells:perform

pooling12SJTUDeepLearningLecture.http://personalpage.flsi.or.jp/fukushima/index-e.htmlInspiredbytheexperimentofHubel&

Wiesel,1959KunihikoFukushima,Neocognitron:ASelf-organizingNeuralNetworkModelforaMechanismofPatternRecognitionUnaffectedbyShiftinPosition,Biol.Cybernetics36,193202(1980)Abitofhistory:12SJTUDeepLearningLecture.Perceptron(1957)Neocognitron(198x)NIN(2013)InceptionV1(2014)InceptionV2/V3(2015)MSRANet(2014)ResNet(2015)ResNetV2(2015)InceptionResNetV2(2016)DenseNet2017…Fast-forwardtotoday:ConvNetsareeverywhereDetectionSegmentation[Farabetetal.,

2012]16SJTUDeepLearningLecture.Con