16-卷积神经网络基础下

DeepLearninganditsApplicationConvolutionalneuralnetworks下Acloserlookatspatial

dimensions:32332x32x3

image5x5x3

filter32convolve(slide)over

allspatial

locationsactivation

map3212828SpatialdimensionsSJTUDeepLearningLecture.33Acloserlookatspatial

dimensions:77x7input

(spatially)assume3x3

filter7SpatialdimensionsSJTUDeepLearningLecture.Spatialdimensions34Acloserlookatspatial

dimensions:77x7input

(spatially)assume3x3

filter7SJTUDeepLearningLecture.onebyone=>5x5

output357Acloserlookatspatial

dimensions:77x7input

(spatially)assume3x3

filterSpatialdimensionsSJTUDeepLearningLecture.7x7input(spatially)assume3x3filterappliedwithstride

23677Acloserlookatspatial

dimensions:SpatialdimensionsSJTUDeepLearningLecture.7x7input(spatially)assume3x3filterappliedwithstride

23777Acloserlookatspatial

dimensions:SpatialdimensionsSJTUDeepLearningLecture.7x7input(spatially)assume3x3filterappliedwithstride

2=>3x3

output!3877Acloserlookatspatial

dimensions:Spatial

dimensionsSJTUDeepLearningLecture.7x7input(spatially)assume3x3filterappliedwithstride

3?3977Acloserlookatspatial

dimensions:doesn’t

fit!cannotapply3x3filter

on7x7inputwithstride

3.SpatialdimensionsSJTUDeepLearningLecture.NNFF40Output

size:(N-F)/stride+

1e.g.N=7,F=

3:stride1=>(7-3)/1+1=

5stride2=>(7-3)/2+1=

3stride3=>(7-3)/3+1=2.33SpatialdimensionsSJTUDeepLearningLecture.Inpractice:Commontozeropadtheborder410000000000e.g.input

7x73x3filter,appliedwithstride

1padwith1pixelborder=>whatisthe

output?(recall:)(N-F)/stride+

1SJTUDeepLearningLecture.Inpractice:Commontozeropadtheborder420000000000e.g.input

7x73x3filter,appliedwithstride

1padwith1pixelborder=>whatisthe

output?7x7

output!SJTUDeepLearningLecture.•43Inpractice:Commontozeropadthebordere.g.input

7x73x3filter,appliedwithstride

1padwith1pixelborder=>whatisthe

output?7x7

output!ingeneral,commontoseeCONVlayerswithstride1,filtersofsizeFxF,andzero-padding

with(F-1)/2.(willpreservesize

spatially)e.g.

F=3=>zeropadwith

1

F=5=>zeropadwith

2

F=7=>zeropadwith

30000000000SJTUDeepLearningLecture.Rememberback

to…E.g.32x32inputconvolvedrepeatedlywith5x5filtersshrinksvolumesspatially!(32->28->24...).Shrinkingtoofastisnotgood,doesn’twork

well.32323CONV,ReLUe.g.65x5x3filters28286CONV,ReLUe.g.

105x5x6filtersCONV,ReLU44….102424Inpractice:CommontozeropadtheborderSJTUDeepLearningLecture.Examplestime:Inputvolume:

32x32x3105x5filterswithstride1,pad

2Outputvolumesize:

?Outputvolume

size:(32+2*2-5)/1+1=32spatially,

so32x32x10Numberofparametersinthis

layer?45SJTUDeepLearningLecture.Examplestime:Inputvolume:

32x32x3105x5filterswithstride1,pad

2(+1for

bias)Numberofparametersinthislayer?eachfilterhas5*5*3+1=76

params=>76*10=76046SJTUDeepLearningLecture.30721Reminder:FullyConnected

Layer32x32x3image->stretchto3072x

110x

3072weightsactivationinput1

number:theresultoftakingadotproductbetweenarowofWandthe

input(a3072-dimensionaldot

product)110Eachneuronlooksatthe

fullinput

volume53SJTUDeepLearningLecture.twomorelayerstogo:

POOL/FC54Pooling

layerSJTUDeepLearningLecture.57

SJTUDeepLearningLecture.PoolinglayerPooling

layer55makestherepresentationssmallerandmore

manageable(arguablymorerobustagainstnoise,andcapturetheinvariantinformation)operatesovereachactivationmap

independently:SJTUDeepLearningLecture.1124567832101234xmaxpoolwith2x2

filtersandstride

2566834MAX

POOLING