21-卷积网络视觉应用上

ComputervisiontasksPARTiDeepLearninganditsApplicationSJTUDeepLearningLecture.1Practices:DataAugmentationSJTUDeepLearningLecture.2Practices:DataAugmentationChangethepixelswithoutchangingthelabelTrainontransformeddataVerywidelyusedHorizontal

flipsRandom

crops/scalesColor

jitter…Especially

useful

for

small

datasetsSJTUDeepLearningLecture.3Practices:DataAugmentationHorizontal

flipsSJTUDeepLearningLecture.4Practices:DataAugmentationRandom

crops/scalesTake

ResNet

as

an

example:Training:

sample

random

crops/scalesPickrandomLinrange[256,480]Resizetrainingimage,shortside=LSamplerandom224*224patchesTesting:averageafixedsetofcropsResizeimageat5scales:{224,256,384,480,640}Foreachsize,use10224*224crops:4corners+center,and

+flipsSJTUDeepLearningLecture.5Practices:DataAugmentationColor

jitterRandomly

jitter

contrastSJTUDeepLearningLecture.6Practices:

Transfer

LearningSJTUDeepLearningLecture.7Practices:

Transfer

LearningSJTUDeepLearningLecture.8Practices:

Transfer

LearningSJTUDeepLearningLecture.9Practices:

Transfer

LearningSJTUDeepLearningLecture.10ComputerVisionTasksClassificationClassification+LocalizationDetectionSegmentationSJTUDeepLearningLecture.11SJTUDeepLearningLecture.12CowGrassSkyTreesThisimageisCC0public

domainGrassCatSkyTreesLabeleachpixelintheimagewithacategorylabelDon’tdifferentiateinstances,onlycareaboutpixelsSemanticSegmentationSJTUDeepLearningLecture.13Full

imageExtract

patchClassify

centerpixelwith

CNNCowCowGrassProblem:Veryinefficient!Notreusingsharedfeatures

betweenoverlapping

patchesFarabetetal,“LearningHierarchicalFeaturesforSceneLabeling,”TPAMI

2013PinheiroandCollobert,“RecurrentConvolutionalNeuralNetworksforSceneLabeling”,ICML

2014SemanticSegmentation:

Sliding

WindowSJTUDeepLearningLecture.14Input:3xHx

WConvolutions:DxHx

WConvConvConvConvScores:CxHx

WargmaxPredictions:Hx

WDesignanetworkasabunchofconvolutionallayerstomakepredictionsforpixelsallat

once!Problem:convolutionsatoriginalimageresolution

willbeveryexpensive

...SemanticSegmentation:

Fully

ConvolutionalSJTUDeepLearningLecture.15Input:3xHx

WPredictions:Hx

WDesignnetworkasabunchofconvolutionallayers,

withdownsamplingandupsamplinginsidethe

network!High-res:D1xH/2x

W/2High-res:D1xH/2x

W/2Med-res:D2xH/4x

W/4Med-res:D2xH/4x

W/4Low-res:D3xH/8x

W/8Long,Shelhamer,andDarrell,“FullyConvolutionalNetworksforSemanticSegmentation”,CVPR2015Nohetal,“LearningDeconvolutionNetworkforSemanticSegmentation”,ICCV

2015Downsampling:Pooling,stridedconvolutionUpsampling:???SemanticSegmentation:

Fully

ConvolutionalSemanticSegmentationIdea:

Fully

ConvolutionalSJTUDeepLearningLecture.16Input:3xHx

WPredictions:Hx

WDesignnetworkasabunchofconvolutionallayers,

withdownsamplingandupsamplinginsidethe

network!High-res:

D1xH/2x

W/2High-res:

D1xH/2x

W/2Med-res:D2xH/4x

W/4Med-res:D2xH/4x

W/4Low-res:D3xH/8

x

W/8Downsampling:Pooling,stridedconvolutionUpsampling:Unpoolingorstridedtranspose

convolutionLong,Shelhamer,andDarrell,“FullyConvolutionalNetworksforSemanticSegmentation”,CVPR2015Nohetal,“LearningDeconvolutionNetworkforSemanticSegmentation”,ICCV

2015SJTUDeepLearningLecture.17Classification+

LocalizationClass

ScoresCat:

0.9

Dog:

0.05Car:

0.01...Vector:4096FullyConnected:4096to

1000BoxCoordinates(x,y,w,

h)FullyConnected:4096to

4SoftmaxLossL2

LossLossCorrect

label:CatCorrect

box:(x’,y’,w’,

h’)+ThisimageisCC0public

domainOftenpretrainedonImageNet(Transfer

learning)Treatlocalizationas

aregression

problem!Multitask

LossSJTUDeepLearningLecture.18ObjectDetectionas

Regression?DOG:(x,y,w,

h)DOG:(x,y,w,

h)CAT:(x,y,w,

h)12numbersDUCK:(x,y,w,h)

ManyDUCK:(x,y,w,h)

numbers!….Eachimageneedsadifferentnumberof

outputs!CAT:(x,y,w,h)

4

numbersObjectDetectionasClassification:

Sliding

WindowSJTUDeepLearningLecture.19Dog?

NOCat?

NOBackground?

YESApplyaCNNtomanydifferentcropsoftheimage,CNNclassifieseachcropasobjector

backgroundObjectDetectionasClassification:

Sliding

WindowSJTUDeepLearningLecture.20Dog?

YESCat?

NOBackground?

NOApplyaCNNtomanydifferentcropsoftheimage,CNNclassifieseachcropasobjector

backgroundObjectDetectionasClassification:

Sliding

WindowSJTUDeepLearningLecture.21ApplyaCNNtomanydifferentcropsoftheimage,CNNclassifieseachcropasobjector

backgroundDog?

NOCat?

YESBackground?

NOProblem:NeedtoapplyCNNtohugenumberoflocations,scales,andaspectratios,verycomputationally

expensive!SJTUDeepLearningLecture.22RegionProposals/Selective

SearchFind“blobby”imageregionsthatarelikelytocontain

objectsRelativelyfasttorun;e.g.SelectiveSearchgives2000regionproposalsinafewsecondson

CPU[1]Alexeetal,“Measuringtheobjectnessofimagewindows”,TPAMI2012[2]Uijlingsetal,

“SelectiveSearchforObjectRecognition”,IJCV

2013[3[Chengetal,“BING:Binarizednormedgradientsforobjectnessestimationat300fps”,CVPR2014[4]ZitnickandDollar,“Edgeboxes:Locatingobjectproposalsfromedges”,ECCV

2014SJTUDeepLearningLecture.23RegionProposals/Selective

SearchEfficientGraph-BasedImageSegmentationHierarchicalGroupingAlgorithmSJTUDeepLearningLecture.24R-CNNGirshicketal,“Richfeaturehierarchiesforaccurateobjectdetectionandsemanticsegmentation”,CVPR

2014.FigurecopyrightRossGirshick,2015;source.SJTUDeep