图像局部特证及其匹配的详细讲解

MatchingwithInvariantFeatures Example BuildaPanorama M BrownandD G Lowe RecognisingPanoramas ICCV2003 Howdowebuildpanorama Weneedtomatch align images MatchingwithFeatures Detectfeaturepointsinbothimages MatchingwithFeatures DetectfeaturepointsinbothimagesFindcorrespondingpairs MatchingwithFeatures DetectfeaturepointsinbothimagesFindcorrespondingpairsUsethesepairstoalignimages MatchingwithFeatures Problem1 Detectthesamepointindependentlyinbothimages nochancetomatch Weneedarepeatabledetector MatchingwithFeatures Problem2 Foreachpointcorrectlyrecognizethecorrespondingone Weneedareliableanddistinctivedescriptor Moremotivation Featurepointsareusedalsofor Imagealignment homography fundamentalmatrix 3DreconstructionMotiontrackingObjectrecognitionIndexinganddatabaseretrievalRobotnavigation other Contents HarrisCornerDetectorDescriptionAnalysisDetectorsRotationinvariantScaleinvariantAffineinvariantDescriptorsRotationinvariantScaleinvariantAffineinvariant Anintroductoryexample Harriscornerdetector C Harris M Stephens ACombinedCornerandEdgeDetector 1988 TheBasicIdea WeshouldeasilyrecognizethepointbylookingthroughasmallwindowShiftingawindowinanydirectionshouldgivealargechangeinintensity HarrisDetector BasicIdea flat region nochangeinalldirections edge nochangealongtheedgedirection corner significantchangeinalldirections Contents HarrisCornerDetectorDescriptionAnalysisDetectorsRotationinvariantScaleinvariantAffineinvariantDescriptorsRotationinvariantScaleinvariantAffineinvariant HarrisDetector Mathematics Changeofintensityfortheshift u v HarrisDetector Mathematics Forsmallshifts u v wehaveabilinearapproximation whereMisa2 2matrixcomputedfromimagederivatives HarrisDetector Mathematics Intensitychangeinshiftingwindow eigenvalueanalysis 1 2 eigenvaluesofM directionoftheslowestchange directionofthefastestchange max 1 2 min 1 2 EllipseE u v const HarrisDetector Mathematics 1 2 Corner 1and 2arelarge 1 2 Eincreasesinalldirections 1and 2aresmall Eisalmostconstantinalldirections Edge 1 2 Edge 2 1 Flat region ClassificationofimagepointsusingeigenvaluesofM HarrisDetector Mathematics Measureofcornerresponse k empiricalconstant k 0 04 0 06 HarrisDetector Mathematics 1 2 Corner Edge Edge Flat RdependsonlyoneigenvaluesofMRislargeforacornerRisnegativewithlargemagnitudeforanedge R issmallforaflatregion R 0 R 0 R 0 R small HarrisDetector TheAlgorithm FindpointswithlargecornerresponsefunctionR R threshold TakethepointsoflocalmaximaofR HarrisDetector Workflow HarrisDetector Workflow ComputecornerresponseR HarrisDetector Workflow Findpointswithlargecornerresponse R threshold HarrisDetector Workflow TakeonlythepointsoflocalmaximaofR HarrisDetector Workflow HarrisDetector Summary Averageintensitychangeindirection u v canbeexpressedasabilinearform DescribeapointintermsofeigenvaluesofM measureofcornerresponseAgood corner pointshouldhavealargeintensitychangeinalldirections i e Rshouldbelargepositive Contents HarrisCornerDetectorDescriptionAnalysisDetectorsRotationinvariantScaleinvariantAffineinvariantDescriptorsRotationinvariantScaleinvariantAffineinvariant HarrisDetector SomeProperties Rotationinvariance Ellipserotatesbutitsshape i e eigenvalues remainsthesame CornerresponseRisinvarianttoimagerotation HarrisDetector SomeProperties Partialinvariancetoaffineintensitychange Onlyderivativesareused invariancetointensityshiftI I b HarrisDetector SomeProperties But non invarianttoimagescale Allpointswillbeclassifiedasedges Corner HarrisDetector SomeProperties QualityofHarrisdetectorfordifferentscalechanges Repeatabilityrate correspondences possiblecorrespondences C Schmidet al EvaluationofInterestPointDetectors IJCV2000 Contents HarrisCornerDetectorDescriptionAnalysisDetectorsRotationinvariantScaleinvariantAffineinvariantDescriptorsRotationinvariantScaleinvariantAffineinvariant Wewantto detectthesameinterestpointsregardlessofimagechanges ModelsofImageChange GeometryRotationSimilarity rotation uniformscale Affine scaledependentondirection validfor orthographiccamera locallyplanarobjectPhotometryAffineintensitychange I aI b Contents HarrisCornerDetectorDescriptionAnalysisDetectorsRotationinvariantScaleinvariantAffineinvariantDescriptorsRotationinvariantScaleinvariantAffineinvariant RotationInvariantDetection HarrisCornerDetector C Schmidet al EvaluationofInterestPointDetectors IJCV2000 Contents HarrisCornerDetectorDescriptionAnalysisDetectorsRotationinvariantScaleinvariantAffineinvariantDescriptorsRotationinvariantScaleinvariantAffineinvariant ScaleInvariantDetection Considerregions e g circles ofdifferentsizesaroundapointRegionsofcorrespondingsizeswilllookthesameinbothimages ScaleInvariantDetection Theproblem howdowechoosecorrespondingcirclesindependentlyineachimage ScaleInvariantDetection Solution Designafunctionontheregion circle whichis scaleinvariant thesameforcorrespondingregions eveniftheyareatdifferentscales Example averageintensity Forcorrespondingregions evenofdifferentsizes itwillbethesame Forapointinoneimage wecanconsideritasafunctionofregionsize circleradius ScaleInvariantDetection Commonapproach Takealocalmaximumofthisfunction Observation regionsize forwhichthemaximumisachieved shouldbeinvarianttoimagescale Important thisscaleinvariantregionsizeisfoundineachimageindependently ScaleInvariantDetection A good functionforscaledetection hasonestablesharppeak Forusualimages agoodfunctionwouldbeaonewhichrespondstocontrast sharplocalintensitychange ScaleInvariantDetection Functionsfordeterminingscale Kernels whereGaussian Note bothkernelsareinvarianttoscaleandrotation Laplacian DifferenceofGaussians ScaleInvariantDetection Comparetohumanvision eye sresponse ShimonUllman IntroductiontoComputerandHumanVisionCourse Fall2003 ScaleInvariantDetectors Harris Laplacian1Findlocalmaximumof Harriscornerdetectorinspace imagecoordinates Laplacianinscale 1K Mikolajczyk C Schmid IndexingBasedonScaleInvariantInterestPoints ICCV20012D Lowe DistinctiveImageFeaturesfromScale InvariantKeypoints AcceptedtoIJCV2004 ScaleInvariantDetectors K Mikolajczyk C Schmid IndexingBasedonScaleInvariantInterestPoints ICCV2001 Experimentalevaluationofdetectorsw r t scalechange Repeatabilityrate correspondences possiblecorrespondences ScaleInvariantDetection Summary Given twoimagesofthesamescenewithalargescaledifferencebetweenthemGoal findthesameinterestpointsindependentlyineachimageSolution searchformaximaofsuitablefunctionsinscaleandinspace overtheimage Methods Harris Laplacian Mikolajczyk Schmid maximizeLaplacianoverscale Harris measureofcornerresponseovertheimageSIFT Lowe maximizeDifferenceofGaussiansoverscaleandspace Contents HarrisCornerDetectorDescriptionAnalysisDetectorsRotationinvariantScaleinvariantAffineinvariantDescriptorsRotationinvariantScaleinvariantAffineinvariant AffineInvariantDetection Aboveweconsidered Similaritytransform rotation uniformscale Nowwegoonto Affinetransform rotation non uniformscale AffineInvariantDetection TakealocalintensityextremumasinitialpointGoalongeveryraystartingfromthispointandstopwhenextremumoffunctionfisreached T Tuytelaars L V Gool WideBaselineStereoMatchingBasedonLocal AffinelyInvariantRegions BMVC2000 Wewillobtainapproximatelycorrespondingregions Remark wesearchforscaleineverydirection AffineInvariantDetection Theregionsfoundmaynotexactlycorrespond soweapproximatethemwithellipses AffineInvariantDetection Covariancematrixofregionpointsdefinesanellipse Ellipses computedforcorrespondingregions alsocorrespond AffineInvariantDetection Algorithmsummary detectionofaffineinvariantregion StartfromalocalintensityextremumpointGoineverydirectionuntilthepointofextremumofsomefunctionfCurveconnectingthepointsistheregionboundaryComputegeometricmomentsofordersupto2forthisregionReplacetheregionwithellipse T Tuytelaars L V Gool WideBaselineStereoMatchingBasedonLocal AffinelyInvariantRegions BMVC2000 AffineInvariantDetection MaximallyStableExtremalRegionsThresholdimageintensities I I0Extractconnectedcomponents ExtremalRegions Findathresholdwhenanextremalregionis MaximallyStable i e localminimumoftherelativegrowthofitssquareApproximatearegionwithanellipse J Mataset al DistinguishedRegionsforWide baselineStereo ResearchReportofCMP 2001 AffineInvariantDetection Summary Underaffinetransformation wedonotknowinadvanceshapesofthecorrespondingregionsEllipsegivenbygeometriccovariancematrixofaregionrobustlyapproximatesthisregionForcorrespondingregionsellipsesalsocorrespond Methods Searchforextremumalongrays Tuytelaars VanGool MaximallyStableExtremalRegions Mataset al Contents HarrisCornerDetectorDescriptionAnalysisDetectorsRotationinvariantScaleinvariantAffineinvariantDescriptorsRotationinvariantScaleinvariantAffineinvariant PointDescriptors WeknowhowtodetectpointsNextquestion Howtomatchthem Pointdescriptorshouldbe InvariantDistinctive Contents HarrisCornerDetectorDescriptionAnalysisDetectorsRotationinvariantScaleinvariantAffineinvariantDescriptorsRotationinvariantScaleinvariantAffineinvariant DescriptorsInvarianttoRotation Harriscornerresponsemeasure dependsonlyontheeigenvaluesofthematrixM C Harris M Stephens ACombinedCornerandEdgeDetector 1988 DescriptorsInvarianttoRotation Imagemomentsinpolarcoordinates J Mataset al RotationalInvariantsforWide baselineStereo ResearchReportofCMP 2003 Rotationinpolarcoordinatesistranslationoftheangle 0Thistransformationchangesonlythephaseofthemoments butnotitsmagnitude Rotationinvariantdescriptorconsistsofmagnitudesofmoments Matchingisdonebycomparingvectors mkl k l DescriptorsInvarianttoRotation Findlocalorientation Dominantdirectionofgradient Computeimagederivativesrelativetothisorientation 1K Mikolajczyk C Schmid IndexingBasedonScaleInvariantInterestPoints ICCV20012D Lowe DistinctiveImageFeaturesfromScale InvariantKeypoints AcceptedtoIJCV2004 Contents HarrisCornerDetectorDescriptionAnalysisDetectorsRotationinvariantScaleinvariantAffineinvariantDescriptorsRotationinvariantScaleinvariantAffineinvariant DescriptorsInvarianttoScale Usethescaledeterminedbydetectortocomputedescriptorinanormalizedframe Forexample momentsintegratedoveranadaptedwindowderivativesadaptedtoscale sIx Contents HarrisCornerDetectorDescriptionAnalysisDetectorsRotationinvariantScaleinvariantAffineinvariantDescriptorsRotationinvariantScaleinvariantAffineinvariant AffineInvariantDescriptors Affineinvariantcolormoments F Mindruet al RecognizingColorPatternsIrrespectiveofViewpointandIllumination CVPR99 Differentcombinationsofthesemomentsarefullyaffineinvariant AlsoinvarianttoaffinetransformationofintensityI aI b AffineInvariantDescriptors Findaffinenormalizedframe J Mataset al RotationalInvariantsforWide baselineStereo ResearchReportofCMP 2003 A Computerotationalinvariantdescriptorinthisnormalizedframe SIFT ScaleInvariantFeatureTransform1 Empiricallyfound2toshowverygoodperformance invarianttoimagerotation scale intensitychange andtomoderateaffinetransformations 1D Lowe DistinctiveImageFeaturesfromScale InvariantKeypoints AcceptedtoIJCV20042K Mikolajczyk C Schmid APerformanceEvaluationofLocalDescriptors CVPR2003 Scale 2 5Rotation 450 SIFT ScaleInvariantFeatureTransform Descriptoroverview Determinescale bymaximizingDoGinscaleandinspace localorientationasthedominantgradientdirection Usethisscaleandorientationtomakeallfurthercomputationsinvarianttoscaleandrotation Computegradientorie