外文翻译原文-一个基于视觉注意模型的由粗到精的影像配准

.RESEARCHPAPER.SCIENCECHINAInformationSciencesDecember2014,Vol.57122302:1122302:10doi:10.1007/s11432-014-5207-4cScienceChinaPressandSpringer-VerlagBerlinHAcoarse-to-fineimageregistrationmethodbasedonvisualattentionmodelFENGJing1,MALong1,BIFuKun2*,ZHANGXueJing3&CHENHe11SchoolofInformationandElectronics,BeijingInstituteofTechnology,Beijing100081,China;2CollegeofInformationEngineering,NorthChinaUniversityofTechnology,Beijing100144,China;3CollegeofInformationTechnology,BeijingUnionUniversity,Beijing100101,ChinaReceivedJune20,2014;acceptedJuly30,2014AbstractImageregistrationisfundamentalandcrucialtoremotesensing.Howevergettinghighlyaccurateregistrationperformanceautomaticallyandfastforlarge-fieldimagesconsistentlyisachallenge.Asaworkaroundtothisproblem,weproposeanewimageregistrationconceptbasedonvisualattentioninthispaper.Thisconceptemploystheadvantagesoffeature-basedorarea-basedmethodstoimprovetheprecisionandeciencyofimageregistration.Thekeyconceptofproposedintegratedschemeistomakeoptimumuseofthehighlyprominentdetailsinthefullscenebymeansofvisualattentioncomputationalmechanism.Totestifythevalidation,comparisonswithotherclassicalmethodsarecarriedoutonreal-worldimages.Theexperimentalresultsshowthattheproposedmethodcaneectivelyperformonmulti-view/multi-temporalremotesensingimageswithoutstandingprecisionandtimesavingperformance.Keywordsremotesensingimageregistration,visualattention,similaritymeasure,partialcorrelation,coarse-to-fineprocessCitationFengJ,MaL,BiFK,etal.Acoarse-to-fineimageregistrationmethodbasedonvisualattentionmodel.SciChinaInfSci,2014,57:122302(10),doi:10.1007/s11432-014-5207-41IntroductionImageregistrationistheprocessofoverlayingtwoormoreimagesofthesamescenetakenatdierentmoments,fromdierentviewpoints,and/orbydierentsensors1.Sincetheregisteredimagecouldprovidecomplementaryinformationfrommulti-dataimages,imageregistrationisregardedasanimpor-tantstepinremote-sensingimageanalysis,whichiswidelyusedinmanyfieldssuchasnationaldefense,changedetection,environmentsurveillance,etc.Duringthepastfewyears,theapplicationsofmulti-view,multi-temporalandmulti-sensorimagesregistrationhaveattractedmoreandmoreresearchersattentionandaprototypesystemisnowavailable.Butacquiringhighaccuracy,goodadaptabilityandnoiserobustnessstillremainachallengeinthestate-of-artalgorithms.Imageregistrationmethodscanberoughlyclassifiedintotwocategories:feature-andarea-basedmethods.Thefeature-basedmethodsextractandmatchsimilarfeaturesfrompairsoftwoormoreimages.Thereforeeectivefeaturedetectionandaccuratefeaturematchingmethodsplayimportantrolesinthisprocess26.Thearea-basedmethodsimplementregistrationdirectlybyutilizingtheinformation*Correspondingauthor(email:)FengJ,etal.SciChinaInfSciDecember2014Vol.57122302:2correlationbetweenreferenceandfloatimages,whichcanberegardedasatemplatematchingprocess79.Area-basedmethodsarewidelyemployedinmedicalimageprocessingfield.Sincethismethodusesglobalinformation,ithashigheraccuracythanthefeature-basedmethodsalthoughitishighlytime-consuming.Forlarge-sceneremotesensingimageregistration,itisdiculttoprocessmassiveimagedatainrealtime.Inordertoacquirereliableandfullyrepresentativesalientfeatures,multiplefeaturedetectorsarerequiredforentiresceneanalysis.Whatismore,forsimilarattributes,tomatchandthenderivematchingfeatures,particulareectivesimilaritymetricsareneededinoutlierremoval.Notably,whencalculatingtheafore-mentionedprocessinafullscene,thecomputationallaborisenormous2.Asasolutiontothisproblem,ContourletTransformcouldbeusedtoreducecomputationonlargescaleSIFT6.Zhangetal.10usedoutlierremovalalgorithmbasedonKNN-TARandproposedapromisinglysimple,robustfeature-point-matchingregistrationmethod.Themaindrawbackoftypicalarea-basedmethodsisthattheygenerallyworkdirectlywiththeentireimageintensityvalues,whichincreasesthecomputationaltimenecessaryfortheregistration.Pluimetal.11proposedtoexploitmulti-resolutionconceptforcomputationalacceleration.Ref.12describedapreprocessingschemethatinvolvesextractionofsta-tisticallycorrelatedregions(ESCR)andenhancementofstatisticalcorrelation(ESCF)toimprovetheperformanceofnormalizedmutualinformation(NMI)basedregistration.Allthesemethodscontributetothereductionincomputationalloadtoanextentandinspeedinguptheregistrationimplementation.Recently,consideringtherobustnessoffeature-basedmethodsandhigh-precisionofarea-basedmethods,researchershavetriedtouseboththemethodsinacombinedway.Gongetal.13presentedacoarse-to-fineschemeformulti-sensorimageregistrationbycombiningSIFTandMI,whichareequivalenttopreregistrationandfine-tuningprocessrespectively.However,apartfromitshighcomputationcomplex-ityandremovalofcontrolpointsoutlier,thedirectuseofSIFTonanentiresceneshowsnoimprovementincomputationaltime.Inthispaper,inspiredbystudiesofvisualattention,weformulatethelarge-fieldremotesensingimageregistrationasacoarse-to-fineprocessandproposeanimageregistrationmodelinlarge-fieldhomologousremotesensingimages.Ourmethodproceedsintwomainstepsthepreprocessingofsalientregionsofinterest(ROI)matchingandthefine-tuningofcross-correlationphaseunderroughparametersobtainedfrompreviousphase.Incontrasttoconventionalarea-basedmethods,ourproposedmethodisrobusttointensitydierences,noise,varyingilluminationandisabletoaccelerateconvergenceaccuratelybyincorporatingitwithsalientfeature-basedgeometricstructuralanalysisinclassicalarea-basedmethod.Thismethodisafeature-to-region,objective-to-pixel,coarse-to-finematchingprocess.Organizedintodierentsections,therestofthepaperdiscussesthismethodindetail.Theproposedcoarse-to-fineregistrationmethodisintroducedinSection2.Section3describestheperformanceofthealgorithm.Section4isconclusion.2ProposedmethodGivenareferenceimageR(x,y)andafloatimageF(x,y),registrationistoconfirmthepropercorre-spondingcoordinatesrelationshipbetweenR(x,y)andF(x,y).Morespecifically,weneedtofindoutanoptimaltransformationmodelthatmatchesthetransformedfloatimageandR(x,y)tothemaximumextentforallpixels.Sincesensorsarefarawayfromtheobservedscene,itisfeasibletousetheanetransformationmodelforlarge-fieldremotesensingimageregistration.ThetransformmatrixcanbedenotedbyT=scosssin0ssinscos0trianglextriangley1,(1)andFT(x,y)=TF(x,y).(2)FengJ,etal.SciChinaInfSciDecember2014Vol.57122302:3Figure1Mainprocessoftheproposedregistrationmethod.Parametersintransformmatrixhavespecificgeometricmeanings.s,trianglex,triangleyrepresentthescale,therotationangle,thehorizontalshift,andtheverticalshift,respectively.Figure1describesthemainprocessoftheproposedregistrationmethod.Forpreprocessingmatching,ROIsareextractedfirstfromoriginalinputimagesbyvisualattentionmodel.TheseROIsarethenmatchedbasedonsimilaritiesoftheirtextureandgeometriccorrelationattributes.Fromthecoarsematching,theroughparametersarederivedforthefine-tuningmulti-dimensionaloptimalsearchingregistration.Adetailedalgorithmisasbelow.2.1ExtractionofROIfeatureThoughthearea-basedmethodshavetheadvantageofhighlypreciseregistrationperformance,thelimi-tationliesinitshighcomputationaltime.Forahugeamountofremotesensingdatatobeprocessed,itisnaturaltoexplorewaysofselectingasubsetofthedataonwhichprocessingcanbedonetoachievetheobjectivesofaparticularapplication.Alargenumberofsubsetextractionmethodsareexploitedinrecentyears1217,butahighlyecientandeectivestrategyisstillwanted.VisualAttention(VA)isabletopresentaHumanVisualSystem(HVS)-likemechanismtomimicthevisualsearchprocessoffindingsalientobjectsinhighlyclutteredscene.Manycomputebasedapproacheshavebeenpro-posedandsomeofthemhavealreadybeenusedinlarge-fieldremotesensingimageforfastsaliencydetection18.WeemployAchantasFrequency-tunedSalientRegionDetection(FSRD)-basedmethod19togetROIs.Thankstothemaximumreservationoftheoriginalspatialresolution,theextractedsubsetspossessmuchmoresignificantdetails,andcangetamoretime-savingsolutionthanothermethods20.Achantaadoptsmeanshiftinobjectdetectiontogetsegmentedobjects,whichconsumestimeforlarge-fieldprocessing.HerewefocusonnormalizedareathatcontainstheconcerningROIobjectratherthantheobjectitself,andusethewinner-take-allnetworktoacquirethesetsofimagepatches.(FormoredetailsofAchantasmodel,pleasereferto:http:/ivrg.epfl.ch/supplementarymaterial/RKCVPR09/index.htm).Tospeeduptheextractprocess,everyROIisdenotedbyacircularareawithfixedradiusr,whererisdeterminedbytheimagescale.Figure2givesanexampleoftheextractionresults.Table1givesaperformancecomparisonbetweentwoextractionmethodsofstatisticalcorrelationpatches.ThecomparisonshowsthattheproposedROIsecientlyfocusonthesalientareas/objects,whichcontainsmostoftheentireinformationinthescene.2.2ROImatching&outlierremovalMatchingsimilarfeaturesfrompairsofimagesisthekeyprocessoffeature-basedimageregistration.Manykindsofsimilaritymeasureshavebeenemployedtoclarifythesimilaritydegreeofimagepatches,e.g.,correlation,mutualinformation,andphasecorrelation.However,thesemetricsarenotrobusttorandomnoise,blurring,andspatialdegradationsandthemostimportantfactoristhattheyarenotinvariantane.Accordingtotheareacharacteristics,wepresentanovelsimilaritymeasureschemewhichincorporatesROIsdetailtextureinformationingeometriccorrelation.Thespecificstepsgoasfollows.FengJ,etal.SciChinaInfSciDecember2014Vol.57122302:4Figure2ExampleofextractedsalientROIpatches.(a)ReferenceimageR0;(b)floatimageF0;(c)R0ssaliencymap;(d)F0ssaliencymap.(e)(i),(j)(n)areReferenceImagesROIpatchsubsetandthatofthoseFloatImagesrespectively.Table1StatisticalinformationcontrastofextractedpatchExtractedpatchsentropy/Extractedpatchsentropy/R0sgrayscaleimageentropyF0sgrayscaleimageentropyESCR12statisticalanalysis5.17/7.124.93/7.05(Conventionalmethod)VAstatisticalanalysis5.68/7.125.76/7.05(Ourmethod)LetRsandFsdenotethereferenceimageR,floatimageFssaliencymaprespectively.Andletri,fj(1lessorequalslantilessorequalslantM,1lessorequalslantjlessorequalslantN)representeachROI.Step(1)ROImatching:CalculateandcompareeachROIsinvariantmomentsEuclideandistancetoimplementsimilaritymeasure21:ForeveryROI,mpqisacertainROIk(x,y)sordinarygeometricmomentmpq=summationdisplayx,ykxpyqk(x,y).(3)The(p+q)sordercentralmomentofkisdefinedaspq=summationdisplayx,yk(xx)p(yy)qk(x,y),(4)wherex=m10m00,y=m01m00.(5)Thenormalizedcentralmomentispq=pq(p+q)/2+100.(6)Thenotherinvariantmomentsaregivenby1=20+02,(7)2=(2002)2+4211,(8)FengJ,etal.SciChinaInfSciDecember2014Vol.57122302:53=(30312)2+(32103)2,(9)4=(30+12)2+(21+03)2,(10)5=(30312)(30+12)(30+12)23(21+03)2+(32103)(21+03)3(30+12)2(21+03)2,(11)6=(2002)(30+12)2(21+03)2+411(30+12)(21+03),(12)7=(32103)(30+12)(30+12)23(21+03)2+(31230)(21+03)3(30+12)2(21+03)2.(13)EachROIsfeaturevectorcanbedenotedbyk,p=(1,2,3,4,5,6,7),krorfandpiorj.(14)Togetmostsimilaritymatchingbetweenfjandri,wecalculatetheEuclideandistanceoffeaturevector.TheROIinfjandriaredenotedrespectivelybym,n,d(m,n)=bardblm,fn,rbardbl,(15)D(i,j)=d11.d1N.dM1aMNMN.(16)Toeliminatethecomplementaryinterferenceinformation,weomittheparticulardmn,whichisbiggerthanempiricaloutlierthresholddthinsimilarmatrix.Then,weachievethemostsimilarpairs(rm,fm)undertheminimalEuclideandistancestandard,where(1lessorequalslantmlessorequalslant,lessorequalslantminM,N).Step(2)Interferenceoutlierremoval.WithStep(1),wecanecientlyremovetheinterferencecausedbyobjectitself.Sincethenon-rigidobjectchangingmayaectitssaliency,oncethesalientsshifttothenon-salients,usingROIsdetectionandvariantmomentsmeasure,wecanaccomplishthistaskmoreeasily.Inremotesensingapplications,however,thechangeinobjectspositionisanotherfrequentlyaskedquestion(FAQ)andStep(1)wouldbenolongerfeasible.Foralarge-fieldscene,theappearanceofobjectsisalwaysstable,whilethegeographicpositionamongthemmaybechanging.ThematchingROIsgeometricrelationfrompairedimagesshouldremainstable,evenifthereisscaledierence.Thatistosay,thestructuralinformationofROIiswithinacertainrangeofchangingscales.Forthisreason,weadoptthistermtoremovetheinterferenceROIoutlier.Theconcretestepsareasfollows.(i)CalculatetheneighborROIsEuclideandistanceof-dimensionalvectorsamongrmandfmsets:r=(d(Or,1,Or,2),d(Or,1,Or,),d(Or,Or,1),(17)f=(d(Of,1,Of,2),d(Of,1,Of,),d(Of,Of,1).(18)whereeveryEuclideandistanceismeasuredbetweentwoROIscoordinatesofthecenterofgravitygivenbyformula(5).(ii)Line-featurebasedsimilaritymeasure:firstconstructtheindependentsimilarity,SSR(structuralsimilarityratio),togetthestructuralchangingscales,i.e.,Sratio=(s1,s1,s)=parenleftbiggd(Or,1,Or,2)d(Of,1,Of,2),d(Or,1,Or,)d(Of,1,Of,),d(Or,Or,1)d(Of,Of,1)parenrightbigg,(19)FengJ,etal.SciChinaInfSciDecember2014Vol.57122302:6SSRi=si.(20)AfteralltheSSRiisobtained,wecandeterminewhethertheparticularpair(ri,fi)isadissimilaroneandwhetheritisdiscardedaccordingtothefollowingrule:braceleftBigg|SSRi11|,|SSRi+11|,removeROIpair(ri,fi),others,retainROIpair(ri,fi),(21)wheredenotestheempiricalthreshold.Asetofstructure-reliableROImatchescanbeobtainedaftertheROImatchingandinterferenceoutlierremovalofthesetwopreprocessingsteps.Theiruniquenessliesinthefollowingfouraspects.(1)Theextractedtiepatchescontainthemostremarkablecontentinthescene,namelytheroughareawithprominentdetails.(2)Abundantmutualinformation.Accordingtosimilaritymatching,thematchingareasincludecoexistinginformation,whichisrobustagainstcomplicatedtransforms,i.e.,anetransformandsimilartransform.(3)Automaticdesign.TheVAmodelemployedinthisschemeisbasedonabottom-upapproachonsalientobjects/featuressearching,andrequiresnopriorknowledgeortask.(4)Real-timeperformance.ROIsearchingisacceleratedbyreplacingmeanshiftwithWTAnetworkforobjectlocating.2.3TransformparameterestimateAfterthestageinSubsection2.2,theconcretesetsofmatchingcandidates(ROIs)acrosstheentirescenecanbeselectedas(rprimem,fprimem),where1lessorequalslantmlessorequalslantprime,primelessorequalslant.Thuswecangetthetransformparametersfromdetaileddescriptionofselectedcandidatesforsubsequentfine-tuningstages.Usingformula(3),wecalculatethecoordinatesofthecenterofgravityofrm,fm,i.e.,Or(xr,yr)andOf(xf,yf).Hencethehorizontalandverticalshiftcanbedefinedasx=xfxrandy=yfyr.UsingthetwomostsimilarROIpaired,therotationanglecanbegivenby=arccosangbracketleftBigOr,1Or,2,Of,1Of,2angbracketrightBigvextendsinglevextendsinglevextendsingleOr,1Or,2vextendsinglevextendsinglevextendsinglevextendsinglevextendsinglevextendsingleOf,1Of,2vextendsinglevextendsinglevextendsingle,0,.(22)Giventhesub-stagestep,thescaleparameterscanbecalculatedbys=s1+s2+sprimeprime.(23)2.4Optimumcross-correlationprocessFindingthemaximumofsimilaritymeasureisthekeyofarea-basedscheme.Howeveritiscomputa-tionallydicultandhighlytime-consumingtoobtainglobalextremesolutionviaexhaustivesearching.Particularly,thecomplementaryinformationfromthepairedimagesandthesmootharea,whichcon-tainslessprominentdetails,makeitmuchmorediculttolocalizethemaxima.Manywindowsofpredefinedsizesorotherpre-selectedpatchesareemployedforsubsequentsearching.Duetothis,cov-eringexactlythesamepartsofthesceneinthereferenceandfloatimagesandendowingroughareawithhighsaliencytoimproveoptimumsearchremainsachallenge.Inthispaper,wepresentanecientpreprocessingsolutionforoptimallysearchinginitialparametersestimatebycombiningmodifiedvisualattentionmodelwithstructuralanalysisofselectedcandidates.Here,giventheinitialparameters,adoptingthePowellmulti-dimensionalsearchstrategy,wecarryoutaniterativeprocess.Foreveryparameter,theoptimaltransformationmatrixissearchedbyBrentsone-dimensionaloptimizationmethod.Locatingthemaximaoflocalcross-correlation(CC),wecangetthebestmatchingtransformmatrixTtocompletethefine-tuningprocess.Theconceptcanbedescribedasfollow:T=argmaxCC(R,T(F),(24)FengJ,etal.SciChinaInfSciDecember2014Vol.57122302:7whereRisthereferenceimage,T(F)isthetransformedfloatimage.Thecross-correlationcalculationisconductedonthesetsofcandidates(patches)rmandthecorrespondingcoordinatesareafprimeminT(F)foreveryiteration.Duetotheprominentsaliencyinrm,thehighstatisticalcorrelationisfirmlyguaranteed.Thecross-correlationbetweensimilarmeasuresisdefinedasCC=primeproductdisplaym=1summationtexti,jrm(rm(i,j)rm)parenleftbigfprimem(i,j)fprimemparenrightbigradicalBigsummationtexti,jrm(rm(i,j)rm)2radicalBigsummationtexti,jrmparenleftbigfprimem(i,j)fprimemparenrightbig2.(25)3ResultsanddiscussionTheproposedapproachwasexperimentallyevaluatedinover100pairsofhighresolutionremotesensingimagesformulti-viewandmulti-temporalanalysis.Forthisapplication,thescaledistortionwaskeptinrelativelysmallvariants,whichisoneofthemainreasonsforusingvisualattentionmodeltoselectthesalientregionsofinterest.Astoaclutteredscene,ourvisualattentionwasautonomouslydrawnbythesalientobjectsinvisualfield.However,thehighscaledistortionmaygreatlyaectthesaliency-guidedprocess.Inthissituation,theempiricalthresholdwassetat0.5.Consideringthehigh-resolutionopticalimagesprominentdetailinformationandtheimportanceoftheaccuracyofpre-matchingforparametersestimate,wefixedarigorousthresholddthat0.9.AcomparisonbetweenESCproposedby12andourswasmade.Besidesweadoptedtwocommonevaluationindicators:TRE(targetregistrationerror)andRMSE(rootmeansquareError)toillustratetheregistrationperformance.Itisalsotobenotedthatfeature-basedmethodSIFTwasconductedusingCandMATLAB,whileESCandouralgorithmwerecodedbyMATLAB.TRE=radicalBig(xx)2+(yy)2+()2+(ss)2,(26)RMSE=radicaltpradicalvertexradicalvertexradicalbt1Nsummationdisplayi,jrm|r(xi,yj)fprime(xi,yj)|2,(27)wherex,y,saretherealparameters,x,y,saretheoptimalestimatevalues,r(xi,yj)andfprime(xi,yj)representthermareasinreferenceandthecorrespondingoverlappedareasintransformedfloatimage,respectively.Ingeneral,thelessTREandRMSE,thebettermatchingaccuracy.3.1Multi-viewopticalairborneimageregistrationexperimentThefirsttwopairsofimageswereprovidedbyI3DEA,ArizonaStateUniversityacquiredfromthesamesensorwithnearlythesametrajectory,whichreflectsthe0.5mresolutionofMesa,Arizonawithrichblocksandbuilding