外文翻译原文-OpenCV兼容实时处理器识别背景前景

AbstractThebackgroundidentificationmethodsareusedinmanyfieldslikevideosurveillanceandtrafficmonitoring.InthispaperweproposeahardwareimplementationoftheGaussianMixtureModelalgorithmabletoperformbackgroundidentificationonHDimages.TheproposedcircuitisbasedontheOpenCVimplementation,particularlysuitedtoimprovetheinitialbackgroundlearningphase.Bit-widthhasbeenoptimizedinordertoreducehardwarecomplexityandincreaseworkingspeed.Theproposedcircuitprocesses221920X1080framespersecondwhenimplementedonVirtex5FPGA.IndexTermsBackgroundidentification,FieldProgrammableGateArray,Objectdetection,OpenCV.I.INTRODUCTIONSurveillanceandtrafficmonitoringapplications1-4relyontheidentificationofrelevanteventsinvideosequences.Objectdetectiontechniqueshavebeenstudiedduringtheyearsanddifferentalgorithmshavebeenproposed.Mainrealtimeidentificationalgorithmsarebasedontheframedifference5-7oronthecomparisonwithareferencemodel(backgroundsubtractionmethods)8-13.Theframedifferencemethods5-7,detectthemovingobjectscomparingconsecutiveframes.Thealgorithmsarefastandsimpletoimplementbuttheoutputdependsonthespeedofthemovingobjects.Thebackgroundsubtractionalgorithmsdetecttheforegroundcomparingtheframewithareferencemodel.Ref.8considersapixelasbackgroundifitkeepsaconstantvalueforarelativelylongtime.Thetechniquefailswhenachangeoftheilluminationariseonanotherwiseconstantscene.Papers9,10useaKalmannfilter,while11proposesaWienerfilter,toadaptthebackgroundmodel.Othermethodsusestatisticalalgorithmstoobtainastatisticalmodelforeachpixelofthebackground,12,13.Foranewframe,thepixelsarecomparedwiththestatisticalmodel.Ifthedifferenceisgreaterthanathreshold,thepixelisclassifiedasforeground.In12analgorithmthatusesasingleGaussiandistributionperpixel,isproposed.Thealgorithmisefficientbutisnotabletodescribeamultimodalbackgroundinwhichshadowsandobjectsshowingrepetitivemotion(e.g.ripplingwavesorwhishingleaves)arepresent.Forthisreason,in13,eachpixelismodeledwithamixtureofGaussiandistributions.Thetechniqueof13isknownasGaussianMixtureModel(GMM)andprovidesgoodperformancesinbothpresenceofilluminationchangesandmultimodalbackground.DuetothegoodperformancestheGMMalgorithmhasbeenselectedasthebackgrounddetectionalgorithmintheOpenCVlibrary15.OpenCV(OpenSourceComputerVision)isaOpenSourcesoftwarelibrarydevelopedbyIntel.OpenCVprovidesacommonbaseofcomputervisioninstrumentsabletoextractrelevantdetailsfromtheimagesandtoprocesstheminautomaticway.TheGMMalgorithmproposedintheOpenCVlibraryisanoptimizedversionofthealgorithmof13thatisparticularlysuitedtoimprovetheinitiallearningphase.MaindrawbackoftheGMMalgorithmistheneedofnumerousnonlinearcomputationsthatlinearlyincreasewiththenumberofGaussiansperpixelandmakerealtimevideoprocessingnotpossiblewithasoftwareimplementation.In13,aframerateof11fpsisobtainedforasmallframesizeof160x120onanSGI02withaRIOOOOprocessor.In14anFPGAimplementationoftheGMMalgorithmisproposed.ThealgorithmdoesnotcomplywiththeOpenCVGMMalgorithmbutimprovestheprocessingcapabilitiesandprocessesrelativelylargeimages(1024x1024at38fps).InthispaperahardwareimplementationoftheGMMalgorithmthatallowsrealtimeprocessingofhighdefinitionvideosandcomplieswiththeOpenCV,isproposed.OurcircuitprocessesHDimagesat22fpswhenimplementedonVirtex5FPGA.Programmablelogicoccupationis5.5%ofthexc5vlx50FPGAwhilepowerdissipationis27.6mW47MHz.II.GAUSSIANMIXTUREMODELTheGMMalgorithmhasbeenproposedbyStaufferandGrimson13withtheaimofefficientlymodelamultimodalbackground.ItdescribesthestatisticofeachpixelusingastatisticalmodelcomposedbyamixtureofKGaussiandistributions.UsingmorethanoneGaussianallowstomodelrealsituations.AsanexampleapointontheframewhoseintensityoscillatesbetweentwovaluesisperfectlymodeledwithtwoGaussians.GreateristhenumberoftheGaussiandistributionsemployedbythealgorithm,higheristheprecisionofthemethodandthecomputationalcomplexity.AdescriptionoftheGMMalgorithmfollows.ForthedetailsOpenCVcompatiblerealtimeprocessorforbackgroundforegroundidentificationM.Genovese,E.Napoli,N.PetraDIBETUniversityofNapoli,ViaClaudio,21-80125Napoli-ItalyEmail:ma.genovesestudenti.unina.it22ndInternationalConferenceonMicroelectronics(ICM2010)978-1-4244-5816-5/09/$26.002009IEEEreferto13andtotheOpenCVdocumentation.A.ParametersUpdateWhenaframeisacquired,amatchconditionwiththeKGaussiansisverifiedforeachpixel.ApixelmatcheswithaGaussianifthedifferencebetweenthemeanofthedistributionandthepixelislowerthan2.5standarddeviations.Thematchconditionisimportanttoestablishifthepixelvaluecanbeconsideredasbackground.Whenthek-thGaussianverifiesthiscondition(matches),itsparametersareupdatedasfollows:,1,2222,1,1,1,1ktktktktktktktktktGtktGtkktktpixelpixelMmatchsummatchsum(1)where,andaremean,varianceandweightforeachGaussian.,GtandaretwodifferentlearningratesandmatchsumisacounterintroducedintheOpenCValgorithmthatwillbedescribedinthefollowing.FortheunmatchedGaussianstheweightsareupdatedaccordingto:,1,ktktkt(2)whilemeanandvarianceareunchanged.EachpixelcanonlymatchasingleGaussian.IfthepixelfallsundermorethanoneGaussianapriorityparameter,namedFitness,selectsthematchedGaussian.TheFitness(F)parameterisgivenby:kkkF(3)WhenaGaussianhasanhighFvalueitrepresentsabackgroundpixelwithhighprobability.TheKGaussiansaresortedindecreasingorderoftheFvalue(thatis,F1F2F3).TheGaussianthatmatcheswiththepixelandhasthehighestFisconsideredasthematcheddistribution.Aspecific“nomatch”updatingprocedureisexecutedwhenthepixeldoesnotmatchanyGaussian.InthiscasetheparametersoftheGaussianwithsmallestFareupdatedas:2,1,1,1_11ktktkktkkpixelvarianceinitmatchsummatchsum(4)wherevariance_initisafixedinitializationvalue.TheweightsoftheotherGaussiansaredecrementedasin(2)whiletheirmeansandvariancesareunchanged.B.BackgroundidentificationThebackgroundidentificationisperformedusingthefollowingalgorithm,whereTisaprefixedthreshold:,1argminbbktkBT(5).Eq.(5)worksaddinginsuccessiontheweightsofthefirstbFsortedGaussians,untiltheirsumisgreaterthanT.TheGaussiansthatverify(5)representthebackground.IfapixelmatcheswithoneoftheseGaussians,itisconsideredasbackgroundpixel,otherwiseitisclassifiedasforeground.III.HARDWAREIMPLEMENTATIONTheGMMcomputationalcomplexityisveryhighandgrowswiththenumberofGaussiansusedforeachpixel.Alsotheaccuracyofthebackgroundidentificationincreaseswiththisnumber.Ithasbeenobservedthatagoodcompromiseisamodelwith3Gaussians.Theproposedcircuitimplementsthesinglechannel,luminancebased,OpenCVimplementationoftheGMMalgorithm.ThecomputationalcomplexityofthealgorithmhasagrowthrateofO(n)wherenistheframesize.ThetargetofthisworkistoprocessHDvideoatmorethan10fps.Itisthenrequiredtoprocess21Mpps.Asoftwareimplementationdoesnotreachtheseperformances.TheproposedHWimplementationoftheGMMhasbeendescribedinVHDLcode.ThetargetdevicesfortheimplementationhavebeenhighperformanceVirtexFPGA.OurGMMprocessorcomplieswiththeOpenCValgorithmallowingasimplereplacementofaslowersoftwareimplementationinarealtimeimageprocessingsystem.A.CircuitTheproposedcircuitisshowninFig.1.Inputdataarethe8bitluminanceoftheinputpixel(Pixel),andthestatisticalmodelofthepixel(Parameters).Theoutputis,fortheinputpixeltheupdatedstatisticalmodel(UpdatedParameters)andthebackground/foregroundtag.AdetailedexplanationofthealgorithmimplementedbyeachcircuitalblockofFig.1isgiveninthefollowing.Fitness:computestheFitnessfactor(3)forthethreeGaussians;Match:verifiesthematchFig.1.BlockdiagramofthebackgroundidentificationcircuitconditionforthethreeGaussians;Controllogic:sortstheGaussiansindecreasingFitnessorderandestablisheswhichGaussianisupdatedasin(1),(2),(4).GaussiansortingisperformedthroughatwobytwocomparisonoftheFfactors.Onlythreecomparatorsandfewlogicgatesareneeded.ParameterUpdate:ifthematchconditionisverified,Weights,MeanandVarianceblocksupdatetheparametersasdescribedin(1),(2).IfnoGaussianmatchesthepixel,No-Matchblockupdatesmean,varianceandweightofthesmallestFGaussianusing(4).OutputSelection:Dependingonwhetherthematchconditionisverifiedornot,thelogicofoutputselectionestablishesthevaluesoftheupdatedparameters.Backgroundidentification:verifiesthebackgroundidentificationconditionshownin(4)andgeneratesthebg/fgmaskforeachframe.TheOpenCValgorithmuses,foreachGaussian,acounternamedmatchsumthatisincrementedwhentherelateddistributionmatcheswithapixelandisusedtoupdatetheweightwhennomatchisverified.In13,thematchsumisnotusedand,whennomatchisverified,theweightisinitializedtoafixed(verysmall)value.OpenCVGMMmethodisfasterwhentheinitialidentificationofthebackgroundisconducted.ItdetectsthebackgroundafteroneframewhiletheGMMalgorithmof13requiresabout35framesasshowninFig2.Theintroductionofthematchsumk,tentailsthesynthesisofthreecountersthatarenotonthecriticalpath.Theonlydrawbackisanincreaseofcircuitarea.B.BandwidthThecrucialpointoftheGMMisthememorysizerequiredtorecordthebackgroundparameters.Foreachpixelmean,variance,weightandmatchsumofeachdistributionmustbeloaded/stored.Dependingontheirwordlengths,memorybandwidthbottleneckscanincur.InthesoftwareOpenCVimplementationtheparametersaredoubleprecision(64bits)floatingpointnumbers.Theparameterbitsforeachpixelaretherefore758.If,asexample,thememorythroughputis128bitpercycle,6clockcyclesareneededinordertoloadtheparameterbitsforeachpixel.Theconsequence,alsoduetothehardwarecomplexity,isthatrealtimevideoprocessingisnotpossibleusingfloatingpointarithmetic.AdetailedexaminationoftheGMMalgorithmrevealsthatmostsignalshavealimiteddynamics.Mean,weightandvariancerangein0,255,0,127,and0,1,respectively.InthiscaseusingafixedpointrepresentationprovidesgoodperformanceswhilereducingHWcomplexityandtherequiredbandwidthtowardsthememory.Thenumberofbitofthefixedpointrepresentationoftheparametersisbasedonboththeirrangeandtherequiredaccuracy.IfthemeanoftheGaussiansisa23bitnumber(7bitarefortheintegerpart,18forthefractionalpart)bothmemorybandwidthandlogicutilizationareveryhigh.Inordertoimprovetheperformancesthewordlengthshavebeenreducedobtaininggoodprecisionwiththeleastpossiblenumberofbits.Itisworthnotingthatthenumberofbitsofthefractionalpartoftheparametersarealsoafunctionofthelearningrate(G,t,).Smallerlearningratesincreasethebitsneededforthefractionalpart.Asexampleif=0.02,atleast6bitsareneededforthefractionalpartof.Usinglowerwordlengths,causesanunderflowthatresultsin=0andimpairstheGMMalgorithm.TherepresentationsusedinourimplementationareshowninTableI,givenwithUm,nnotation.Um,nindicatesanunsignedfixedpointnumberwhere2mistheweightofthemsband2-nistheweightofthelsb.Withregardtothematchsum,ithasbeenobservedthat4bitspermatchsumallowgoodperformanceswithreducedlogicutilization.Withtheproposedrepresentations116bitsareemployedforeachpixel,greatlyreducingtherequiredmemorybandwidthforrealtimeimageprocessing.Toprocess10fpswithframesizeof1920X1080thememorybandwidthis287MB/s.Moreover,theimagesinFig.3showthatthedoubleprecisionfloatingpointimplementation(Fig.3a)isverysimilartotheproposedimplementation(Fig.3b).C.NonlinearfunctionsDifferentnonlinearfunctionsarerequiredfortheimplementationoftheGMM.Asexample,thefitnessfactor,theweightin(4),andthelearningrateGrequirethebinaryinversionoperation.Theactualchoice,inordertominimizecircuitcomplexityandmaximizecircuitspeed,hasbeentheuseofROMimplementationofthenonlinearfunctions.TheROMimplementationisonlyfeasibleifthenumberofinputbits(andhenceofthedifferentROMentries)islimited,sinceFig.2.Numberofframesrequiredforbackgroundidentification.Solidline:proposed,OpenCVcompliantimplementation;dashedline:GMMalgorithmof13TableIRepresentationsusedinourimplementationParameterRepresentationBit#=0.02U-6,94k,tU7,412k,tU13,-212k,tU-1,88Fig.3.Background/Foregroundframes.(a)ObtainedwithdoubleprecisionfloatingpointGMM(b)Proposedoptimizedimplementation.LUToccupationexponentiallyincreaseswiththenumberofinputbits.MainadvantageoftheproposedHWimplementationisthattheinputdatahaveanoptimizedfixedpointrepresentationthatreducesthenumberofinputbitswhileprovidingareliableandefficientcircuit.Asexample,thecalculationoftheFitnessfactor,hasbeenimplementedwithaROMandamultiplier.TheROMstorestheinverseofthestandarddeviation.Ifthevarianceandtheinverseofthestandarddeviationarerepresentedon12and8bits,respectively,thesizeoftheROMfortheFitnesscomputationis212X8bits.ThelogicutilizationforVirtex5xc5vlx50is243Slicesoutof28800.Workingfrequencyandpowerare191.6MHzand1.87mW,respectively.D.ReconfigurabilityAspreviouslynoted,thenumberofbitsoftheGaussiansparametersmustbechosenaccordingtothevaluethatisafunctionoftheconsideredbackground(slowbackgroundchangingscenesrequirelowervalues).Theproposeddesignishighlyparameterizedandallowsastraightforwardmodificationoftherepresentationoftheparameters.Thisallowsafastadaptionofthecircuittonewapplicationsandisofutmostimportanceforcommercialapplicationssinceprovidesoptimaldesignswiththelowestpossiblenumberofbits.E.ResultsandPerformacesTheproposed,notpipelined,implementationoftheOpenCValgorithmhasbeensynthesizedandimplementedonvariousXilinxFPGA.TheonlysequentialelementsaretheFFsthatsynchronizeinputandoutputdata.Thecircuithasbeentestedusingartificialvideo,computeranimatedvideoswithsimplebackgroundandusingvideosequencestakenfromrealsurveillancecameras.Thecircuitperformsoptimallyandrunssmoothlywithoutshowingreliabilityproblems.TableIIshowstheperformancesofdifferentcircuitimplementationsvaryingthevalueandthenumberofbitsoftherepresentations.Asshown,alltheproposedimplementationssatisfythe10fpsrequirement.Fordecreasingvaluesthehardwarecomplexityincreaseswhilespeeddecreases.Howeverforlowerthan0.002hardwarecomplexity,aswellasthespeedofthecircuit,donotincreaseinasensibleway.TheuseofacheaperVirtex-4FPGAreducesthespeedofthecircuitbyanamountequalto27%.AsaconsequenceitmightbeusefultoexploitpipeliningtoincreaseperformanceinordertouseevencheaperFPGAs.IV.CONCLUSIONSThispaperpresentsanhardwareimplementationoftheGMMalgorithmusedintheOpenCV.TheproposedcircuitimplementedonVirtex5(xc5vlx50Speedgrade-3)allowsamaximumworkingfrequencyof47MHzandishenceabletoprocess22fpsforanHDvideowithframesize1920X1080.Thecircuithasreducelogicutilizationof1572slicelut(5.5%oftheavailableLUT).Thedynamicpowerdissipationis27.6mW.HavingimplementedtheOpenCValgorithmthecircuitallowsaveryfastinitializationofthebackground.ThealgorithmhasbeenimplementedusingafixedpointarithmeticinsteadofthedoubleprecisionfloatingpointrepresentationusedintheOpenCVlibrary.Inthispaperitisshownthattheprecisionoftheresultingsystemisunchanged.REFERENCES1D.Gutchess,M.Trajkovi,E.Cohen-Solal,D.LyonsandA.K.Jain,Abackgroundmodelinitializationalgorithmforvideosurveillance”,inProc.EighthIEEEInternationalConferenceonComputerVision,vol.1,Vancouver,BC,July2001,pp.733-740.2I.Haritaoglu,D.HarwoodandL.S.Davis,“Afastbackgroundscenemodelingandmaintenanceforoutdoorsurveillance”,inProc.ofthe15thInternationalConferenceonPatternRecognition,vol.4,Barcellona,Spain,2000,pp.179-183.3B.Gloyer,H.K.Aghajan,K.Y.Siu,andT.Kailath,“Video-basedfreewaymonitoringsystemusingrecursivevehicletracking,”inProc.SPIESymp.ElectronicImaging:ImageandVideoProcessing,1995,pp.173-180.4L.Vibhaetal,MovingVehicleIdentificationusingBackgroundRegistrationTechnique”,Proc.oftheIntern.MultiConferenceofEngineersandComputerScientists,vol.1,HongKong,2008,pp.572-577.5Z.Chaohui,D.Xiaohui,X.Shuoyu,S.ZhengandL.Min,“AnImprovedMovingObjectDetectionAlgorithmBasedonFrameDifferenceandEdgeDetection”,inProc.Fo