PowerPoint 演示文稿 JDL.doc

PowerPoint 演示文稿 JDL A GenericVirtual ContentInsertion SystemBasedon VisualAttention Analysis223H.Liu1,2,S.J iang1,Q.Huang1,2,C.Xu2,31Institute ofComputing Technology,Chinese Academyof Sciences2China-Singapore Institute of DigitalMedia3National Labof PatternRecognition,InstituteofAutomationOutline?Motivation?Related work?The proposedVirtual ContentInsertion(VCI)(VCI)system?Experimental results?Conclusionsxx-11-25.jdl.ac.2Virtual ContentInsertionxx-11-25.jdl.ac.3?Convenient?Changeable?Cost lessToconstruct ageneric VCI systemChallenge?Advertisement insertionVS.Augmentation?Software basedVS.Hardware basedl?Challenge?Insertion time?Insertion place?Insertion method?Insertion contentRelated workInsertion Time?Insert advertisementsinto videoprologue?Be neglected?Insert thead intointeresting segments?Our method?Temporal attention?Higher AttentiveShots?K.Wan,C.Xu,“Automatic ContentPlacement inSports Highlights”,ICME,xx.xx-11-25.jdl.ac.5Related workInsertion Place?Static region?Color consistentregion?Visual relevancemeasure?Lower informativeregion?Our method?Spatial attention?Lower AttentiveRegion?C.Xu,etc.,“Implanting VirtualAdvertisement into Broadcast SoerVideo”,PCM,xx.?K.Wan,etc.,“Automatic ContentPlacement inSports Highlights”,ICME,xx.?Y.Li,etc.,“Real TimeAdvertisement Insertionin BaseballVideo Basedon AdvertisementEffect”,ACM Multimedia,xx.xx-11-25.jdl.ac.6Related workInsertion Method?Challenge?Camera parametersunknown?Existing methods?Structure ofthe scene?Predefined landmarks?Our method?Affine transformation?Global MotionEstimation?X.Yu,etc.,“Inserting3D ProjectedVirtual ContentintoBroadcastTennis Video”,ACM Multimediaxx.?C.Xu,etc.,“Implanting VirtualAdvertisement intoBroadcast SoerVideo”,PCM,xx.xx-11-25.jdl.ac.7Related workInsertion Content?Improve thead effect?Decrease intrusion?VideoSense?Textual relevance?Local visualand auralrelevanceT.Mei,X-S.Hua,L.Yang,S.Li,“VideoSense-Towards EffectiveOnline VideoAdvertising”,16th ACMInternational Conferenceon Multimedia,pp:1075-1084,xxxx-11-25.jdl.ac.8Outline?Motivation?Related work?The proposedVCI system?Experimental results?Conclusionsxx-11-25.jdl.ac.9Temporal AttentionAnalysisxx-11-25.jdl.ac.10Temporal Attention?Basic idea?The moredifferent aframe/shot/video clipis tothe precedingones,the moreprobable forint tobe atteded?Measure?Noveltyxx-11-25.jdl.ac.11?,1min,ititbindiff F FH H?1txx-11-25.jdl.ac.12bin?1,tttitii tlNol FdiffFFw FF?HAS Detection?Shot novelty?1,ttititi tlNol SdiffS Sw SS?The longera shotis,the moreit isprobable tobe attendedxx-11-25.jdl.ac.13?tttAV SLNolS?xx-11-25.jdl.ac.14Spatial AttentionAnalysis?Static attention?Spatio-temporal attention?Motion saliencyxx-11-25.jdl.ac.15?Static noveltyStaticsaliency calculationMotionsaliency calculationStatic novelty calculationStaticsaliency mapsMotionsaliency mapsStatiovelty maps+Video FramesAttentionMapsStatic Saliency (1)?Psychological basis?Contrast?Information theoryOd?Our method?Contrast andinformation theory?Calculation?Property ofreceptive fieldxx-11-25.jdl.ac.16?20,0,DoGNN?Static Saliency (2)?Perceptive unit?Pixel/block?Region?Object?Color quantizationxx-11-25.jdl.ac.17Static Saliency (3)?ContrastI ftidit?1,KkiCon kdf kfiG ik?xx-11-25.jdl.ac.18?Information density?Saliency?logID kpf k?Sal kConkID k?Motion Saliency?Motion VectorSpace?HSV colorspaceAngleHMagnitudeS?xx-11-25.jdl.ac.19gTextureV?Static Novelty (1)xx-11-25.jdl.ac.20Static Novelty (2)?Statiovelty:An events importancealong temporalaxis?Distance:KL?1XlogtttMxNol tMxdxMx?xx-11-25.jdl.ac.21?1Xtxx-11-25.jdl.ac.22Static LARDetectionxx-11-25.jdl.ac.23Dynamic LARDetection t?1LMtttMAM Pwt AMP?,111k kttMkPHP?xx-11-25.jdl.ac.24xx-11-25.jdl.ac.25Affine transformation111xVyf?111rV V?222rVV?312rrr?123Arrr?1100A PPA PAP?xx-11-25.jdl.ac.26222xVyf?Global MotionEstimation?1,1000ttk kAkPHAPPAP?xx-11-25.jdl.ac.27Outline?Motivation?Relatedwork?The proposedVCIsystem?Experimental results?Conclusionsxx-11-25.jdl.ac.28Experiment DataSetTest VideoNo.VideoGenreShotTime1Friendssituation edyxx1:25Children atsituationxx-11-25.jdl.ac.292Children atHousesituation edyxx4:483A Datewith LuYuInterviewxx0:494Adventure tothe westOutdoorteleplayxx5:48Sum-80072:50Experiment DataSet-Virtual Contentxx-11-25.jdl.ac.30Temporal attention&HAS (1)xx-11-25.jdl.ac.3100.51Attention curveNoticeRateTemporal attention&HAS (2)?Noticing rate:?Consistency:the inN010203040500Shotxx-11-25.jdl.ac.32similarity betweenattention curveand noticingcurve?min,kcosAC kNCk?No.Consistency10.7520.7930.8440.82Temporal attention&HAS (3)?Relationship betweennoticing rateand attentionvaluexx-11-25.jdl.ac.33Spatial attention&LAR?Invited theusers toevaluate thebrands he/she hasnoticed inthe video.rate ofGOOD11nniiiiRgNgN?xx-11-25.jdl.ac.34VideoGOODNEUTRALBAD172.2519.138.62270.8723.136.00366.2525.008.75470.3825.624.00Mean69.9423.226.84Variance6.678.555.20Static InsertionDemoDynamic InsertionEvaluation?Subjective evaluation?Criteria1.Is the results deformationconsistent withthe scene?xx-11-25.jdl.ac.362.Does theinserted VCfollow thecamera motion?3.To whatdegree theuser issatisfied withtheresult?Scores:1?5123456024VCI ResultEvaluationMeanVarianceDynamicInsertion Resultxx-11-25.jdl.ac.37Conclusion?Main contribution?A genericvirtual cont