数据挖掘与内容搜索

1、数字视频技术数字视频技术-视觉数据挖掘与内容搜索视觉数据挖掘与内容搜索数字视频技术数字视频技术数据挖掘数据挖掘The process of extracting patterns from data.The process of analyzing data from different perspectives and summarizing it into useful information.图像与视频中的数据挖掘图像与视频中的数据挖掘图像与视频中的数据挖掘图像与视频中的数据挖掘图像数据挖掘系统模型图像数据挖掘系统模型ImagesDatabaseIndex and StorageFeat

2、ure ExtractionQuery ResultQuery ImageLookup对象识别与语义解读有本质不同对象识别与语义解读有本质不同Here word recognition is solved, we can access the meaning of the words. But yet, we are far from having solved language understanding. Detecting objects is just one small piece of understanding scenes (it might not even be the h

3、ardest). Images and sequence tell stories, and the structure of those stories are as complex as sentences, paragraphs and books.“老板坐车去了机场，但是因为交通堵塞，他没能赶上飞机。”Vision as a source of semantic informationslide credit: Fei-Fei, Fergus & Torralba Object categorizationskybuildingflagwallbannerbuscarsbusfaces

4、treet lampslide credit: Fei-Fei, Fergus & Torralba Scene and context categorization outdoor city traffic slide credit: Fei-Fei, Fergus & Torralba Qualitative spatial informationslantedrigid moving objecthorizontalverticalslide credit: Fei-Fei, Fergus & Torralba rigid moving objectnon-rigid moving ob

5、ject内容搜索内容搜索 场景认识的核心技术场景认识的核心技术Techniques for understanding a visual scene Video Shot Detection Video Object Segmentation Blob Detection Object class recognition Context Semantic内容搜索内容搜索 场景认识的系统构架场景认识的系统构架内容搜索内容搜索 场景的认识场景的认识1) Object representation based on intrinsic features:Localfeaturesno carClas

6、sifierp( car | VL )2) Detection strategy:SkyMountainBuildingscars3) The scene representation内容搜索内容搜索 图像特征图像特征Low Level:ColorTextureEdge/ShapeObject Level:Regions内容搜索内容搜索 场景的认识场景的认识AthleteHorseGrassTreeSaddleWindAthleteRockGrassTreeSkyRopeAthleteSnowTreeSkySnowboardLarge # of uninitialized imagesSmal

7、l # of automatically initialized imagesVisual Text CNrORNFXArNtZSTSkyAthleteTreeMountainRockClass: Rock climbingAthleteMountainTreeRockSkyAscent SkyAthleteWaterTreesailboatClass: SailingAthleteSailboatTreeWaterSkyWindLearningModelRecognitionTreeAthleteSnowboardSnowClass: SnowboardingAthleteSnowboard

8、TreeSnowSkyPowder场景的认识场景的认识 应用示例应用示例内容搜索内容搜索 对象的认识对象的认识内容搜索内容搜索 对象的认识对象的认识内容搜索内容搜索 对象的认识方法对象的认识方法内容搜索内容搜索 对象的认识方法对象的认识方法内容搜索内容搜索 对象的认识方法对象的认识方法注意问题注意问题 Image similarity is subjective and context-dependent. In addition, we are using low-level image features. (semantic gap) Thus, it is VERY difficult

9、to express the users concept by these features.对象的认识对象的认识 例子例子对象的认识对象的认识 应用示例应用示例内容搜索内容搜索 行为的认识行为的认识 Traffic analysisObject classification内容搜索内容搜索 行为认识的方法行为认识的方法 Behaviour and Event Analysis行为认识行为认识 应用示例应用示例行为认识行为认识 应用示例应用示例挑战与方向挑战与方向“In general, current systems have not yet had significant impact on society due to an i