基于KINECT点云采集系统开题报告

1、基于KINECT的点云采集系统1. 研究的目的、意义由于三维激光扫描的软硬件水平日趋成熟和大众化，相应的三维点云采集技术越来越受关注的研究热点，广泛应用于计算机动画、医学图像处理、文物保护、地形勘探、游戏开发和数字化媒体创作等领域。点云采集技术是近来数据采集的一个发展热点，因为其具有快速性、穿透性、不接触性、实时性、动态性、主动性、全数字性、高密度、高效率等许多传统的常规测量技术所不具有的优点，所以被广泛的应用于很多领域，具有广阔的发展前景和应用需求。近年来三维激光扫描设备在效率、精度和价格方面得到了极大地发展，同时为了满足人们的需求三维点云采集技术也成为了研究的重点。在实际测量中，由于光的线

2、性传播特性，三维激光扫描设备在一个视角下对于具有复杂形状的物体的某些区域或者背面往往存在视觉盲点，需要通过多次不同视角下的测量才能实现完整的模型数据采集；由于扫描设备测量范围有限，对于大尺寸物体或者大范围场景，不能一次性完整测量，必须分块测量。上述的问题导致测量结果往往是多块具有不同坐标系统且存在噪声的点云数据，不能够完全满足人们对数字化模型真实度和实时性的要求，因此对三维点云采集技术有重要的研究意义。2. 国内外研究现状在中国，政府和科研机构均开始高度关注大数据，工信部发布的物联网“十二五”规划上，把信息处理技术作为四项关键技术创新工程之一提出。在进行三维数据采集时，有很多不确定的因素都会使

3、得采集时引入噪声，在使用数据时，我们要尽可能的降低噪声所造成的影响，有些学者通过滤波的方式进行处理。陈晓霞等在进行数据点的筛选时主要进行了两个步骤的操作，首先在预处理时利用了密度聚类这种方式，然后通过VRML实时交互的显示功能来展现。张毅等利用K领域结合核函数，来对点云数据进行噪声的去除。在当前三维点云数据配准研究领域中，使用最为广泛的算法当属ICP 算法了。ICP 算法自 1992 年由 Besl 和 Mckay提出之后便在自动配准方面得到了广泛的引用。但是由于传统的 ICP 算法的效率并不高，而且对初始值要求很高，容易陷入局部收敛等缺陷，很多学者对 ICP 算法进行了改进。Hans Mar

4、tin Kjer 等利用基于曲率的方法进行抽样加速配准效率；孙谦等利用法向量内积加权，但是由于人为的因素对最后的配准精度与效率造成了一定的影响；贺永兴等提出了一种基于邻域特征的配准方法；蒋成成等利用 Delaunay 三角剖对ICP 算法进行了一定的改进。为了使数据模型的复杂度更适合于有限的计算机资源，必须对数据模型进行简化。对三维点云数据的精简方法主要可以概括为两大类：一类是对三维点云数据进行拓扑划分利用拓扑关系进行精简，另一类是根据特征信息来选取代表点从而进行精简。曲率精简法是典型的直接通过代表点选取来进行精简的方法。国外的 Martin 等人在1997 年提出均匀网格法这种方法，但是这种

5、方法的局限性在于它经常会误删除掉一些特征数据；Chen 等根据法向量来进行精简，但是对点云数据要求比较局限；Lee 等的改进方法对于物体表面特征的保留起到了一定的作用，但是时间开销太大。国内的张丽艳等人也是利用法向量来精简点云，该方法对于物体特征保留虽然不错，但是由于实验中的某些参数是根据经验值来选择的，这个方法的可操作性就不强了；朱冒冒等人是通过二次精简来进行改进的，相对来说精简的比较合理了；史宝全等提出的聚类精简算法虽然在保留点云特征上面已经不错了，可计算量有所增加；杜晓晖等人提出的混合算法简化效率就比较高了，但是时间略有下降。3. 拟采取的研究路线（1）点云数据的采集 本文提出了一种基于

6、kinect的点云采集系统设计方案，该方案以kinect为核心，利用模块化C+模板库PLC（点云库）中提供的通用采集接口，可以直接获取到实际坐标空间的三维信息，三维坐标信息保存为点云数据，提高了点云数据的采集速度。本方案的采集系统是由局域网内的一台计算机负责kinect点云数据的采集，它将采集到的图像深度信息转换为实际空间上的点的三维坐标信息，利用了kinect的深度成像原理，采用OpenNI开放自然交互框架来抓取kinect设备中的点云数据。（2）点云数据去噪声处理由于受人为扰动、光照、扫描设备本身的缺陷等因素的影响，采集到的数据会受到噪声污染，需要进行去噪声处理，根据噪声在各个方向上扩散方

7、式不同我们可以采用各向同性和各向异性算法对噪声进行处理。（3） 点云数据的参数化表示利用三角网格参数化将原始模型上的数据点映射到给定的参域上，建立点云数据到参数域上新的点云集合Q之间的一种对应关系&：G>Q，并且要求在一定意义下几何变形达到极小。（4） 点云数据的可视化处理 针对三维模型的几何特征，提出基于三角形简化的多分辨率复杂三维模型生成算法，生成多尺度的三维点云数据结构，同时构建对应的多分辨率纹理特征模型在建立几何与纹理尺度关联的基础上，采用R+树的索引机制实现三维模型的分块存储，建立静态多层次（LOD）的三维模型分块数据结构。在进行三维场景浏览时，依据客户端现场范围裁剪和

8、网络传输效率，在服务端快速检索和获取相应的静态LOD三维模型数据，自适应分块传输到客户端，实时生成符合视觉要求的动态LOD三维模型，达到最佳的可视化效果。4. 文献综述Kinect 是微软公司研制的一款体感外设，最初是针对其游戏主机 XBOX360 推出的一套外设产品，适用领域也仅限于游戏领域。但是其高科技的含量以及该装置本身的创意在发售后的两年内开始逐渐应用于许多领域，随着近来 kinect for windows 这款针对 windows 平台的研发设备的推出，目前世界上尤其是国外的一批人工智能科学家，人机交互，体感互动工程师和研究小组等等也纷纷在对 kinect 的应用领域做探究和研发。

9、 Kinect 最初的开发代号称为 Natal，之后正式更名为Kinect。Kinect 技术是微软公司基于高端研究得出来的电子科技产品，是微软在依靠人工智能解决复杂问题的过程中产生的一个副产品，这就是 Kinect 的来历。除了体感设备已经比较普及的游戏领域外，Kinect 的应用和实验性应用正在快速发展，下面通过几个领域已经出现了的应用来探讨 Kinect 的应用领域。（1）虚拟应用。欧洲时装店 Topshop 在莫斯科旗舰店安装了一种全新的试衣间，这种虚拟的试衣间利用了当前最先进了两种技术：增强现实（augmented reality，AR）和微软 Kinect 体感外设，你无需试穿就能

10、见到真实的试衣效果。（2）3D 建模雕塑工具。经过国外一些小组的实验，多台 Kinect 可以用作 3D 摄像机并进行 3D 建模。在一个名为 Blablab LAB 的小组的街头实验中，通过使用三台 Kinect 为游客进行扫描建模，然后使用 Rep Rap 3D 打印机制作出一个迷你的雕塑。（3）机械控制遥控机器人。使用 Kinect 作为机器人的头，通过 Kinect 检测周围环境，并进行 3D 建模，来指导机器人的行动。 因为机器人的可应用领域非常广泛，低廉成本的 kinect机器人可以代替传统机器进行一些不需要很高精度的危险地区或者地底高空等恶劣环境下的测量与勘察作业。（4）虚拟实验

11、医学领域。Kinec 在医学领域中，可代替医生进行尸检，研究人员只需要对着空气做手势或者语音，就可控制 3D 图片放大缩小旋转等功能。根据测量探头的组成方式不同,被测对象表面数据的获取主要包括接触式和非接触式测量两类。接触式测量的代表性设备是三坐标测量机(CMM)。但是由于接触式测量设备与被测物体接触,不可避免的使被测物体产生变形,因此测量误差较大。非接触式则应用光学及激光原理进行激光扫描或光学扫描等,不存在受力变形产生的误差。大多数实用的非接触式测量仪器都采用结构光照明技术,投影仪器发出结构照明光束,接收器接受由被测三维表面返回的光信号。由于三维面形对结构照明光束产生的空间或时间调制,因此可

12、以通过适当的方法从观察光场中解调出三维面形数据。激光三维扫描设备采集到的数据,是大量的三维点坐标的集合。由于点的数量巨大,其数据被形象的成为点云数据。点云数据采集过程一般为:将仪器与电源、微机连接并开启,打开数据识别和处理软件。建立定点参照目标,并开启扫描仪坐标系统的自动识别功能,建立三维坐标系统。在当前坐标系统内,对采集范围内的实体进行数字采集,并建立三维图形。一次采集完毕后,更换仪器地点,通过定点参照物重新识别当前坐标,进行数据的多次采集,并自动完成数据的空间合并。对扫描得到的云点数据进行先期处理,包括对模型的分割、修剪、移动、旋转、缩放等等。通过开放的数字接口,对当前模型数据进行转换,使

13、其与后期三维设计软件和开发软件兼容、并行和共享。同样我们也可以利用Atos扫描仪进行点云数据的采集。Atos三维扫描仪是一种带有两个CCD摄像机和一个中央投影单元的光学三维扫描仪。它的中央投影单元部分配备了一个白色的投射灯泡和一个可规则滑动的复杂光栅。Atos扫描仪的传感器被固定在一个三脚架上,并可以十分方便的沿四轴方向转动。测量时,投射灯泡将规则变化的光栅投影到被测工件表面产生的摩尔条纹,摩尔条纹的变化被CCD镜头记录下来,并转送到计算机,经过处理以后得到两个CCD镜头分别拍摄到的两张“三维”照片。由于两个CCD镜头可以感知高达440,000个象素,所以每一单幅照片可以采集到1.3万个有效数

14、据点。Atos软件可以在瞬间处理这1.3万个数据并精确的标定出其三维空间坐标值。在Atos扫描仪进行测量,即点云采集的过程中,误差的产生是难以避免的,但如果误差累积到一定程度,就无法达到精度的要求。因此,正确的测量顺序应该是由中部向四周逐渐扩展测量,这样做所得到的误差是最小的。 点云数据采集的工作特点：（1）多幅性。各种数据采集系统,由于测量范围的限制,所得的点云是一幅一幅进行测量的,一般情况下,一幅点云的测量并不能包含工件上所有需要的点,所以最终工件完整点云数据的获得需要利用多次测量的多幅点云进行拼合。这种拼合在测量系统中一般有自动拼合和人工拼合两种方式,也有相应的软件。自动拼合方式有边测量

15、边拼合,例如在光学三维扫描系统ATOS中的利用相邻两幅点云的共同参考点进行拼合,以及利用数码相机定位、经TRITOP软件处理生成的整体参考点数据与包含特定参考点的单幅点云数据拼合;也有利用工件表面特征在测量完成后进行自动拼合的,例如利用Geomagic软件进行的多幅点云的自动拼合。（2）工件的多样性。在测量中,工件的形状、尺寸以及工件中不同部位的精度不同,决定着需要采取不同的测量策略和测量手段来进行测量。一般情况下,对于工件需要进行几何反求的部分,例如汽车饭金、各种铸件外形、各种注塑件等,由于其功能各异,因而形状各异;由于其模型材料各异,有的是用油泥塑造的,有的是塑料件,有的是钢件,有的是玻璃

16、件,有的是海绵或橡胶件,因而表面形态各异,功能不同,复杂程度各异。（3）数据采集要求的多变性。在进行数据采集时,要明确所采集数据的用途,所采集的数据与产品中哪些部分相关或在空间以产品中哪些部件的相应部分为参考点,以决定数据采集是对产品部件单独进行,还是在产品装配中包含相关部件来进行。这在进行数据采集前首先要予以明确,以减少不必要的返工。数据采集中对工件不同部位精度要求不同,例如对发动机机体数据采集时,对用于定位的孔的尺寸精度要求较高。性材料构成的工件不可采用接触式测量,而须采用非接触测量。参考文献: 1 乔思航，程志权，陈寅，等基于三个Kinect 的个性化人体重建系统仿真学报，2013，25

17、(10)，2408 - 24112 李国镇基于 Kinect 的三维重建方法的研究和实现D北京：北京交通大学，20123 韦羽棉，尚赵伟基于 Kinect 的旋转刚体三维重建方法计算机与现代化，2014(5)：89 - 98 4 罗元,谢彧,张毅. 基于 Kinect 传感器的智能轮椅手势控制系统的设计与实现J.机器人,2012,(01).5 刘鑫，许华荣，胡占义. 基于GPU和 Kinect 的快速物体重建. 自动化学报，2012，38(8)：1288-1297.6 周瑾，潘建江，童晶，等.使用 Kinect 快速重建三维人体. 计算机辅助设计与图形学学报，2013，25(6)：873-87

18、9.7 宋诗超, 禹素萍，许武军基于 Kinect 的三维人体扫描、重建及测量技术的研究天津工业大学学报，2012，31(5)：34 - 41.8 孙晶晶，王金变，管玉基于三维扫描技术的人体测量天津工业大学学报，2012，31(5)：30-33. 9 宋诗超基于Kinect 的三维人体建模与测量的研究上海：东华大学，2013.10 朱德海点云库 PCL 学习教程北京：北京航天航空大学出版社，201211 余涛Kinect 应用开发实战：用最自然的方式与机器对话北京：机械工业出版社，201312 陶丽君基于深度信息的实时头部姿态估计厦门：厦门大学，201313 Henry P, Krainin

19、M, Herbst E, etal. RGB-D mapping: Using depth cameras for dense 3D modeling of indoor environments. In Proceedings of the International Symposium on Experimental Robotics(ISER), 2010.14 Newcombe R A, Izadi S, Hilliges O, etal. Kinect Fusion: Real-Time Dense Surface Mapping and Tracking, in IEEE ISMA

20、R, IEEE, October 2011.15 Izadi S, Kim D, Hilliges O, etal. Kinect Fusion:Real-time 3D reconstruction and interaction using a moving depth cameraJ. In Symposium on User Interface Software and Technology(UIST), 2011.5.5. 外文文献翻译The Microsoft KINECT: A Novel Tool for Psycholinguistic Research Rinus G. V

21、erdonschot1, Héloïse Guillemaud2, Hobitiana Rabenarivo2, Katsuo Tamaoka31. Waseda Institute for Advanced Study, Waseda University, Tokyo, Japan 2. Graduate School of Engineering, Nagoya University, Nagoya, Japan 3. Graduate School of Languages and Cultures, Nagoya University, Nagoya, Japan

22、Received 29 May 2015; accepted 26 June 2015; published 30 June 2015 Abstract The Microsoft KINECT is a 3D sensing device originally developed for the XBOX.The Microsoft KINECT opens up many exciting new opportunities for conducting experimental research on hu-man behavior. We investigated some of th

23、ese possibilities within the field of psycholinguistics(specifically: language production) by creating software, using C#, allowing for the KINECT to be used in a typical psycholinguistic experimental setting. The results of a naming experiment using this software confirmed that the KINECT was able

24、to measure the effects of a robust psycholinguistic variable (word frequency) on naming latencies. However, although the current version of the software is able to measure psycholinguistic variables of interest, we alsodiscuss several points where the software can still stand to be improved. The mai

25、n aim of this paper is to make the software freely available for assessment and use by the psycholinguistic community and to illustrate the KINECT as a potentially valuable tool for investigating human behavior, especially in the field of psycholinguistics. Keywords Language Production, Psycholingui

26、stics, KINECT, Psychological Research Tool Introduction The way we interact with technology is rapidly changing. While we were once limited to keyboards and poin-and-click devices, we can now interact with technology using our whole body. The rapidly decreasing cost of 3D sensing technologies (such

27、as the Microsoft KINECT), even allows us to interact with technology through facial expressions and voice information. Although this technology offers exciting new opportunities for experimental research onhuman behavior, the actual implementation of these novel technologies is still in its infancy.

28、 This paper highlights a potentially important role for KINECT technology in a particular area concerning the study of human behavior, namely language production (a subfield of psycholinguistics). This paper is structured as follows: First, we provide a brief background on the existing research and

29、theoretical models of language production, and summarize how dependent variables (such as naming latencies and accuracy) are usually obtained. Second, we introduce several important features of the KINECT sensor and review their potential applications within experimental psycholinguistic research. S

30、ubsequently, we discuss the C# software developed by our lab (all code freely downloadable), which implements the KINECT device to an experimental paradigm by depicting a characteristic experimental situation found in psycholinguistics. Next, we present experimental data within a genuine experimenta

31、l setting by testing 34 participants on a word-frequency paradigm by using the KINECT and validate this data by using an established method in the field (i.e., by voice key). Finally, we point out particular shortcomings of the current version of the software and avenues for resolving these shortcom

32、ings and implementing the KINECT in future research, both on language production and in general.1. Short Background on Language Production Research Although the KINECT offers advancements for behavioral research in many fields, this paper focuses on how the KINECT can benefit research on language pr

33、oduction (a part of experimental psycholinguistics). Within thelanguage production literature, there are several theoretical models that describe the way speech is produced: starting from ideas in our head and ending with the actual pronunciation of words (e.g. Dell, 1986; Levelt, Roelofs, & Mey

34、er, 1999). Most of the experimental data supporting these models comes from chronometric research (i.e. measuring reaction time latencies) using basic “triggering devices” such as buttons and voice keys (i.e.electronic circuits initiating a pulse if an input volume crosses a certain threshold). Typi

35、cal experimental paradigms used in language production research either show a particular stimulus on the screen or present a stimulus auditorily and wait for the participant to name a particular target out loud. The time it takes from seeing (or hearing) the stimulus to naming it out loud is called

36、the reaction time (RT) and serves as the main dependent variable together with the accuracy of the response. However, classic lab equipment such as voice keys only capture RTs for the onset of a single word at a time, and the difference between speech and other (irrelevant) sounds (e.g. coughing) ca

37、nnot be distinguished without time consuming post-hoc (or online) manual response checking (although there is freely available software which substantially eases and optimizes this task such as Check Vocal; Protopapas, 2007). This is because voice key triggering will simply occur if the input volume

38、 crosses a certain threshold. Additionally, data will be usually lost if the voice input does not exceed that threshold (e.g.when a participant speaks softly for instance). Moreover, voice keys have no semantic capabilities, which again instigate a need for manual response checking. Finally, some qu

39、estions have arisen about the reliability of voice keys. For example, when speaking, even after phonemes are produced it may take the voice key varying amounts of time to detect them, since some sounds take more or less time to initiate (e.g. /z/ versus /p/; see Kessler, Treiman, & Mullennix, 20

40、02; Sakuma, Fushimi, & Tatsumi, 1997). It is therefore reasonable to state that paradigms found in experimental psycholinguistics can be limited by particular aspects of experimental equipment.2. The Microsoft KINECT Device In contrast to devices designed to be implemented for scientific use onl

41、y, the KINECT is a device (costing roughly 200 USD) developed by Microsoft to be used with video games (e.g. on XBOX and Windows). The KINECT enables users to interact with a computer via gestures and voice commands.The KINECT (v1)1 contains an infrared (IR) emitter and IR depth sensor (640 × 4

42、80 pixels) for 3D tracking, a RGB camera (1280 × 960 pixels) to acquire high-quality RGB color video (both the IR depth sensor and the RGB camera operate at 30 fps) and a microphone array, which contains four microphones for capturing sound. The IR emitter emits infrared light in a predetermine

43、d “speckle pattern” (which are in fact small dots of infrared light that fall on everything in front of the KINECT camera). The IR depth sensor perceives these patterns and determines depth by looking at the displacement of specific dot patterns (e.g. on objects close to the KINECT the dot pattern w

44、ill be spread out, but on far objects the dot pattern will be much denser). Additionally, as there are four microphones, it is possible to accurately retrieve the spatial location of the sound source (e.g. a person speaking), as well as being able to record what is spoken. Furthermore, by using an a

45、ccelerometer it is possible to determine the current orientation of the KINECT and the integrated tilt motor can be used to track objects or people within the room.For research in language production, one particularly important feature of KINECT is its ability to track thehuman face . Microsoft has

46、made a so-called Software Development Kit (SDK; current version for KINECT v1 is 1.8) available which contains numerous programming routines to track a human face in real time. This SDK can measure roughly 100 points (including so-called “hidden points”) resulting in real-time face-tracking. Thus, t

47、he KINECT is able to build a detailed model of the human face, called a face mesh, using sets of triangles and lines. 3. Opportunities Offered by the KINECT for Research in PsycholinguisticsNaturally, the most important issue for researchers is how the KINECT can contribute to their research. The fo

48、llowing list, though incomplete, offers five potential ways we believe the KINECT could advance language production research: 1) The KINECT can track lip movements in real-time, allowing researchers to obtain detailed information on the speech planning process even before actual speech sounds are ut

49、tered. By focusing on the distances between particular points on the lips and face, in combination with the speech recognition pack (found in the SDK), it is possible to determine the onset and offset of individual words. In this paper we report our preliminary efforts to build a novel program that

50、detects the detection of the beginning and end of individual words, by tracking lip movements.2) Another exciting feature is that the KINECT is able to track more than one person over time, which would allow for language experiments to take place in a more natural, conversational setting.3) The KINE

51、CT has the potential to perform basic eye tracking, allowing researchers to assess approximately where participants are looking on a screen. Experimental paradigms may benefit from these additional behavioral measures, which could indicate, for instance, whether participants are engaged in the task

52、at hand, and, if so, which parts of the screen they are mainly fixating on. 4) The KINECT comes with advanced voice recognition (including language packs for many major languages), allowing for automatic post-hoc accuracy checking (see examples in the SDK provided by Microsoft).5) It has been shown

53、that the KINECT is able to on-line track and interpret body gestures and basic emotions,allowing for another dimension to be added to the dependent measures in a psycholinguistic experiment.4. A First Attempt to Implement the KINECT into the Area of Language Production As far as we know, there is no

54、 previous language production literature that utilizes the KINECT. This paper therefore represents the first attempt in this field to integrate the KINECT into the daily practice of a psycholinguistics lab. In this paper we focus on implementing the first of the five abovementioned points, that is,

55、the tracking of lip movements in real-time to gather information on the speech planning process.As there are no previous instances for comparison (again, as far as we know), we set out to program a working version of the KINECT software (using C#) to display experimental stimuli and measure a psycho

56、linguistic variable of interest. We aim to keep the code open and freely available for other researchers to use and adapt to their own insights. Obviously, when running the program, an attached KINECT for Windows, including the SDK is required (and Visual Studio is needed when adapting the code). To

57、 accommodate those who do not have this setup we provide a short video demonstrating the program online. Furthermore, the program(executable and source code) is provided .Notice that we provide the complete working directory in this file to have everything available to experienced programmers (for t

58、hose who simply want to run the program the executable can be found in /bin/x86/debug/FaceTrackingBasicsWPF.exe). The KINECT SDK v1.8 needs to be installed as well.Although the KINECT is able to track more than one person, in this initial stage of program development, only a single person is tracked

59、 during an experiment. The current version of the program is able to:1) Randomly display a word (taken from an Excel file) to a participant.2) Use the KINECT to determine the visual on- and offset of the word relative to its initial presentation (i.e.lip/face points) in real-time.3) Use the KINECT to detect the auditory on- and offset of the word relative to its