英文翻译651024419.doc

基于动态图像的目标跟踪与匹配算法研究概述基于图像的绘制技术（IBR技术）是一个新兴的从图像视频的密集采样中绘制真实场景的技术。最近，提出了一种基于一类动态图像的物体跟踪的方法，即所谓的全景视频。本文提出了一种使用水平集法的自动对象跟踪方法。与以往的只利用全局特征的做法相比，我们的这种跟踪方法既利用了图像序列的全局特征又利用了局部特征，达到了更好的跟踪对象的效果。由于可能存在物体边界分割误差，我们采用了Bayesian方法进行自动匹配。此外，我们还采用了类似于MPEG4的基于对象压缩全景视频的的算法，它包含了阿尔法图像、深度图像和全景视频中以分割图像为基础的不同的对象纹理流。实验结果表明，该方法可以获得满意的结果。1、 简介基于图像的绘制技术（IBR技术）是一个新兴的、有前途的从图像视频的密集的采样中逼真恢复出背景和对象的技术。IBR技术的核心是全光函数1，它发展形成了先进的虚拟现实与可视化系统的新框架。IBR技术的另一个重要优势是卓越的图像质量，它提供了三维模型的建立，特别是非常复杂的现实世界场景。无论现场的复杂性有多高，它都能够提供计算量较少的渲染方法。捕获，压缩和有效的渲染是IBR研究的基本问题1。文献中呈现了对象的渲染方法和一个以动态图像为基础的对象压缩，提出了所谓的全景视频。该全景视频具有一个四维全光功能，它通过捕获得到视频，该视频经常是沿着一系列的分割段，而不是放置在静态光场二维平面。主要的动机是为了减少在捕捉动态图像时的超高维度和过多的硬件成本。尽管简化成本，全景视频还是可以提供一系列场景，视差和连接图像光线变化的阵列。在基于对象的方法中，在较大差异下，不同对象划分为不同的层渲染和压缩。一个重要优势表现在顶端的光线区域，在大环境下的渲染质量可以得到显著改善4。此外，通过对对象级别的全景视频编码，可以实现最好的效果，如可扩展性的内容，差错控制，及与个别对象的交互性IBR技术等等。在基于对象的技术中一个重要步骤是把在视频流中的对象分割成一层层或者基于图像有着不同深度值的物体。目标提取方法的提出是基于Lazy snapping，这一采样是半自动性质的。要为分割全景视频减少分割时间，一个可能的方法是获取在使用特定半自动下一个视频的初始分割，在随后的时间内依靠跟踪技术来分割不同场景的对象。为此，本文提出利用水平集的自动目标跟踪方法方法。我们的方法利用局部和全局的图像特征序列，而不是在文献中只利用全局性等特点，达到更好的跟踪对象的结果。在全景视频中的物体被提取出来后，他们能够用已估计的深度图像单独的重建出来。由于深度的不连续性和分割差错，通常执行匹配。基本上，前景色F，背景色B，和不透明度component（也称为混合阿尔法图像）来估计物体边界周围的每个像素。使用阿尔法和纹理映射可以方便估计原始图像或其他全景视频图像中基于背景的对象。由于其良好的性能，我们在中采用Bayesian方法。并且，MPEG - 4类似的基于对象的压缩算法也可以压缩了基于对象的全景视频，这包含了阿尔法图像，深度图像和基于不同全景视频中图像对象的纹理。本文的结构如下：在2中论述采集系统的建立。第3节讨论目标跟踪算法。第4节中提出匹配算法和基于对象的压缩算法。第5节得出实验结果。最后，在第6节中得出的结论。2、 全景视频采集系统图1显示了全景视频系统用于捕捉动态场景。该系统包含了两个线性矩阵，两张图像是存取于6 JVC公司DRDVP9ah摄像机。在相机之间的两个连续线性矩阵间距15cm，矩阵之间的角度可以灵活调整。可连接更多矩阵在一起形成较长的视频。由于视频是录制的，该系统也更便于捕捉户外的动态场景。1万字动态全光场景随着各线阵相机矩阵被捕获，并且，用户可以凭借其意愿继续用摄像机延伸其线性矩阵。多元线性矩阵的使用使用户有更多的观看体育赛事和其他生活活动的自由。该系统是一个权衡简单和自由观看之间的设计。当然，也可使用其他配置的相机。该相机校准使用方法7。为了使用此方法来校正我们的相机矩阵，我们设计了一张巨大的参考网格，以便可以让所有的摄影机同时采集。使用提取的内在和外在的相机参数，摄像机的视频可以纠正的渲染。捕获后的视频数据存储在磁带可通过FireWire接口传输到电脑。所有这些组件是相对便宜，他们可以很容易地扩展。3、 利用水平集方法的目标跟踪像之前所提到的，有显著差异的不同目标被分割成一层层并分别被压缩。这避免了在目标边界导致的深度不连续。在该方法中，首次应用半自动的方式对目标进行初始分割。这种跟踪技术随后被应用到分割其他视频流和随后的瞬时的目标对象。我们的方法是基于水平集方法和几何偏微分方程（PDE）的。在目标跟踪，图像分割和图像分析中对PDE和曲率驱动的使用，可以得到超过过去数年的大衰减数值。其基本思路是通过一个给定的变形曲线，曲面，或图像得出其偏微分方程，并达成了作为本偏微分方程的稳态解的预期效果。这个问题也可以被看作是减少一定的能量函数： (1)作为一个曲线函数或C曲面。下标表示，函数是从给定的图像1计算得到的。通常，是设计来衡量所需的曲线从C在点x的偏差。为了尽量减少函数(1)，可以用变分方法来转换成为为偏微分方程函数（PDF）。一个用C的必要条件是局部最低的函数是 。一般的数值方法是先用一个初始曲线和一个虚构的时间变量t演变得出偏微分方程，决定于如下的： (2)然而，因为可能存在某些偏微分方程的奇异点，传统的有限差分法不适合求解方程(2)。一个求解方程(2)的重大突破是由Sethian and Osher13提出的，该方法通常被称为水平集方法。水平集方法的基本思想是代表“隐形式，如零水平集或更高维函数isophone”曲线或曲面。更正式的说，曲线的时间演化函数代表了一个嵌入函数： (3)其中C是一个给定的实常数。(2)可改写为如下的PDE函数： (4)其中在的正方向下的流速和它派生如上的。最初的曲线是用C = 0设置水平，即零水平的，其时间过程的计算方法是求解以下公式，之后在很小的时间间隔的离散化步骤： (5)其中是一个在等式右边近似相当于(4)的函数。最佳解决方案是在足够大的n值时偏微分方程收敛，并得到其解。对于我们的目标跟踪问题，我们定义为下面的能量函数的C： (6)其中设计和 分别控制的扩大和收缩曲线C的位置和。如果我们假定像素值是独立的高斯分布分别，其中和 的分别指曲线的内外，那么可以证明这样得到的偏微分方程可以写为（由于篇幅所限省略细节）： (7)其中，为正参数，是像素（x，y）的值，在指向曲线C内，代表曲线C以外，这是从使曲线平滑和连续的衍生的。有两种不同的方法确定和 ：基于全局和基于局部的不同的方法。基于全局的方法是在中通过利用所有像素驱曲线C，其中的的意思是表示的曲线C内的所有像素,是曲线以外的所有像素。全局的方法有许多好处，例如处理速度快和对噪声不敏感。然而，物体边界的一些细微特征进行跟踪时可能会丢失。图2显示了一个利用全局为基础的方法的追踪结果的例子。可以看出，女孩的右侧外的曲线，因为它意味着相比身体更近似背景。相反，局部的方法是使用一个窗口内的平均值，而不是局部的所有图像的像素。在文献，利用基于局部的方法，和 被设定如下：，其中是对所有整数对（i，j），| i | m and | j | m和曲线C内的像素（x+i，y + j），其中是对所有整数对（i，j），使得| i | m和| j | m和像素(x + i, y + j)以外的曲线最低值。显然，这种方法利用图像的局部特征，以应付有能量分布不均匀的对象。不幸的是，这种方法对噪声非常敏感，因为只有一个像素是确定和 两者。在这里，我们建议运用和 把基于全局和基于部分的方法的优点都结合起来： (8)4、 对象的匹配和压缩如前所述，由于周围的边界误差可能的分割和间断有限取样深度，它是首选的计算图像的对象和背景的成员较弱的相关性。换句话说，边界像素被假定为一个从前景色和背景的相应像素的线性组合： (9)在那里我，F和B是像素的复合变量，前景和背景颜色，是像素的不透明变量或alpha映射。利用这个模型，它可能匹配在不同场景和其他背景下的特定对象。数字模拟匹配（-图像）是由Porter和Duff于1984年提出的。在自动匹配中，所有变量，F和B必须估计，问题是从I中要找到，F和B中最有可能的估计，这可以制定后验概率P(F, B,| I )。使用贝叶斯规则，我们有： (10)由于优化参数P(I)是独立的，后者可以被丢弃。此外，如果F，B，被假定为独立，则(10)可以写成： (11)可以从上对于一个基于图像的对象渲染过程中，我们将还需要一个alpha图像和一个从传统的纹理图片或图像的几何信息中得到的深度图像。通过上述方法讨论alpha图像产生的自动匹配，它们是用于组成和渲染基于图像对象的。传送此全景视频信息需要有效地储存和压缩。我们已经采取了基于对象的编码方案，它有着许多符合MPEG - 4标准的概念，只是它支持的图像和EM的深度压缩能够得到更好的编码效率16。更确切地说，全景视频的视频流被分主流（MPEG - 4的编码算法没有提及）和其他流组。主流然后提供空间预测到相邻的二级视频流，它采用在时间上和空间上的预测以便在视频流中得到更好地冗余度。由于阿尔法图像和深度图想类似于纹理图像，两图像作为纹理图像亮度分量是采用相同的编码方式的。我们现在提出建议制度的实验结果。5、 实验结果我们评价建议跟踪方法的性能是用IBR技术计算Dance序列的。对于每一帧，初步曲线C是前一帧的跟踪结果，第一帧对象曲线是手动得到的。第一级设置轮廓变化是通过使用窄带方法，其中(7)中使用了速度函数。为计算局部灰度，窗口的大小m是固定为6。参数，是不固定的，在他们所依赖的是每个像素的和 。图2显示了在基于全局方法的跟踪结果。图3(a)-(i)表明使用我们的方法对Dance序列跟踪的结果，其中得到了很好的边界划定。对Ball用我们的方法序列跟踪结果显示在图4(a)-(h)。从结果可以看出，该方法的结果可以让更合理的对对象进行与非均匀的能量分布。基于图像对象的自动匹配结果见图5。图5(a)及(b)显示了一个叫做“dance”基于图像的对象部分，及其相关的alpha映射序列的计算。图5(c)和(d)显示的基于图像的对象实例效果图，有两个不同的背景或场景铺垫。图6显示了所提出的使用VM的速率控制算法计算得到的舞蹈序列编码压缩结果的纹理和形状。记曲线“MPEG - 4”代表使用没有空间预测的MPEG - 4的结果，而那些由“SP-3”和“SP-5”代表的使用空间预测算法分别记3和5视频流编码结果。可以看出，提出基于对象的编码方案已在PSNR（1分贝）效能上比的直接对视频流对象应用MPEG - 4得到相当大的改善。该算法的详细信息将报告其他地方。6、 结论我们提出了一个基于一类动态图像的自动目标跟踪的的算法调用方法的基础上的水平集的全景视频。自动铺垫与贝叶斯方法，以提高在深度和可能的分割不连续的误差下的渲染质量，它允许我们到不同全景视频中复现对象。此外，类似于MPEG 4的基于物体的跟踪算法还制定了压缩的基于对象的全景视频，其中包含了阿尔法图像，深度图像和的基于不同全景视频中图像对象的纹理。实验结果表明该方法有较好的实用性，质量和灵活性。7、 鸣谢作者在此感谢 Qing Wu 和 King-To Ng 在得出结果中所做的工作。8、 参考文献1 H. Y. Shum, S. B. Kang and S. C. Chan, “Survey of Imagebased Representations and Compression Techniques,” IEEE Trans. on CSVT, vol. 13, no. 11, pp. 1020-1037, Nov, 2003.2 Z. F. Gan, S. C. Chan, K. T. Ng and H. Y. Shum, “An Object-Based Approach to Plenoptic Videos,” to appear in Proc. of IEEE ISCAS2005.3 S. C. Chan, K. T Ng, Z. F. Gan, K. L. Chan and H. Y. Shum, “The Plenoptic Videos: Capturing, Rendering and Compression,” in Proc. of IEEE ISCAS2004, vol. 3, pp. 905-908, May, 2004.4 H. Y. Shum, J. Sun, S. Yamazaki, Y. Li and C. K. Tang,“Pop-up light field: An interactive image-based modeling and rendering system,” ACM Trans. on Graphics, vol. 23, issue 2,pp. 143 -162, April 2004.5 T. F. Chan and L. A. Vese, “Active Contours Without Edges,” IEEE Trans. on Image Processing, vol.10, no.2, Feb,2001.6 Y. Y. Chuang, B. Curless, D.Salesin, and R. Szeliski, “A Bayesian Approach to Digital Matting,” in Proc. of IEEE CVPR2001, vol.2, pp.264-271, 20017 Z. Zhang, “A Flexible New Technique for Camera Calibration,” IEEE Trans .on PAMI, vol. 22(11), pp. 1330-1334,2000.8 S. J. Osher and R. P. Fedkiw, “Level Set Methods and Dynamic Implicit Surfaces,” Springer Verlag, 2002.849 N. Paragios and R. Deriche, “Geodesic Active Contours and Levels Sets for Detection and Tracking of Moving Objects,” IEEE Trans. on PAMI, vol.22, pp. 266-280, 2000.10 G. Sapiro, “Geometric Partial Differential Equations and Image Analysis,” Cambridge University Press, Cambridge, U.K.,2001.11 J. A. Sethian, “Level Set Methods: Evolving Interfaces in Geometry, Fluid Mechanics, Computer Vision and Materials Sciences,” Cambridge University Press, Cambridge, U.K., 1996.12 A. R. Mansouri, “Region Tracking via Level Set PDEs without Motion Computation,” IEEE Trans. on PAMI, vol.24,no.7, pp.947 961, July, 2002.13 S. J. Osher and J. A. Sethian, “Fronts Propagation with Curvature Dependent Speed: Algorithms Based on Hamilton-Jacobi Formulations” J. Compute. Phys.79, pp.12-49, 1988.14 A. Yilmaz, X. Li and M. Shah, “Object Contour Tracking Using Level Sets,” in Proc. of ACCV2004, Korea, 2004.15 T. Porter and T. Duff, “Compositing Digital Image,” In SIGGRAPH84, pp. 253-259, July, 1984.16 S. C. Chan, K. T Ng, Z. F. Gan, K. L. Chan and H. Y.Shum, “The Compression of Simplified Dynamic Light Fields,”in Proc. of IEEE ICASSP2003, vol. 3, pp. 653-656, April, 2003. Object Tracking and Matting for A Class of Dynamic Image-based RepresentationsAbstractImage-based rendering (IBR) is an emerging technology for photo-realistic rendering of scenes from a collection of densely sampled images and videos. Recently, an object-based approach for a class of dynamic image-based representations called plenoptic videos was proposed. This paper proposes an automatic object tracking approach using the level-set method. Our tracking method, which utilizes both local and global features of the image sequences instead of global features exploited in previous approach, can achieve better tracking results for objects, especially with non-uniform energy distribution. Due to possible segmentation errors around object boundaries, natural matting with Bayesian approach is also incorporated into our system. Furthermore, a MPEG-4 like object-based algorithm is developed for compressing the plenoptic videos, which consist of the alpha maps, depth maps and textures of the segmented image-based objects from different video plenoptic streams. Experimental results show that satisfactory renderings can be obtained by the proposed approaches.1. IntroductionImage-based rendering (IBR) is an emerging and promising technology for photo-realistic rendering of scenes and objects from a collection of densely sampled images and videos. Central to IBR is the plenoptic function 1, which forms a new framework for developing sophisticated virtual reality and visualization systems. Another important advantage of IBR is the superior image quality that it offers over 3D model building, especially for very complicated real world scenes. It also requires much less computational power for rendering, regardless of the scene complexity. The capturing, compression and effective rendering are the fundamental problems in IBR research 1. In 2, an object-based approach for rendering and the compression of a class of dynamic image-based representations called plenoptic videos was proposed. The plenoptic video is a 4D plenoptic function and it is obtained by capturingvideos, which are regularly placed along a series of line segments, instead of a 2D plane in the static light fields 2, 3. The main motivation is to reduce the large dimensionality and excessive hardware cost in capturing dynamic representations. Despite the simplification employed, plenoptic videos can still provide a continuum of viewpoints, significant parallax and lighting changes along line segments joining the camera arrays. In the object-based approach, objects at large depth differences are segmented into layers for rendering and compression. An important advantage is that the rendering quality in large environment can be significantly improved, as demonstrated by the pop-up lightfields 4. In addition, by coding the plenoptic videos at the object level, desirable functionalities such as scalability of contents, error resilience, and interactivity with individual IBR objects, can be achieved.An important step in the object-based approach is to segment the objects in the video streams into layers or image-based objects with different depth values. The object extracting method proposed in 2 is based on Lazy snapping, which is semi-automatic in nature. To reduce the segmentation time for segmenting plenoptic videos, one possibility is to obtain an initial segmentation of the video at a particular view using semi-automatic tools and rely on tracking techniques to segment the objects at different views and at subsequent time instants. Towards this end, an automatic object tracking approach using the level-set method is proposed in this paper. Our method, which utilizes both local and global features of the image sequences instead of global features exploited in 5, can achieve better tracking results for objects, especially with non-uniform energy distribution.After the objects in a plenoptic video have been extracted, they can be rendered separately using the estimated depth map. Due to depth discontinuity and segmentation errors, matting 4, 6 is usually performed. Basically, the foreground color F, the background color B, and the opacity component (also called the alpha map for mixing) of each pixel around the object boundary are estimated. Using the alpha map and texture estimated, it is convenient to composite the image-based objects onto the background of the original or other plenoptic videos. The Bayesian approach in 6 is adopted in our system because of its good performance. Furthermore, a MPEG-4 like object-based compression algorithm is also developed for compressing the object-based plenoptic videos, which consist of the alpha maps, depth maps and textures of the segmented image-based objects from different video plenoptic streams.The paper is organized as follows: the construction of the proposed capturing system is described in Section 2. The proposed object tracking algorithm is discussed in Section 3. The matting algorithm and the proposed object-based compression algorithm are presented in Sections 4. Experimental results are presented in Section 5. Finally, conclusions are drawn in Section 6.2. The Capturing System for Plenoptic VideoFigure 1 shows the proposed plenoptic video system used to capture dynamic scenes. This system consists of two linear arrays of cameras, each hosting 6 JVC DRDVP9ah video cameras. The spacing between successive cameras in the two linear arrays is 15cm and the angle between the arrays can be flexibly adjusted. More arrays can be connected together to form longer segments. Because the videos are recorded on tapes, the system is also more portable for capturing outdoor dynamic scenes. Along each linear camera array, a 4D simplified dynamic light field is captured, and the users viewpoints are constrained along the linear arrays of video cameras. The use of multiple linear arrays allows the user to have more viewing freedom in sport events and other life performance. The proposed system represents a design trade off between simplicity and viewing freedom. Other configurations can also be employed. The cameras are calibrated using the method in 7. In order to use this method to calibrate our camera array, a large reference grid was designed so that it can be seen simultaneously by all the cameras. Using the extracted intrinsic and extrinsic parameters of the cameras, the videos of the cameras can be rectified for rendering. After capturing, the video data stored on the tapes can be transmitted to computers through FireWire interface. All these components are relative inexpensive and they can readily be extended to include more cameras.3. Object Tracking Using Level-set MethodAs mentioned earlier, objects at large depth differences are segmented into layers and are compressed and rendered separately. This helps to avoid the artifactssegmented into layers and are compressed at object boundaries due to depth discontinuities. In the proposed method, an initial segmentation of the objects is first obtained using semi-automatic approach. Tracking techniques are then employed to segment the objects at other video streams and subsequent time instants. Our method is based on the level set method or geometric partial differential equations (PDE). The use of PDE and curvature-driven flows in tracking, segmentation and image analysis has received great attenuation over the last few years 8-12. The basic idea is to deform a given curve, surface, or image according to the PDE, and arrive at the desired result as the steady state solution of this PDE. The problem can also be viewed as minimizing a certain energy function: (1)as a function of a curve or surface C. The subscript indicates that the energy is computed from the given images I. Usually, F(C, x) is designed to measure the deviation of the desired curve from C at point x. To minimize the functional in (1), the variational approach can be employed to convert it to a partial differential function (PDF). A necessary condition for C to be a local minimum of the functional is U (C) = 0 I . A general numerical approach is to start with an initial curve C0 and let it evolve over a fictitious time variable t according to a PDE, which depends on the derivativeU (C) I as follows: (2)However, conventionally finite difference methods are unsuitable to solve (2), because the PDE might be singular at certain points. A major breakthrough in solving (2) is due to Sethian and Osher 13, and the method is commonly referred to as the level-set method. The basic idea behind the level-set method is to represent a curve or surface in “implicit form” such as the zero level sets or isophone of a higher dimensional function. More formally, the time evolution of curves C(x, t) is represented as the level-set of an embedding function (x, t) : (3)where c is a given real constant. (2) can be rewritten as a PDE of (x, t) as follows: (4)where is the velocity of the flow in the normal direction and it is derived fromU (C(t) above. The initial curve C0 is associated with the level set with c=0, i.e. zero level set, and its time evolution is computed numerically by solving the following equation for (t) , after discretizing at a sufficiently small time interval or step t : (5)where G( , x) is an appropriate approximation of the right hand side of (4). The desired solution is obtained when the PDE converges at sufficiently large value of n. For our object tracking problem, we define the following energy function for curve C: (6)where Cinside(x,y) and Coutside(x,y) are two functions designed respectively to control the expansion and contraction of the curve C at location (x,y), and Length(C) measures the length of the curve. If we assume that the pixel values are independent and Gaussian distributed with means cin and cout respectively inside and outside the curve, then it can be shown that the PDE so obtained can be written as (details omitted due to page limitation): (7)where , and are positive parameters, ( x, y) u is the value of pixel (x, y), in c denotes the driving force inside the curve C, and out c represents driving force outside the curve C. The third term, which is derived from Length(C), makes the curve smooth and continuous.There are two different methods for determining in c and out c : global-based and local-based methods. The global-based method which is adopted in 5 utilizes all the pixels to drive curve C, where in c denotes the mean of all pixels inside the curve C, and out c is the mean of all pixels outside the curve C. The