基于JND的立体感知视频编码(引言和结论).doc

基于JND的立体感知视频编码Lei Zhang, Qiang Peng, Qiong-Hua Wang, and Xiao Wu, Member, IEEEAbstractStereoscopic video coding is a key technology for the development of three-dimensional television (3DTV). Nonetheless, the task remains highly challenging owing to the large amount of 3DTV. Traditional stereoscopic video coding is found to be ineffective in reducing the unnecessary stereoscopic perceptual redundancy. Thus better compression efficiency is highly desired in stereoscopic video coding. In this paper, we propose a foveation weight model based on auto-stereoscopic display, and incorporate it into a stereoscopic just-noticeable-distortion (JND) model to give more accurate estimation of the stereoscopic perceptual effect. In order to utilize this model to improve the stereoscopic video coding efficiency, we present a block-adaptive residue pre-processing method based on stereoscopic JND model. Extensive experiment results demonstrate that the proposed scheme can efficiently reduce the unnecessary perceptual redundancy without visual quality degradation.Index TermsJust-noticeable-distortion (JND), perceptual video coding (PVC), stereoscopic摘要：立体视频编码是3DTV发展中的一项关键技术。然而，这项工程巨大的挑战是3DTV庞大的数据量。传统的立体视频编码在减少不必要的立体感知冗余方面效率低下。因此立体视频编码迫切需要更加高的压缩效率。本文提出一种基于自动立体显示的中心凹权重模型，将其结合到一个立体JND模型中，更准确地估计立体感知效应。为了利用这个模型提高立体视频编码效率，我们给出基于立体JND模型的块自适应残差预处理方法。大量的实验证明所提出的方案可以在不降低视觉质量的同时有效地减少不必要的感知冗余。1. IntroduceExtending visual sensation to the third dimension has been investigated over decades. Stereoscopic video can provide users a sense of depth perceptual by present a pair of stereoscopic images to a viewers left and right eyes. It can give users vivid information on the scene structure. With the development of three-dimensional television (3DTV), stereoscopic video coding is drawing more and more attention. Compared to mono-video systems, the stereoscopic video systems require double bandwidth. Thus, an efficient coding scheme is necessary for transmitting stereoscopic videos. Recently, several studies on stereoscopic video coding have been performed to improve the coding efficiency, which take into account the statistical properties among inter-views. Nevertheless, the stereoscopic visual perceptual features are usually neglected, since traditional video codec based on the signal-processing techniques does not explore the visual perceptual properties. Although the peak signal-to-noise ratio (PSNR) is widely used as the video quality evaluation criteria of video codec, PSNR has been criticized for not correlating well with the perceptual of human visual system (HVS) 1.延伸到三维的视觉感受已经研究了几十年。立体视频通过向观看者的左右眼呈现一对立体图像为用户深度感。它可以给用户带来生动的场景结构信息。随着立体电视（3DTV）的发展，立体视频编码吸引了越来越多的关注。与单视频系统相比，立体视频系统需要双倍带宽。因此，为了传输立体视频需要一个有效的编码方案。最近，一些关于提高编码效率的立体视频编码的研究已经实施，其中考虑到视点间的统计特性。尽管如此，立体视觉感知特性通常被忽视，因为传统的基于信号处理技术的视频编解码器并没有探讨视觉感知特性。尽管峰值信号信噪比（PSNR）作为视频编解码器的视频质量评价标准被广泛地使用，但是由于不能很好地和人眼视觉系统的感知特性良性结合而遭受异议1。It is well known that HVS cannot perceive all changes in images/videos due to its underlying physiological and psychological mechanism 2. There exist a lot of visual perceptual redundancy in images and videos. Since the ultimate video quality is evaluated by human, it is meaningful to incorporate HVS into the stereoscopic video coding. It is expected that better coding performance could be achieved by discarding unnecessary perceptual redundancy and protecting important visual information.众所周知的，HVS由于其基本的生理和心理机制无法察觉图像/视频中所有的变化2。图像和视频中存在大量的视觉感知冗余。由于最终的视频质量是由人眼进行评价，结合HVS的立体视频编码是有意义的。期望可以通过丢弃不必要的感性冗余和保护重要的视觉信息以达到更高的编码性能。Two major issues have to be addressed in designing a stereoscopic video coding system based on HVS. The first issue is to establish more accurate HVS models to provide stereoscopic perceptual detection. Just noticeable distortion (JND) 3 is a promising way to model the perceptual redundancy, which accounts for the maximum distortion threshold that HVS does not perceive. Many HVS features, such as luminance, texture, temporal and foveation masking, have been utilized to determine the JND models 410. However, all these JND models are built for mono-video, which are incompetent for discarding the stereoscopic perceptual redundancy. According to the structure and property of auto-stereoscopic display, current stereoscopic perceptual can only be perceived at the ideal distance and in the correct position 11. This indicates that the best stereoscopic viewing position based on the auto-stereoscopic display can be combined with the foveation perceptual feature to determine the stereoscopic perceptual redundancy。基于HVS的立体视频编码系统设计要解决两个主要问题。第一个问题是要建立更精确的人类视觉模型，提供立体知觉检测。JND3是一个有前途的方式来模拟感知冗余，JND表示HVS不能感知的最大失真阈值。许多HVS特性，如亮度，纹理，时空和中心凹掩蔽，被用来确定JND的模型4 - 10。然而，所有这些JND的模型是专为单视频，用于去除立体感知冗余存在不足之处。根据自动立体显示器的结构和属性，目前立体感知只能在理想的距离和正确的位置才能被感知到11。这表明，最好的基于自动立体显示器的立体观看位置可以与中心凹感知特性相结合，确定立体感知冗余。The second issue is to incorporate HVS models into the stereoscopic video codec to improve the coding performance. Many HVS-based codecs have been proposed for this purpose 1218. Unfortunately, most of these methods can be viewed as a special case of region-of-interest image processing, which improves the image quality of important visual regions and decreases the quality of other regions. The objective of this paper is to adopt a perceptual coding scheme for stereoscopic perceptual video coding, which aims to remove the perceptual redundancy without the perceivable loss of the reconstructed video quality. Furthermore, the perceptual coding scheme should be established in image-domain that is consistent with the representation of stereoscopic perceptual features. Considering these factors, the residue preprocessing method 16 using the perceptual feature to adjust residue information is a suitable way to guide the stereoscopic perceptual video coding task.第二个问题是将HVS结合到立体视频编解码器中以提高编码性能。已为此提出了许多人类视觉系统为基础的编解码器12 - 18。不幸的是，这些方法大多数可以被视为感兴趣图像处理的特例，提高了重要的视觉区域的图像质量，但降低了其他地区的质量。本文的目的是采用感知立体感知视频编码方案，旨在消除重建视频质量的感知冗余的同时不感知质量。此外，感知编码方案应建立在图像域从而和立体感知特性保持一致。考虑到这些因素，利用感知特性的残留预处理方法16，调整残留信息引导感知立体视频编码是一个合适的方式。In this paper, the main contribution is the novel idea of utilizing the stereoscopic perceptual features based on auto-stereoscopic display to improve the stereoscopic coding efficiency. Firstly, to characterize the property of stereoscopic perceptual in auto-stereoscopic display, we utilize the best stereoscopic viewing position to establish a foveation weighting model. Integrating the foveation weighting model with the basic JND models, an image-domain stereoscopic JND model is presented for efficient stereoscopic visual perceptual detection. Secondly, we adopt the residue preprocessing method based on stereoscopic JND model to reduce the unnecessary perceptual redundancy. Since the image distortion and bitrate will be altered after residue preprocessing, we present a block-adaptive residue pre-processing parameter determining method by minimizing the overall rate-distortion cost to achieve better perceptual coding performance.本文的主要贡献是提出一种新的利用基于自动立体显示立体的立体感知特性提高立体编码效率的想法。首先，在自动立体显示器上定义立体感知的特性，我们利用立体的最佳观看位置，建立foveation加权模型。将foveation加权模型结合到基本的JND模型中，提出一种基于图像的立体JND模型进行高效的立体感知检测。其次，我们采用基于立体JND模型的残差预处理方法减少不必要的感性冗余。由于图像失真和比特率在残差预处理后将改变，我们提出了一个块自适应残留预处理参数确定方法，通过减少整体成本，以达到更好的感知编码性能。The remainder of this paper is organized as follows. In Section II, we give a brief description of related work. Section III, discusses a foveation weighting model and stereoscopic JND model based on auto-stereoscopic display. A block adaptive residue preprocessing scheme based on stereoscopic JND model is presented in Section IV. Experiments are evaluated in Section V. Finally, we summarize this paper with a conclusion in Section VI.本文其余部分的安排如下：在第二节中，我们给了相关工作的简要说明。第三节，讨论了foveation加权模型和基于自动立体显示的立体JND模型。第四节在立体JND的模型的基础上提出了块自适应残留预处理方案，第五部分是实验分析，最后，第六节是结论。6.结论In this paper, we utilize the stereoscopic perceptual features of auto-stereoscopic display to develop a foveation weight model and present a stereoscopic JND model for stereoscopic perceptual detection. A stereoscopic perceptual video codec is presented to provide better stereoscopic coding performance. The main advantage of the proposed scheme is to remove the perceptual redundancy without the perceivable loss of the reconstr