第12章MPG视频编码II

1、第12章 mpeg视频编码iislide 1 目录mpeg-4概述可视对象编码合成对象编码mpeg-4 overviewslide 3mpeg-4 可视对象编码的特点综合性：自然音视频对象与合成音视频对象的集成交互性：选择播放，超链等高效率的压缩编码：1/51/10的mpeg2码率，几乎相同的质量mpeg-4可视对象的编码slide 5第1代视频编码the smallest entity in a picture is a pixel with its associated texture (color), and motionmessage to be coded for every pix

2、el: texture (color) + motionslide 6第1代视频编码的不足 与人的视觉本质不同 不易控制场景中的不同对象潜力有限slide 7第2代视频编码将一个场景分为一系列组成对象，对每个对象分别编码slide 8第2代视频编码slide 9第2代视频编码the smallest entity in a picture is an object with its associated shape, texture (color), and motionmessage to be coded for every pixel: shape + texture (color) +

3、 motionslide 10mpeg-4的音视场景slide 11mpeg-4音视场景的描述在mpeg-4中，音视场景采用基于对象的描述方式，场景由媒体对象以层次方式组合而成（树），叶节点是初级(primitive) 媒体对象，例如: 静止图像 (固定不变的背景), 视频对象 (没有背景的说话人) 音频对象 (说话人所发出的声音); 其他，如文本和图形. 初级媒体对象可以是自然的，也可以是人造(合成)的, 可以是 2维,也可以是3维. 使用bifs的（binary format for scenes）语言来对场景的组成、场景中的音视对象的时空关系进行描述slide 12mpeg-4的音视场景

4、假想的观察者位置视频复合投影平面场景坐标系用户输入下载的数据/控制复合流上载的数据/控制复合流场景人2d背景家具演示地球仪讲台声音教师(场景的逻辑结构)slide 13mpeg-4 场景描述的优点可以集成各种对象，无缝地集成自然媒体(源于麦克风、摄象机等)与人造媒体(计算机生成) 、实时信息与存储信息， av0可以是单双多声道音频信息、单双多镜头2d3d视频信息。提供更强的交互能力，场景中的对象（人、桌子、地球仪、白扳、人的声音）以及多媒体演示声音均作为单个对象而独立编码，用户可以有选择地与其中某(几)个对象交互。具有良好的重用性，可重新组合音视对象 avo (audio visual obj

5、ect)构造新场景。slide 14bifs 示例示例slide 15mpeg-4视频流结构视觉对象序列（vs:visual object sequence）视频对象（vo:video object）视频对象层（vol:video object layer）视频对象平面组（gov: group of vop）视频对象平面（vop:video object plane）slide 16vop的编码vop的描述：形状(shape)、运动(motion)、纹理(texture)。muxbuffertexturecodingmotioncompensationmotionestimationprevi

6、ous reconstruction vop+-shapecoding vop of arbitrary shape vop of arbitrary shape shape infomotion infotexture info输入vopvop编码器slide 17基于vop的运动补偿mc-based vop coding in mpeg-4 again involves three steps: motion estimation. mc-based prediction. coding of the prediction error. only pixels within the vop

7、 of the current (target) vop are considered for matching in mc.to facilitate mc, each vop is divided into many macro blocks (mbs). mbs are by default 1616 in luminance images and 88 in chrominance images.slide 18motion compensationslide 19padding an example of repetitive padding in a boundary macrob

8、lock of a reference vop: (a) original pixels within the vop, (b) after horizontal repetitive padding, (c) followed by vertical repetitive padding.slide 20motion vector目标vop中的每个宏块在参考vop中寻找一个最佳匹配宏块。n the size of the mb. map(p; q) = 1 when c(p; q) is a pixel within the target vop, otherwise map(p; q) =

9、 0.运动矢量编码与h.263类似，采用预测编码1010),(),(),(),(nknllykxmapljykixrlykxcjisadslide 21texture codingtexture coding in mpeg-4 can be based on: dct or shape adaptive dct (sa-dct).i. texture coding based on dct in i-vop, the gray values of the pixels in each mb of the vop are directly coded using the dct followe

10、d by vlc, similar to what is done in jpeg. in p-vop or b-vop, mc-based coding is employed it is the prediction error that is sent to dct and vlc.slide 22texture coding（cont.）coding for the interior mbs: each mb is 1616 in the luminance vop and 88 in the chrominance vop. prediction errors from the si

11、x 88 blocks of each mb are obtained after the conventional motion estimation step.coding for boundary mbs: for portions of the boundary mbs in the target vop outside of the vop, zeros are padded to the block sent to dct since ideally prediction errors would be near zero inside the vop. after mc, tex

12、ture prediction errors within the target vop are obtained.slide 23shape adaptive dct7,.,0)(16) 12(cos2)()(70uifuiucufi10)(2) 12(cos)(2)(niifnuiucnufn优点：不会产生多余的系数n缺点：需要额外的模板记录最初的形状nshape adaptive dct (sa-dct) is another texture coding method for boundary mbs.n due to its efctiveness, sa-dct has been

13、adopted for coding boundary mbs in mpeg-4 version 2.slide 24 shape adaptive dct(cont.)slide 25shape codingmpeg-4 supports two types of shape information, binary and gray scale.binary shape information can be in the form of a binary map (also known as binary alpha map) that is of the size as the rect

14、angular bounding box of the vop. a value 1 (opaque) or 0 (transparent) in the bitmap indicates whether the pixel is inside or outside the vop. alternatively, the gray-scale shape information actually refers to the transparency of the shape, with gray values ranging from 0 (completely transparent) to

15、 255 (opaque).slide 26分割出来的前景图像作为一个任意形状的vo进行编码只在视频序列的第1帧画面时传输1次,保存在背景缓冲器中, 此后仅仅传输描述镜头运动的8个参数sprite coding在编码前从一系列的视频画面中把背景图像抽出并拼合而成使用8个参数，对背景进行仿射变换，重建出每一帧画面的背景mpeg-4的合成对象编码slide 282d mesh codinguniformmeshdelaunayslide 29coding of delaunay triangulationexcept for the first location (x0, y0), all sub

16、sequent coordinates are coded differentially that is, for n1,dxn = xn xn1; dyn=ynyn1; and afterward, dxn, dyn are variable-length coded.slide 302d mesh motion codinga new mesh structure can be created only in the intra-frame, and its triangular topology will not alter in the subsequent inter-frames enforces a one-to-one mapping in 2d mesh motion estimation.for any mop triangle (pi, pj, pk), if the motion vectors for pi and pj are known to be mvi and mvj, then a prediction predk will be made for the motion vector of pk and this is rounded to