视讯编码技术与硬体架构设计

1、Video Coding Techniques and Architectures DesignDepartment of Electronic Engineering, FJUVideo Coding Technique and Architectures Design1Video Coding Techniques and Hardware Architecture Design視訊編碼技術與硬體架構設計 Chapter 1IntroductionVideo Coding Techniques and Architectures DesignDepartment of Electronic

2、 Engineering, FJUVideo Coding Techniques and Architectures Design2SyllabusInstructor: Shyue-Kung Lu (呂學坤) FJU EE Building, Room 707. TEL: 29031111-3800, email: .twRecommended Texts1.Handouts 2. Papers References 1. Vasudev Bhaskaran and Konstantinos Konstantinides, “Image and Video Com

3、pression Standards,” Kluwer Academic Publishers, 2nd Edition. 2. Iain E. G. Richardson, “H. 264 and MPEG 4,” Wiley, 2004. 3. VLSI Implementations for Image Communications (authors: Peter Pirsch.) 4. Reuse Methodology Manual for System-On-A-Chip Designs by Michael Keating, Pierre Bricaud (Hardcover -

4、 June 1999) 5. Peter Kuhn, “Algorithms, Complexity Analysis and VLSI Architectures for MPEG-4 Motion Estimation,” Kluwer Academic Publishers. 6. 戴顯權、陳瀅如、王春清、”多媒體通訊,” 紳藍出版社，2003 7. 吳炳飛、胡益強、蘇崇彥、瞿忠正， ”JPEG 2000 影像壓縮技術”，全華科技圖書，2003(Grading)1. Homework & Class Participation : 10%2. Architecture Discu

5、ssion and Mid-term Report (Present at least one journal paper): 15%3. Mid-term exam 25%4. Final-term exam 25%3. Term Project and Presentation (Present at lest one paper and implement the proposed architecture) : 25%Video Coding Techniques and Architectures DesignDepartment of Electronic Engineering,

6、 FJUVideo Coding Techniques and Architectures Design3Topics for Term Project and Final Projectq Block Based(8 x 8) DCT/IDCTq Motion Estimation and Compensation (Architectures)q Motion Estimation and Compensation (Algorithms)q Variable-Length Coding (Huffman Coding, Arithmetic Coding)Video Coding Tec

7、hniques and Architectures DesignDepartment of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design4Contentsq Compression Fundamentals Video Formats and Quality Video Coding Conceptsq Method and Standards for Lossless Compressionq Fundamentals of Lossy Image Compressionq JPEG &

8、amp; JPEG 2000q Fundamentals of Lossy Video Compressionq A Family of VLSI Designs for the Motion Compensation Block-Matching Algorithmq MPEG Video Standards & MPEG-4 Visualq Hardware Architecture of Entropy Coding q Overview of MPEG-4 and H.264Video Coding Techniques and Architectures DesignDepa

9、rtment of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design5Digital ConvergenceVideo Coding Techniques and Architectures DesignDepartment of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design6Multimedia Technology for Human Lifeq From office to home

10、 and the outdoorsq From large devices to portable devicesq From specific people to everybodyVideo Coding Techniques and Architectures DesignDepartment of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design7Natural Video Sceneq A typical “real world” or natural video is compos

11、ed of multiple objects each with their own characteristic shape, depth, texture and illumination.q Characteristics of a typical natural video scene that are relevant for video processing and compression include spatial characteristics (texture variation within scene, number and shape of objects, col

12、our, etc.) and temporal characteristics (object motion, changes in illumination, movement of the camera or viewpoint and so on)Video Coding Techniques and Architectures DesignDepartment of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design8Captureq A natural scene is spatial

13、ly and temporally continuous.q Spatial sampling The most common format for a sampled image is a rectangle with the sampling points positioned on a square or rectangular grid (pixel).q Temporal Sampling A moving video image is captured by taking a rectangular “snapshot” of the signal at periodic time

14、 intervals. Sampling at 25-30 complete frames per second is standard for television pictures. Frame rates below 10 frames per second are some times used for very low bit-rate video communications but motions clearly jerky and unnatural at this rate.Video Coding Techniques and Architectures DesignDep

15、artment of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design9Frames and Fieldsq A video signal may be sampled as a series of complete frames (progressive sampling) or as a sequence of interlaced fields (interlaced sampling).Video Coding Techniques and Architectures DesignDe

16、partment of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design10Colour Spacesq A monochrome image requires just one number to indicate the brightness or luminance of each spatial sample.q Colour images, on the other hand, require at least three number per pixel position to r

17、epresent colour accurately.q In the RGB colour space, a colour image sample is represented with three numbers that indicate the relative proportions of Red, Green, and Blue.q The human visual system (HVS) is less sensitive to colour that to luminance (brightness). It is possible to represent a colou

18、r image more efficiently by separating the luminance from the colour information and representing luma with a higher resolution than colour.Video Coding Techniques and Architectures DesignDepartment of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design11YCbCrq The YCbCr colo

19、ur space and its variations (sometimes referred to as YUV) is a popular way of efficiently representing colour image.q Y is the luminance (luma) component and can be calculated as a weighted average of R, G, and B:Y = krR + kgG + kbBq The colour information can be represented as colour difference (c

20、hrominance or chroma) components.Cb = B Y, Cr = R Y, Cg = G - YVideo Coding Techniques and Architectures DesignDepartment of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design12YCbCr (Cont.)q YCbCr has an important advantage over RGB, that is the Cr and Cb components may be

21、represented with a lower resolution than Y because the HVS is less sensitive to colour than luminance.q ITU-R recommendation BT.601 defines kb = 0.114 and kr = 0.299.Y = 0.299R + 0.587G + 0.114BCb = 0.564(B-Y)Cr = 0.713 (R-Y)R = Y + 1.402CrG = Y-0.344Cb-0.714CrB = Y + 1.772CbVideo Coding Techniques

22、and Architectures DesignDepartment of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design13YCbCr Sampling Formatsq Sampling patterns for Y, Cb and Cr supported by MPEG-4 Visual and H.264. 4:4:4 sampling 4:2:2 samplingVideo Coding Techniques and Architectures DesignDepartment

23、of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design144:2:0 Samplingq 4:2:0 sampling is sometimes described as 12 bits per pixel.Video Coding Techniques and Architectures DesignDepartment of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design15Interl

24、aces Video SequenceTop fieldTop fieldTop fieldBottom fieldBottom fieldBottom fieldVideo Coding Techniques and Architectures DesignDepartment of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design16Video Frame Formatsq CIF: Common Intermediate FormatFormatLuminance resolution(

25、horiz. vert.)Bits per frame(4:2:0, eight bits per sample)ApplicationsSub-QCIF128 96147456Mobile multimedia applicationsQuarter CIF (QCIF)176 144304128VideoconferencingMobile multimedia applicationsCIF352 2881216512Videoconferencing4CIF704 5764866048Standard Definition TV, Video Coding Techniques and

26、 Architectures DesignDepartment of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design17The Role of Video Codingq If we want to store video in CD-ROM 30 fps 720 480 resolution Generate data at 20.736 MBytes/sec Only 31 seconds of video can be stored on 650 Mbytes CD-ROM.q We

27、need compression!q Compression is the core of a digital video system.Video Coding Techniques and Architectures DesignDepartment of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design18Core Technology-Compression (Coding)Video Coding Techniques and Architectures DesignDepartme

28、nt of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design19Video Coding TechnologyVideo Coding Techniques and Architectures DesignDepartment of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design20Statistical Redundancyq Image, video, and audio signals

29、 are amenable to compression due to the following factors: Spatial Correlation Within a single image or a single video frame, there exists significant correlation among neighbor samples. Spectral Correlation For data acquired from multiple sensors (such as satellite images), there exists significant

30、 correlation among samples from these sensors. Temporal Correlation There is significant correlation among samples in different segments of time.Video Coding Techniques and Architectures DesignDepartment of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design21Statistical Redu

31、ndancy (Cont.)q There is considerable information in the signal that is irrelevant from a perceptual point of view.q Some data tend to have high-level features that are redundant across space and time, that is, the data is of fractal nature.For a given application, compression schemes may exploit an

32、y one or all of the above factors to achieve the desired compression data rate.Video Coding Techniques and Architectures DesignDepartment of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design22Generic Compression Systemq Compression ratio (Cr) = source coder input size/sourc

33、e coder output size.q A more commonly used notion for size is the bits needed to represent one second of video.SourceCoderChannelCoderChannel DecoderSource DecoderEncoderDecoderVideo Coding Techniques and Architectures DesignDepartment of Electronic Engineering, FJUVideo Coding Techniques and Archit

34、ectures Design23Compression TaxonomyImage, Video, and Audio Compression Methodsq Model-Based Liner-predictive Coding (LPC) AR, ARMA Modeling Polynomial Fitting Fractals Object-Based Otherq Waveform-Based Lossless LossyVideo Coding Techniques and Architectures DesignDepartment of Electronic Engineeri

35、ng, FJUVideo Coding Techniques and Architectures Design24Compression Taxonomy (Cont.)q Lossless Compression Statistical Gilbert Fano Huffman Otherq Lossy Spatial Domain, Time Domain Delta Modulation PCM DPCM Vector Quantization Other Universal Arithmetic Coding Lempel-Ziv (LZ) Coding Pattern Matchin

36、g Other Frequency Domain Filter-Based Subband, Wavelet Transform-Based Fourier Karhunen-Loeve (KL) Hadamard DCTVideo Coding Techniques and Architectures DesignDepartment of Electronic Engineering, FJUVideo Coding Techniques and Architectures Design25Lossless Compressionq In many applications, the de

37、coder has to reconstruct the original data without any loss.q This is also referred to as a reversible process.q In lossless compression, for a specific application, the choice of a compression method involves a tradeoff between coding efficiency, coding complexity, and coding delay.Video Coding Techn