外文翻译空间变化运动模糊的隔行扫描ccd图像复原

外文原文Image restoration for interlaced scan CCD image with space-variantmotion blursabstractWhen the speeds of objects in a scene exceed the temporal resolution of the camera shutter, motion blurs will occur. Since objects are often moving in different directions at different speeds, the degradation of a CCD image is often characterized by space-variant motion blurs. Image restoration algorithms for space-variant motion blurs are available for progressive scan CCD images, but not for interlaced scan images. To address the space-variant image restoration for interlaced scan images, a novel three-step image restoration scheme is proposed. Firstly, one interlaced scan image is divided into odd eld and even eld images. Secondly, these two eld images are further segmented into rectangular blocks and the motion vectors are computed based on these rectangular blocks using an efcient block matching algorithm. Thirdly, image restoration is performed using a blind deconvolution algorithm in the odd or even eld image. The nal restored image is obtained by combining the restored odd and even eld images. The scheme is illustrated by restoring a space-variant blurred moving vehicle image and a synthetic blurred image.1. IntroductionWhen the speeds of different objects in a scene exceed the temporal resolution of the camera shutter, the output of a CCD camera is usually degraded by motion blurs subject to physical and technical limitations. Depending on the imaging process, image degradation caused by motion blur can be classied as either a space-invariant or a space-variant distortion. The space-invariant motion blur corresponds to the cases in which the image degradation model does not depend on the positions in the CCD image. This type of motion blurred image is often a result of camera motion during the imaging process. Image restorations for space-invariant motion blurs have been addressed by manyschemes 14. The goal is to nd the point spread function (PSF) of the blurring system and then apply deconvolution techniques to restore the ideal images. For space-variant motion blur, the PSF, which causes the degradation, is a function of position in the image. This type of motion blur usually appears in an image containing fast-moving objects with different motions recorded by a static CCD camera. Image restoration for space- variant motion blur remains a big challenge and to date has been addressed only by a few researchers.Previous work on space-variant image restoration is relatively sparse. The main space-variant image restoration schemes include coordinate transforms 5, sectional processing 68, iterative lters 9,10, 2D Kalman lters 11,12, projection onto convex sets (POCS) 13, and multi-channel variational 14,15. The coordinate transformation scheme is limited to types of motion for which transformations can be derived to change the space-variant problem into a space-invariant one. This restoration has been done for rotational motion blurs and some types of geometrically induced motion blurs such as produced in satellite photography. In sectional schemes, the image is sectioned into rectangular regions, where each section is restored using a space-invariant method, such as the maximum a posterior lter 6 or the modied Landweber iterative lter 7. An accurate estimation of the local motion vector for each section is required in sectional schemes, and this estimation is often performed using adjacent image frames from an image sequence. An iterative lter is presented for space-variant image restoration in Ref. 10, where the distortion process was separated for several layers or images, and a regularized iterative lter has been used. Patti et al. applied the reduced order model Kalman lter (ROMKF) to space-variant image restoration and found that the scheme was computationally expensive for a motion blurred image even with moderate-size blurs. The multi-channels 14,15 are applicable to degraded images with complicated motion blurs and out-of-focus blurs when the scene has depth, using multiple blurred images.All of these image restoration schemes do not specically consider the effects of interlaced eld images (i.e., the odd and even eld images of one interlaced scan CCD image), although fogel et al. indicated the need for the consideration of interlaced scan effects as a future research topic 7. Therefore, these image restoration schemes may not provide satisfactory restoration results for interlaced scan CCD images, while good restoration results can be achieved for progressive scan CCD images. To address this problem, we propose a novel space-variant image restoration scheme for interlaced scan CCD images based on sectional processing. This novel scheme not only outperforms sectional processing schemes proposed for interlaced scan CCD image restoration 68 but also is time efcient since only one image frame is required in this scheme, in contrast to image sequence required in those sectional schemes 68. In our scheme, only one image frame is required for the image restoration. In fact, if multiple image frames are used in a motion estimation procedure, the motion offset between the odd and the even elds of one image frame and the motion offset between the adjacent image frames will mix together. This motion offset mixture may result in large motion estimation errors. Therefore, only one image frame is used in our scheme.2. Image restorationOur image restoration scheme for space-variant motion blurs is based on the sectional processing scheme. As is the case of all sectional image restorations, our novel scheme assumes the motion to be locally linear in a region of a reasonable size (e.g. a small rectangular block region). This assumption is reasonable since neighboring points will usually be moving at similar speeds. Therefore, the PSF in every small rectangular block can be approximately space-invariant and then some image restoration schemes for space-invariant linear motion blur can be used in every small rectangular block.Our scheme consists of three steps. First, only one interlaced scan image frame is required to extract the odd eld image and the even eld image. Second, the motion vector estimation is performed in every rectangular block unit using the odd eld and the even eld images, which requires a time-efcient block matching algorithm. Third, the PSF computed with the estimated motion vector in every rectangular block unit will be used as an initial PSF estimation for the subsequent blind deconvolution image restoration. This PSF initialization can obviously both accelerate the blind deconvolution speed and improve the image restoration result.2.1. Odd and even eld image extractionThe extraction of odd or even eld image is the rst step in our scheme. The odd or even eld image is extracted from the interlaced scan image frame I(x,y), y = 0,1, y, N 1 so that Io(x,y) =I(x,y), y = 2k, k = 0,2, y, N/2 1; Ie(x,y)= I(x,y), y = 2k +1, k = 0,1,2, y,N/2 1. Then Io(x,y) or Ie(x,y) is enlarged to form Io1(x,y) or Ie1(x,y) that is identical with I(x,y) in size using an interpolation algorithm. A nearest-neighbor interpolation, a bilinear interpola-tion or a bicubic interpolation can be used in the interpolationalgorithm.2.2. Motion vector estimationsAfter the odd and even eld images are extracted and resized,they are divided into a matrix of rectangular block units. Each rectangular block unit is also called as macro block. Using a block matching algorithm, every macro block in the resized even eld image is then compared with the corresponding macro block and its adjacent neighbors in the resized odd eld image to create a motion vector that stipulates the movement of a macro block from one location to another. The computed motion vector in every macro block will be used in the subsequent PSF estimation as described in Section 2.3. In a block matching algorithm, two parameters, the size of macro block and the search parameter p, must be set with reasonable values. For example, one macro block is taken as a square of side 16 pixels, and the search parameter is 7 pixels in Fig. 1. The matching of one macro block with another is based on the output of a cost function such as the mean absolute difference (MAD) or the mean squared error (MSE). The macro block that results in the least cost is the one that matches the closest to current block.There are some fundamental block matching algorithms,which include the exhaustive search (ES), three step search(TSS), new three step search (NTSS) 16, simple and efcientsearch (SES) 17, four step search (FSS) 18, diamond search (DS)19 and adaptive rood pattern search (ARPS) 20. In general, DS and ARPS algorithms prove to be good block matching solutions,since they provide excellent Peak-Signal-to-Noise-Ratio (PSNR) results, close to those results of ES, while they drop the searches by more than an order of magnitude. Therefore, the DS algorithm is employed in the block matching in every macro block of theresized odd or even eld image. Fig. 1. One macro block for block matching. Fig. 2. DS procedure; this gure shows the LDSP and SDSP and also shows an example path to motion vector ( 4, 2) in ve search steps, which include four times of LDSP and one time of SDSP.In the DS algorithm, the search point pattern is changed from a square to a diamond, and there is no limit on the number of steps that the algorithm can take. DS uses two different types of xed patterns: Large diamond search pattern (LDSP) and small diamond search pattern (SDSP). These two patterns and the DS procedure are illustrated in Fig. 2. The rst step uses LDSP, and if the least cost is at the center location, we jump to the fourth step. The consequent steps, except for the last step, are similar to step 1 and use LDSP, while the number of points where the cost function is checked is either 3 or 5, as illustrated in Fig. 2. The nal step uses SDSP around the new search origin, and the location with the least cost is the best match. Since the search pattern is neither too small nor too big and there is no limit to the number of steps, this algorithm can nd global minimum very accurately. The end result can achieve a PSNR close to that of ES while computational expense is signicantly decreased.2.3. Blind deconvolution image restorationA motion vector (D1x, D1y) for every macro block between the resized odd and even eld images (i.e.Io1(x,y) and Ie1(x,y) is computed based on the above-mentioned DS algorithm. Accord-ing to the CCD imaging principle, the time interval between the odd and even eld images is the summation of the charge integration time of the odd or even eld image and its blanking time. Therefore, the charge integration time of the odd or eveneld image is 0.92 times the time interval between the odd and even eld images, subject to the PAL mode standard. Consequently, one motion vector for every macro block in the odd or even eld image can be computed approximately as follows: (1)where is the motion vector for the corresponding macro block between the resized odd and even eld images.On the other hand, image blur arises from the relative linear motion between the CCD camera and the moving object, resulting in a degraded version of the same image. Generally, the relation between the observed image g(x,y) and its uncorrupted version f(x,y) is as follows 21: （2） where h is the point spread function (PSF) that convolves an original image and n is the additive noise function. The general form of linear motion blur function is given as follows 22: （3）As seen in Eq. (3), linear motion blur depends on two parameters: motion length L and motion direction f. In every macro block of the resized odd or even eld image, these two parameters can be computed simply by the motion vector obtained with Eq. (1).After the PSF is obtained in every macro block of the resized odd or even eld image, each macro block is restored by a blind deconvolution image restoration 23. This restoration scheme maximizes the likelihood that the resulting image, when convolved with the resulting PSF, is an instance of the blurred image, assuming Poisson noise statistics. This blind deconvolution can be used effectively when no information about the blurring and noiseis known. This scheme restores the degraded image and the PSF simultaneously, by using the degraded image and the initial estimated PSF as two input parameters and an iterative processsimilar to the LucyRichardson algorithm. In this scheme, it is very important to use the initial PSF estimation as one input parameter. In fact, an accurate initial PSF estimation can achieve a good image restoration result with only a few iterations. Therefore, the PSF computed using the above-mentioned motion estimation is set asan accurate initial PSF in the blind deconvolution restoration in every macro block, which not only accelerates restoration speed but also improves the restoration result.In fact, single-image blind deconvolution is considered the most ill-posed problem, since it must estimate the PSF and the original image. Previous schemes assume parametric models for the PSF such as a low-pass lter 24 or a sum of normal distributions 25. Fergus et al. 26 shows that blur kernels are often complex and sharp and uses ensemble learning to recover a blur kernel. A variational method is used to approximate the posterior distribution and then RichardsonLucy is applied for deconvolution. Shan et al. 27 create a unied probabilistic framework for both blur kernel estimation and original image restoration, which effectively avoids local minima and ringing artifacts. However, these two blind deconvolution schemes based on probabilistic frameworks fail to restore the blurred image from slight camera rotation or non-uniform object motion (i.e. space- variant motion blur). Fig. 3. One motion blurred moving car image. Fig. 4. Restored car image using our scheme with 5 blind deconvolution iterations and block size 16 16In our scheme, the complex probabilistic-framework-based PSF estimations 26,27 are not employed, since the PSF computed using the DS motion estimation algorithm and the odd and even eld images are usually accurate enough to be an initial PSF in the blind deconvolution restoration in every macro block. Therefore, this scheme not only accelerates restoration speed but also improves the restoration result.After the blind deconvolution restoration is performed in every macro block of the resized odd or even eld image, we can get the restored, resized odd or even eld image denoted asor Then each macro block after the motion compensation with the motion vector (D1x, D1y) in can be obtained as follows: (4)Therefore, the motion compensated odd eld image obtained in this way is denoted as Finally, the restored interlaced scan CCD image is formed as follows: （5）3. Results and comparisons3.1. Real image restorationIn this Section, a comparative image restoration experiment is performed, using our image restoration scheme and for comparison using another sectional processing 7. A fast-moving car image sequence is taken and recorded by an interlaced scan CCD camera One image frame of this image sequence is selected and shown in Fig. 5. Restored car image using our scheme with 10 Fig. 7. Restored car image using scheme of Ref.Blind deconvolution iterations and blocksize1616. 7 Fig. 6. Restored car image using our scheme with Fig. 8. Restored car image using odd and even Macro block size3232and 5 iterations field extraction,motion computation and scheme . of Ref27Fig. 9. Image restoration comparison for the wheel sub-image: (a) original wheel image; (b) restored wheel image by scheme of Ref. 7; (c) restored wheel image by scheme of Ref. 27; (d) restored odd eld image by our scheme; (e) restored even eld image by our scheme and (f) restored nal image by our scheme.Fig. 10. Computed motion vectors with respect to different 1584 macro blocks: (a) vertical components of the computed motion vectors and (b) horizontal components of he computed motion vectors.Fig. 3 (the image size is 704 576). In Fig. 3, a fast-moving car is moving horizontally in front of the static background wall. This moving car can be approximately divided into two different parts with different motions. The large part with a horizontal linear motion is the car bodys side face except the wheel. The small part is the moving wheel with a complicated motion, which combines a horizontal linear motion with a rotation motion. First, our scheme is used to restore this degraded image with space-variant motion blurs. The DS block matching algorithm is used here in the motion vector estimation for each macro block.The size of macro block is 16 16 and the search parameter p = 20.In our scheme, the size of macro block should be small enough to ensure that each macro block in the wheel part can be restored approximately with a space-invariant linear motion blur model. In fact, it is assumed that in each macro block in the wheel part, the combined motion can approximately be seen as a space-invariant linear motion, if the size of each macro block is small enough. Moreover, it can also be assumed that the curve motion between the matching blocks in the wheel part can approximately be seen as a linear motion. Although these approximations will result in PSF computation error in every macro block, as described in the third step of our scheme, the PSF computed using the above-mentioned linear motion approximations is accurate enough to be set as an initial PSF in the blind