[机械模具数控自动化专业毕业设计外文文献及翻译]【期刊】基于稀疏表示的多聚焦图像融合与恢复-外文文献

884 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 59, NO. 4, APRIL 2010Multifocus Image Fusion and RestorationWith Sparse RepresentationBin Yang and Shutao Li, Member, IEEEAbstractTo obtain an image with every object in focus, we al-ways need to fuse images taken from the same view point with dif-ferent focal settings. Multiresolution transforms, such as pyramiddecomposition and wavelet, are usually used to solve this problem.In this paper, a sparse representation-based multifocus imagefusion method is proposed. In the method, first, the source image isrepresented with sparse coefficients using an overcomplete dictio-nary. Second, the coefficients are combined with the choose-maxfusion rule. Finally, the fused image is reconstructed from thecombined sparse coefficients and the dictionary. Furthermore,the proposed fusion scheme can simultaneously resolve the imagerestoration and fusion problem by changing the approximatecriterion in the sparse representation algorithm. The proposedmethod is compared with spatial gradient (SG)-, morphologicalwavelet transform (MWT)-, discrete wavelet transform (DWT)-,stationary wavelet transform (SWT)-, curvelet transform (CVT)-,and nonsubsampling contourlet transform (NSCT)-based methodson several pairs of multifocus images. The experimental resultsdemonstrate that the proposed approach performs better in bothsubjective and objective qualities.Index TermsImage fusion, image restoration, sparserepresentation.I. INTRODUCTIONNOWADAYS, image fusion has become an important sub-area of image processing. For one object or scene, mul-tiple images can be taken from one or multiple sensors. Theseimages usually contain complementary information. Image fu-sion is the process of detecting salient features in the sourceimages and fusing these details to a synthetic image. Throughimage fusion, extended or enhanced information content can beobtained in the composite image, which has many applicationfields, such as digital imaging, medical imaging, remote sens-ing, and machine vision 15.As an example of fusion that is relevant to this paper, opticaimaging cameras suffer from the problem of finite depth offield, which cannot make objects at various distances (fromthe sensor) all in focus. Therefore, if one object in the sceneis in focus, then the other objects at different distances fromthe camera will be out of focus and, thus, blurred. The solutionManuscript received January 17, 2009; revised May 22, 2009. First publishedOctober 30, 2009; current version published March 20, 2010. This workwas supported in part by the National Natural Science Foundation of Chinaunder Grants 60871096 and 60835004, by the Ph.D. Programs Foundationof the Ministry of Education of China under Grant 200805320006, and bythe Key Project of the Chinese Ministry of Education under Grant 2009-120. The Associate Editor coordinating the review process for this paperwas Dr. Cesare Alippi.The authors are with the College of Electrical and Information Engineering,Hunan University, Changsha 410082, China (e-mail: ;shutao_).Digital Object Identifier 10.1109/TIM.2009.2026612to get all the objects focused in one image is multifocus imagefusion technique. In this technique, several images of a sceneare captured with focus on different parts. Then, these imagesare fused with the hope that all the objects will be in focus inthe resulting image 59.There are various methods available to implement imagefusion. Basically, these methods can be categorized into twocategories. The first category is the spatial domain-based meth-ods, which directly fuse the source images into the intensityvalues 1013. The other category is the transformeddomain-based methods, which fuse images with certain fre-quency or timefrequency transforms 1, 4, 5.Assuming that F() represents the “fusion operator,” thefusion methods in the spatial domain can be summarized asIF= F(I1,I2,.,IK). (1)The simplest fusion method in spatial domain just takesthe pixel-by-pixel average of the source images. However, thismethod often leads to undesirable side effects, such as reducedcontrast 1. If the source images are not completely registered,then a single pixel-based method, such as spatial gradient (SG)-based method 10, always results in artifacts in the fusedimage. Therefore, some more reasonable methods were pro-posed to fuse source images with divided blocks or segmentedregions instead of single pixels 1113. However, the block-based fusion methods usually suffer from blockness in thefused image 11. For the region-based method, the sourceimages are first segmented, and the obtained regions are thenfused using their properties, such as spatial frequency or SG.The segmentation algorithms, usually complicated and timeconsuming, are of vital importance to the fusion quality 13.A more popular method that has been explored in recentyears is by using multiscale transforms. The usually usedmultiscale transforms include various pyramids 1417, dis-crete wavelet transform (DWT) 1, 5, 18, complex wavelet19, 20, ridgelet 21, curvelet transform (CVT) 22, andcontourlet 23. The transformed domain-based methods can besummarized asIF= T1(F (T(I1),T(I2),.,T(IK) (2)where T() represents a multiscale transform, and F() meansthe applied fusion operator.Pyramid decomposition is the earliest multiscale transformused for image fusion 1417. In this method, each sourceimage is first decomposed into a sequence of images (pyra-mid) in different resolutions. Then, at each position in thetransformed image, the value in the pyramid with the highest0018-9456/$26.00 2009 IEEEYANG AND LI: MULTIFOCUS IMAGE FUSION AND RESTORATION WITH SPARSE REPRESENTATION 885saliency is selected. Finally, the fused image is constructed us-ing the inverse transform of the composite images. The wavelettransform-based fusion methods employ a similar scheme to thepyramid transform-based methods. However, the performanceof multiresolution transform-based methods is limited owing tothat most of the multiresolution decompositions are not shiftinvariant, which is brought by the underlying down-samplingprocess 1, 5, 19. The shift-invariant extension of theDWT can yield an overcomplete signal representation, whichis suitable for image fusion 2426. Further, many advancedgeometric multiscale transforms, such as CVT, ridgelet, andcontourlet, have been explored in recent years and shownimproved results 2123, 2729. However, because thefused image obtained by transform domain-based algorithmsis globally created, a little change in a single coefficient of thefused image in the transformed domain may cause all the pixelvalues to change in spatial domain. As a result, undesirableartifacts may be produced in the fusion process using themultiresolution transform-based methods in some cases 3.Obviously, effectively and completely extracting the under-lying information of the original images would make the fusedimage more accurate. Different from multiscale transforma-tions, the sparse representation using an overcomplete dictio-nary that contains prototype signal atoms describes signals bysparse linear combinations of these atoms 3034. Two maincharacteristics of sparse representation are its overcompletenessand sparsity 32. Overcompleteness means that the numberof basis atoms in the dictionary exceeds the number of im-age pixels or signal dimensions. The overcomplete dictionarythat contains rich transform bases allows for more stable andmeaningful representation of signals. Sparsity means that thecoefficients corresponding to a signal are sparse, that is to say,only “a few descriptions” can describe or capture the significantstructure information about the object of interest. Benefitingfrom its sparsity and overcompleteness, sparse representationtheory has successfully been applied in many practical appli-cations, including compression, denoising, feature extraction,classification, and so on 3234. Recent studies have shownthat common image features can also be accurately describedby only a few coefficients or “a few descriptions” 32. Usingthe few coefficients as the salient features of images, we designa sparse representation (SR)-based image fusion scheme. Ingeneral, sparse representation is a global operation, in the sensethat it is based on the gray-level content of an entire image.However, the image fusion quality depends on the accuraterepresentation of the local salient features of source images.Therefore, a “sliding window” technique is adopted to achievebetter performance in capturing local salient features and keep-ing shift invariance.In the proposed method, the source images are first dividedinto patches, which lead to “a small size” of overcompletedictionary to every patch. Second, the patches are decomposedby prototype signal atoms into their corresponding coefficients.The larger the coefficient is, the more salient features itcontains. Third, the “choose-max” fusion rule is used tocombine the corresponding coefficients of the source images.Finally, the result image is reconstructed using the combinedcoefficients previously obtained. In sparse representation, thedictionary (atoms) is often created by a prespecified set offunctions, such as discrete cosine transforms (DCT), short-timeFourier transforms, wavelet, CVT, and contourlet. Morecomplex dictionary can be obtained by learning from a givenset of signal examples 32.Note that most of the image fusion methods are based on theassumption that the source images are noise free. Therefore,these fusion algorithms can produce high-quality fused imagesif the assumption is satisfied. However, practically, the imagesare often corrupted by noise during acquisition or transmissionprocesses. For multiresolution-based methods, they usually de-noise the source images, first, by setting all the coefficientsbelow a certain threshold to zero and keeping the remainingcoefficients unchanged. Then, the filtered images are fused. Oneadvantage of our proposed method is that it can simultaneouslycarry out denoising and fusion of noisy source images.The rest of this paper is organized into five sections. InSection II, the basic theory of sparse representation is pre-sented. In Section III, we propose the fusion scheme with sparserepresentation theory and discuss how to simultaneously carryout image restoration and fusion. Numerical experiments anddiscussions are detailed in Section IV. Both advantages anddisadvantages of the proposed schemes, together with somesuggestions about the future work, are given in Section V.II. BASIC THEORY OF SIGNAL SPARSE REPRESENTATIONSparse representation is based on the assumption that naturalsignals can be represented or approximately represented asa linear combination of a “few” atoms from dictionary 30.For signals Rn, sparse representation theory suggests theexistence of a dictionary D RnT, which contains T pro-totype signals that are referred to as atoms. For any signalx , there exists a linear combination of atoms from D thatapproximates it well. More formally, it is x , s RTsuch that x Ds. The vector s contains “coefficients” of x inD. It usually assumes that Tn, implying that the dictionaryD is redundant. The solution is generally not unique. Findingthe smallest possible number of nonzero components of sinvolves solving the following optimization problem:minsbardblsbardbl0subject to bardblDs xbardbl (3)where bardblsbardbl0denotes the number of nonzero components in s.The above optimization is an NP-hard problem and can onlybe solved by systematically testing all the potential combina-tions of columns 31. Thus, approximate solutions are con-sidered instead. In the past decade, several pursuit algorithmshave been proposed to solve this problem. The simplest algo-rithms are the matching pursuit (MP) 30 and the orthogonalMP (OMP) algorithms 32. They are greedy algorithms thatsequentially select the dictionary atoms. These methods involvethe computation of the inner product between the signal anddictionary columns.The MP algorithm aims to learn x Ds as follows: Let,denote the inner product. Initialize the residual function r0asr0= x. (4)886 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 59, NO. 4, APRIL 2010Then, loop over all prototype signal atoms and select the indextlof the best atom function in D astl= arg maxtangbracketleftbigrl, dtangbracketrightbig(5)with a coefficient along that dimension ass (tl)=angbracketleftBigrl, dtlangbracketrightBig. (6)Then, update the residual functionrl+1= rl s (tl) dtl(7)and repeat untilbardblrlbardbl2=vextenddoublevextenddoublevextenddoublevextenddoublevextenddoublex lsummationdisplayi=1s (ti) dtivextenddoublevextenddoublevextenddoublevextenddoublevextenddouble2 (8)or the iteration reaches the priori set number.Because the computation of the MP algorithm is intensive,in this paper, the OMP is used. The OMP method updates theresidual asrl+1= x PspanbraceleftBigdt1, dt2,.,dtlbracerightBigx (9)where P denotes the orthogonal projection onto a subspace.The OMP algorithm, in parallel, applies the GramSchmidtorthogonalization upon the chosen atoms for the efficient com-putation of projections. Ideally, the number of iterations is equalto the number of nonzeros in s. Other well-known pursuitapproaches include the basis pursuit (BP) 35 and the focal un-derdetermined system solver (FOCUSS) 33. The BP replacesthe lscript0-norm with an lscript1-norm in (3), whereas the FOCUSS usesthe lscriptp-norm with as a replacement for the lscript0-norm.III. IMAGE FUSION METHODA. Sparse Representation for Image FusionSince the sparse representation globally handles an image,it cannot directly be used with image fusion, which dependson the local information of source images. In our method, wedivide the source images into small patches and use the fixeddictionary D with small size to solve this problem. In addition,a sliding window technique is adopted to make the sparserepresentation shift invariant, which is of great importance toimage fusion.We assume that source image I is divided into many imagepatches. As shown in Fig. 1, to facilitate the analysis, the jthpatch with size n n is lexicographically ordered as a vectorvj. Then, vjcan be expressed asvj=Tsummationdisplayt=1sj(t)dt(10)where dtis an atom from a given overcomplete dictionary,and D =d1 dt dT, which contains T atoms. sj=sj(1) sj(t) sj(T)Tis the sparse representationobtained in (3).Fig. 1. Selected image patch and its lexicographic ordering vector.Fig. 2. Schematic diagram of the proposed SR-based fusion method.Assume that the vectors responding to all the patches inimage I are constituted into one matrix V. Then, V can beexpressed asV =d1d2dTs1(1) s2(1) sJ(1)s1(2) s2(2) sJ(2).s1(T) s2(T) sJ(T)(11)where J is the number of image patches. Let S =s1, s2, sJ. Then, (11) can be expressed asV = DS (12)where S is a sparse matrix.B. Proposed Fusion SchemeAssume that there are K registered source images I1,.,IKwith size of M N. Then, the proposed fusion scheme basedon image sparse representation, shown in Fig. 2, takes thefollowing steps.Use the sliding window technique to divide each sourceimage Ik, from left-top to right-bottom, into patches of sizen n, i.e., the size of the atom in the dictionary. Then, all thepatches are transformed into vectors via lexicographic ordering,and all the vectors constitute one matrix Vk, in which eachcolumn corresponds to one patch in the source image Ik.Thesize of Vkis (n n) (M n +1) (N n + 1).For the jth column vector vkjin Vk, its sparse representationis calculated using the OMP method. The OMP iterations willstop when the representation error drops below the specifiedtolerance. Then, we get a very sparse representation vector skjfor vkj.YANG AND LI: MULTIFOCUS IMAGE FUSION AND RESTORATION WITH SPARSE REPRESENTATION 887Then, the activity level of skjresponding to the kth sourceimage Ikis obtained asAkj= bardblskjbardbl1. (13)Fuse the corresponding columns of all sparse representationmatrix S1,.,Sk,.,SKof the source images to generateSFaccording to their activity levels. The jth column of SFis obtained assFj= skj,kj= arg maxkj(Akj). (14)The vector representation of the fused image VFcan becalculated byVF= DSF. (15)Finally, the fused image IFis reconstructed using VF.Reshape each vector vFjin VFinto a block with size n nand then add the block to IFat its responding position. Thiscan be seen as the inverse process of Fig. 1. Thus, for eachpixel position, the pixel value is the sum of several block values.Then, the pixel value is divided by the adding times at itsposition to obtain the reconstructed result.C. Restoration and FusionIn practice, the source images for fusion are often corruptedby noise during the acquisition or transmission process. Intraditional methods, image restoration and image fusion areseparately treated. Little effort has been made to combinethem 36. The other benefit of the proposed scheme is tosimultaneously conduct image restoration and fusion by usingthe advantage of sparse representation in image restoration.Assume that image I is contaminated by additive zero-meanwhite Gaussian noise with standard deviation .Forthejthpatchs vector of image I vj, its denoising solution using themaximum a posteriori estimator issj= minsjvextenddoublevextenddoublesjvextenddoublevextenddouble0subject tovextenddoublevextenddoublevj Dsjvextenddoublevextenddouble2C (16)where C is a constant 34. Then, all the patchessparse representation vectors constitute the sparse coefficientmatrixS.Then, for source images I1,.,IK, the corresponding over-lapped patches and vectoring matrix are V1,V2,.,VK.Their sparse representationsS1,S2,.,SKare obtained ac-cordin