Texture Features for Browsing and Retri of Image Data.doc

Texture Features for Browsing and Retrieval of Image Data 图像数据的浏览和检索的纹理特征B.S. Manjunathi and W.Y. Ma Abstract-Image content based retrieval is emerging as an important research area with application to digital libraries and multimedia databases.抽象图像内容检索的技术正逐渐成为重要的研究领域。在数字图书馆和多媒体数据库中具有重要的应用。 The focus of this paper is on the image processing aspects and in particular using texture information for browsing and retrieval of large image data. 本文的重点是图像处理方面，特别是使用纹理信息的大图像数据的浏览和检索。We propose the use of Gabor wavelet features for texture analysis and provide a compirehensive experimental evaluation. 我们建议使用Gabor小波特征纹理分析，并提供了全面的实验评估。Comparisons with other multiresolution texture features using the Brodatz texture database indicate that the Gabor features provide the best pattern retrieval accuracy.与其它采用Brodatz纹理数据库的多分辨率纹理特征比较表明，Gabor特征模式检索的准确性最高。 An application to browsing large air photos is illustrated. Index Terms-Digital libraries, image database, content-based image retrieval, texture analysis, Gabor wavelets. 应用于浏览大型航空照片说明了数字图书馆，图像数据库，基于内容的图像检索，纹理分析，Gabor小波。+ 1 INTRODUCTION RETRIEVAL of image data based on pictorial queries is an interest- ing and challenging problem. The recent emergence of multimedia databases and digital libraries m,ikes this problem all the more important. While manual image annotations can be used to a cer- tain extent to help image search, the feasibility of such an ap- proach to large databases is a questionable issue. 1引言检索的图像数据的基于图案查询是一个兴趣和挑战性的问题。多媒体数据库和数字图书馆的出现，这个问题显得更为重要。而手动图像注释一定程度上可以帮助图像搜索，这种方法对大型数据库的可行性仍是一个问题。In some cases, such as face or texture patterns, simple textual descriptions can be ambiguous and often inadequate for database search. The objective of this paper is to study the use of texture as an image feature for pattern retrieval. An image can be considered as a mosaic of different texture regions, and the image features asso- ciated with these regions can be used for search and retrieval. 一些情况下，如人脸或纹理图案，简单的文字说明是模糊的，往往是不足够的数据库搜索。本文的目的是研究纹理的纹理图像检索的图像特征。图像可以被视为一个马赛克的不同纹理领域的图像特征，也影响这些可用于搜索和检索的领域。A typical query could be a region of interest provided by the user, such as outlining a vegetation patch in a satellite image. The input information in such cases is an intensity pattern or texture within a rectangular window. See Fig. 6 for an example of a texture based browsing application. Texture analysis has a long hktory and texture analysis algo- rithms range from using random field models to multiresolution filtering techniques such as the wavelet transform.一个典型的查询可能是用户提供的感兴趣的区域，例如在卫星图像中概述了植被斑块。在这种情况下的输入信息是一个矩形窗口内的强度图案或纹理。图6为基于纹理的浏览应用程序的示例。纹理分析有着悠久的历史，纹理分析算法的研究范围从使用随机域模型多分辨率滤波技术，如小波变换。 Several re- searchers have considered the use of such texture features for pat- tern retrieval 181, 191. This paper focuses on a multiresolution representation based on Gabor filters. The use of Gabor filters in extracting textured image features is motivated by various factors. The Gabor representation has been shown to be optimal in the sense of minimizing the joint two-dimensional uncertainty in space and frequency 4.一些学者认为帕特-纹理特征的检索系统 181， 191。本文重点研究了基于Gabor滤波器的多尺度表示。Gabor滤波器提取纹理特征的使用是出于各种因素。Gabor表示已被证明具有在空间和频率上最优的最小关节二维不确定性。 These filters can be considered as orien- tation and scale tunable edge and line (bar) detectors, and the sta- tistics of these microfeatures in a given region are often used to characterize the underlying texture information. Gabor features have been used in several image .analysis applications including texture classification and segmentation l, 141, image recognition 51, 81, 131, image registration, and motion tracking 151. 这些过滤器可以被视为导向性和规模可调的边缘线（条）探测器，和站在一个给定的区域，这些微统计经常被用来表征底层纹理信息。Gabor特征已被用于在多个图像。分析应用包括纹理分类和分割1， 141，图像识别 51， 81 ， 131，图像配准，和运动跟踪 151。The main contributions of this paper are summarized below: 1) A simple texture feature representation based on Gabor features is proposed, and a filter design strategy is suggested to reduce the redundancy in the representationThe authors are with the Depavtrnent of Electrical and Computer Engineering, University of California at Santa Barbara. 本文的主要贡献如下：1）提出了一个简单的纹理特征表示基于Gabor特征，以及滤波器的设计策略建议减少冗余.作者是加州大学圣芭芭拉分校电气与计算机工程学院的工程师。Santa Barbava, CA 93106-9560. E-mail: , weiiplah.ecc,.. Manuscript received Dec. 16,1994. Recommended for acceptance by R. Picard. For information on obtaining reprints of this article, please send e-mail to: transpamicomputer.or, and reference IEEECS Log Number P96055. Santa Barbava，CA 93106-9560。电子邮件：，wei iplah。ECC，。来稿12月161994。建议由R. Picard验收。在获取文章转载信息，请发送电子邮件至：transpami 计算机。或，参考ieeecs日志数p96055。2) An adaptive filter selection algorithm is proposed which can facilitate fast image browsing. 3) A detailed comparison with the performance of three other multiscale texture features is provided. Our proposed repre- sentation compares favorably in terms of feature computa- tions and retrieval accuracy. 4) Finally, an application to browsing large air photos is dem onstrated. 2）提出了一种自适应滤波器选择算法，便于快速图像浏览。三）详细的比较与其他3个多尺度纹理特征的性能。我们提出的代表条件就计算与检索而言更具准确性。4）最后，一个应用于浏览大型航空照片被显示。3)2 TEXTUREFEATUREEXTRACTION 2.1 Gabor Functions and Wavelets A two dimensional Gabor function g(x, y) and its Fourier trans- form G(u, v) can be written as: where e,( = 1/2zo, and ou = 1/2zo,. Gabor functions form a complete but nonorthogonal basis set. Expanding a signal using this basis provides a localized frequency description. A class of self-similar functions, referred to as Gabov wavelets in the following discussion, is now considered. Let g(x, y) be the mother Gabor wavelet, then this self-similar filter dictionary can be obtained by appropriate dilations and rotations of g(x, y) through the generat- ing function: gmil(x, y) = LG(x, y), U 1, m, n = integer x= a-(xcosO+ysinO), andy= a-*(-xsinO+ycosO),2纹理特征提取2.1个Gabor函数和小波一二维Gabor函数g（x，y）及其傅里叶变换形式G（u，v）可以写为：其中E，（= 1 / 2zO，和欧= 1 / 2zO，。Gabor函数形成一个完整的但非正交基组。扩展的信号，使用此基础上提供了一个本地化的频率描述。一类自相似的功能，称为gabov小波在下面的讨论中，现在被认为是。设G（x，y）是母亲，Gabor小波，那么这种自相似过滤词典可以通过适当的扩张和旋转的G（x，y）获得通过生成-功能：GMIL（x，y）= L“G（X，Y），u1，m，N =整数x = -”（xcoso + ysino），安迪= A *（- xsino + ycoso）， (3) where O = nzfK and K is the total number of orientations. The scale factor U- in (3) is meant to ensure that the energy is inde- pendent of m. 2.2 Gabor Filter Dictionary Design The nonorthogonality of the Gabor wavelets implies that there is redundant information in the filtered images, and the following strategy is used to reduce this redundancy.（3）O = nzfk和K为取向的总数。规模因子U”（3）是为了确保能源独立悬挂的。2.2 Gabor字典的设计Gabor小波的非正交性意味着在滤波后的图像有冗余的信息，和下面的策略来减少这种冗余。 Let U, and U, denote the lower and upper center frequencies of interest. Let K be the number of orientations and S be the number of scales in the mul- tiresolution decomposition. Then the design strategy is to ensure that the half-peak magnitude support of the filter responses in the frequency spectrum touch each other as shown in Fig. 1. This re- sults in the following formulas for computing the filter parameters ou and ou (and thus cr, and cy). 让U，和U表示，较低的和上的中心频率的兴趣。设K是取向数及在多分辨率分解尺度的数量。然后，设计策略是，以确保一半的峰值幅度的滤波器响应的频率频谱的支持，如图1所示。计算滤波器参数欧和欧在以下公式重新结果（因此CR，和CY）。where W = U, and m = 0, 1, ., S - 1. In order to eliminate sensi- tivity of the filter response to absolute intensity values, the real (even) components of the 2D Gabor filters are biased by adding a constant to make them zero mean (This can also be done by setting G(0,O) in (2) to zero). 在哪里=U，和m= 0，1，1。为了消除森西-绝对强度值的滤波器响应性，真正的（甚至）的二维Gabor滤波器组件的加入使他们零均值恒定偏置（这也可以通过设置G做（0，O）在（2）零）。2.3 Feature Representation Given an image I(x, y), its Gabor wavelet transform is then defined to be (5) where * indicates the complex conjugate. It is assumed that the local texture regions are spatially homogeneous, and the mean pm, and the standard deviation cmn of the magnitude of the trans- form coefficients are used to represent the region for classification and retrieval purposes: 2.3特征表示给定的图像（x，y），它的Gabor小波变换定义为（5）其中*表示复共轭。它是假定局部纹理区域的空间均匀，且平均PM，和标准偏差CMN的跨幅度系数是用来表示区域的分类和检索的目的：W,(XJY) = p(l!Yl)m& - X1.Y - Yl)dx,dYlr P, = Jjlwm.(xy)ldxdy, and omn = Jjj(l(,y) - ,)dxdy. (6) A feature vector is now constructed using pm, and omn as fea- ture components. In the experiments, we use four scales S = 4 and six orientations K = 6, resulting in a feature vector (7) = Po0 000 Pol . P35 0351. 。W,(XJY) = p(l!Yl)m& - X1.Y - Yl)dx,dYlr P, = Jjlwm.(xy)ldxdy, and omn = Jjj(l(,y) - ,)dxdy. （6）特征向量是现在使用PM，和OMN作为特征分量。在实验中，我们使用四个尺度S= 4和六个方向k = 6，导致在一个特征向量（7）= po0 000 PolP35 0351。I - 2.3.1 Distance Measure: Consider two image patterns i and 1, and let 7) and 7) represent the corresponding feature vectors. Then the distance between the two patterns in the feature space is defined to be where a(pL,) and a(o,) are the standard deviations of the respective features over the entire database, and are used to normalize the individual feature components. 2.3.1距离测度：考虑两图像模式I和1，让7“）7”）表示相应的特征向量。然后定义在特征空间中的两个模式之间的距离是在a（pl,）和a（o,），是在整个数据库中的各自的功能的标准偏差，并用于规范化的单个特征组件。2.4 Retrieval Performance 2.4.1 Texture Database The texture database used in the experiments consists of 116 dif- ferent texture classes. Each of the 512 x 512 images is divided into 16 128 x 128 nonoverlapping subimages, thus creating a database of 1,856 texture images. A query pattern in the following is any one of the 1,856 patterns in the database.2.4数据库检索性能2.4.1纹理纹理数据库用于实验包括了116个不同的纹理类。每个512512图像分为16个128128不重叠的子图，从而创造1856个纹理图像数据库。在数据库中的1856个模式中的任何一个查询模式。 This pattern is then proc- essed to compute the feature vector as in (7). The distance d(i, j), where i is the query pattern and j is a pattern from the database, is computed. The distances are then sorted in increasing order and the closest set of patterns are then retrieved. In the ideal case all the top 15 retrievals are from the same large image. 这种模式是程序的计算特征向量为（7）。距离d(i, j)从数据库中的查询模式i和数据j的距离计算距离，然后进行排序，增加顺序和最接近的模式，然后检索。在理想的情况下15大检索来自同一个大的图像。The perform- ance is measured in terms of the average retrieval rate which is defined as the average percentage number of patterns belonging to the same image as the query pattern in the top 15 matches. We observe that the use of omn feature in addition to the mean improves the retrieval performance considerably. This perhaps explains the low classification rate of the Gabor filters reported in 3 where only the mean value was used. On the average 74.37% of the correct patterns are in the top 15 retrieved images. The per- formance increases to 92% if the top 100 (about 6% of the entire database) retrievals are considered instead (i.e., more than 13 of the 15 correct patterns are present). Some retrieval examples are shown in Fig. 2. A detailed comparison with other texture features is given in Section 4. 性能测量中，被定义为属于相同图像作为查询模式中的前15名的匹配图案的平均百分比数的平均检索率方面。我们观察到，除了平均使用OMN特征显着改善的检索性能。或许正是这个报告在3仅使用平均值，其中Gabor滤波器的低分类率。在正确的模式，平均74.37，在排名前15位检索到的图像。将性能提高到92，如果前100名（约6，整个数据库）检索被认为是代替（即超过13的15个正确的模式是存在）。一些检索例子示于图2.与其他纹理特征详细的比较见第4节。(4 Fig. 2. Texture browsing using Gabor features. Examples shown are some of the difficult patterns to analyze. The average retrieval rate is shown in the parentheses: (a) Texture browsing interface with all 116 textures and retrievals for D42, lace (50%), (b) D43, swinging light bulb (54%), (c) D23, beach pebbles (54%), and (d) D91, clouds (25%). The first image in each of (b)-(d) represents the query image. Note that some of the incorrect matches actually look quite similar to the query pattern. 3 ADAPTIVE FILTER SELECTION In many cases it is desirable to reduce the image processing time to the extent possible while not seriously affecting the overall per- formance. One way to reduce the computations is to select the Gabor filters in a pattern dependent way. Authorized licensed use limited to: National Taiwan University. Downloaded on October 7, 2008 at 23:29 from IEEE Xplore. Restrictions apply.4所示。 2.纹理浏览器使用的Gabor特征。所示的例子是一些困难模式的分析。平均检索率示于括号：（一）纹理浏览界面与所有116纹理和检索为D42，花边（50），（B）D43，摆动灯泡（54），（三）D23，沙滩卵石（54），和（d）D91，云（25）。在各（b）的第一图像- （d）示查询图像。需要注意的是一些不正确的匹配实际上看起来颇为相似，查询模式。3自适应滤波器选择在许多情况下，期望减少可能的图像处理时间的范围内，而不会严重影响整体性能。以减少计算的一种方法是在一个图案相关的方式选择的Gabor滤波器。授权许可使用限制于：国立台湾大学。在23:29从IEEE Xplore数据库下载的2008年10月7日。An insight into the discrimination quality of individual features can be obtained by considering the average intraclass to inter-class distance ratio for each of the components. This ratio is about 0.25 on the average, with higher frequency components providing a better discrimination 16. In database search one is often inter- ested in finding out how much of the search space can be elimi- nated by using a particular feature. Suppose we want to keep all the 15 correct textures belonging to the same pattern in the set of retrieved images. The average threshold is about 30%, and as in the previous case, high frequency components have a better dis- crimination. More details can be found in 161. These experiments indicate that each feature component can individually provide useful discrimination measure which can be further improved by selectively choosing these filters based on the query pattern. The selection scheme described here uses the spectral informa- tion in conjunction with the average database image properties to select a subset of filters. The pur-pose is to identify salient query image properties which best diijtinguish it from the database items. Fig. 3 shows a schematic diagram of this method. The dif- ference between the spectrum of the input image pattern and the average spectrum provides information about salient spectral characteristics of the given image. Let where F&uf(n, v) is the Fourier transform of the input image pat- tern. F, (U, v) and FUu,(u, v) are the mean and variance associated with the distribution of Fourier transforms of all image patterns in the database. D(u, U) basically is the energy of the difference nor- malized by the variance associated with each frequency compo- nent (U, U). Each filter is evaluated based on the total diffevence en- ergy within its spectral coverage: where G,(u, U) is the frequency response of the filter g,(x, y) . The larger the value i, is, the better the performance of the filter will be for pattern retrieval. Thus the filters can be ordered based on cmx. Fig. 3 shows an example where the input pattern has strong orientation preference which distinguishes the pattern from much of the database images. Using only the top four filters, one can retrieve on the average about 50% of the correct patterns in the top 15 retrieved patterns 1161. Such a strategy may be reasonable in many image database applications and amounts to a significant savings in image processing computations. Fig. 4 shows some examples wherein different number of filters are used. Computing the entire Gabor feature vector of (7) takes 9.3 sec- onds of CPU time (in MATLAB on a SUN-Sparc20, see Table 2). Search and retrieval takes about 1.02 seconds. Using only four feature components requires 2.3 seconds of feature extraction time including the adaptive filter selection (0.7 seconds), and about 0.1 seconds for search and retrieval. 4 EXPERIMENTAL RESULTS A detailed comparison with some of the other recently proposed multiresolution texture image features are made in this section. For the Gabor feature case, the 24 x 2 component feature vectors are used. An application to browsing large air-photos is illustrated. 4.1 Comparison with Other Texture Features The comparisons are made with the conventional pyramid- structured wavelet transform (TWT) features, tree-structured wavelet transform (TWT) features, and the multiresolution simul- taneous autoregressive model (MR-SAR) features. The filter coeffi- cients used for computing PWT are the 16-tap Daubechies or- database + input FFT pattern #2 #4 selected filters Fig. 3. An example of the filter selection strategy. Fig. 4. Examples of retrieval using a different number of filters selected by the adaptive filter selection strategy. The upper-left image in each block is the query image (D83: woven matting), and the top 15 matches are shown in row scan order with increasing distance. Top: Results using one filter; bottom: results using two filters. With four fil- ters, all the correct 15 patterns are retrieved for this case. thogonal wavelets 6 (same as the ones used in 3 for the TWT). The 128 x 128 image pattern is decomposed into three levels (4 x 3 = 12 bands) of the wavelet transform. The mean and standard de- viation of the energy distribution corresponding to each of the subbands at each decomposition level are used to construct a (12 x 2) feature vector. In 3, decomposition of image subbands at each level is based on energy considerations and this results in a tree structured de- composition where different patterns have different structures. For pattern retrieval applications, it is convenient to have a fixed structure. A fixed decomposition tree can be obtained by sequen- tially decomposing the LL, LH, and HL subbands. The HH band is not decomposed as this often does not lead to stable features. A three level decomposition results in 52 (4(1 + 3 + 9) subbands. As in the PWT, the mean and standard deviation in each subband are used to construct a 52 x 2 component feature vector. The third set of feature used are the MR-SAR model features 171. Previous work 191, 201 indicate that the MR-SAR features at levels 2, 3, and 4, provide the best overall performance. At each level, five parameters are computed to represent the texture, thus requiring a total of 15 feature components. The Mahalanobis dis- tance is used to compare the feature vectors. 4.1.1 Summary of Comparisons Table 1 provides a summary of the experimental r