Abstract

We introduce a novel texture description scheme and demonstrate it with our fast similarity search technique for content-based retrieval and navigation applications. The texture representation uses a combination of edge and region statistics. It is compared with the Multi-Resolution Simultaneous Auto-Regressive Model and Statistical Geometrical Features techniques using the entire Brodatz texture set and on a collection of more complex texture images obtained from a product catalogue. In both cases, the edge based representation gives the best classification.

Introduction Texture analysis has been widely studied, and a large number of approaches have been developed. Amongst these methods, some of the popular models include Markov Random Field (MRF) [6], Simultaneous Auto-Regressive (SAR) [15], Gabor Filters [5], Wold Transform [13], and Wavelets. Texture analysis techniques may be described in three main categories: structural, statistical, and structural statistical approaches. The structural methods use the geometrical features of texture primitives as the texture features. However, they involve a lot of image pre-processing procedures to extract texture primitives. Mostly these methods are time-consuming, and often only regular textures can be recognized but the features are normally rotation-invariant. Statistical methods are the dominant approach for texture matching, and they can work well on regular, random and quasi-random textures. Not many researchers have developed texture analysis techniques using combined statistical structural methods. Chen et al. [4] use the statistical geometrical features to classify textures from the entire Brodatz texture album [3], and the method shows a good performance for classification.

Several studies [2] [5] [7] [18] [16] have shown that using edge information in the texture features can achieve good classification performance. In this paper, we propose a novel edge based method which achieves a high classification rate with the entire Brodatz texture database. One of our objectives was to develop a texture matching technique which is effective with complex textures such as those in commercial furniture catalogues (see figure 3). We have compared our edge based method, the Statistical Geometrical Features (SGF) [4] and the Multi-Resolution Auto-Regressive Model (MR-SAR) [15] using the entire Brodatz texture database and a set of complex textures from the catalogue. In both cases, the new Edge Based method gives the best retrieval results.

Edge Based Texture Classification   Various methods have been developed by researchers to extract edge information from a texture image. These include Gabor filters by Coggins et al. [5] and Generalized Co-ocurrence matrices by Davis et al. [7]. Patel et al. [16] calculate edge direction using 3 3 masks then use rank order statistics to produce the texture features. Our approach to Edge Based texture feature calculation begins like that of Patel et al. but where they provide only edge information, our method also captures details of regions with no edge information, as these too can contribute valuable information to the texture features. We also introduce other low and high level texture measures as described below.

We calculate grey value variances of 4 different directions (0 , 45 , 90 , 135 ) from a 3 3 mask. The direction with the minimum variance is chosen as the label on the centre pixel of the mask. However, some areas in an image may have no edge information, and these can also be used as part of the texture features. Before the direction from a mask is determined, we must decide whether there is any edge information inside the mask. To do this we calculate the sum of differences in each 3 3 window.

where is the mean grey level value of the entire window, , and is the grey level value of pixel .

When the direction is decided, then the labelling is performed as 0 - Horizontal (H), 45 - Right Diagonal (RD), 90 - Vertical (V), and 135 - Left Diagonal (LD).

Figure 1a shows an original texture image, and figure 1b shows that accurate edge retrieval is accomplished. The light areas in 1b indicate a blank label and the darker areas are labelled with a direction.

  
Figure 1: Effect of edge extraction from an image

After processing the entire image, the ratio for each edge direction and plain region is calculated as a fraction of the total number of labels:

 

where D is one of the labels Horizontal, Vertical, Right Diagonal, Left Diagonal, and Blank. is the ratio of label D that appears in the image. is the number of appearances of D in the image.

Edge information alone is not sufficient for a complete description of the texture; the contrast across each edge can be relevant. If an edge direction of a mask is found, the contrast ( ) of that direction, D (excluding Blank), within the mask is calculated as:

where and are the mean grey levels of the pixels on either side of the determined direction. If the mask is classified as Blank, then the mean grey level value is computed instead.

When is summed up through an entire image, then equation 3 is applied to normalize into the range [0-1] by dividing by the maximum possible greylevel value, e.g. 255. The values of and can be regarded as lower level features.

 

where is the maximum intensity level.

The higher level texture edge features are evaluated by using the conditional probability between the edge direction of the centre of the mask and the surrounding locations. Figure 2 shows a matrix of conditional probabilities which has a similar form to the Generalized Co-occurrence Matrix suggested by Davis et al. [7]. The differences are the use of conditional probabilities and the inclusion of plain region statistics.

  
Figure 2: Conditional Probability Matrix of Edge Information

For example, is the conditional probability of getting Vertical labels in a 3 3 window given the central pixel is Horizontal. This is computed by counting the number of appearances of vertical edge labels in the surrounding location divided by the area of mask (excluding the centre pixel). Each entry in the conditional probability matrix is accumulated according to the labels of central and surrounding location in the mask. A 5 5 probability matrix is then generated and normalized into the range [0-1] by dividing with .

Similarity Measurement In [15], Mao et al. reported that using a large number of parameters will cause an effect on severe averaging over power discriminatory features. When comparing textures, contributions to the similarity measure from edge information should be weighted according to the fractions of those edges occuring in the images, i.e. by the ratios, . If two images have a high ratio on horizontal edges, their similarity value on horizontal edges should also increase. If two images have high and low ratios on edge properties, then the weight is taken as the average between both ratios. In this case, we can match two images based on their similarity and dissimilarity. The weight is evaluated as:

where and are the ratios of the general term D of the two different images. The similarity, s, is the sum of the squared Euclidean distances of the conditional probabilities and the contrasts, multiplied with the weights.

where D and are the general terms of H, V, LD, RD, and B. This measure uses a weighted combination of contrast across edges and conditional probability of edge directions and plain regions to assess the similarity between two images.

Brodatz Texture Database Each Brodatz image is digitized into a 512 512 256 grey level image, and cut into 16 subimages of 128 128. A total of 1792 (112 16) images are produced from the texture album. Eight out of each set of 16 subimages are randomly taken, texture features are extracted and pre-indexed into a training database. The rest of the subimages are used for testing. A similar experiment was also performed by Manjunath et al. [14]. They used a nearly complete set of Brodatz textures (except D31, D32, and D99) for comparing between Gabor wavelets, MRSAR, pyramid-structured and tree-structured wavelet transform. The MRSAR result was fractionally lower than the Gabor wavelets with 73% and 74% respectively.

Complicated Texture Database Thirty texture patterns (11 classes) were extracted from a commercial furniture catalogue. Figures 3a - 3d show some of the patterns which are categorised into groups such as, Abbey Stripe, Georgia Damask, Tournament Stripe, etc. Some of the images were taken directly from texture samples in the catalogue, others were extracted from the pictures of furniture.

  
Figure 3: Examples of commercial complicated texture patterns

Results Two different techniques with good matching capabilities, SGF [4] and MRSAR [15], are chosen to compare with our method. For each test image, we calculate the Euclidean distance between feature vectors to retrieve the top 15 nearest matches out of the 896 features in the image database. If all the 8 subimages from the same orginal texture image are retrieved, the testing image scores a 100% retrieval rate. If 7 subimages are retrieved, the result is 87.5% and so on. This experiment is repeated three times and the results are averaged to even the random selection of the test set and sample sets in the Brodatz database. For all the 8 test images of each class, we averaged the classification rate. The number of Brodatz textures that scored a classification rate in a certain percentage range are presented in table 1. Our method clearly outperforms the other two methods in that more than half of the entire Brodatz textures have an accuracy between 90% - 100%. Our method also shows that an 83% correct classification rate is obtained on average over all the Brodatz textures compared to 75.5% and 71.4% achieved by MRSAR and SGF respectively.

 

 

: Content Based Retrieval from a lace texture

Some textures have very low classification success with all matching techniques tested; for examples D43, D44, and D58. This is due to a high inhomogoenity pattern spread over the whole original uncropped image. A preliminary manual examination of other textures for which the very best matches are not from the parent image suggests that the best matches are visually similar to the query textures. For example, some of the nearest matches of D54 subimage query are classified to D05, and their appearances are visually similar. Further investigation of the quality of the result order is in progress.

  
Table 1: The classification results of all the 112 Brodatz textures

For the complex textures experiments, we made the number of nearest matches considered for a certain class equal to double the number of images in the class since each class has different numbers of samples. The entries in table 2 represent the number of classes which score in each accuracy percentage range. Although these numbers are very low in this particular test, they also suggest that the Edge Based method is performing better than the other two approaches.

  
Table 2: The classification result of complex textures

 

 

: Content Based Navigation from a commercial furniture complex pattern

Content-based Retrieval and Navigation A hypermedia package [12], Multimedia Architecture for Video, Image and Sound (MAVIS), is being developed at the University of Southampton which is capable of content based retrieval and navigation for non-text media. In this section, the edge based texture classification is demonstrated with MAVIS for retrieval of similar complex furniture textures and navigation to different media using links based on texture matching.

To index multidimensional image features, the R-tree [9] is one of the popular choices and Beckmann et al. [1] developed the R*-tree which improved the space utilization compared against R-tree. In other popular content-based retrieval applications such as QBIC [8], the R*-tree is also used for indexing image features.

For efficient content-based retrieval, we use the Hilbert R-tree [10] for fast multi-dimensional indexing and retrieval. It has been shown that it outperforms the R*-tree. We have compared the performance between Hilbert R-tree and the R*-tree with an image database. The results showed that using Hilbert R-tree accesses significantly fewer nodes, has easy implementation, and is much less time consuming for building an indexing tree. We have experimented with k nearest neighbour queries for R-tree by Roussopoulos et al. [17] which gives a good performance. However, it is possible to improve on the k nearest neighbours search for a clustered image database. In [11], we showed that less data comparison can be achieved than with Roussopoulos et al. k nearest neighbour search on image features data and less computation can be obtained for faster retrieval.

We use a subset of features (lowlevel: Ratio and Contrast) indexed by a Hilbert R-tree, and enlarged k+t nearest neighbours are searched with normal Euclidean distance measurement as the similarity measure. Then the weighted Euclidean distance measurement is performed among these k+t retrievals using the full feature vectors described above. With t = 40, the accuracy of classifying all Brodatz textures drops down to around 3% compared against sequential normal k nearest neighbours search with full features set.

Figure 4 shows content-based retreival of a Brodatz lace texture (D40), and the results show a high accuracy retrieval with 1792 images stored in a database. All the 15 subimages are retrieved in the top 20 nearest matches.

In the MAVIS system it is possible to author generic links [12] from images to other parts of the information space using texture as the key. Once authored, the link may be followed from similar instances of the texture. In the next example, generic links have been authored from a texture patch to an image of a sofa with a similar texture. Also a link has been authored to some text describing the texture. Figure 5 shows an application using a similar commercial pattern (Tournament Stripe) to navigate to other information with related content. An ordered list of links is displayed in the Image links window with their iconic images. The related text information of the furniture pattern is shown in the txt window when the text media link (in Image links window) is selected and the Follow link button is clicked. A sofa image (in mavis_img_viewer window) with the same furniture pattern is located with similar actions.

Conclusion

A new texture classification technique has been proposed which uses edge and plain region information to characterize a texture. The texture method has been compared to MRSAR and SGF with the entire Brodatz texture database. The results show that our method outperforms the other two methods; more than half of the entire texture database is matched with 90%-100% reliability. On average, our method achieves at least 83% matching accuracy over all the Brodatz textures. With complex commerical textures, our method also gives a better classification rate. We demonstrate content based retrieval and navigation with the edge based texture scheme which provides an accurate method for content-based multimedia applications. Currently, our method is not rotation and scale invariant but modifications to include rotation invariance are in progress.