Unsupervised Method of Object Retrieval Using Similar Region Merging and Flood Fill

— In this work; we address a novel interactive framework for object retrieval using unsupervised similar region merging and flood fill method which models the spatial and appearance relations among image pixels. Efficient and effective image segmentation is usually very hard for natural and complex images. This paper presents a new technique for similar region merging and objects retrieval. The users only need to roughly indicate the after which steps desired objects boundary is obtained during merging of similar regions. A novel similarity based region merging mechanism is proposed to guide the merging process with the help of mean shift technique. A region R is merged with its adjacent regions Q if Q has highest similarity with R among all Q's adjacent regions. The proposed method automatically merges the regions that are initially segmented through mean shift technique, and then effectively extracts the object contour by merging all similar regions. Extensive experiments are performed on 22 object classes (524 images total) show promising results.


INTRODUCTION
CLASS-SPECIFIC (or category-level) object segmentation is one of the fundamental problems in computer vision.Although a human can delineate the object boundaries with much ease, segmenting images is not as ease for a computer.Its goal to segment an image into regions with each region solely containing object(s) of a class.As object segmentation requires that each segmented region to be a semantic object, it is much more challenging than traditionally object segmentation [1,2,3,4].There has been a substantial amount of research on image segmentation including clustering based methods, region growing methods [5], histogram based methods [6], and more recent one such as adaptive thresh-hold methods [7], level set methods [8], graph based methods [4,9] etc.Despite many years of research, unsupervised image segmentation techniques without human interaction still do not produce satisfactory results [10].Therefore semi-supervised segmentation methods incorporating user interactions have been proposed [11,12,13,14,15] and are becoming more and more popular.For instance, in the active contour model (ACM) i.e. snake algorithm [11], a proper selection of initial curve by user lead to a good convergence of the true object contour.
In order to do semantically meaningful image segmentation, it is essential to take priori (e.g.object part configuration [16], or class fragments [17]) information about the image into account.
The low level image segmentation methods, such as mean shift [18,19], watershed [20] and super pixels [21], usually divide the image into small regions.These low level segmentation methods provide a good basis for the subsequent high level operations, such as region merging.As a popular segmentation technique for color images, mean-shift [19] can have less segmented parts in comparison to watershed and super pixels [15,21,22] while preserving well the edge information of the objects.Because of less number of segmentation, the statistical features of each region, which will be exploited by the proposed unsupervised similar region merging method and object detection can be more robustly calculated and then be used in guiding the region merging process.
In this paper, we proposed unsupervised similar region merging method based on initial segmentation of mean shift.The proposed method will calculate the similarity of different regions and merge them based on largest similarity.The object will then extract from the background when merging process ends.Although the idea of region merging is first introduced by [23] this paper uses the region merging for obtaining the contour for object and then extracting desired object from image.The key contribution of the proposed method is a novel similarity based region merging technique, which is adaptive to image content and does not requires a present threshold.With the proposed region merging algorithm, the segmented region will be automatically merged and labeled, when the desired object contour is identified and avoided from background, the object contour can be readily extracted from background.The proposed algorithm is very simple but it can successfully extract the objects from complex scenes.
The rest of the paper is organized as follows; section 2 presents the proposed region merging algorithm.Section 3 performs extensive experiments to verify the proposed method and analysis.Section 4 concludes the paper and section 5 experimental results for different color spaces, different initial segmentation and comparison of proposed method with various existing algorithms.

II. SIMILARITY REGION MERGING
In proposed method, an initial segmentation is required to partition the image into homogeneous region for merging.www.ijacsa.thesai.orgFor this we use any existing low level image segmentation methods e.g.watershed [20], super-pixel [21], level set [24] and mean-shift [18,19] can be used for this step.In this paper we use mean-shift method for initial segmentation because it has less over segmentation and well preserve the object boundaries.For the initial segmentation we use the mean shift segmentation software the EDISON system [25] to obtain the initial segmentation map.Fig. 1. shows an example of mean shift initial segmentation.For detailed information about mean shift and EDISON system, please refer to [18,19,25,26].In this paper we only focus on the region merging.A. Similarity Measure After mean shift initial segmentation, we have a number of small regions.To guide the following region merging process, we need to represents these regions using some descriptor and define a rule for merging.A region can be described in many aspects, such as texture [27], shape and size [c] and color edge [28] of the regions.Among them color descriptor is very useful for representation of the object color features.In the context of region merging based segmentation, color descriptor is more robust than other feature descriptors because shape and size feature is vary lots while the colors of different regions from the same object will have high similarity.Therefore we use color histogram represent each region in this paper.The RGB color space is used to compute the color histogram of each region in this paper.We uniformly quantize each color channels into 16 levels and then the histogram is calculated in the feature space of 4096 bins.Next problem is how to merge the region based on their color histograms so that the desired object can be extracted.The key issue in region merging is how to determine the similarity between different segmented regions of image so that the similar regions can be merged by some logic control.Therefore we need to define a similarity measure Formula (1) between two regions R and Q to accommodate the comparison between various regions, for this there are some well known statistical metrics such as Euclidean metric, Bhattacharyya coefficient and log-likelihood ratio statistic [29].
Here we use Bhattacharyya coefficient [29,30,31,32] to measure the similarity between two regions say R and Q is: Where HistR and HistQ are the normalized histogram of R and Q, respectively and superscript u represents the u th element of them.
(√ ) (√ ) The higher the Bhattacharyya coefficient between R and Q is the higher the similarity between them i.e. smaller the angle θ.The geometric explanation of Bhattacharyya coefficient actually reflects the perceptual similarity between two regions.If two regions have similar contents then their histogram will be very similar, and their Bhattacharyya coefficient will be very high i.e. angle between histogram vectors is very small.Certainly it is possible that two different regions may have different histogram, such case happen very rare.Similarity measure between two regions we use Bhattacharyya similarity which works well in proposed region merging method.The Bhattacharyya descriptor is a very simple yet efficient way to represent similarity between regions.However other color spaces e.g.HSV, YCbCr etc. and other distance measure such as the Chernoff, Euclidean and Manhattan are also be adopted that for the region merging.In section 3 we present examples by using HSV, YCbCr color spaces and Manhattan distance.Results will be similar to those by using the RGB color space and Bhattacharyya descriptor.

B. Similarity Based Merging Rule
It is still a challenging problem to extract accurately the object contour from the background.The conventional region merging methods are merging two adjacent regions whose similarity is above based on threshold [32].These methods are difficult because of threshold selection.A big threshold will lead to incomplete merging belonging to object, while a small threshold will cause over-merging.Moreover it is difficult to detect when region merging process should stop.Proposed region merging method will start from any random segment part and start automatic region merging process.The entire region will be gradually labeled as either object region or background region.The lazy snapping cutout method proposed in [15], which combine graph cut with watershed based initial segmentation, is actually a region merging method.It is controlled by max-flow method [33].In this paper we present an adaptive similarity based merging technique of regions either in foreground or in background.
Let Q be the adjacent region of R and denoted by ̅̅̅ * + its set of Q's adjacent regions.Using Bhattacharyya coefficient calculate similarity among Q's adjacent regions ̅̅̅ * + . Obviously R will be one of the adjacent regions of SQ.If the similarity between R and Q will be maximum then region R will be merged in region Q.We will use merging rule according to the formula defined as: Equation ( 2) is the merging rule which establish the basis of proposed region merging process.Important advantage of ( 2) is that it prevents the use threshold for merging control, and the a b (1) = (2) (3) www.ijacsa.thesai.orgBhattacharyya coefficient is the inner product of the two histogram vectors and it is robust to small noise and variations.The automatic region merging process cover the all part of segmented image, and after every step of merging we will whether we want to work on this image or not.Therefore in the automatic region merging process object regions will have high probabilities to be identified as object.

C. The merging process
The whole object retrieval process is working in two stages.In first stage similar region merging process is as follows, our strategy to merge the small segmented image which is start with any randomly selected and merge this with any of its adjacent regions with high similarity.Some two step supervised merging process used in [34,35] for image pyramid construction.Different from [34,35] proposed method used image segmentation and it is unsupervised technique of region merging.We will merge segmented image regions with their adjacent regions as: if for each region Q we will set its adjacent regions SB i=1, 2, …, r.If the similarity between any Rj for any i= j is maximum i.e.The above procedure is implemented iteratively.Note that to each and every iterative step we will see whether the desired object is retrieve or not.Specifically the segmented region is shrinking; we will stop iteration when desired object is found.
After the first stage i.e. when full part of object boundaries or likely to appear which is seems in every step we apply second stage of algorithm for this we select a input point on the object and expand this using four connectivity of pixels by using well known Flood Fill method.

Input:
(1) the image (2) the initial mean shift segmentation of input image Output: desired object While there is a merging up to object contour 1.First stage of merging of initial segmented image (by mean shift method) using similar merging rule.2. After step one number of regions are minimized and again apply similar region merging rule, this is and iterative procedure.3.After retrieving object contour go to step (4).  1) what is the guarantee that the automatic similarity merging method will converge after a certain extent?To answer this question we will prove a proposition stated below.
Proposition1.The Similarity region merging method in section 2.3 will converge i.e. every region in the image will be merge after a certain extent.
Proof.If a region Q has the maximal similarity with region R then region R will be merged with region Q i.e.Q= QUR, in the first stage of proposed method this procedure is repeatedly and number of segmentation in the image is finite so the desired contour of object is obtained after a certain extent i.e. after kth iteration.
From above analysis we see that the number of regions in image (after mean segmentation) is N (say) it will decrease in the process if iterative region merging.The whole algorithm will stop and all segmented region is in either object or in background.
Therefore proposed algorithm converges and it will be label all the region of image.

III. EXPERIMENTAL ANALYSIS
The proposed similarity region merging method is an unsupervised method, since it will automatically merge the regions and it will labels every regions either object or background.
In section 3.1 we will first shows the unsupervised similarity region merging method qualitatively by several representative examples; in section 3.2 we compare proposed method with well-known hybrid graph model, graph cut and normalized cut; in section 3.3 we test our proposed method for different color spaces and different distance metrics.

A. Experimental analysis and Results
Fig. 2. shows an example of how unsupervised similarity region merging method extract object contour in complex scene.After initial segmentation by mean shift, automatic segmentation merging starts and after every step we test our merging results and also after which stage of merging we want   The execution time object retrieval using unsupervised similar region merging and flood fill depends upon a number of factors, including size of image, the initial mean shift segmentation results etc.We implement unsupervised similar region merging and flood fill algorithm in the MATLAB (R 2008a) 7.6 programming environment and run it on a PC with P4 2.80 GHz CPU and 1.0 GB RAM.
Table 1 shows the running time of proposed method on testing different types of images e.g.bird and airplanes etc.

B. Comparison with HGM and N-cut
In this section we compare the object retrieval method using unsupervised similarity region merging and flood fill method with hybrid graph model (HGM) and well known Normalized cut [4].Since the original graph cut segmentation is a pixels based method (GCP) for a fair comparison of proposed method, we extended the original pixel based graph method (GCP) to a region based graph cut (GCR) i.e. the nodes in the graph are mean shift segmented region instead of original pixels.
Table 2 shows the comparison of the three methods on testing different types of images e.g.bird and airplanes etc.We can see that proposed unsupervised region merging method achieves the best results in comparison to others, while (GCR) performs better result in comparison to (GCP).
It can be seen that (GCR) will miss some object regions and wrongly label background regions as object regions.To quantitatively compare the three methods, as shown in table 3, we mutually labeled the desired objects in the test image and took them as ground truth.After this we compute true positive rate (TPR) and false positive rate (FPR) for these segmentation results.The TPR is defined as the ratio of number of correctly classified object pixels to the number of total object pixels, and FPR is defined as the ratio of number of background pixels but classified as object pixels to the number of ground pixels.Obviously, higher the TPR and lower the FPR that method is better.

C. Unsupervised region merging under different color spaces, distance metrics and initial segmentation
Although RGB space and Bhattacharyya distance are used in proposed method, other color spaces and metrics are also used.In this section, we present some example to verify the performance of unsupervised region merging and flood fill method.We first test the effect of color space on the region merging result.In this experiment RGB color space is converted into HSV and YCbCr.The Bhattacharyya coefficient is calculated for the histogram of these color spaces.Fig. 4.
shows the unsupervised region merging on the images birds and airplanes and after that we use flood fill method on HSV and YCbCr space for extraction of object.
The Fig. 4(b) shows the initially segmented images in the HSV color space and Fig. 4(c), 4(d) and 4(e) shows the finally segmented object contour and Fig. 4(f) is mask of object and finally Fig. 4(g) shows object retrieve by using unsupervised region merging and after that we use flood fill algorithm for object retrieval.We can see that the results are same as those by using RGB color spaces with Bhattacharyya distance.Fig. 5 shows the segmentation results on the images of birds and airplanes.We can see that the results are same as those by Bhattacharyya distance.
At last we test the unsupervised similar region merging method with some other initial segmentation.Besides the mean-shift, watershed [20] and super-pixels [21] are another popular initial segmentation method.Super-pixels partition the images into more over segmentation in comparison to mean shift.In this experiment super-pixel method is used for initial segmentation.Section 5 shows the results of airplanes and birds.It can be seen that watersheds and super-pixel gives the similar results as mean shift.

IV. CONCLUSION
In this paper proposed a class specific object segmentation method using unsupervised similar region merging and flood fill algorithm.The image is initially segmented using meanshift segmentation and automatic start of merging with any random segmented region and after each merging we check whether the object contour is obtained or not, if at any particular stage of merging object contour is obtained then use flood fill algorithm and click with mouse which object we want to extract.The proposed scheme is simple yet powerful and it is image content adaptive.
In future we can extract multiple objects from input image by using unsupervised method as well as supervised method by merging similar regions using some metric.Extensive experiments were conducted to validate the proposed method of extracting single object from complex scenes.The proposed method is efficiently exploits the color similarity of the target.The proposed method provides a general region merging framework, it does not depend initially mean shift segmentation method or other color image segmentation methods [20,24,25,36] can also be used for segmentation.Also we can use appending the different object part to obtaining complete object from complex scene, and also we can use some supervised technique also.

V. EXPERIMENTAL RESULTS USING PROPOSED METHOD (FOR DIFFERENT COLOR SPACE AND INITIAL SEGMENTATION)
To see how Similar Region Merging Flood Fill produces promising segmentation results in the case that there is a large variation in shape (including position, size, profile and pose) within an object class.We refer to the Fig. 6, 7, 8, 9, 10, 11.The task is to segment an airplane from the background scene.To segment a new image that may contain object of several classes, we use initial mean shift segmentation method to segment the image into K regions in which all containing (s) of object class.We assume that K is known a priori for each test image.In this section, we present the example to verify the performance of proposed method under the different color spaces like HSV, YCbCr etc. and for different initial segmentation like watershed [20], super pixels [21].
We first test effect of color space on region merging result.In this experiment RGB color space is converted into HSV, YCbCr color spaces.Fig. 6. shows the object retrieval from RGB space, where as Fig. 10 and Fig. 11 shows the object retrieval from HSV and YCbCr color spaces respectively.Also Fig. 12. and Fig. 13.shows that object retrieval using different initial segmentation besides mean shift, watershed [20] is another important method of initial segmentation.Different from mean shift it partitions the image into more number of regions.Fig. 12. and Fig. 13.shows the result on bird and horse.Due to large number of regions in the initial segmentation of images using watersheds [20] its running time is more in comparison to mean shift initial segmentation and also results shows that the retrieve object after processing in similar fashion is not good as comparison with mean shift.

Fig. 1
Fig. 1 Original Image Mean shift segmentation Rj are merged into one region and new region is same leveled by

Fig
Fig.2 Region merging Process 1st stage merging Initial segmentation

Fig. 3
Fig. 3(g)  shows the extracted object using the two steps object retrieval method.

Fig
Fig.4 Original image Initial segmentation (HSV color space)

Fig
Fig.5 Original image object (HSV color space)

Fig
Fig. 6 Original image Mean shift segmentation 1st stage merging 2nd stage merging (RGB image)

Fig
Fig. 8 Original image Mean shift 1st stage merging 2nd stage merging (HSV image) segmentation

Fig
Fig. 13.Original Watershed Object contour Object Mask Object Image Segmentation

Table - 2
: Evaluation of results on 12 different class of image fg www.ijacsa.thesai.org