Wakes-Ship Removal on High-Resolution Optical Images based on Histograms in HSV Color Space

Ship detection on optical remote sensing images is getting great attention; however, some images called wakesship have not been taken into account yet. Current works in ship detection are focusing on in-shore detection where ships are in calm; furthermore, their methods get high Intersection Over Union (IoU), above 70%, but when computing IoU using only wakes-ship images the ratio is 22%. In this paper, it is presented a new framework to improve ship segmentation on wakes-ship images. In order to achieve this goal, it was considered HSV color space and histograms. First, ship detection was done using state-of-the-art ship detection methods. Second, bin histograms in HSV color space was used to get the colors that rely on wakes. Finally, the removal of wakes from ships was done using some discriminative properties. In this way, the increase of the IoU performance at wake-ship segmentation goes from 22% to 63%, which is an improvement of 186%. Keywords—Wakes-ship removal; optical remote sensing; ship detection; HSV color space; histograms; intersection over union


I. INTRODUCTION
Ship detection on optical satellite images is attracting great interest with the growing use of optical remote sensing images in recent years [1] and because of its importance in maritime security, fishery management and other applications [1].Most of the state-of-the-art works in ship detection are using Convolutional Neural Network (CNN) as the benchmark [2]- [5] and they are getting high detection ratio, results are above 85% in precision and recall.
There are two kinds of images used in ship detection task, one is off-shore as shown in Fig. 1, where ship detection task appears to be easy and the other is in-shore, Fig. 2, which shows great difficulty when it comes to ship detection because of the land, harbor, and other issues that may happen in optical remote sensing images [1], [6].Most of the works are focused on in-shore detection since it represents a big challenge.
The tests were run on HRSC20161 dataset released by (Lui's et al.,2016) [6], it contains 1680 high-resolution optical images of ships in in-shore and off-shore collected from different sources.Due to the difficult task in in-shore ship detection, most of the images used on current works are inshore, where many ships are in calm and lying next to the harbor as shown in Fig. 2.
In this paper, it is wanted to address the problem with offshore images, Fig. 1.There are two kinds of images in offshore, images where ships are moving and generating wakes  around them, Fig. 1a, in this paper they are called as wakesship images and the others are static-ship images, Fig. 1b, where ships are not moving, thus, they are not generating wakes.These two kinds of images could be seen in Fig. 1.
While ship detection in static-ship images, Fig. 1b, appears to be easy.However, it is difficult to detect ships when the sea part has noise.On the other hand, there are two problems that have not been taken into account yet on wakes-ship images, Fig. 1a.First, most of the methods on these images detect the ship, but including the wakes, thus, that means that they get low Intersection Over Union (IoU2 ), and because of the vast use of in-shore images where static-ships presence is dominant, they get high IoU.To show this behavior, tests were run using current ship detection methods on static-ship images and the results have shown that they got high IoU.Getting high IoU is important because it allows to get the right measurements from ships such as length and width and consequently it helps to ship classification task [5], [7], [8].
A research [9] approached this problem using clustering kmeans and watershed segmentation, they tested their method in a small dataset of 54 images, also they point it out how important is the segmentation in this kind of images.Another recent research [10] remarks the importance of segmentation on ship-wakes images, they used CIELAB color space to segment ship from wakes as a preliminary study.Both works were done using remote sensing images, but with low resolution.
To tackle the problem with wakes-ship images, in this paper, it was used bin histograms generation in HSV 3 color space to remove the wakes from ships, but before that, a ship detection method was needed because it allowed knowing whether there are ships with wakes on the image or not.Two ship-detection methods and Otsu method were used to deal with ship detection as described in Section II-B, consequently, they were compared each other at IoU on wakes-ship images.So, there are two stages in the proposed framework in this paper, at first stage, it was used a ship detection method, it allowed knowing that there exist a ship on the image.The second stage, once the segmented image has been taken with only wakes and ship, the removal of wakes was done by changing the image into HSV color space and by generating HSV histograms on this color space to evaluate the colors that rely on the wakes.Fig. 3 shows the images that were taken in the entire process of our framework.Fig. 3: Images obtained at every step of the entire process of the framework, from original wakes-ship images until the wakes-ship removal and ship segmentation.

The remainder of this paper is organized as follows. Section
This section is covered.First, the HRSC2016 dataset is described, furthermore, the kind of images that have been taken into account in this paper, then it is described two ship detection methods used in this paper and the evaluation of them when applied to wakes-ship images.

A. HRSC2016 Dataset
The HRSC2016 dataset [6] contains images from two scenarios, ships far from the harbor and ships next to the harbor, they are typically called off-shore and in-shore images, respectively.All of the images were collected from famous harbors.The resolution of the images is between 2-m and 0.4m.
From these two groups of images; most of the images are in-shore, a small number of images are off-shore, (Liu et al.) explains it is because of the big challenge that in-shore ship detection demands.From the total 242 off-shore found images, it has been selected 50 wakes-ship images, which means, ships are moving and generating wakes around them, like in Fig. 1, the rest of the images the ships are next to the harbor in calm and without wakes.In summary, from the 242 off-shore images, there have been taken 50 wakes-ship images and 192 static-ship images.They were used both off-shore images to test the accuracy of IoU of ship detection methods.

B. Ship Detection Methods
It was used OTSU method as a baseline and two ship detection methods [11], [12] to compare, these methods were implemented and tested in Liu et al. [6], [13], both of them focuses on off-shore ship detection, so they were not prepared for wakes-ship images.
Tang's method [12] is based on sea-land segmentation with some improvements, they include ship location, feature representation, and classification, but only ship location criteria was used in this paper.The other method, Liu's ship extraction method [11] is based on "V" shape ship-head detection and it is well suited for high-resolution optical images.
To know the accuracy of these segmentation methods, the metric Intersection Over Union (IoU) used in PASCAL VOC 2007 challenge was applied, which is described as follows, S represents the bounding-box area of the original image and S' represents the bounding-box area of the proposed method.When this ratio is above 50%, it means a good detection rate.
After running tests on wakes-ship images using these ship detection methods presented previously, it has been seen that these methods performed with a high Intersection over Union(IoU) ratio on the 192 static-ship images, between 62% and 73% as shown in Table I.However, when applied only using wakes-ships images, the result of the Intersection over Union (IoU) ratio was between 22% and 32% as shown in Table II.These results show that segmentation on static-ship images appears to be easy; however when static-ship images present noises, false alarm removal is required.While in wakes-ship images the methods do not perform well since the wakes generated by ships have too much noise, furthermore, IoU ratio is lower because of the presence of wakes around the ship.

III. METHODOLOGY
In this paper, it is proposed a new framework for wakesship removal of off-shore images on high-resolution optical images as shown in Fig. 4. By going throughout this framework, the removal of wakes from ships can be done, thus, to get the right measurements from ships, such as length and width could be used for ship classification task.Classification of ships is beyond the needs and scope of this paper.
The proposed framework is divided into two stages.First, ship-detection segmentation is the stage where the ship with wakes is taken from original images; current ship detection methods aforementioned were used at this stage, during the test results applying Tang's method [14] excelled, it could be seen in Fig. 5 along with other methods used.The second stage, the wakes-ship removal task is done, where the use of the segmented image performed previously takes place, by converting such image in HSV color space.To deal with colors that rely on wakes, the generation of histograms in HSV color is performed, such histograms clearly indicate the wakes color values, next the separation of wakes from ship is done by deleting the values obtained from histograms and consequently generating a small group of pixels.Finally, false alarms are removed to get the ships.The next subsections explain in detail every single step of the second stage of our framework, Fig. 4.
Fig. 4: The methodology presented in this paper with two stages.Ship-detection Segmentation, where the ship detection task is done and Wakes-ship Removal, where the removal of wakes is performed.

A. Wakes-Ship Segmentation Image and HSV Color Space Conversion
First, getting the wakes-ship segmented image was needed; most of the methods for ship detection provide one, they detect the ships, but including the wakes as shown in Fig. 5.It is necessary to remark, these methods are not prepared for wakesship images since they wanted to detect ships in in-shore, but according to the tests carried out they still detect the ship in off-shore.Liu's method [11], Tang's method [12] and Otsu's method as a baseline were used.From these methods, the proposed framework outperformed with Tang's method since it provides the mask of the complete ship and wakes, in addition, it allows getting a better representation of HSV histograms of HSV wakes colors.Fig. 5: Several ship detection methods detect the ship, but including the wakes around them.So, a removal of them is needed.
Once the mask for ship and wakes were obtained, conversion of the image in HSV color space using the method proposed in [14], [15] is performed.HSV color space has been largely used for image segmentation and image retrieval [14], [15].H(Hue) is defined as the angle in the range [0, 2π] starting at red axis with red at 0, green at 2π/3, blue at 4π/3 and red at 2π again, the transformation of these angles to values between 0-1 was done.S(Saturation) is measured as a radial distance from the central axis with a value between 0 and 1, it defines the brilliance and intensity of a color and V(value) defines the lightness or darkness of a color, it goes from 0 to 1.
From these three values, after running some tests, H(hue) and V(value) were the only values that clearly indicated the presence of the colors that rely on wakes.Unlike S(saturation), even its value was ranged from 0 to ∞ it did not indicate big changes and did not cause many troubles.So H(hue) and V(value) values were used to generate histograms as seen in Fig. 6 and 7.

B. Getting HSV Wakes Color Values from Histograms and Wakes Removal
Second, to get the colors that rely on wakes, the differentiation of the wakes colors from ship colors was done by generating histograms in HSV color space for H(hue) and V(value) as shown in Fig. 6.Tang's method excelled combined with the proposed framework because it segments both ships and wakes without damaging the integrity of the ship and taking a good proportion of wakes.Although Liu's method has better detection ratio and performs well in static-ship images, it deletes most of the wakes around the ship and part of the ship in wakes-ship images, so the HSV values obtained are hard to normalize since most of the wakes pixels were removed from the image.In order to get the right H(hue) and V(value) wakes values, a normalization of these values was needed and also adding an extra tolerance for each value when deleting the pixels from wakes-ship images.So the chosen number of bins for H(hue) was set 16 as suggested in [14].Saturation values did not cause many problems on the tests as explained before, even when its value was changed between 0 and ∞, thus, it was not taken into account.V(value) value was set to 18 bins empirically, Fig. 7 shows these histograms.
Once the values were normalized, the calculation of the HSV wakes values to delete them from the image was carried out.In order to accomplish that, setting the HSV values I f rom(h,s,v) , from which starting off deleting the pixels until reaching I to(h,s,v) HSV values.This range is calculated for each image as it is written below.

C. General Thresholding and False Alarm Removal
Third, after the removal of the HSV wakes pixels, the pixels left form multiple small connecting regions.General thresholding was applied to the entire image, then 4-connected neighborhood components were used to get all the regions left.Finally to remove the false alarms that may exist it has been used three discriminative properties, area, compactness and length-width as in [11], [16].
1) Area: on the tests, it has been noticed that just calculating the largest area among all the regions on the 50 wakes-ship images it has been gotten 96% accuracy, which means the elimination of almost all the wakes were done by breaking them down.2) Compactness : It measures the circular similarity degree, and it is as follows.
Where P represents the perimeter and A represents the Area.3) Length-Width : Due to the most of the ships are long and thin, this simple ratio can eliminate false alarms.
Where L is the length of the bounding-box and W is the width.

IV. EVALUATION
The proposed method was tested on 50 wakes-ship images from HRSC2016 dataset, the results are shown in Table III, which shows that ship detection methods on wakes-ship images get better performance when combined with our method.Table III shows the importance of taking into account the wakes around ships since the more wakes pixels it has, the more HSV representative wakes values it gets and it is easy to normalize their values.Liu's method did not excel in Table III

V. CONCLUSION AND FUTURE WORKS
In this paper, it has been proposed a new framework for wakes-ship removal by generating histograms in HSV color space.It has been used current ship detection methods.By doing this, the only task that remains is the removal of wakes from ships.In order to do that a complete analysis of the colors that rely on wakes was carried out by generating histograms of HSV values to get representative wakes HSV value colors.The elimination of the was done by eliminating pixels values near the mode(Mo) in histograms, next the elimination of false alarms and the extraction of the ship was done by using some discriminative properties.
The results show that the average success rate of Intersection over Union (IoU) goes from 22% to 63% which is an improvement of 186%, IoU above 50% means a good detection rate.It is also important to mention that on some images where the integrity of the ship has been affected by using ship detection methods, it was impossible to recover the pixels.Another observation, when applied our framework to the image without a ship detection method, the HSV values represent almost all the sea values, so it is hard to get the right wakes values, thus, it is important to know whether the image contains a ship with wakes or not.
In order to get a better representative HSV values from wakes, it is necessary to collect more wakes-ship images, in that way, a supervised learning model for wakes-ship removal in HSV color space could improve the results for others ship detection methods.

Fig. 1 :
Fig. 1: There are 242 images in off-shore in HRSC2016.(a) Wakes-ship images, there are 50 of them.(b) Static-ship images, there are 192 of them.

Fig. 2 :
Fig. 2: 1438 in-shore images out of 1680 from HRSC2016.current ship detection works focus on these images since it represents a big challenge.

Fig. 6 :
Fig. 6: Histograms in Hue and Value color space without normalization.

Fig. 7 :
Fig. 7: Histograms in Hue and Value for Hue with 16 bins, and Value with 18 bins.The mode in each figure represents the Hue and Value for wakes values.

TABLE I :
Comparison Intersection Over Union (IoU) among Ship-Detection Methods on the 192 Static-Ship Images in Off-Shore

TABLE II :
Comparison Intersection Over Union (IoU) among Ship-Detection Methods on the 50 Wakes-Ship Images in Off-Shore as it did in TableIIbecause it was prepared to work on high-resolution images, furthermore, it deletes part of the wakes from images, which difficulties the task of normalizing since the HSV wakes values are almost the same as HSV ship values.

TABLE III :
Improvement for Ship Detection Methods, Getting Higher IoU by Removing Wakes from Images