Underwater Optical Fish Classification System by means of Robust Feature Decomposition and Analysis using Multiple Neural Networks

Live fish recognition and classification play a pivotal role in underwater understanding, because it help scientists to control the subsea inventory in order to aid fishery management. However, despite technological progress, fish recognition systems still have many limitations on observing fish. Difficulties in visualizing optical images can arise due to external attenuation, scattering properties of water. Optical underwater imaging systems can also have detection problems such as changing appearance/orientation of objects, and changes in the scene. In this paper, we propose a new object classification system for underwater optical images. The proposed method is based on robust feature extraction from fish pattern. A specific preprocessing method is used in order to improve the recognition accuracy. A mean-shift algorithm is charged to segment the images and to isolate objects from background in the raw images. The training data is processed by Principal component analysis (PCA), where we calculate the prior probability inter-features. The decision is given using a combined Bayesian Artificial Neural networks (ANNs). ANNs will calculate non linear relationship of the extracted features, and the posterior probabilities. These probabilities will be verified in the last step in order to keep (or reject) the decision. The comparison of results with state of the art methods shows that the proposed system outperforms most of the solutions in different environmental conditions. The solution simultaneously deals with artificial and reel environment. The results obtained in the simulation indicate that the proposed approach provides a good precision to make distinguish between different fish species. An average accuracy of 94.6% is achieved using the proposed recognition method. Keywords—Fish recognition; Optical image analysis; scene understanding; principal component analysis; non-linear artificial neural networks


I. INTRODUCTION AND RELATED WORK
Oceans and seas are very fragile environments that harbor millions of plant and animal species.Morocco has a coastline stretching over 3,500 km of coastline (in the Atlantic Ocean and the Mediterranean Sea), a maritime area of about 1.2 million km 2 , a fishing potential estimated by FAO at around 1.5 million tons (renewable every year) [1].However, some species between them, victims of intensive fishing, are threatened with extinction [2].Today, the exploitation of these resources is an obligation.Nowadays, the underwater ecosystem can be protected by utilizing different monitoring systems.Some of these systems consider the automatic observation and visualization of fish [3].Automatic fish monitoring use different sensors such as cameras and sonars, the detection and classification of different species is done automatically instead of manual annotation.Computer vision and pattern recognition techniques offer powerful methods, which can tackle with the uge collected sensors data.Actually, the most underwater monitoring systems are based on optical images (captured by camera) and the exploitation of data.The big advantage of the use of cameras is a low cost in data gathering.
A major difficulty in processing underwater images is the attenuation of light (fig.I).This last reduces visibility range to about twenty meters in clean water and less than five meters in turbid water.In addition, images captured underwater are suffered due to many problems: first the rapid attenuation of light requires the attachment of a light source to the vehicle providing the lighting necessary [4], [5], [6].Unfortunately, artificial lights tend to illuminate the scene in a non-uniform way producing a bright spot in the center of the image and the poorly lit surrounding area.Furthermore, many technologies have been built in this context in order to develop sophisticated systems in underwater fish detection and recognition [7], [8], [9], [10].Authors in [11] use an automatic computer assisted by underwater video analysis for long term observation.[12] develop a fish detection method that involves the training of a classifier based on features extracted from fish and samples of other object types.Statistics about specific oceanic fish species distributions, namely discrimination and independence are given in [13] in order to aid in the feature selection process.
Choosing which features are to be used in the identification process has a major influence on the results of the study.In [14] and [15], the authors used shape and texture as features for the classification, which were derived from an active appearance model.The Principal Component Analysis (PCA) is used for fish cataloging.The classification accuracy obtained was about 76%.However, this method deal with very limited fish species group and its extracted features use only shape and texture.McGrath et al. [16] developed a fish identification system where the features are based on color, texture, and shape taken from the video sequences of four species.K-nearest neighbor is used on feature selection and fish classification.Correct classification rate about 70.6% is achieved.However, this method is limited in use as the images were taken on dead fish.Therefore, it is difficult to apply it in a real underwater environment and in real-time.Spampinato et.al [17] acquire underwater live fish recognition using a Balance-Guaranteed Optimized Tree.This method extracts many features, which sum up a combination of color, shape, and texture properties from fish pattern.Those features, however, do not perform well for noisy images.The method obtain 4% improvement of the average recall (AR) compared to the flat SVM classifier.Qin et.al [18] propose a system based on two feature extraction techniques: SIFT and Principal PCA, and two immunological algorithms: Artificial Immune Network and Adaptive Radius Immune Algorithm.The system achieves a 92% of success rate for six species of dead fish.
In all previous works, the accuracy is based on the proportion of correct recognitions while robustness means recognizing fish in a complex environment.In this paper, we present a new method for fish recognition using cameras.We focus our study to offer the submarine biologist methods to better exploration of marine resources without prior knowledge of underwater environment.The proposed approach, instead of building techniques in order to perform the classifiers structure itself, we consider it as a blackbox and focus on the extraction of robust features.
In next, this article is organized as follows.In Section 2, system overview is described where we discuss each step of the approach under investigation.In Section 3, experimental results are shown, and Section 4 concludes this work.

II. SYSTEM OVERVIEW
The main goal of fish classification is to derive methods for the cataloging of species underwater.Given a set of data from fish D = {d 1 , ..., d n }, we look for its categories c = {c 1 , ..., c m }.A raw data is represented by a feature vector, which will predict the fish category (Fig. 2).
The method is divided into two modes: offline and online.The role of the offline mode is to calculate the prior probabilities of the training data in order to make a preliminary distinction between species according to each feature.Offline  Training data processing.The training data is made offline as pretreatment stage.In this stage, the prior probabilities according to each feature are calculated.F 0 , F 1 , • • • Fn are the features used in the offline mode.For example the length of fishïs one of the features used.mode is described in the next subsection The goal of the online mode is to classify fish from the scene.In this mode, the system selects the detected fish and recognizes them in real time.The online mode is discussed in Section 2.2.
A. The Offline Mode 1) Initial probability: In this stage (Fig. 3), the dataset has to be trained.The aim of this step is to calculate the prior probability for fish species according to their features.If we consider c to be a set of fish categories, and m the number of categories, the initial probability of each category is defined as follows: with P (c j ) is the probability for the j-th category; 2) Prior probability: Unfortunately, in the practice, the previous probability is not uniform.Accordingly, the probability in (1) cannot help us to make a preliminary differentiation between species in the dataset.For this purpose, we compare the fish features according to their categories in order to find differences between them.A feature denotes a certain attribute After careful analysis, we reached the conclusion that the third specie surpasses in length the other two.Therefore, in the identification process, when we have to consider a big fish, for example, there is a higher probability it would belong to the third category.
that is considered important to describe a fish category to other one.For example: shape, length, texture, number of tails,... are some kind of characteristics (features).
In fact, the main objective of this stage is to find relevant data in many different features that deal with large quantities of information.In other words, this part take charge to calculate the probabilities of the training data according to each feature F i , in order to make prior distinction between fish species.For example, Fig. 4 illustrates the results of a comparative analysis, based on length, of three types of fish, undertaken with the aid of box plots.Used to show overall patterns of response for a dataset, box plots provide a useful way to visualize the characteristics of a large group of different fish.The fish categories represented in Fig. 4 are: Salmon (1), Blowfish (2), and Parrotfish (3).Note that the red marked symbols illustrate the median values of each category.
As you can see in Fig. 4, the population of the third category surpasses the one in the first and second category.Moreover, the second category is nearly as big as the first one.Accordingly, the prior probability in (1) will be changed.Precisely, in case the detection process contains large fish (for example an individual fish measuring more than 60 cm), the prior probability of the categories in relation with the length will be as This process is applied to all features in our fish dataset according to their categories.In next, the following subsection will deal with the online mode.

B. The Online Mode
In order to be able to identify fish using the online mode, a six stage process is proposed, as it can be seen in Fig. 5.Further on, we will divide this unit into four sections.The first one, which we will entitle subsection 2.2.1.,deals with the raw images recorded from the scene.These images will be preprocessed and further on segmented in order to separate the fish body from the rest of the image.subsection 2.2.2, on the other hand, deals with the preprocessing stage, while the segmentation one is discussed in subsection 2.2.3.When the fish is detected (in subsection 2.2.4), features of interest will be extracted.Then, feature vectors are generated.This process is described in Section 2.2.5.In last, subsection 2.2.6, stands as the classificatory one, when the cataloging, through a fish division (or partition) technique, using mixed neural networks, takes place.

1) Data acquisition:
The underwater platform is designed to detect and record optical signals over an extended deployment.The system is also able to take in the input images and video sequences.Fig. 6 illustrates some examples of raw scenes.
2) Preprocessing: Image analysis requires to be preprocessed regardless of the level of noise that the take displays.This noise should be eliminated.The objective of this subsection is to reduce the relevant noise, by ameliorating the visual appearance of the image.Furthermore, when looking at the underwater images, we could find two major problems: smoothness/noise.For this reason, we use a novel method of image denoising and enhancement suggested in [19] in order to improve the visibilty in raw images.In [19], authors model underwater environment by two overlapping processes: a Poisson distribution, and Gaussian mixture GM distribution.It's called Poisson-Gauss mixture distribution.The distribution was defined as (in [19]): where λ : a > 0 real number.z : Event.
x : Realization of Poisson-distribution. y : the realization of Gaussian-distribution f .m : number of Gaussians distributions.C i : covariance matrix of the i-th Gaussian.µ i : mean of the i-th Gaussian.α i : the mixture coefficient.
3) Segmentation: Mean shift is introduced by Fukunaga and Hostetler [20], this procedure, which stands as a powerful, non-parametric, iterative algorithm, was developed in order to be applied in many fields of computer vision.Its purposes are various.Mean shift associates these segments with the nearby pixels of the dataset probability density function.For each segment, it defines a window around it, and then it computes the mean of the data points.Then it shifts the center of the window to the mean and repeats the algorithm until it converges.After each iteration, the window shifts to a denser region of the image.As a result, mean shift segments images into different regions.Thereafter, the regions corresponding to fish will be extracted.

4) Feature extraction:
Feature extraction is a type of dimensionality reduction that efficiently represents appealing parts of an image as a compact feature vector.This approach is useful when the size of the image is large, and a reduced feature representation is required to rapidly complete tasks as image matching and retrieval.Feature detection and extraction are often combined to solve common computer vision problems, as well as object detection, content-based image retrieval, face detection and recognition, and objects classification.
Challenges: Automatic fish recognition is a difficult undertaking.In over thirty years of research in computer vision, progress has been limited.The main challenge is the amount of variation in visual appearance.A feature detector must take into account all specific characteristics of each category, and also with the specificity of visual imagery that exists underwater.For example, fish varies in size, shape, color, and in small details-such as the shape of the head, texture, and the position or number of fins.Furthermore, the lighting, surrounding scenery, algae, and an object's position affect its appearance.A feature detection algorithm must also distinguish fish species from all other visual patterns that can occur.
Formulation: The purpose of this stage is to extract as much information as possible from the data obtained in the previous stage.These features are expected to characterize different properties of structures and objects in each source of data.After feature extraction, a large amount of valuable information is obtained.
In this stage, the model traces useful object parts in each testing image.We split the fish body into three categories: head, middle, and tail.Fig. 7 depicts an example of this distribution.Size, orientation, shape, color, texture, and specific features of each part are extracted as a main group of features.As its name indicates it, the particular features represent a specific characteristic of a certain part of the fish body.In other words, it defines a special feature related only to a specific part of the fish's body.For example, the "position of the mouth" is a particular characteristic which is situated only in the head part.It can be terminal, inferior or superior.Therefore, the feature vectors will not be identical.These categories are later on used as class labels in the classification stage.In addition, we will study each part of the fish body.There are four major scenarios (or parts) that we desire to study: a whole body, fish tail, middle part, and fish head.

a. First scenario: The whole body
In the first scenario, we have set out to determine a large set of features (Fig. 8).For each fish species, we have gathered more than fifty different characteristics that are divided into four groups.These groups and their corresponding numbers of features are the following: Fig. 8. Feature extraction from a whole body of fish.This example shows a part of the feature vector in accordance with the whole body of the fish.Most of the features extracted here, are also used in the other scenarios.Fig. 9. Feature extraction from fish tail.We note that the feature vector does not resemble the others.
The gathering of these features is performed in all scenarios.In the other scenarios, we will approach only the extrafeatures.

b. Second scenario: Fish tail
This section refers to the "fish tails" particularities, such as the shape.There are different kinds of tail shapes: heterocercal, forked, lunate, emarginated, truncated, rounded, or pointed.These details play a critical role in the differentiation of species, and it can improve the final decision.Fig. 9 shows an excerpt from the feature vector for the tail.

c. Third scenario: Middle part
The middle part is very important because it contains the texture and some fish fins.In this part, we will focus our attention on the number of fins, visual appearance, and texture.The features discussed in the first scenario will be used as well.Fig. 10 shows a part of the feature vector concerning the middle part.
d. Fourth scenario: Fish head  The head of the fish displays specific features, as well.For example, "the position of the mouth" can be: terminal (mouth oriented in the middle), superior (mouth oriented upwards), or inferior (mouth oriented downward).Fig. 11 illustrates an excerpt of the feature vector of the head.5) Fish identification: It is performed using Artificial Neural networks (ANNs) [21].Ann defines interaction between elements (nodes or neurons).A special function is responsible to compute the output of nodes.In the practice, ANNs are composed of three main layers, namely, the input, hidden and output layer (Fig. 12): • Input layer: They take as input n real values I 1 , I 2 , ..., I n .These values represent feature vector of part of fish.• Hidden layer: It is the feature combination function which calculates the summation n i=1 I i w i where w i is the input weight w 1 , w 2 , ..., w n .
• Output layer: It's the activation function, in which we compute the posterior probabilities of fish categories.
layers are composed by one or more nodes.It's symbolized as small circles.The flow of information, from one node to the other, is figured by oriented lines.

III. CLASSIFICATION RESULTS
The classification was carried out using the ANNs combined artificial neural network.Fig. 13 shows the processing operation for input features according to fish sub-divisions, where each body part of a fish is treated independently.In which each region will be analyzed according to the input values obtained by the feature vector of each part (cluster).Subsequently, the resulting probabilities obtained by each of them will be combined to have a value.The latter will be considered as the overall decision that presents the final decision.
The main purpose of this section is to extract pertinent information from each body part in order to obtain final decision by combining each ANN output (final probability of each Neural network).The ANN model can be represented as follows: where  t is the aggregated single network model prediction.
w k is the combination weight of the k-th network.
In order to build data fusion of several ANNs, we define a combination function after getting the output posterior probabilities (from each ANN) (in Fig. 13).It can be drawn from the formula of Bayesian probability combinations.
Let us suppose that we have H 1 , H 2 , • • • H K in the Bayesian inference process, where H K represents hypothesis that can illustrate an observation E (or an event).Thus, the posterior probability of hypothesis H k being true, given the evidence E, P (H k |E), can be defined as in (4).
where P (H j ) is the prior probability of hypothesis H j being true and P (E|H j ) is the probability of observing evidence E given that H j is true.II show hypothetical examples based on the "head shape" of the fish.In these examples, we note that the feature "shape of the head" is symbolized as f 0 , and we Algorithm 1: The proposed pseudo code algorithm.

Table I and Table
The application of classification is usually a trade-off between low numbers of false alarms (false positives FP) and high numbers of correct detections (true positives TP).According to statistics, the numbers of true and false positives are described using the following formula: where l is the number of classes.
Different combinations of both measures can be plotted as ROC (Receiver Operating Characteristic).The number of true negative TN is not exactly defined.The amount of false detection can be produced.Furthermore, it is often not possible to estimate the correct number of background objects.For this reason, another measure is used to describe the number of false alarms: A total accuracy, which is defined as the percentage of correctly classified fish.Usually, accuracy is represented as a real value between 0 and 1.

Accuracy = Number of correct decisions Number of total decisions (15)
The first evaluation only assesses the performance of our fish detection system, which consists of two classifiers.When evaluating each class separately, the classification performance for each class could be concluded: The individual class precision/recall is represented in Fig. 14.The suggested approach achieves a good accuracy in fish distinction.As shown in Table.III, an accuracy average of 94.6% (dataset1: 99.4% and dataset2: 89.9%), and the matching speed of 0.26 seconds for several test sample images have been obtained by the proposed approach.The proposed approach 99.4% 89.9% 94.6%

IV. DISCUSSION
In order to measure the robustness of our method, the proposed approach is tested in two environmental conditions: clair and turbid environment.The proposed approach is also compared with two methods: Ada-boost (proposed by Hsu et.al [23]) and hierarchical tree (proposed by Fan et al. [24]), and applied to the same data.As shown in Table IV, we note that our method is about 5% better than hierarchical tree.The average was about 94.6% using the proposed approach against 89.7% by hierarchical tree.The Ada-boost method achieves 79.2% accuracy.Furthermore, in many applications, the ROC curve provides more interesting information about the quality of learning than just the error rate.Fig. 15 (a) and (b) show ROC diagram of the three methods on the database 1 and 2. The curves in red represents the proposed method, in orange the hierarchical tree, and in green the Ada-boost method, respectively.As shown in Fig. 15, curves in "green" and "orange" are lower than the "red" one.It means that the number of true positive is important compared to the others methods.As results,we conclude that the proposed approach outperforms these two methods.*d.set1: dataset 1 "clair environement", *d.set2: dataset 2 "turbid environement".

V. CONCLUSION
The present paper described an algorithm employed in fish classification based on robust fish division/sectioning/partition and feature extraction.Our goal was to develop a system that detects and recognizes an array of different types of fish in images and videos, including various structures in underwater images.Each segment (or part) forms a sub-classifier and represents various local properties of the fish.Bayesian mixed ANNs are used to classify each part.Probabilities are given by each ANN and they were combined and recalculated in order to reach a conclusion.Experimental results show a fine accuracy in the fish classification system.We note that the observed precision of 94.6% may be considered a satisfactory estimation, as it provides a reasonable approximation of fish classification, the varying environmental conditions in an open unconstrained space and the changeable status of the sensors used.However, the use of static features could limit the application of the proposed algorithm.In the future work, we plan to build a novel fish recognition system using deep learning methods, that can provide more dynamicity and flexibility in the feature selection process.

Fig. 1 .
Fig. 1.Underwater environment affect seriously to the optical images.

Fig. 3 .
Fig.3.Training data processing.The training data is made offline as pretreatment stage.In this stage, the prior probabilities according to each feature are calculated.F 0 , F 1 , • • • Fn are the features used in the offline mode.For example the length of fishïs one of the features used.

Fig. 4 .
Fig. 4. box plot of the training data.The following example compares the length of three different fishes: c1-Salmon, c2-Blowfish, and c3-Parrotfish.After careful analysis, we reached the conclusion that the third specie surpasses in length the other two.Therefore, in the identification process, when we have to consider a big fish, for example, there is a higher probability it would belong to the third category.

Fig. 6 .
Fig. 6.Example of scenes used in the classification.

Fig. 7 .
Fig. 7. Fish decomposition.The image is divided into three parts.The first part illustrates the head, the second one represents the body, and the third one the tail.The size of the body is twice the size of the head, and correspondingly the tail.

TABLE I .
HYPOTHESIS ACCORDING TO THE FEATURE "shape of the head".