Category Decomposition Method for Un-Mixing of Mixels Acquired with Spaceborne Based Visible and Near Infrared Radiometers by Means of Maximum Entropy Method with Parameter Estimation Based on Simulated Annealing

Category decomposition method for un-mixing of mixels (Mixed Pixels) which is acquired with spaceborne based visible to near infrared radiometers by means of Maximum Entropy Method (MEM) with parameter estimation based on Simulated Annealing: SA is proposed. Through simulation studies with spectral characteristics of the ground cover targets which are derived from spectral library and actual remote sensing satellite imagery data, it is confirmed that the proposed method works well.


INTRODUCTION
Linear models for un-mixing are based on: (1) a maximum likelihood method (Settle, 1996 [1]; Matsumoto et al., 1991 [2]); (2) a least squares method with constraints (Chang, 2003 [3]; Arai et al., 1995 [4]); (3) spectral feature matching (Mazer, 1988 [5]); (4) a partial space projection technique (Chang et al., 1998 [6]; Arai et al., 2002 [7]); (5) a rectangular partial space method (Harsanyi et al., 1994 [8]).Other than these, there is standard method; Principal Component Analysis (PCA) based un-mixing.In essence, the PCA based method is almost identical least square method without constraint.The least squares method with constraints estimates a mixing ratio vector based on an end-member's spectral feature vector, using the generalized inverse matrix or the least squares, where the convex combination conditions act as a constraint.The spectral feature matching method searches and selects two or more spectral features out of many in a spectral database, and compares the mixing ratios and the spectral feature of the mixed pixel: mixel of interest.
Further studies of un-mixing methods are required for appropriate end-member determination, improvement of accuracy, reduction of processing time, etc.This paper mainly focuses on the improvement of un-mixing accuracy, and estimation of accuracy of mixing ratio.
Un-mixing methods based on partial space projection (Chang, et al., 1998 [6]; Arai et al., 2002 [7]), and rectangular partial space (Harsanyi et al., 1994 [8]), can make the spectral features of a target category more conspicuous, by mapping and combining the spectral feature of the mixel with a subspace that intersects perpendicularly with the other spectral features.The methods can also perform dimensionality reduction.Moreover, the un-mixing method based on an orthogonal subspace has a comparatively good accuracy of unmixing, and is widely used.Furthermore, it is equivalent to un-mixing based on maximum likelihood, and also to the method of least squares.Independent component analysis (ICA) is a technique that decomposes a given mixel into highly independent components in the spectral feature space (Parra et al., 2000 [9]).Specifically, it is assumed that a mixing ratio can express the spectral features of a mixel by taking into consideration an end-member's spectral features and their variation.The statistical model for a component and the ratio of each component are estimated by unsupervised learning.Therefore, it is a method that uses ICA with constraints.Moreover, since the orthogonal subspace method becomes ideally independent in the spectral feature after projection, it is also equivalent to ICA.For this reason, this paper shall examine un-mixing method based on the orthogonal subspace.
The subspace method with learning process has already been proposed as an image classification technique (Erkki, 1983 [10]).The basic idea is as follows: if the coordinate-axis of the subspace in the orthogonal subspace method is rotated, then classification accuracy will be improved, resulting in an appropriate angle for a high classification performance; the spectral features of a pixel are then mapped and classified into the orthogonal subspace of this optimal coordinate-axis angle.
The following section describes the proposed category decomposition (un-mixing) method based on Maximum Entropy Method: MEM with parameter estimation by using Simulated Annealing: SA followed by some experiments.Then conclusion is described together with some discussions.www.ijarai.thesai.org

A. Conventional Method
Mixed pixel: Mixel is defined as the pixel which is composed with plural ground cover materials.Also un-mixing is defined as the method which allows estimation mixing ratios of the ground cover materials in the Mixels in concern.In particular for hyper-spectral radiometer data, un-mixing can be done utilizing spectral feature of the ground cover materials with a consideration of atmospheric influences.Figure 1 shows example of the spectral characteristics of the ground cover materials which are included in the Mixels in concern.There are specific absorption characteristics to the ground cover materials.Specific absorption characteristics can be observed with hyper-spectral radiometers.Absorption wavelength intervals, depth of the absorption, and center wavelength of absorption can be used as spectral absorption characteristics and also used as features for discrimination of types of ground cover materials and also for un-mixing.Example of the spectral characteristics of the ground cover materials Fig. 2.
Feature vectors of the spectral characteristics of ground cover materials Figure 2 shows the vectors, sp1 and sp2 of the end member of the ground cover materials together with the unknown vector of the Mixel in concern (dotted line).Mixing ratios of sp1 and sp2 can be estimated with the ratio of c and d.
There is another conventional method so called "SVD method".Hyper-spectral data represented by vector Y can be expressed as follows: where m denotes mixing ratio vector of each ground cover target no.1 to m, while Z denotes the spectral characteristics of the ground cover target such that if the inverse matrix of Z is existing then the mixing ratio vector can be estimated as follows, It is, however, not always true that the inverse matrix exists.In order to solve this problem, regularization techniques with constraints, a prior information, etc., have been proposed.One of those is the generalized inverse matrix, or so called "Moore-Penrose" inverse matrix that is derived from Singular Value Decomposition (SVD).If Y is expressed with SVD as follows: (3) where u and v are orthogonal vectors, then the Moore-Penrose generalized inverse matrix Y + is expressed by the following equation: Therefore, if u and v can be calculated then the generalized inverse matrix can also be calculated.This method is referred to the conventional SVD based method hereafter.

B. Proposed Un-Mixing Method
The proposed un-mixing method is based on Maximum Entropy Method: MEM which is described in the next section.MEM allows finding maximum probability of the unknown vectors of the Mixels in concern with a possible combination of ground cover materials.The basic idea of the proposed unmixing method is that mixing ratios of the ground cover materials in the Mixels in concern can be estimated at the maximum probability of the unknown vectors of Mixels in concern.The maximum probability is equal to maximum entropy.Therefore, MEM can be utilized for this process.
Through maximizing process, unknown parameters have to be determined.The unknown parameters are in both sides of the equation.Therefore, iteration process is required to determine the unknown parameters.Although there are many iteration methods, Most Steepest Descent method, Conjugate Gradient method, etc., there is only one method, Simulated Annealing: SA method that gives a global optimum solution.The other methods give local minima, not the global optimum.Therefore, SA is used for parameter determination.

C. Maximum Entropy Method
Maximum Entropy Method: MEM is used for learning processes for maximizing combined entropy in two layered neural network www.ijarai.thesai.org(5) (6) where x, y notes input and output signals, or images while w denotes weighting coefficients of two layered neural network.Θ denotes a threshold.g(v) is called sigmoid function.
The combined entropy can be expressed with the equation ( 7).(7) where J denotes Jacobian matrix as shown in equation ( 8) (8) Because the second term of the equation ( 7) is constant, the following equation ( 9) has to be maximized.( 9) It used to be said that blind separation can be done by Maximum Entropy Method (MEM) theoretically if the mixed voice sounds is distributed as Generalized Gauss Function (GGF) and if the mixing ratio of the target voice sounds is greater than a certain level.It, however, used to be distributed as non-GGF distribution and the mixing ratio of the target voice sounds is not so high.Then it becomes difficult to separate the target voice sounds from the mixed voice sounds signals.In order to fit the of mixed voice sounds to GGF distribution, the method using high frequency component of MRA is proposed [11].
In general, if the wavelet transformation is applied to the target voice sounds and the other voice sounds individually, then irregularly distributions are used to be obtained for low frequency component.Turns out, high frequency component derived from the wavelet transformation for the mixed voice sound signals shows GGF distributions.Therefore high frequency component of the target voice sounds become separable to the other voice sounds because learning converged neural network may output the high frequency component of the target voice sounds and the other voice sounds if the high frequency components of the target voice sounds and the other voice sounds are input to the neural network.Thus the histogram of the high frequency component of the mixed voice sounds after the wavelet transformation is followed by the GGF distribution after the learning process of the neural network is converged.
If low frequency components of the target voice sound and the other voice sound signals are input to the neural network, the low frequency components of the target voice sound and the other voice sound are output from the neural network after the learning process is converged results in the target voice sound is reconstructed based on inverse wavelet transformation because the mixing ratio of the target voice sound and the other voice sound is not changed.
The separability of the proposed method depends on how the histogram of the high frequency component is similar to the GGF distribution and the kurtosis of the histogram.In general, the GGF likeness is not so bad while the kurtosis is not so good.The blind separation method proposed here is to improve the separability through enhancement and improvement the GGF likeness and the kurtosis.

D. Simulated Annealing
Simulated annealing utilizes Metropolis algorithm (Metropolis et al. 1953 [12]), in which some trades that do not lower the mileage are accepted when they serve to allow the solver to "explore" more of the possible space of solutions.Such "bad" trades are allowed using the criterion that (10) where is the change of distance implied by the trade, is a "synthetic temperature," and is a random number in the interval . is called a "cost function," and corresponds to the free energy in the case of annealing a metal.In the case of temperature parameter would actually be the , where is Boltzmann's Constant and is the physical temperature, in the Kelvin absolute temperature scale.If is large, many "bad" trades are accepted, and a large part of solution space is accessed.Objects to be traded are generally chosen randomly, though more sophisticated techniques can be used.

A. Simulation Method
From the Unites State of America Geological Survey: USGS web site so called "Spectral Library", spectral characteristics (surface reflectance) can be retrieved for huge number of ground cover materials.In the library, two ground cover materials which show a good correlation between both spectral characteristics are chosen.Also two ground cover materials which show a poor correlation between both are selected.Moreover, ground cover materials which show a middle level of correlation are used.These are shown in Figure 3. 437 spectral channels ranged from 500 nm to 2500 nm of wavelength region of spectral characteristics are used.Examples of the selected two ground cover materials of spectral characteristics which show a good a middle level and a poor correlation between both spectral characteristics Using these spectral characteristics, un-mixing by means of the proposed MEM based un-mixing method is attempted with changing mixing ratio as well as additive noises.0.5 of mixing ratio of Mixels are generated for simulation study.
Root Mean Square Error: RMSE between designated and estimated mixing ratios is evaluated.The RMSE is evaluated for both the conventional SVD based method and the proposed method.

B. Simulation Method
RMS error of the un-mixing error between designated mixing ratio and estimated mixing ratio is evaluated as function of additive normal distributed noises which is generated using random number generator with zero mean and standard deviation of "noise" based on Messene Twister for the three cases of correlation, poor, middle level and maximum.
The evaluation results are shown in Figure 4.

RMS error
Figure 4 shows RMS errors for conventional SVD method and the proposed MEM based method.As shown in Figure 4, www.ijarai.thesai.orgRMS error of the proposed MEM based method is always lower than that of the conventional SVD based method.Therefore, it is said that the proposed un-mixing method is superior to the conventional method.
It also shows that RMS error depends on correlation.Namely, it is hard to estimate mixing ratio when the Mixel is composed with highly correlated spectral characteristics of ground cover materials for both the conventional and the proposed un-mixing methods.
Convergence processes for the iteration are shown in Figure 5.  Convergence processes for the iteration of the proposed MEM based method for a various noises and correlation between ground cover materials of the Mixels for simulation study In accordance with the number of categories which is changed from the poorly correlated ground cover materials from two to 10, RMS error the proposed MEM based method is increased as shown in Figure 6.Correlation coefficients of 10 of combinations of ground cover materials are as follows, 0.990 0.893 0.796 0.676 0.589 0.465 0.329 0.202 0.096 Fig. 6.
Relation between the number of categories (ground cover materials) and RMS error for the proposed MEM based un-mixing method.
The relation of Figure 6 shows that RMS error is increased with increasing of the number of ground materials from two to six, then it is gradually increased form the number of ground materials is six.This trend is almost same for the other cases of middle level of correlation as well as highly correlated ground cover materials.
RMS error also depends on mixing ratio.Figure 7 shows relation among RMS errors for the proposed MEM based method (a) and the conventional SVD based method (b), noise, and mixing ratio.RMS errors of the proposed un-mixing method are almost lower than those of the conventional method.RMS error decreases in accordance with increasing of mixing ratio.Also RMS error increases in accordance with increasing of noise.IV.CONCLUSION Category decomposition method for un-mixing of mixels (Mixed Pixels) which is acquired with spaceborne based visible to near infrared radiometers by means of Maximum Entropy Method (MEM) with parameter estimation based on Simulated Annealing: SA is proposed.
Through simulation studies with spectral characteristics of the ground cover targets which are derived from spectral library and actual remote sensing satellite imagery data, it is confirmed that the proposed method works well.
Through simulation studies with USGS provided spectral reflectance, it is found that RMS error depends on mixing ratio.In particular, relation among RMS errors for the proposed MEM based method and the conventional SVD based method, noise, and mixing ratio shows that the proposed method is superior to the conventional method.RMS error decreases in accordance with increasing of mixing ratio.Also RMS error increases in accordance with increasing of noise

Fig. 1 .
Fig. 1.Example of the spectral characteristics of the ground cover materials (a)Poor Correlation (R=0.0009) Fig. 3.Examples of the selected two ground cover materials of spectral characteristics which show a good a middle level and a poor correlation between both spectral characteristics Fig. 4.RMS error

Figure 5 (
a) shows noise dependency on RMS error while Figure5(b) shows correlation dependency on RMS error.Much longer time is required for convergence process for greater noise and highly correlated ground cover materials.
Fig. 5.Convergence processes for the iteration of the proposed MEM based method for a various noises and correlation between ground cover materials of the Mixels for simulation study

Fig. 7 .
Fig. 7.Relation among RMS errors for the proposed MEM based method (a) and the conventional SVD based method (b), noise, and mixing ratio.