Context Classification based on Mixing Ratio Estimation by Means of Inversion Theory

A contextual image classification method with a proportion estimation of the pixels composed of several classes, Mixed pixels (Mixels), is proposed. The method allows us to check the connectivity of separated road segments, which are observed frequently as discontinuity of roads in satellite remote sensing imagery. Under the assumption of almost same proportions for the Mixels in the discontinuous portion of road segments, a proportion estimation method utilizing Inverse Problem Solving is proposed. The experimental results with the simulation data including observation noise show 73.5~98.8(%) of improvements in terms of proportion estimation accuracy (Root Mean Square: RMS error), compared to the results from the previously proposed method with generalized inverse matrix. Also, usefulness of contextual classification based on the proposed proportion estimation was confirmed for the investigation of connectivity of roads in remotely sensed images from space. Keywords—Search engine; fuzzy expression; knowledge base system; membership function; mixed pixel: Mixel; context information; inverse problem solving


I. INTRODUCTION
Pixels constituting a satellite image are rarely composed of one ground covering class (Pure Pixe1) and are generally composed of a plurality of classes. These are called Mixel (Mixed Pixel), and the heterogeneous (Heterogeneous) area at the boundary between homogeneous (Homogeneous) areas is constituted by Mixel [1]. These are factors that reduce the image classification accuracy. It is not so difficult to classify Pure Pixel with high precision, and how to classify Mixel with high accuracy is a problem in image classification [2].
The author examined the context classification of Mixel in the heterogeneous area here. The spectral reflection characteristic of Mixel is considered as a mixture of the spectral reflection characteristics of multiple classes of Pure Pixel that constitute it and is given as a linear combination function weighted by the mixing ratio of each class. For this reason, in Mixel, if the spectral reflection characteristics of the class of Pure Pixel and the mixture ratio of Mixel that are constructed in any way from the image are known, the pixel is equivalent to being classified.
Mixel class mixing ratio estimation method based on the concept of feature mixing, using inverse matrix, generalized inverse matrix, quadratic programming [2]- [4], least squares method [5], surrounding 8 pixels. Considering the state of, the one obtained by linear regression has been proposed [6]. In addition, when calculating the spectral reflection characteristics of Pure Pixe1 from the image, the conventional method considers the residual of the observation vector and its estimated value in consideration of the scattering reflection characteristics of the observation target, fluctuation of observation conditions, and dispersion due to measurement error. While the solution was obtained under the condition of minimizing, a method of improving the estimation accuracy by applying the least square method to the class mixture ratio has been proposed [7].
The author is studying the expertly of image classification, focusing on road discontinuities that are scattered on satellite images [8], and trying to develop a knowledge base to check the connect ability of segments of discontinuous roads As the state of the discontinuous part and its surrounding pixels used in this knowledge base [9], the class mixture ratio that constitutes those pixels is taken up, the above method is improved and estimated [10], and it is used as contextual information. By using it, a method of checking road continuity is proposed.
In the following section, related research works and research background including motivation of the research are described. Then, the proposed context classification method is described followed by experimental method together with experimental results. After that, concluding remarks and some discussions are described.
II. RELATED RESEARCH WORKS Classification by re-estimating statistical parameters based on auto-regressive model is proposed for purification of training samples [11]. Meanwhile, multi-temporal texture analysis in TM classification is proposed for high spatial resolution of optical sensor images [12]. On the other hand, Maximum Likelihood (MLH) TM classification considering pixel-to-pixel correlation is proposed [13].
Supervised TM classification with a purification of training samples is proposed [14] together with TM classification using local spectral variability is proposed [15]. A classification method with spatial spectral variability is also proposed [16] together with TM classification using local spectral variability [17].
Application of inversion theory for image analysis and classification is proposed [18]. Meanwhile, polarimetric SAR image classification with maximum curvature of the trajectory in eigen space domain on the polarization signature is 44 | P a g e www.ijacsa.thesai.org proposed [19]. On the other hand, a hybrid supervised classification method for multi-dimensional images using color and textural features is proposed [20].
Polarimetric SAR image classification with high frequency component derived from wavelet multi resolution analysis: MRA is proposed [21]. Comparative study of polarimetric SAR classification methods including proposed method with maximum curvature of trajectory of backscattering cross section in ellipticity and orientation angle space is conducted and well reported [22].
Comparative study on discrimination methods for identifying dangerous red tide species based on wavelet utilized classification methods is conducted [23]. On the other hand, multi spectral image classification method with selection of independent spectral features through correlation analysis is proposed [24]. Image retrieval and classification method based on Euclidian distance between normalized features including wavelet descriptor is proposed [25].
On the other hand, wavelet Multi-Resolution Analysis: MRA and its application to polarimetric SAR classification is proposed [26]. Meanwhile, object classification using a deep convolutional neural network and its application to myoelectric hand control is proposed and evaluated its performances [27].
Image classification considering probability density function based on simplified beta distribution is conducted [28]. Also, Maximum Likelihood; MLH classification based on classified result of boundary mixed pixels for high spatial resolution of satellite images is proposed [29]. Fig. 1(a) shows the topographic map of the intensive study area and Fig. 1(b) is a part of the image of the area including Saga city and Ushizu town acquired by LANDSAT-5 Thematic Mapper: TM in May 1986. This area is almost composed of paddy fields, as evident from the topographic map shown in the figure, and includes urban areas, residential areas, roads, railways, and rivers. Focusing on roads, there are road discontinuities determined by the relationship between the road width and the instantaneous field of view (IFOV) of the TM, the scanning direction and the angle between the roads, and the like.

A. Motivation of the Research
The pixel of the discontinuous portion is located between the paddy fields in the periphery thereof in the spectral space and is rather close to the paddy field. Therefore, the pixel is classified into the paddy field using only the spectral information. The result of road extraction using only this spectrum information has discontinuous parts as shown in Fig. 2.
In order to examine whether these discontinuous pixels can be judged as roads, first, for these road segments extracted from the image, line likeness and directionality as texture information, which is one of spatial information, are used.
As a result, it was confirmed that it was a line element and the directionality was also consistent. However, in this study, which aims at automatic classification by a computer using knowledge base, it is not reliable to judge spectrally different pixels as the same road only by matching spatial information. For this reason, the class mixture ratio of pixels in the discontinuous part of the road and the class mixture ratio of surrounding pixels in the road segment are estimated as contextual information, and these are comprehensively determined to determine whether they are on the same road.  45 | P a g e www.ijacsa.thesai.org Specifically, Hough transform is applied to the image, road segments are re-extracted, and road segments at discontinuous portions are extracted, and pixels that appear to be roads are extracted by expanding the area with 3x3 windows around the extracted pixels. Then, their class mixture ratio was estimated.
As an example of this, Fig. 3 shows the result of adapting a part of the above image.

B. Class Mixing within Pixels using Conventional Generalized Inverse Matrix Estimation Method of Ratio
Let be I is the M-dimensional observation vector of the pixel composed of N classes, B is the mixture ratio vector of those classes, and the spectral reflectance characteristics of the ground object corresponding to those classes, i.e., Pure Pixel in spectral space Assuming that the spectral vector of is A, it can be represented by the following linear combination function.

I=AB
(1) This I is known because it is an observation vector. For A, it is known if a basic class is set in advance and their spectral reflectance characteristics are measured. However, the spectral reflection characteristics of the object measured on the ground are different from those of the same object appearing in the satellite image. In other words, the measured values are far from ground-based measured values due to the influence of the atmosphere and the ground surface.  In order to avoid this, the distribution of the basic class in the spectral space was obtained from the satellite image, and this average vector was determined as a representative of the spectral reflection characteristics of each class. Here, Eq. (6) is obtained by solving Eq. (1) under the condition that minimizes the norm between the observed value and the estimated value in Eq. (5), and the mixture of the set basic classes in Mixel is obtained.

A. Estimation of Mixture Ratio of Pixel almost equal to Mixture Ratio of Preceding Pixel
In the situation currently set, it is considered that the class mixture ratio between adjacent pixels of continuous boundary pixels may be substantially the same. For example, an asphalt road crossing a large paddy field as shown in Fig. 4 is a typical example. It is considered that the class mixture ratio of such continuous boundary pixels is often almost the same between adjacent pixels. Therefore, for the pixel of interest, a constraint that minimizes the norm from the class mixture ratio of the boundary pixel adjacent to the pixel of interest is further added to Equation (5) to improve the estimation accuracy of the class mixture ratio of the pixel of interest. We propose a method to achieve. Here, considering not only the average value but also the variance, it results in a nonlinear optimization problem, but the solution in this case has already been proposed, so please refer to it. 46 | P a g e www.ijacsa.thesai.org In addition, as pointed out in [7], the representativeness of the spectral reflectance characteristics of this basic class should be examined in consideration of physical observation conditions. However, in estimating the class mixture ratio proposed here, even if the mixture ratio is estimated assuming that a "road-like class" in which the pixel occupancy of the road is dominant is set from the beginning, It was judged that it was sufficient to judge the continuity of the road by estimating the mixture ratio of the discontinuous part of the existing road and the pixels around them, and it was decided not to depend on the representativeness.
If a vector representing the class mixture ratio of the adjacent boundary pixels is X, an expression representing the norm of the target pixel at the mixture ratio B can be written as follows.
From Eq. (11), From the above, an expression representing the norm between the vector representing the class mixture ratio of the adjacent boundary pixels and the mixture ratio of the pixel of interest can be written as follows.
Eq. (1) is solved using this and the constraint condition (Eq. (14)) that minimizes the square error to obtain Eq. (16).

B. Determining the Strength of Constraints
In Eq. (16), if r is set to zero, it is equivalent to the method using the generalized inverse matrix, and conversely, if this is increased, the class mixture ratio vector of the pixel of interest approaches the class mixture ratio vector of the pixel preceding it without limit. Here, as a method for optimizing r, a method is considered in which the distance in the spectral space between adjacent boundary pixels is used as an index, and r is reduced if the distance is long, and is increased if the distance is short.
In other words, it is assumed that the pixel of interest for which the intra-pixel class mixture ratio is to be estimated is almost the same as the class mixture ratio estimated before it, so if some degree of similarity between pixels is known, it is used as an index. Then, the constraint condition can be changed. Although various similarities can be considered, the Euclidean distance in the spectral space is taken as an example here.
An example is shown below in Fig. 5 showing the relationship between the Euclidean distance in the spectral space between adjacent boundary pixels and the optimal r that minimizes the Root Mean Square: RMS error.
This figure plots the estimation accuracy of the class mixture ratio when the Euclidean distance between adjacent pixels is changed by 5 and the constraint condition factor r is changed by 2,000, and the thick solid line is the locus of the minimum value. In this example, the number of classes is set to 3, and their mixture ratio is set to 0.2, 0.0, and 0.8, respectively. In addition, the statistical noise between adjacent pixels is changed from the position in the spectral space to the center of gravity of the closed surface with the average value vector of each class as the vertex, and observation noise is reduced so that S / N becomes 10.
As is clear from Fig. 5, the estimation accuracy (Root Mean Square: RMS error) of class mixture ratio depends on r and Euclidean distance. In an extreme case, if the Euclidean distance between adjacent pixels is 0, a large constraint condition is applied, and the estimation error can be made as close to 0 as possible. Conversely, when the Euclidean distance is large, the estimation accuracy is better if no constraint condition is applied. That is, it becomes equivalent to the mixture ratio estimation itself by the generalized inverse matrix.

C. Class Settings and their Number
In general, in estimating the class mixture ratio, the larger the number of classes, the lower the estimation accuracy of the mixture ratio. Therefore, we considered limiting the number of classes as much as possible. Here, the pixels at the discontinuous portion of the road of interest are boundary pixels that are spectrally heterogeneous and exist at the boundaries of different class regions. The image has spatial information based on the macroscopic structure (for example, a network of roads, railways, rivers, etc.), which can be extracted by an edge operator or the like. Furthermore, it is confirmed that the boundary pixels are spectrally boundary by the homogeneity test [1] and the like.
There are several types of pixels of the set class around the boundary pixel (at most, the number of pixels around eight pixels around the boundary class). When estimating the class mixture ratio based on Eq. (16), the smaller the number of set classes, that is, the smaller the number of unknowns, the better the estimation accuracy, but only for the boundary pixels such as roads. Then the number of classes will not be so large. When determining the number of classes to be set, the frequency distribution of the number of classes of eight neighboring pixels around the boundary pixel may be checked and determined in advance.

A. Simulation Data
Set the number of classes to 3, and Mixel was simulated as a parameter. The generated random number is signaled so that the S / N ratio becomes 10. Table I shows a comparison among the proportion estimation accuracies derived from Generalized Inverse Matrix Method (GIMM), the Least Square Method with a Prior Proportion (LESP) in case of which the Euclidian distance between the pixel of interest and the previously estimated boundary pixel is 3.
Since the proposed method is based on the generalized inverse matrix method and adds constraints, the simulation results are used to estimate the generalized inverse matrix and the estimation accuracy and RMS error of the method proposed in this paper. As a result, as shown in Table I, the proposed method showed that the RMS error in estimating the mixture ratio was 73.5-98.8% better than that using the generalized inverse matrix.

B. Actual Data
The data of the area including Saga city and Ushizu town acquired in May 1986 by LANDSAT-5 / TM was used. The area is composed of paddy fields, including urban, residential, road, railway and river networks. This is called Saga data here.
Focusing on the area surrounded by the solid line in Fig. 2, three pixels are selected from each segment of the paddy field located away from the road, the paddy field on the side of the road, the road, and the unconnected part, and their mixture ratio was estimated. Fig. 6 shows the selected pixel position and Table II shows the estimation results.
The results showed that the discontinuous pixels had a large vegetation ratio. When we went to the corresponding place and investigated the situation, we confirmed that the branches and leaves of the trees on the road covered the road.
The matrix A representing the spectral reflectance characteristics of each class in Eq. (15) was obtained by extracting pixels considered to be representative of the class from the image and averaging them. However, since the extracted pixels themselves are often Mixel, an estimated from these is poor in the representativeness of each class of paddy field, road, and vegetation.
In particular, the pixels corresponding to the road class are greatly affected by the ground covering having spectral reflectance characteristics other than roads, and do not represent the true road spectral reflectance characteristics. However, even if the mixture ratio is estimated assuming that a "road-like class" in which the pixel occupancy is dominant is set from the beginning, the discontinuous portion of the road under consideration and their mixture ratio estimation Is considered to be sufficient for the determination of road continuity according to.

VI. CONCLUSION
The method of estimating the class mixture ratio in adjacent boundary pixels proposed in this paper is more effective than the conventional method using the generalized inverse matrix when the class mixture ratio of successive boundary pixels is almost equal. The estimation accuracy is excellent. That is, if the Euclidean distance between consecutive boundary pixels is 3 and the S / N due to observation noise is 10, the RMS value of the estimation error of the proposed method is reduced to about 1/4 to 1/81 of that of the generalized inverse matrix.
It was confirmed that it could be reduced. This method is effective especially when the Euclidean distance between consecutive boundary pixels is short, such as a road running in a large paddy field or in an urban area, that is, when the class mixture ratio of the boundary pixels is about a bit. If this precondition does not hold and the distance between consecutive boundary pixels is long, it is equivalent to a generalized inverse matrix.
This estimation method can be applied not only to one of the knowledge bases using context information such as "trees wear on the road" as an example, but also to more knowledge bases.

VII. FUTURE RESEARCH WORKS
The proposed method is adopted in the real earth observation satellite imagery data, and it is a future subject to realize a more usable context-based classification method.