Image Prediction Method with Nonlinear Control Lines Derived from Kriging Method with Extracted Feature Points Based on Morphing

Method for image prediction with nonlinear control lines which are derived from extracted feature points from the previously acquired imagery data based on Kriging method and morphing method is proposed. Through comparisons between the proposed method and the conventional linear interpolation and widely used Cubic Spline interpolation methods, it is found that the proposed method is superior to the conventional methods in terms of prediction accuracy.


INTRODUCTION
Time series of images which are acquired with unequal interval can be converted to en equal interval of the time series of images.The proposed conversion method is based on morphing [1] method 1 utilizing feature points extracted from the time series of images.Control lines required for morphing method are defined by the extracted features.Then the time series of images at the designated time is interpolated with the adjacent images.In this process, nonlinear control lines are created in accordance with Kriging [2]- [15] method 2 .Through experiments with satellite imagery data, it is confirmed a validity of the proposed method in comparison to the conventional linear interpolations and widely used Cubic Spline 3 interpolation methods.
The following section describes the proposed method followed by some experiments.Then conclusion is described with some discussions.

A. Process Flow of the Proposed Method
The proposed method is for conversion from unequally interval of time series images to equally interval of time series images.Therefore, some interpolations are necessary.There are many interpolation methods such as Cubic Spline http://ja.wikipedia.org/wiki/%E3%82%B9%E3%83%97%E3%83%A9%E3%82%A4%E3%83%B3%E6%9B%B2%E7%B7%9A interpolation (Appendix), Hermitian 4 interpolation, Bezier 5 interpolation, Lagrange 6 interpolation and so on.The proposed method is based on morphing method utilizing feature points extracted from the time series of images.Control lines required for morphing method are defined by the extracted features.Then the time series of images at the designated time is interpolated with the adjacent images.In this process, nonlinear control lines are created in accordance with Kriging method.

B. Morphing Method
Morphing method is "metamorphosing" 7 which allows interpolation from the objective shape to the other objective shapes smoothly.The basic idea of the morphing method is the following, shapes in an image represented on a rubber sheet is modified with pin movements elastically as shown in Figure 1. Figure 1 shows an example of the basic idea of morphing method.In the figure, the point "p" in the previous image moves to point "p'" with the weighted mean of the following two points, "p 1 '" and "p 2 '" where "p 1 '" is orthogonally projected point of "p" against the line of A 1 '~B 1 ' while "p 2 '" is orthogonally projected point of "p" against the line of A 2 '~B 2 '.

C. Kriging Method
Kriging method allows estimation of future point locations with known point locations probabilistically.If only one known point location used, then future point location is exactly same location.If we know that a future point is situated at the center of two known points, the future points is situated at the point of median.Thus the future point, is estimated with equation ( 1) by using the known points, . ( can be determined with the condition of equation ( 2) where is defined as the distance between the future point,0 and the known points, j while is defined as the distance in between the known points, i and j.
Meanwhile, estimation error covariance of semi-variogram, is defined in equation ( 4).(4)   where a and b are coefficients while L denotes distances.Therefore, if the coefficients are determined with the known points, then the distance is known.Thus can be determined results in estimation of future points.

A. Imagery Data Used
Visible/Infrared Spin Scan-Radiometer: VISSR onboard Geostationary Meteorological Satellite: GMS acquired time series of imagery data are used for experiments.Figure 2 shows four time series of VISSR images of South East China Sea areas which were acquired in 1994.These are called image "A", "B", "C", and "D", respectively.
Although VISSR images are taken with one hour interval, interpolations are required for cloud tracking.For time being, clouds appear and disappear as well as move from one to the others as shown in Figure 2.
Also one hour interval would not be enough for cloud tracking, in particular, for the clouds which grow quite rapidly.Therefore, interpolations are necessary.

B. Experimental Procedure
Assuming some of the time series of VISSR/GMS image is missing, interpolations are attempted for the supposed missing image by using the rest of given time series of images.Tie points are extracted from the two images which are acquired at the two closest times to the time at which the image is missing in accordance with the correlation between two images.By using the location of the extracted tie points, the location of the corresponding points on the missing image are estimated with the proposed and the other conventional interpolation methods, linear interpolation and Cubic Spline interpolation.Using the corresponding points, morphing method is applied to the two images which are acquired one previous and one after time to the missing image.Then the missing image is created as a result of the morphing.
Interpolation accuracy is evaluated with the difference between the interpolated and the original images which is supposed to be missing because the original image is known.In the evaluation process, two lines are estimated by using two corresponding points in between two previously acquired and just one after the image of which the image is missing.Evaluations of the difference are done for two cases of which the second and the third VISSR/GMS image are missing.

C. Experimental Results
Figure 3 (a) shows interpolated and predicted the second and the third images based on the conventional linear interpolation method while Figure 3 (b) shows those for the www.ijarai.thesai.orgCubic Spline interpolation method, respectively.Meanwhile, Figure 3 (c) shows interpolated and predicted the second and .thethird images based on the proposed method.Root Mean Square Error: RMSE between the original and the interpolated and predicted images is evaluated.RMSE of the prediction of the second image "2" is 67.0, while that of the third image "3" is 53.5 for the linear interpolation method.Meanwhile, RMSE of the image "2" and "3" is 55.5, and 52.4,respectively for Cubic Spline interpolation method.On the other hands, RMSE of the image "2" and "3" is 54.1, and 51.9, respectively for the proposed Kriging utilized method.As the results, the proposed Kriging utilized method is superior to the conventional linear interpolation and Cubic Spline interpolation methods.1.There are four tie points, "A", "B", "C", and "D" on each image.Using the tie points on images "1" and "3", tie points on image "2" is estimated while tie points on image "3" are estimated by using the tie points on images "2" and "4", respectively.The estimated tie point locations are shown in Table 2.   Then RMS error between the actual image and the interpolated and predicted image by the proposed Kriging utilized method and the other interpolation method can be evaluated.Table 3 shows the results from the evaluation for the image size of 100 by 100 pixels, the original image size.It shows that the proposed Kriging method utilized interpolation method is superior to the Linear and Cubic Spline interpolation methods.In more detailed RMS error is evaluated for 5 by 5 pixels of a small portion of images of which the tie points are centered of the portion of images for the cases to interpolate, predict images, "2" and "3".www.ijarai.thesai.orgTable 4 shows the result.Through comparisons between the proposed Kiriging utilized method is superior to the other conventional methods for all the cases.Meanwhile, RMS error between the original and interpolated images by using the actual tie point locations is evaluated for each 5 by 5 pixels portion images whose center is the tie point.Table 5 shows the results from the evaluation.Through the comparison between Table 4 and 5, it is found that the estimated tie points are very close to the actual points because RMS errors between Table 4 (Kriging) and 5 are almost same.

IV. CONCLUSION
Method for image prediction with nonlinear control lines which are derived from extracted feature points from the previously acquired imagery data based on Kriging method and morphing method is proposed.Through comparisons between the proposed method and the conventional linear interpolation and widely used Cubic Spline interpolation methods, it is found that the proposed method is superior to the conventional methods in terms of prediction accuracy.

Figure 1
Figure 1 Basic idea of the morphing method

Figure 2
Figure 2 Time series of VISSR/GMS images used

Figure 3
Figure 3 Interpolated and predicted images 52) D(48,49) Linear interpolation D(44,76) D(48,53) Cubic Spline D(44,62) D(48,52) Kriging D(44,48) D(48,49) APPEXDIN: CUBIC SPLINE INTERPOLATION n-th Spline function is defined as a n-th order polynomial function for a finite area and is a continuous function for all the defined areas which ensure continuously differentiable by n times.The third order spline function is defined as follows, Assuming the n points, the third order of spline function, for the region, can be expressed as follows, From the condition of interpolation, the following equations have to be existed, The first and the second derivatives have to be continuous.Therefore, These are rewrite as follows, If the following equation is assumed, then Thus So that, In this equation, there aren-1ofunknownparameters, www.ijarai.thesai.orgAlso the number of the given equations is n-3.Therefore, other two conditional equations are required.is essentially natural spline function.Therefore, Thus, And Thus all the required coefficients for spline function are obtained.

TABLE I .
ACTUAL TIE POINT LOCATIONS: (X,Y) DENOTES X AND Y

TABLE II .
ESTIMATED TIE POINT LOCATIONS

TABLE III .
RMS ERROR OF THE ESTIMATED TIE POINT LOCATIONS

TABLE IV .
RMS ERROR OF THE ESTIMATED TIE POINT LOCATIONS

TABLE V .
RMS ERROR OF THE ESTIMATED TIE POINT LOCATIONS