A New Corner Detection Operator for Multi-Spectral Images

Corner detection is a crucial image processing technique that has a wide range of application, including motion detection, image registration, video tracking, and object recognition. Most proposed approaches for corner detection are based on gray-scale images, despite it has been shown that color information can greatly improve the quality of corners detection. This paper aims to introduce a new operator that identifies the secondorder image information for multi-spectral images. The operator is developed using the multi-spectral gradient and differential structures of the image. Consequently, the eigenvectors of the proposed operator are used for detecting corners. A comparative study is conducted using synthetic and real images, and the result confirms that the proposed approach performs better compared with two other approaches for detecting corners. Keywords—Corner detection; multi-spectral; operator


I. INTRODUCTION
Corner points are considered as important structural elements for extracting features of local images. The word corner is commonly referred to as a point of interest in the image with abruptly changing intensity and/or contours in all directions at the same time. Detection of such points are popular in a wide range of applications, such as motion tracking, images matching, robot navigation, object detection and recognition and image registration [1], [2], [3], [4], [7], [9], [8]. Although the corners can clearly be recognized by the human vision system, the automated detection of the exact corner location is a non-trivial task. A good corner detector must fulfill a number of eligible criteria, i.e., discern between real and false corners, reliably identify corner positions, be robust in terms of noise and efficiency.
Numerous forms of corner detection have been published in literature during the past few decades. [7], [10], [6], [5], [11], [12]. Majority of these methods can be classified into two categories depending on whether the method is contour-or intensity-based. In what follows, a review of some corner approaches from both categories. Dating back to 1977, Moravec [13] proposed one of the early corner detection algorithms, in which the corner point is described as a point of low self-similarity. The Moravec's idea was developed to propose the Harris' algorithm by using the first order derivatives to approximate the second derivatives [14]. Later, the operator used in Harris' algorithm was extened to space-time [15]. Mikolajczyk and Schmid [16] proposed a corner detector based on the Harris corner detector and the Gaussian scale space representation. The Harris's algorithm was developped for three-dimensional multi-spectral images based on correlation [17]. A multi-scale point detector based on the Gabor Wavelet principle is presented [18]. Recently, an adaptive corner detection method based on deep learning is proposed [19].
Most of these approaches are built on the assumption that the corners correspond to abrupt changes in contour directions, and are based on this observation by examining the first and second derivative of the image to locate the corners. In case of mono-spectral images (gray level), the first derivative is roughly computed by the gradient vector and the second derivative by the Hessian matrix. In case of multi-spectral images, the images are considered as dimension two differential manifold. Thus, multispectral contours are identified by the eigenvalues and eigenvectors of the metric tensor estimated by the product of the transposed Jacobian matrix with itself [20]. For the extraction of multi-spectral information of second order, the basic approach is first to separately calculate the Hessian matrix of each band, then realize a direct Hessian matrix sum to produce the final second order differential matrix [21]. However, by performing a direct sum, the terms of the Hessian matrix may be opposite signs, so that the amount may lead to the cancelation of the second derivatives. To solve the problem, a quaternion-based method formed by the Hessian matrix of each band was proposed [22]. However, this method has proved time consuming, because the calculation of the eigenvalues requires the singular values of the quaternion decomposition.
To the best of our knowledge, an operator that detects a multi-spectral image's second-order information has not yet been proposed. From the excesses of the multi-spectral gradient and the differential structure of the image, a multispectral operator to identify the second-order information of color images is developed initially, then, the eigenvalues of this operator is used for detecting corners.

II. NOTATION AND PRELIMINARIES
Given an m−bands image defined by I : R 2 → R m , that maps a point (x, y) in the image plane to an vector I(x, y) = (I 1 (x, y), · · · , I m (x, y)) in R m . The variations of an image are evaluated by the change in the image values in an infinitesimal displacement. This could be represented by the differential The squared norm of dI, which indicates how much the image value varies in any direction, is given as Using tensor notation, Di Zenzo [20] introduced the first definition of the gradient of a multi-spectral image. Let J be the jacobian matrix of I. Then, the metric tensor is approximated by the matrix Where At each point on the image, there are two main quantities to be know; the direction along which the function I has the maximum rate of change, and the absolute value of this maximum rate of change. The variations of multispectral image I are extreme in the directions of the eigenvectors of the matrix tensor J T J. If the maximum and minimum contrasts (corresponding to the largest and smallest eigenvalues of J T J) are represented by λ max and λ min , respectively, then, such extreme values can be calculated as Let N be the associated eigenvector to λ max . This vector coresponds to the direction of the maximum variation and it is given by If the multi-spectral image as a surface in the space denoted by The image is, thereby, a two-dimensional surface embedded in R m . At each point p(x, y) ∈ I I , let T p I denotes the tangent plane of the surface I I generated by the two vectors U = ∂I ∂x et V = ∂I ∂y . Therefore, the vector N may be represented in the basis ( U , V ) in the form: N (x, y) = cos(θ) U + sin(θ) V .
as shown in figure 1.

III. DESCRIPTION OF THE NEW MULTI-SPECTRAL OPERATOR
In the case of multi-spectral image, the components I 1 = R, I 2 = G and I 3 = B are identified. As noted above, the vector N points to the direction of the maximum change of the multi-spectral contour and orientation angle depends of the coordinate (x, y). Given that the corners are localized on the points that correspond to an abrupt change in the orientation of the contour, at first step, the Jacobian matrix J N of N with variables x and y is calculated. Then the eigenvalues of the matrix J T N J N are incorporated in a decision rule to locate the corners. Taking into account the fact that the two vectors I x and I y which generate the tangent plane T p I change from one point to another as shown in Figure 1. Applying the chain rule to (2), the Jacobian matrix J N of the vector N at point (x, y) can be formulated as 2 × 3 matrix in term of the two vectors J 1 and J 2 as follows where and The indices correspond to the first and second derivatives with repect to x and y. θ x and θ y are the partial derivatives of the angle θ given by Here the second-order derivatives of R •• , G •• and B •• are computed by convolution of the color channels R, G and B with a second order Gaussian derivative mask. The powerful Gaussian property guarantees the existence and continuity of the second derivatives of I.
In the Jacobian matrix (3), the presence of terms of second derivatives of the three bands of the image, not as a direct sum, but instead as combination of rotation angles and their derivatives, as well as the terms of the first derivative of the image. Now, to quantify the change in the vector N , consider the matrix J T n J N that can be interpreted as the matrix that approximates the metric tensor of the space formed by the vectors N . As in the case of the eigenvalues of matrix (1) that determine the multi-spectral contour, the eigenvalues λ max and λ min of the matrix J T N J N quantify variations on this contour, which allows to identify the corners which are characterized by an abrupt change of the orientation of the contour. As rule decision to detect the corners, the following function [14] is used: Where κ is the sensitivity setting. The smaller the value of κ, the more likely it is to detect the corners with acute angles.

IV. EXPERIMENTAL AND DISCUSSIONS
In this section, the results of the proposed corner detector on two different applications are presented, the first application is to detect corners in an image, and the second application is to track corners in a video sequence. In this section, a comparison of the proposed method of corner detection with two standard methods in literature is reported; the first one is an extension of the harris's detector [14] to color images (HC) proposed in [23], this method is based on approximation of the auto-correlation of the gradient in different directions. The second approach is based on local and global curvatures (CLG) for the detection of corners on gray-scale images [24]. A comparative study of different corner detection methods [25] show that these two approaches demonstrate good performance compared to other approaches in the literature. For the calculation of derivatives in expressions (1) (3) and (III), a Gaussian bypass filters of first and second orders are applied, the best results are obtained when the sensitivity setting (4 ) is attached to the value 0.05.

B. Qualitative Comparison
In Figure2 and figure 3, the results of three approaches on two synthetic images are reported, while in figure4 and figure 5 the comparative corner detection methods are applied on three real images. Note that the detection of corners is almost perfect in computer graphics for the proposed approach and the comparative detection methods based on the color (HC), however, the method (CLG) could not properly detect many corners in the image, especially, those of the white hexagon.
The difference between the three methods is more visible when applied to real image in Figure 4. The method (CLG) furthermore, while detecting a majority of corners, it also detects many false corners including the texture of plants and grass, as for the approach (HC), it could not detect a few important corners in the image, especially those formed by shadows and points which three or more contour regions meet. The approach proposed in this paper has led to good results of detection on real images, because most of the corners are located all in minimizing false detection.
Illumination has a strong influence on images and consequently on corners detection. Hence, images of the same scene under different illuminations can be very dissimilar, which greatly affects the detection of interest points like corners. In figure 6.a one more illumination source was added in the image on the right. There were 116 corners detected in the original image and only 118 detected in the right image, while in Figure 6.b the orientation of the illumination was changed, a difference between the four corners between the original and www.ijacsa.thesai.org modified images is noticed. Adding illumination invariance seems to have a relatively small effect in this example shown in Fig 7,8.

C. Quantitative Comparison
On noisy images, the corner detection methods are applied on a noisy artificial image and on a noisy real laboratory image. The artificial and real images illustrated in figure 9 and figure 9 are generated by adding a Gaussian noise with standard deviation σ = 20. As illustrated in 9 and figure 9, it can be observed that the proposed corner detection approach detect all corners from the noisy image. The Table 1 presents a quantitative comparison of the corner detecting methods. It can be clearly seen from the mean localization error values that the proposed approach performs well in corner localization accuracy.

D. Location and Tracking of Points of Interest
Points of interest such as the corners are commonly used for tracking objects [26]. From the perspective of an automatic analysis of facial expressions, the proposed corner detector is applied in this paragraph to track the points of interest in the face in a video sequence. An automatic analysis of facial expressions system generally consists of three main phases; face detection, components extraction and finally the classification of facial expression. Extraction of facial components passes mainly through the localization and eye-tracking in the sequence video [27]. As a first step, the method of locating eyes in the face using the method proposed by Viola and Jones [28] is applied, after this initial stage, the proposed operator is used to generate corners, these feature points are used for eye tracking in the video by following the approach proposed by Shi and Tomasi [26]. Figure 10 shows an example application of the proposed operator for eye tracking. Initially in Figure 10(a), an initial location is performed for eyes detection method based on a cascade of classifiers boosted [28], then a tracking feature points is performed to estimate the position of his points in the following images, as illustrated in figures 10(a) -10(f ).
In a third application reported in Figure 11, the point cloud generated to track each eye is filtered to keep the two most important corners in both ends of the eye, these two points are essential to the realization of an automatic analysis system of facial expressions [27]).

V. CONCLUSION
In this paper, a new operator for corner detection is introduced. Initially a multi-spectral operator was developed to identify the second-order information of color images, based on its multi-spectral gradient and differential structures. As a by-product, the eigenvectors of this operator are used to detect corners. For testing the proposed approach, two methods that are considered among the most efficient methods were chosen. Experiments on synthetic and real images show a better detection of the corners by the proposed method. These preliminary results are very promising and encouraging.