Eye Detection Based-on Color and Shape Features

This paper presents an eye detection technique based- on color and shape features. The approach consists of three steps: a rough eye localization using projection technique, a white color thresholding to extract white sclera, and an ellipse fitting to fit the ellipse shape of eye. The proposed white color thresholding utilizes the normalized RGB chromaticty diagram, where white color objects are bounded by a small circle on the diagram. Experimental result shows that the proposed technique achieves a high eye detection rate of 91%.


INTRODUCTION
Nowadays researches on facial features detection and recognition attract attention for researchers, due to their significant contribution in the Human Computer Interface (HCI) applications.One of the interesting topics is an eye detection technique which is employed for activating mobile phone [1]; phoning, reading and browsing [2]; monitoring driver fatigue [3]; and improving human-computer dialogue [4].
Basically, eye detection technique is divided into three methods [5]: a) Shape-based, b) Appearance-based; c) Hybrid shape and appearance-based.In shape-based method, the eye shape such as eye edges, pupil, and eye corners are used as the features.In appearance-based method, the photometric appearance of eye is used for detection.Hybrid method combines both shape and appearance methods to exploit their advantages.
In [6], the eye is detected by extracting the pixels darker than surrounding and grouping them according to their geometric centers.The best eyeball is selected by applying a set of geometric constraints.In [7], an ellipse fitting method is employed to detect the potential eye region.The antropological characteristics of human eye, such as the width and height ratio of the eye and the orientation of the major axis are used to verify the detected eye.The horizontal projection is employed in [8] to find the eyebrows.Further the region below is extended to locate the pupil.The generalized projection function is proposed in [9].It combines the integral projection, variance projection, and hybrid projection functions.In the method, pupil is located by applying the vertical and horizontal projections.
In [10], a cascaded classsifier trained by AdaBoost algorithm is employed to detect multi-view face and eye.To detect eye, they proposed four-step detection, i.e. eye's candicate area restriction, eye's candidates detection using an eye classifier, eye pair candidates using an eye pair classifier, and an eye pair decision based on features extracted from previous steps.Instead of rectangle feature which is used in [10], a pixel-pattern-based texture feature is used in [11].They employed Adaboost and Support Vector Machine classifier to classify the eye and non-eye image patches.In [12], wavelet and Neural Network is used to classify an eye or non-eye region.
In [13] and [14], shape and appearance are combined to generate an eye model which is invariant to scale and rotation changes.In [13], a color model is employed for coarse-scale eye tracking and grayscale appearance is used for precise localization.
Previous works as decribed above mostly utilize intensity image for detecting eye.Color is usually employed for skin color modeling [13].In eye detection, the Infrared light (IR) is the dominant use of color [15], [16].This method relies on the fact that when IR light falls on the eye, it produces the bright pupil effect.However, it has a drawback that a special geometric arrangement of IR light emitting diodes (LEDs) should be prepared for producing the desired bright pupil effect.
In this paper, a color thresholding technique is proposed to detect eye.It takes an advantage of the white color of eye's sclera [17].Further the shape feature, i.e. ellipse shape, is utilized to find the precise location of eye.The method assumes that a face region has been obtained by a face detection technique.The proposed eye detection consists of three steps: a) Once a face is detected, the rough location of eye is localized by horizontal projection of the gradient image; b) White color thresholding is employed to extract eye's sclera; and c) An ellipse detection technique is applied to detect the precise eye location.
The rest of paper is organized as follows.Proposed technique is described in Section 2. Section 3 discusses the experimental results.Finally conclusion is covered in Section 4.

A. Overview
Fig. 1 illustrates the overview of proposed system.Prior to eye detection, the face detection is applied to locate a face.It employs the popular Viola-Jones face detector [18].Since the Viola-Jones detector requires a grayscale image as the input, it needs to convert the RGB color image to the grayscale image www.ijacsa.thesai.orgas shown in the figure.After face is detected, the eye detection algorithm starts by localizing the rough eye location using horizontal projection technique of the gradient image.By analyzing peaks of the horizontal projection, the rough location of eye is obtained.Then the original color image is cropped according to the rough eye location.In the cropped image, white color of eye's sclera appears as the significant clue for detecting eye.Thus white color thresholding is applied to detect the sclera.It creates a blob image (binary image) contains the white and non-white objects.Since the shape of eye (sclera) is ellipse then an ellipse detector is applied to detect the boundary of eye's sclera.

B. Rough Eye Localization
It is observed from the face image that vertical gradient of the image is dominated with horizontal edges, due to the appearance of the facial's components such as eyebrow, eyelid, and lip as shown in Fig. 2(b).Therefore, if the gradient image shown in the figure is projected onto vertical axis (horizontal projection), then there will exist peaks on the projection represent the location of eyebrow, eyelid, and lip.Since the aim here is to localize eye, thus the projection is applied to the upper-half face only.Fig. 3 shows the gradient images with their horizontal projections of the upper-half part.Fig. 3(b) shows that maximum peak of the projection denotes the location of the eyelid.However this situation does not always hold.In Fig. 3(d), maximum peak of the projection denotes the location of eyebrow.Fortunately, from the experiments, the location of eyelid could be found as follows.It first finds the two maximum peaks of the projection.From these two maximum peaks, find the one which the vertical position is lower than the other one, and assign it as the vertical position of eyelid.

C. White Color Thresholding
Color thresholding is usually used for skin color segmentation [13], [19].Previous work [19] shows that the normalized RGB color space is effectively used for skin color segmentation.Skin region is defined on the normalized RGB chromaticity diagram as shown in Fig. 4. In the figure, skin region is bounded by five lines i.e.: a) line g=2, b) line g=r-0.4,c) line g=0.4,d) line g=-r+0.6,e) line-c.A small circle denoted as line-c is used to exclude white pixels.Contrary, in the proposed eye detection, the aim is to extract the white color of eye's sclera.Therefore the white color thresholding could be considered as extracting the pixels which the r and g components are inside on the small circle.The proposed method is described in the following.The binary image of extracted white pixels is denoted as Ithr and obtained by the following equations: (14) (15) ( 16) where is the maximum value of a and b Fig. 5 shows the thresholded image obtained by above equations.Using Eqs. ( 14)-( 17), the binary image (Ithr) is obtained as the shown in Fig. 5(b), where the black color represents the extracted objects, in this case white objects, while white color represents non-extracting object.From the figure, it could be seen that the method also extracts non-white objects outside the face.It is confirmed by [19] and could be observed from Fig. 4 that the above formulas will not extract skin color.Thus the skin-face region shown in Fig. 5(b) is in white color (not extracted).Since the white color thresholding is applied on rough eye region where only eye's sclera is in white color, the thresholding method will work appropriately.The normalized RGB color has drawback that r and g component of the dark and bright color may have the same value such as addressed in [19].Suppose a dark pixel with the value of R=0.1, G=0.1, and B=0.1.The chromaticity values of this pixel are r=0.33 and g=0.33.In other hand, a bright pixel with the value of R=0.9, G=0.9, and B=0.9 will have the value of r=0.33 and g =0.33.
Therefore those two pixels will be considered as the same color in the chromaticity diagram.From Fig. 5(b), it is observed that the black color of the hair is also extracted.To overcome the problem, the following rule is introduced.The pixels extracted by Eqs. ( 14)-( 17) are assigned as the white pixels if the following formula is satified.(18) Using Eq. ( 18) the black hair is not extracted as shown in Fig. 5(c).

D. Ellipse Detection
The proposed white color thresholding extracts the white color of eye, thus the extracted blob image does not make solid ellipses, but they contain holes (i.e.pupils) as shown in Fig. 6.To find the precise boundary of the eye, an ellipse detection technique is employed.The ellipse detection works with edge images, thus it needs to detect edge pixels of the blob.A Sobel edge detector is employed to find the edge pixels.After edge pixels are extracted, ellipse fitting method [20] is applied to detect the ellipses.The ellipse fitting method [20] uses the least square criterion to fit an ellipse from edge pixels.Let an ellipse is expressed as (19) The five coefficients a 1 , a 2 , a 3 , a 4 , a 5 are estimated by the least square estimator from the given edge pixels (points) (x 1 ,y 1 ),...,(x N ,y N ) by minimizing the squares sum of an error between the edge pixels and the ellipse using the following formula The estimated five coefficents are then converted to the parameters of ellipse, i.e.: orientation, center coordinate, semimajor axis, and semi minor-axis.Fig. 7 shows the detected ellipses on the left and right eyes superimposed on the blob image.From the figure, it is shown that the method is able to detect the boundary of eye.It is worth noting that blob image contains hole inside it, thus the edge pixels contain the outer and inner part of the blob.However the ellipse fitting method fits the outer ellipse appropriately.

III. EXPERIMENTAL RESULTS
To verify the proposed eye detection method, the algorithm is implemented using MATLAB and tested on two hundred face images taken from [21].The dataset contains frontal face images with varying lighting.
In the experiments, three approaches are evaluated: a) Eye detection based-on Haarcascade Classifier [18] (HC); b) Proposed rough eye localization (PL); c) Proposed eye detection (PD).It is noted here that eye detection based-on Haarcascade Classifier detects the bounding box of eye, but the proposed eye detection detect the precise location of eye (eye's sclera) acoording to the shape.Therefore these two methods could not be compared directly.In addition to the proposed eye detection, the proposed rough eye locatization is evaluated here for fair comparison to the eye detection based-on Haarcascade Classifier.
The true detection rate and false detection rate are calculated for comparison.The true detection rate is defined as the total number of detected eye's pair divided by the total number of tested images, while the false detection rate is defined as the total number of detected non-eye objects divided by the total number of tested images.By this definition, if only one eye (not a pair of eye) is detected, then it is considered as not detected.Table I shows the comparison results.
The true detection rate of HC is the highest, but the false detection is also high.It is a typical result obtained by the Haarcascade classifier, i.e. a high true detection rate, but a false detection occurs.The true detection rate of PL and PD are lower compared to the HC, but they have no false detection.From the observation, the lower true detection of PL is caused by the misdetection of eye due to the position of left and right eyes which are not in a horizontal line.This result relates to the approach employed in the proposed rough eye localization which assumes that the position of two eyes should be in a horizontal line as described in Section B. From the table, there is a deviation between PL and PD.It implies that the some eye images obtained by the rough localization technique could not be detected by white color thresholding and the ellipse detection techniques properly.From the observation, it is caused by two cases : a) The white color thresholding fails to extract eye pixels, thus there are no www.ijacsa.thesai.orgellipse detected; b) The white color thresholding extract the spurious pixels which yield the wrong ellipses.The high true detection rate of PL is comparable to the one of HC.It shows that the proposed white color thresholding works effectively.The advantages of the proposed method compared to HC are: a) There is no false detection; and b) The eye is detected precisely according to the eye's shape.Fig. 8 illustrates some of the detection results, where the figures in the left, center, and right columns represent the results of HC, PL, and PD respectively.In the first and third rows, all three methods detect or localize the eye properly.But the PD localizes eye precisely, in the sense that it detect the ellipse boundary of eye, while the detected boundaries of HC and PL are in rectangle boxes.From the third row, it is shown that the detected eyes obtained by the HC are not in a horizontal line due the skew position of the left and right eyes.However the detected boundaries of left and right eyes are always in the horizontal line.In the second and fourth rows, the non-eye objects are detected by HC (false detection), but not detected by PL and PD.In the fourth row, there is an eye glasses in the image.The HC and PL locate the eyes properly, but PD could not detect eye properly, i.e. the detected ellipses do not fit the eye properly.As shown in the figure, the glasses create the white shadows which result the wrong ellipse detection.

IV. CONCLUSION
In this paper, an eye detection technique is presented.It first localizes the rough position of eye using projection technique.Then a white color thresholding based-on the normalized RGB chromaticity diagram is employed.It extracts the white color of eye's sclera.Finally the boundary of eye is found by an ellipse fitting method.The method shows a high performance in rough eye localization of 93.5 % and the true eye detection rate of 91% is achieved.
In future, the performance of proposed method will be improved and will be extended for detecting the driver's fatigue by measuring the degree of openess of the eye.Further the real time implementation using video camera will be conducted.

Figure 1 .
Figure 1.Overview of proposed eye detection.

Figure 7 .
Figure 7. Detected ellipse on the left and right eyes.

Figure 8 .
Figure 8. Eye localization and detection results.