Efficient Eye Blink Detection Method for disabled-helping domain

In this paper, we present a real time method based on some video and image processing algorithms for eye blink detection. The motivation of this research is the need of disabling who cannot control the calls with human mobile interaction directly without the need of hands. A Haar Cascade Classifier is applied for face and eye detection for getting eye and facial axis information. In addition, the same classifier is used based on Haarlike features to find out the relationship between the eyes and the facial axis for positioning the eyes. An efficient eye tracking method is proposed which uses the position of detected face. Finally, an eye blinking detection based on eyelids state (close or open) is used for controlling android mobile phones. The method is used with and without smoothing filter to show the improvement of detection accuracy. The application is used in real time for studying the effect of light and distance between the eyes and the mobile device in order to evaluate the accuracy detection and overall accuracy of the system. Test results show that our proposed method provides a 98% overall accuracy and 100% detection accuracy for a distance of 35 cm and an artificial light. Keywords—eye detection; eye tracking; eye blinking; smoothing filter; detection accuracy


I. INTRODUCTION
In the recent years due to the rapid advancement in the technology there has been a great demand of human computer or mobile interaction (HCI or HMI).Eye blink is a quick action of closing and opening of the eyelids.Blink detection is an important enabling component in various domains such as human computer interaction, mobile interaction, health care, and driving safety.For example, blink has been used as an input modality for people with disabilities to interact with computers and mobile phones [1].
In Viola [2] the chain of single-feature filters, Haar Cascade Classifier for identifying sub-region image is used.With the fast calculation of integral image technique, it can work in real time.
Eye tracking provides an almost seamless form of interaction with the modern graphical user interface, representing the fastest non-invasive method of measuring user interest and attention.While the mouse, keyboard, and other touch-based interfaces have long reigned as the primary input mediums associated with the field of human computer interaction, as advances continue to improve the cost and accuracy of eye tracking systems they stand poised to contend for this role [3].An open and close eye template for blink pattern decisions based on correlation measurement is used in [4].The method was specifically useful for people with severely paralyzed.A real-time eye blinking detection was proposed based on SIFT feature tracking with GPU based implementation [5].
An efficient method is proposed in [6].A method is based on image processing techniques for detecting human eye blinks and generating inter-eye-blink intervals.A Haar Cascade Classifier and Camshift algorithms for face tracking and consequently are applied for getting facial axis information.Adaptive Haar Cascade Classifier from a cascade of boosted classifiers based on Haar-like features using the relationship between the eyes and the facial axis applied for positioning the eyes.The algorithm results show that the proposed method can work efficiently in real-time applications.
An EyePhone application which is developed in [7] is a system that capable of driving mobile applications/functions using only the user's eyes movement and actions (e.g., wink).EyePhone tracks the user's eye movement across the phone's display using the camera mounted on the front of the phone.The results indicate that EyePhone is a promising approach to driving mobile applications in a hand-free manner.
An efficient eye tracking system is presented in [1,8] having a feature of blink detection for controlling an interface that provides an alternative way of Communication for the people who are suffering from some kind of severe physical disabilities the proposed system uses pupil portion for tracking the movement of eyes.
The outline of the paper is as follows.In section II, the proposed methods are presented.Section III studies the test results of a real time application for two cases: normal light and artificial light condition with and without using filter.Conclusions and future remarks are described in section IV.

II. PROPOSED EYE BLINK DETECTION
In figure 1 the major steps of the proposed Eye Blink to Control Mobile Phones EBCM are shown.www.ijacsa.thesai.org

A. Frame Capturing
The first step of the proposed EBCM application is the initialization.After taking a short video of the participant's face using the front camera of the Samsung mobile.A process Frame method will be used to create the frames from the captured video.Afterwards the colored frames will be converted to gray scale frames by extracting only the luminance component as shown in figure 2.

B. Face Detection
The Haar classifier is used in EBCM algorithm for face detection.Haar classifier rapidly detects any object, based on detected feature not pixels, like facial feature.However, the area of the image being analyzed for a facial feature needs to be regionalized to the location with the highest probability of containing the feature.By regionalizing the detection area, false positives are eliminated.As the result, the face is detected and marked with color rectangle and will be used later to approximate an axis of the eyes for eye detection step.

C. Eye Detection
To detect the eye, first, the Haar cascade classifier should be trained, in order to train the classifiers, the AdaBoost algorithm and Haar feature algorithms must be implemented, two set of images are needed.One set contains an image or scene that does not contain the object.
The EBCM used all detected elements from the Haar Cascade Classifier, and the result show the detected eye in color rectangle as shown in figure 3.

D. Eye Tracking
The corneal-reflection and pupil-center are the two eye's parts that are the most important parts to extract the features that will be used in EBCM method.These features help us in tracking the eyes movement.By identifying the center of the pupil and the location of the corneal reflection, the vector between them is measured.Besides, with further trigonometric calculations, point-of-regard can be found.The EBCM method succeeded in making the face and the eye's pupil moved together in the same direction synchronously and with the same direction.Let suppose that X is the human face which has been detected, P1 and P2 are two points related to the left eye, and they are moving synchronously with the movement of X as shown in figure 4.

E. Eye Blinking
Eye blinking and movement can be detected with relatively high reliability by unobtrusive techniques.Though, there are few techniques discovered for the active scene where the face and the camera device move independently and the eye moves freely in every direction independently of the face.Although, care must be taken, that eye-gaze tracking data is used in a sensible way, since the nature of human eye movements is a combination of several voluntary and involuntary cognitive processes.

Case 1: Without Filtering
The frames that have been detected earlier will be used in this step to find the status of the eye, if it is open or close.The algorithm gets 15 frames to identify the correct position of the eye.To determine the frame's pixels threshold, a binary threshold using the following equation has been applied.The threshold is initialized to 70 after experimentation we found that 70 is the best number to use.
If the intensity of the pixel src(x,y) in the frame is higher than threshold, then the new pixel intensity is set to a maxVal.Otherwise, the pixels are set to zero. Figure 5 depicts the flowchart of the binarization process of the given frames.
In the binary frames, 0 represent the black color and 1 represent the white color for each pixel.These frames will go through a series of operations to convert all points of black and gray to zeroes and determine the length and width of the part under the eyebrows.If the index is not equal to zero, the points gray will increase.Otherwise, the number of black points will increase and this process help to discover whether the eye is open or closed .Consequently if the black points greater than 3, it means the case is open, otherwise the eye is close as shown in figure 6.

Case 2: With Filtering
After detecting the eye blink successfully, the same steps that have been applied before will be used but with applying the Medium Blur Filter on the binary frames.The major objective of smoothing or blurring image is to decrease the noise.Such noise reduction is a typical image pre-processing method which will improve the accuracy of detection.
After applying the median filter on the frames, the algorithm will check if at least one black pixel appears.If there is no black pixel, the threshold value will increase and follow the same sequence, but if there is more than one black pixel, the process will terminate and get the state of eye value as that threshold.www.ijacsa.thesai.org

F. Mobile Phone Controlling with Eye Blinking
The last step of the EBCM algorithm is controlling the mobile activity by making a phone call depending on the result of previous step.If the state of the eye is close which means the human eye blinked, a call phone to specific phone number will be made, otherwise no phone call will be made as shown in figure 7. The EBCM application used ACTION_CALL action to trigger built-in phone call functionality available in Android device.

G. Performance Parameters
The main parameters that affect the proposed EBCM are the distance and the lighting.If the distance between the user's eye and the mobile phone is long the process of detecting will be difficult or impossible in some cases.The light also affects the process of detecting the eye either in a normal light condition or in an artificial light.
To find the accuracy of the eye detection in the proposed algorithm the following equations has been used [6].Where TP is the number of frames that are correctly detected eye blinks (true positive); FN is the number of frames that show eye blinks but the program is not detected (false negative); FP is the number of frames that are reported as eye blinks but they are not (false positive); and TN is the number of frames that are correctly reported as no blinks (true negative).

III. EXPERIMENTAL RESULTS
The test are conducted by using Intel(R) Core(TM) 2 Due CPU P8400 ,2.10 GHZ, 32 Bit processer with 4.00 GB of RAM.
The simulation program is compiled by using Eclipse Platform Juno (4.2) that supports Java code, OpenCV library [9,10] (as an image processing and computer vision tools), and windows applications, which benefits from its features to design a user interface for android mobile application.The mobile that held the tests is Samsung Galaxy S3 that has Android version 4.3 with model number GT-19300.Finally, the software executed under Windows7 as the operating system.
The Performance parameters that are used in order to evaluate the proposed (EBCM) method are: Figure 8 present a screen shot of our application.9 shows the test results of overall and detection accuracy versus distance under the normal light using a median filter.

C. Best Light Condition (with Filtering)
In this case the detection accuracy rate is very optimal because of the effect of the filter as shown in figure 10.

IV. CONCLUSIONS AND FUTURE WORKS
Eye blink detection is a very challenge problem for controlling mobile phones in a real time application.This is due to the movement of eyes and the variation of light for different distances from the mobile camera.The proposed method provides 8% of accuracy improvement for eye detection and blinking.When an artificial light is used the overall and detection accuracy are 98% and 100% respectively for a distance equal to 35 cm.Each frame takes an average of 71 ms for time execution which is very efficient for real time application.
The future work will be improving the security level of the proposed eye tracking system using voice recognition algorithm and adapting the application for a non-frontal face.


The distance between the volunteer's eye and the mobile camera are taken into account, ranging (15, 20, 25, 30, 35, 40, 45) cm. The light condition o Normal room light or, o High quality artificial room light.

Fig. 8 .
Fig. 8. Disable volunteer during application testA.Normal Light Condition (without Filtering)TableIpresents the test results of the case that the normal light is used without applying a filter.

Fig. 10 .
Fig. 10.Optimal detection accuracy Table I presents the test results of the case that the normal light is used without applying a filter.

TABLE I .
OVERALL AND DETECTION ACCURACY VS.DISTANCE