Study of Face Recognition Techniques : A Survey

With the rapid growth in multimedia contents, among such content face recognition has got much attention especially in past few years. Face as an object consists of distinct features for detection; therefore, it remains most challenging research area for scholars in the field of computer vision and image processing. In this survey paper, we have tried to address most endeavoring face features such as pose invariance, aging, illuminations and partial occlusion. They are considered to be indispensable factors in face recognition system when realized over facial images. This paper also studies state of the art face detection techniques, approaches, viz. Eigen face, Artificial Neural Networks (ANN), Support Vector Machines (SVM), Principal Component Analysis (PCA), Independent Component Analysis (ICA), Gabor Wavelets, Elastic Bunch Graph Matching, 3D morphable Model and Hidden Markov Models. In addition to the aforementioned works, we have mentioned different testing face databases which include AT & T (ORL), AR, FERET, LFW, YTF, and Yale, respectively for results analysis. However, aim of this research is to provide comprehensive literature review over face recognition along with its applications. And after in depth discussion, some of the major findings are given in conclusion. Keywords—Face recognition; illuminations; partial occlusion; pose invariance


I. INTRODUCTION
The 21 st century is a modern and scientific era in which a lot of progress has been achieved as to expedite humans for accomplishing their tasks.In support of above statement, nowadays use of computer technology has been an integral part of life.Computers are being used in pyramids of applications, which range from simple to complex problem solving methods.Among such contributions face recognition technology has emerged as useful tool to recognize features of faces through their inherent traits.And it has been one of the most researched areas in the field of pattern recognition and computer vision.However, due to its wide use in multitude of applications such as in biometrics, information security, law enforcement access control, surveillance system and smart cards.But it possesses many challenges for researcher that needs to be addressed.Face an object depends on facial expressions, which constitute meaningful features.For instance, pose invariance, illuminations and aging which are potential areas that require further investigation over previous work.The result of previous researches reveals that facial expressions are changing with respect to aging; therefore, they could not be permanently modeled in face recognition.The face recognition problem can be categorized into two main phases: 1) face verification and 2) face identification.For example, in real time system, face verification identifies the same person in the scene, and face identification who is this person in that scene.In the first phase it locates a face in an image.Similarly, in the second stage, it extracts features from an image for discrimination.After that they are matched with face database images in order to recognize correct face image as shown in Fig. 1.However, some existing recognition methods for authentication suffer from lack of reliability.For instance, smart cards, wallets, keys, tokens use PINs and passwords which are very difficult to remember.In addition, these passwords and codes can easily be forgotten; also these magnetic cards can be misplaced, even robbed or reproduced.As a result this makes them illegible.Contrary to biological characteristics and traits of an individual person they cannot be stolen, forgotten or misplaced [1].
Biometric recognition system can be built through various techniques.However, most commonly used are fingertips and iris methods.These require individual's participation or involvement to access the system.Moreover, latest systems provide participant access without its intervention.Among such methods face recognition is one of the most viable technique in which an individual can easily be captured and monitored through the system.Face recognition databases vary with controllable photos to uncontrollable videos, where LFT is used for controllable photos and YTM is used for videos.Face recognition system comprises of three main modules: 1) pre-processing, 2) feature selection, and 3) Classification.
Human beings possess natural ability to recognize hundreds of faces by their visual system and cognition.That makes them recognize familiar faces even after a long period of time.Building an intelligent system similar to human perception system is still an active area of research.The researchers have suggested numerous algorithms and methodologies for recognizing a face in an effective and efficient manner.For this purpose, they have focused on detection and recognition of traits and features for individuals such as nose, eyes, mouth, face shape position, size, and beside relationship among traits and features.Furthermore, ongoing research in face recognition tries to develop such systems that could work well in an effective and efficient manner in multitude of real-world applications.In addition, many scholars have proposed and concluded that the accuracy of face recognition can further be improved through the use of 3D faces [6].The rest of this paper is arranged as follows: Section II describes challenges related to the face recognition.Section III explains face recognition databases.Section IV elaborates methods and techniques in face recognition.Section V illustrates widely used applications in face www.ijacsa.thesai.orgrecognition.Finally, Section VI gives conclusion and future directions.

A. Aging
Aging is an inevitable natural process during the lifetime of a person as compared to other facial variations.Aging effect can be observed under main three unique characteristics: 1) The aging is uncontrollable: It cannot be advanced or even delayed and it is slow and irreversible.
2) Personalized aging Signs: Every human passes through different aging patterns.And these rely on his or her genes and many other factors, such as health, food, region, and weather conditions.
3) The aging signs depend on time: The face of person at a specific age will affect all older faces, but unaffected in younger age.

B. Partial Occlusion
Occlusion refers to natural or artificial obstacles in an image.It can be a local region of the face along with different objects such as sunglasses, scarf, hands, and hair.They are generally called partial occlusions.Partial occlusions correspond to any occluding object.And the occlusion less than 50% of the face is considered to be a partial occlusion.The approaches to face recognition with partial occlusion are classified into following three categories: 1) Part Based Methods, 2) Feature based methods and 3) Fractal-Based Methods [4].Many areas of image processing have been impacted by partial occlusion such as recognition by ear is occluded due to earrings.Occlusion affects the performance of a system when people deceive it either by the use of sunglasses, scarves, veil or by placing mobile phones or hands in front of faces.In some cases, other factors like shadows due to extreme illumination also act as occluding factors.Further, local approaches are used to deal with the problem of partially occluded faces which divide the faces into different parts [5].However, this problem can be overcome by eliminating some of the features which create trouble while accurate recognition in the image.Mostly local methods are based on feature analysis, in which best possible features are detected and then they are combined.Another approach that can be applied for this purpose is near holistic approach in which occlude features, traits and characters are eradicated and rest of the face is used as valuable information.Different techniques are being developed by the researchers to cope up with this problem [7], [8].

C. Pose Invariance
Pose variance is yet another hurdle in achieving a successful face recognition system.People pose differently every time they take a picture.There is no standardized rule for taking a pose.Therefore, it makes more difficult to distinguish and recognize the faces from images with varying poses.Pose variations degrade the performance of the facial features.In addition, many systems work under inflexible imaging conditions and as a result it affects the quality of gallery images.The methods dealing with variation in pose can be divided into two kinds i.e. multi-view face recognition and face recognition across pose.Multi-view face recognition can be considered as an annexure of frontal face recognition in which gallery image of every pose is considered.On the other hand, across a pose in face recognition, yield face with a pose which has never been exposed before to a recognition system.A good face recognition approach should provide good pose tolerance and capability to recognize different poses.Several issues in this regard are still open such as lack of perceptive subspace pose variant images.And many of research have been devoted to deal with this issue [9]-[13].However, none of them has achieved 100% accuracy yet.There are some other methods and approaches that are being used to tackle similar problem of face recognition.Furthermore variance and changes in pose can be divided into three classes, namely: 1) general algorithms, 2) two dimensional methods for face recognition, and 3) three dimensional models [14].

D. Illuminations
Illumination is an observable property and effect of light.It may also refer to lightning effect or the use of light sources.Global illuminations are algorithms which have been used in 3D computer graphics.Illumination variation also badly affects the face recognition system.Thus it has been turned an area of attention for many researchers.However, it becomes tedious task to recognize one or more persons from still or video images.But it can be quite easy to extract desired information from images when they are taken under a controlled environment along with uniform background.Also there are three methods that can be implemented to deal with illumination problem.They are gradient, gray level and face reflection field estimation techniques.Gray level transformation technique carries out in-depth mapping with a non-linear or linear function.Gradient extraction approaches are used to extract edges of an image in gray level.As illumination is a factor that heavily affects the performance of recognition system obtained via face images or videos.These techniques are developed to suppress the effect of illumination [9]-[13].

III. FACE RECOGNITION DATABASES
This database of faces was previously called The ORL Database of Faces and it has a set of face images taken at the AT & T lab.This database was used for face recognition project which was carried out with the support of Speech, Vision and Robotics Group of the Cambridge University, Department of Engineering.It includes 10 different images each having 40 diverse subjects.However, for some subjects, the images were acquired under various conditions, i.e. variable light, facial expressions: smiling/sad, open/closed eyes and facial details (glasses/without glasses).These images were obtained with a dark consistent background having position in upright, frontal.Details of the databases used in face recognition are described as under.

A. AR Database
AR database was created by Computer Vision Center (CVC), University of Alabama at Birmingham.It comprises of over 4000 colour images of 126 people's faces.And they are divided into 70, 56 man women, respectively.Images feature frontal view faces with different facial expressions, www.ijacsa.thesai.orgillumination conditions and occlusions (sun glasses, hair styles and scarves).The images of a single person were collected on two different days with a difference of 14 days.This database is available online and can be accessed without any cost for research and academic purposes.

B. FERET Database
The FERET database is being used in facial recognition for system evaluation.The Face Recognition Technology (FERET) program is executed by joint collaboration between the Defense Advanced Research Projects Agency (DARPA) and the National Institute of Standards and Technology (NIST).DARPA released a high-resolution, 24-bit color version of these images in 2003.And it was tested over 2,413 still face images, representing 856 individuals.However, the main motive behind development of FERET database was to facilitate algorithm development and evaluation.Thus initially, it requires a common database of facial images in order to develop and test for the purpose evaluation.After that, complications in image mentioned by the images should enhance.

C. LFW Database
Labeled Faces in the Wild (LFW) is a database of face photographs which was mainly developed for the comprehension of unconstrained face recognition problem.The data set has over 13,000 images of faces obtained via web.And each face is labeled with the name of the person whose picture was captured.However, roughly 1680 of the pictured people contained two or more distinct photos in the data set.However, the main constraint on these face images is that they were detected by the Viola Jones face detector.They are organized into four sets of LFW images, in which one is an original and other three are different types of aligned images.The aligned images have "funneled images" LFW-a, which includes an unpublished method of alignment, and "deep funneled" images.Among them, LFW-a, and the deep funneled images provide higher results over most of the face verification algorithms for original images and funneled images.

D. YouTube Face Database (YTF)
YTF database consists of face videos which were developed for unconstrained face recognition.In this database shortest and longest clips are 48 frames and 6,070 frames respectively.And the average length of a video clip is 181.3 frames.And all the videos were taken from YouTube.An average of 2.15 videos is available for different subjects.

E. Yale Database
The Yale Face Database includes 165 grayscale images in Graphical interchange Format (GIF) of 15 individuals.They are divided into 11 images/subject, each having different facial expression or configuration: center-light, with/glasses, no glasses, happy, sad, sleepy, normal, surprised, and wink.Left-light, right-light.
Yale face database is available in two volumes Yale face A known as yalefaces and Extended Yale face database B. In this database there are 15 different subjects (14 males and 01 female).These comprises of different conditions in facial images such as variations in an expression like sad or normal and happy, etc.This also depends upon other lighting conditions which consists of left, right or center light, and picture having glasses and non-glasses were included.Extended Yale face database is a dataset of 2414 images of 38 subjects.No variation in expression and no occlusions are found in the images but more focus is on extracting feature apt to illumination and they are available in cropped version.

A. Eigen Faces
The word eigenface coined by German -Eigen wert‖ The -Eigen‖ literally mean characteristic and -wert‖ mean value.Eigenface is well establish algorithms that was used to recognize a feature in a face image.It is based on Principle Component Analysis (PCA) [14].In this method the fundamental concept is to recognize the face by taking its unique information about the face in question.Then encode it to compare with the decode result of previously taken image as shown in Fig. 2. In eigenface method, decoding is performed with the calculation of eigenvector and then it is represented as a matrix.However, Eigenface based face recognition systems is only suitable for images having the frontal faces but some researches identify a face with different poses have also been made [1].Analyzing different results drawn from the researchers the accuracy ratio has been much improved in recent years as compared previous results.It is expected to have an effective and efficient output in upcoming years.A comparative study of the analysis results obtained by different researchers by applying face recognition techniques on the basis of Eigen Faces is given in Table I. www.ijacsa.thesai.org

B. Artificial Neural Networks (ANN)
ANN provides an effective feature recognition technique, and it has been widely used after emergence of Artificial Intelligence.This consists of network, where neurons are arranged in the form of layers.Accuracy of face recognition has been boosted with the aid of better deep network architectures and supervisory methods.And recently few remarkable face representation learning techniques are evolved [15].Using these techniques, deep learning (Fig. 3) has got much closer to human performance.For evaluation LFW face verification dataset has been used on tightly cropped face images [15].However, the learned face representation could also add significant intrapersonal variations.One of the most viable feature of Neural Networks is it lessens the complexity.It learns from the training samples and then works fine on the images with changes in lighting conditions and increases accuracy [1].The main drawback of the neural network is a more time is needed for its training.
Initially Training is precursor step to get the desired results from the system as user point of view.
After feature extraction, classifiers for face recognition such as the Radial Basis Function and Feed Forward Neural Network (FFNN) are the implemented.Moreover, study reveals that ANNs achieves improvement over face recognition [2].The following comparative study in Table II shows an accuracy ratio obtained through the use of ANNs.

C. Support Vector Machine (SVM)
SVM is the kind of supervised learning algorithm that uses data for classification and regression analysis.SVM provides advantages of being effective in high dimensions.SVM can be implemented to recognize the faces after facial feature extraction [3].SVM can yield better outcomes when the large quantity of data set is selected directly with training (Fig. 2).However, Least Square Support Vector Machine (LS-SVM) [50], [51] is among the popular one in SVM types that is being successfully utilized for face recognition task.This provides advantage of fast computation, speed along with high recognition rate [1].Component-based SVM classifier [16] is another variant of SVM in face recognition.The Support Vector Machine (SVM) classifier is a most widely used technique that is being implemented on a wide range of classification problems.Mostly these problems are in high dimensions and they are not linearly separable.SVM is useful in the advent of dealing with very high dimensional data.Researchers worked on SVM for classification of face recognition and got better results as shown in the below Table III.D. Gabor Wavelet Dennis Gabor in 1946 introduced a tool for signal processing in noise removal and named as Gabor filter.Gabor wavelets technique is being widely used for face tracking and position estimation in face recognition.While an image representation using the Gabor wavelet transform provides both the spatial relations and spatial frequency structure.As shown in Fig. 4, it has a characteristic that allow it to delineate the properties of spatial localization, spatial frequency selectivity, and orientation [17], [18].Gabor Wavelets works well over extraction of edge and shape information and it represents the faces in a compact way which is more similar to the feature based methods [19].The main advantages of Gabor Wavelets Transform are face feature reduction and its global feature representation in face recognition [21], [29], [30].Table IV shows comparative analysis of Gabor filters by different researchers.

E. Hidden Markov Models
Hiddel Markov Model (HMM) is another statistical modelling technique in which the system undergoes in Markov process with hidden states (Fig. 5).This model was proposed in 1960 and provided a significant contribution towards speech recognition.HMM is well established method in reinforcement learning, temporal pattern recognition, and bioinformatics applications.Currently it is being implemented to recognize face expressions.Also it can be applied to the video sequences for face recognition.It needs a sequence of 1D and 2D images for experimental purposes [21]; but firstly these images should be converted into a chronological sequence of 1D or spatial.However, model consists of two processes, in which first Markov Chain process having a finite number of states is not viewed explicitly.While in other process each state constitutes a set of probability density function connected with it [1].Although for research, generally 5-state HMM is designed for face recognition system.5-state HMM is grouped into five facial features such as eyes, nose, mouth, chin and forehead for frontal view face images as shown in Fig. 6 [47].But the number of states can be added or removed which depends upon system's requirement.Fig. 5. Three states of transition from left to right for HMM [25].www.ijacsa.thesai.orgIn other case, 7-State HMM [48] provide more details which boosts the performance of the face recognition system.Various researchers have worked to get satisfactory outcomes by applying different algorithms in this model.However, a new enhanced model The Adaptive Hidden Markov Model (AHMM) [49] is proposed by authors to find out the issues of identifying the faces via a video sequence.The comparative result in Table V depics the accuracy ratio of HMM.

V. APPLICATIONS OF FACE RECOGNITION
There are many applications where face recognition techniques are successfully used to perform a specific task.Few of them are described as under:

A. Access Control
Access control allows the authorized group of users to access the personal account by logon through their email account using computer accessing bank account through ATM machine.But using face recognition system face pictures are taken under natural conditions such as frontal face images.Such kind of systems yields optimal accuracy without any intervention from the user.These automatic face recognition systems are also used to view and control a user activity on PC or ATM machine; for example, when users leave the PC without proper closing their files and folders for a predetermined time.Then system halts until user again logon and is recognized.In this case, only legitimate persons are allowed to access account.

B. Security
Security is a most important precursor at all places.Computer security is being carried out with use of face recognition application.In this regard, image database is being used for investigation purposes [1]; for instance, searching image for authentication of licensed drivers to search missing peoples, immigrants in law enforcement agencies, General identity verification [1], Electoral registration, banking, electronic commerce, identifying newborns, national IDs, passports, employee IDs.

C. Surveillance
The word surveillance has been derived from a French phrase which means "watching over".Here (sur means "from above‖ and veiller means "to watch").Surveillance is used to monitor the individual's behavior, activities, or beside other related information for ensuring the people safety.This can be achieved by means of electronic equipment i.e. closed-circuit television (CCTV) cameras) or interception of electronically transmitted information.Surveillance system offer no. of benefits to different organizations.For example, it is being used by governments for intelligence gathering, control the crime, monitoring the process, person, crowd or object, or the inquiry of crime.However, on the other side, surveillance is often considered as a violation of privacy, and in such cases it is often criticized by civil societies, groups and activists.Liberal democracies have laws which bounds local governments and law agencies to use surveillance, usually restricting them in those circumstances where public safety is compromised.Legitimate organizations have often been imposed such domestic restrictions.However, international surveillance is similar among all types of countries.Moreover researchers are trying to achieve more improved and contented results through the use of latest algorithms in face recognition.

D. Time & Attendance
Biometric Time Attendance technologies have been used for Access Control solutions and these are among the latest solutions over traditional systems [23].In this technology, users are required to expose their face into the machine's camera by making a certain distance and remove any physical contact with the device.This eliminates any possibility of being tempering or machinery alteration through its noncontact method procedure.Face Recognition system captures specific features from a human face and record it in the form of mathematical template as depicted in Fig. 7.In order to recognize the face, facial image is normalized as to line-up eyes and mouth.Then it performs matching with mathematical vectors from database.Finally face recognition system verifies face and allows for marking attendance or access transaction.These machines could also be implemented for other solutions, where biometric identification/verification is required; such as canteen management, salary distribution, and social services.www.ijacsa.thesai.org

E. Pervasive Computing
The aim of pervasive computing is to create a sensor based network as to make smart devices.Hence, sensor network is used to collect, process and send data, and eventually, it can understand its surroundings and improves the human capability and quality of life.However, pervasive computing uses wireless communication and networking technologies, mobile devices, wearable computers, embedded systems, Radio Frequency Identification Devices (RFID) tags, middleware and software agents.Pervasive computing is being widely used in number of applications, for instance in energy, consumer, healthcare, production, military, and logistics.
One of the examples of pervasive computing is a smart Watch developed by Apple Watch.It informs a user for incoming phone call and allows him to complete the call using watch [1].

VI. CONCLUSION AND FUTURE DIRECTIONS
Study of face recognition has remained a striving area for researchers for many years.In this paper, a comprehensive study was performed over different face recognition methods.After detailed analysis it revealed that PCA is best suited technique when dimension of features is higher for original face images, whereas eigen faces image features method work well for frontal face recognition.Among face recognition methods, the most popular are Neural Networks, Support Vector Machine, Sparse Representation based Classification (SRC), Linear Regression Classification (LRC), Regularized Robust Coding (RRC) and Nearest Feature Line ((NFL).These methods provide better results when the image dimension is under 150 or more.Furthermore, it is suggested that PCA, SVM, NN and Eigen methods still need to be researched so that more satisfactory results could be achieved for face recognition.Moreover, in this paper we also mentioned state of the art face recognition image database and face technology benefits in various applications.However, main findings of this research are highlighted as under:  The development trends and achievements in the realm of face recognition shows that a lot of researchers have been carried out in last four decades.
 Currently, face recognition system has been implemented for many real-time applications, but still it suffers from several challenges that need to be addressed in order to design a well-established face recognition system.
 Developed face recognition techniques could be analyzed over varying facial expression i.e. under varying lighting conditions and pose.And evaluation could be performed using benchmark and latest face databases.
 Similarly to the face image recognition, the video image recognition is more complicated that needs be researched.
Further, it is suggested that for recognition of video images, YouTube Faces could be analyzed for evaluation.Furthermore, recognition of emotional human behavior has emerged recently as a promising [52] research area for scholars that should be exploited in future.Finally, it is concluded that still there remains a gap in terms of study in face recognition system that requires to be filled in order to improve its accuracy and efficiency.

TABLE I .
COMPARATIVE STUDY OF FACE RECOGNITION TECHNIQUES BASED ON PCA

TABLE IV .
COMPARATIVE STUDY OF FACE RECOGNITOIN METHODS BASED ON GABOR WAVELETS