Emotion Classification Using Facial Expression

Human emotional facial expressions play an important role in interpersonal relations. This is because humans demonstrate and convey a lot of evident information visually rather than verbally. Although humans recognize facial expressions virtually without effort or delay, reliable expression recognition by machine remains a challenge as of today. To automate recognition of emotional state, machines must be taught to understand facial gestures. In this paper we developed an algorithm which is used to identify the person’s emotional state through facial expression such as angry, disgust, happy. This can be done with different age group of people with different situation. We Used a Radial Basis Function network (RBFN) for classification and Fisher’s Linear Discriminant (FLD), Singular Value Decomposition (SVD) for feature selection.


INTRODUCTION
Human beings express their emotions in everyday interactions with others.Emotions are frequently reflected on the face, in hand and body gestures, in the voice, to express our feelings or liking.Recent Psychology research has shown that the most expressive way humans display emotions is through facial expressions.Mehrabian [1] indicated that the verbal part of a message contributes only for 7% to the effect of the message as a whole, the vocal part for 38%, while facial expressions for 55% to the effect of the speaker's message.
Emotions are feeling or response to particular situation or environment.Emotions are an integral part of our existence, as one smiles to show greeting, frowns when confused, or raises one's voice when enraged.It is because we understand other emotions and react based on that expression only enriches the interactions.Computers are "emotionally challenged".They neither recognize other emotions nor possess its own emotion [2].
To enrich human-computer interface from point-and-click to sense-and-feel, to develop non intrusive sensors, to develop lifelike software agents such as devices, this can express and understand emotion.Since computer systems with this capability have a wide range of applications in different research arrears, including security, law enforcement, clinic, education, psychiatry and Telecommunications [4].There as been much research on recognizing emotion through facial expressions.
In emotional classification there are two basic emotions are there, Love-fear.Based on this we classify the emotion into positive and negative emotions.The six basic emotions are angry, happy, fear, disgust, sad, surprise.One more expression is neutral.Other emotions are Embarrassments, interest, pain, shame, shy, anticipation, smile, laugh, sorrow, hunger, curiosity.
In different situation of emotion, Anger may be expressed in different ways like Enraged, annoyed, anxious, irritated, resentful, miffed, upset, mad, furious, and raging.Happy may be expressed in different ways like joy, greedy, ecstatic, fulfilled, contented, glad, complete, satisfied, and pleased.Disgust may be expressed in different ways like contempt, exhausted, peeved, upset, and bored.
In emotional expression face, If we are angry, the brows are lowered and drawn together, Vertical lines appear between the brows, lower lid tensed, eyes hard stare or bulging, lips can be pressed firmly together with corners down or square shape as if shouting, nostrils may be dilated, the lower jaw juts out.
If we are happy, corners of the lips are drawn back and up, mouth may or may not be parted, teeth exposed, a wrinkle runs from outer nose to outer lip, cheeks are raised, lower lid may show wrinkles or be tense, crows feet near the outside of the eyes.Disgust has been identified as one of the basic emotions.Its basic definition is "bad taste" secondarily to anything which causes a similar feeling, through the sense of smell, touch and even of eyesight.The well defined facial expression of disgust is characterized by furrowing of the eyebrows, closure of the eyes and pupil constriction, wrinkling of the nose, upper lip retraction and upward movement of the lower lip and chin, drawing the corners of the mouth down and back This research describes a neural network based approach for emotion classification.We learn a classifier that can recognize three basic emotions.

II. RELATED WORK
There are several approaches taken in the literature for learning classifiers for emotion recognition [2] [6].
In the static Approach, the classifier classifies each frame in the video to one of the facial expression categories based on the tracking results of that frame.Bayesian network classifiers were commonly used in this approach.Naïve Bayes classifiers were also used often.Because of this unrealistic approach some used Gaussian classifiers.
In the Dynamic Approach, these classifiers take into account the temporal pattern in displaying facial expression.Hidden Markov Model (HMM) based classifiers for facial www.ijacsa.thesai.orgexpression recognition has been previously used in recent works.Cohen and Sebe [4] further advanced this line of research and proposed a multi-level HMM classifier.This field has been a research interest for scientists from several different tracks like computer science, engineering, psychology, and neuroscience [1].In this paper, we propose a complete emotional classification using facial expression.The network is well trained with the various emotions of the face with different age group of people from different situation.When given an input, the RBF network with the help of FLD, trained with various emotions of the face expression with that of training examples and produces the output.This Existing research was developed by Paul Ekman [3].He is a psychology perspective.In the early 1990s the engineering community started to use these results to construct automatic methods of recognizing emotions from facial expressions in images or video [4] based on various techniques of tracking [5].An important problem in the emotion recognition field is the lack of agreed benchmark database and methods for compare different methods performance.Cohnkanade database is a step in this direction [2].
Since the early 1970, Paul Ekman and his colleagues have performed extensive studies of human facial expression [7].They found evidence to support universality in facial expressions.These "Universal facial expressions" are those representing happiness, sadness, anger, fear, surprise, and disgust.They studied facial expressions in different cultures, including preliterate cultures, and found much commonality in the expression and recognition of emotions on the face.However, they observed differences in expressions are governed by "display rules" in different social contexts.For Example, Japanese subjects and American subjects showed similar facial expressions while viewing the same stimulus film.However, in the presence of authorities, the Japanese viewers were more reluctant to show their real expressions.On the other hand, babies seem to exhibit a wide range of facial expressions without being taught, thus suggesting that these expressions are innate.
Ekman and Friesen [3] developed the Facial Action Coding System (FACS) to code facial expressions where movements on the face are described by a set of action units (AU s).FACS provides the mechanisms to detect facial movements by human coders.Action Units are a set of actions that corresponds to muscle movement such as raising lower lips, blinking, Biting lip, blow.Each AU has some related muscular basis.FACS consists of 44 action units.It does not contain any emotion specific so it must be sent to other system to recognise the emotions.This system of coding facial expressions is done manually by following a set of prescribed rules.The inputs are still images of facial expressions, often at the peak of the expression.This process is very time-consuming.Ekman's work inspired many researchers to analyze facial expressions by means of image and video processing.By tracking facial features and measuring the amount of facial movement, they attempt to categorize different facial expressions.[8] has used these "basic expressions" or a subset of them.In [9], Pantic and Rothkrantz provide an in depth review of many of the research done in automatic facial expression recognition in recent years.The work in computer-assisted quantification of facial expressions did not start until the 1990s.

Recent work on facial expression analysis and recognition
Mase [10] used optical flow (OF) to recognize facial expressions.He was one of the firsts to use image-processing techniques to recognize facial expressions.Lanitis [11] used a flexible shape and appearance model for image10 coding, person identification, pose recovery, gender recognition, and facial expression recognition.Black and Yacoob [12] used local parameterized models of image motion to recover nonrigid motion.Once recovered, these parameters were used as inputs to a rule-based classifier to recognize the six basic facial expressions.
Yacoob and Davis [13] computed optical flow and used similar rules to classify the six facial expressions.Rosenblum, Yacoob, and Davis [14] also computed optical flow of regions on the face then applied a radial basis function network to classify expressions.Essa and Pentlands [15] used an optical flow region-based method to recognize expressions.Donato et al. [16] tested different features for recognizing facial AUs and inferring the facial expression in the frame.Otsuka and Ohya [17] first computed optical flow, then computed the 2D Fourier transform coefficients, which were used as feature vectors for a hidden Markov model (HMM) to classify expressions.The trained system was able to recognize one of the six expressions near real-time (about 10 Hz).Furthermore, they used the tracked motions to control the facial expression of an animated Kabuki system [18].Martinez [19] introduced an indexing approach based on the identification of frontal face images under different illumination conditions, facial expressions, and occlusions.
A Bayesian approach was used to find the best match between the local observations and the learned local features model and an HMM was employed to achieve good recognition even when the new conditions did not correspond to the conditions previously encountered during the learning phase.Oliver et al. [20] used lower face tracking to extract mouth shape features and used them as inputs to an HMM based facial expression recognition system (recognizing neutral, happy, sad, and an open mouth).These methods are similar in that they first extract some features from the images, then these features are used as inputs into a classification system, and the outcome is one of the pre selected emotion categories.They differ mainly in the features extracted from the video images and in the classifiers used to distinguish 11 between the different emotions.

III. PROBLEM DEFINITION
There are three main factors to construct a Facial Expression Recognition system, namely face detection, facial feature extraction, and emotion classification.An ideal emotion analyzer should recognize the subjects regardless of gender, age, and any ethnicity.
The system should be invariant to different lightening conditions and distraction as glasses, changes in hair style, facial hair, moustache, beard, etc. and also should be able to "fill in" missing parts of the face and construct a whole face.It should also perform robust facial expression analysis despite www.ijacsa.thesai.orglarge changes in viewing condition, rigid movement, etc.A good reference system is the human visual system [4].The current systems are far from ideal and they have a long way to achieve these goals.

IV. THE SYSTEM SETUP
The design and implementation of the Emotion classification using facial expression System can be subdivided into three main parts: Image Detection, Recognition technique which Includes Training of the images, Testing and then result of classification of images.
We used real time database instead download the images from existing databases.This is because of noise free and easy use.We are using Radial basis function network which is capable of handling noisy images also.Its gives better result than back propagation neural network.For Face Data, The face slides are part of the facial emotion database assembled by Ekman and Friesen [21].In Proposed method, the pictures in the database were tested on different age group of people from my relatives.We used four individual's one child, one adult, one middle aged women, one aged women.They are expressing one of the six emotions -happy, sad, fear, anger, surprise, disgust.We used three basic emotions such as happy, anger, disgust.We asked them to express their emotions for different situations.Figure 1, Figure 2, Figure 3 shows some of the training images.

B. Recognition technique
It aims at modelling the face using some mathematical representation in such a way the feature vector can be fed into a classifier.The overall performance of the system mainly depends on the correct identification of face or certain facial features such as eyes, eyebrows and mouth.After the face is detected, there are two ways to extract the features: Holistic Approach, Analytic Approach.In Holistic, raw facial image is subjected for feature extraction.While in analytic, some important facial features are detected.Here we used Holistic Approach, as that it means we send a raw image as an input without any feature selection.We fix the block size as fifty.The images are fed into fifty rows and fifty columns.

C. Statistical Method
After reading the image, the image must be analysed for duplication.So that correlation of the matrix will be find.Since the correlation matrix for each image is square, we can Calculate Eigen vector and Eigen value for each matrix.These are very important so that it gives useful information about the data.

1) Eigen Vectors and Eigen Value :
In Eigen vectors any vector change in magnitude but not in direction is called as Eigen vector.In Eigen values, the magnitude that the vector is changed is called an Eigen value.Where A is n x n matrix.X is the length of n column vector.λ is a scalar.It's an Eigen value and x is the Eigen vector.
"The Eigen values for angry image1:" 0.1369, 0.1371, 0.1372, 0.1373, 0.1371, 0.1368, 0.1366, 0.1375, 0.1382, 0.1285, 0.1394, 0.1402, 0.1406, 0.1408, 0.1412, 0.1417, 0.143"."The Eigen values for angry image2:" 0.1368, 0.1371, 0.1374, and 0.1372."The Eigen values for angry image3:" 0.1367, 0.1371, 0.1375 and 0.1371.It is important to notice that these eigenvectors are both unit eigenvectors that is their lengths are both 1.This is very important for FLD.In math packages, when asked for eigenvectors, will give you unit eigenvectors.It turns out that the eigenvector with the highest Eigen value is the linear Component of the data set [22].After getting the Eigen values we calculate the mean values for all the images, in order to get highest Eigen values.The highest Eigen value must contain important feature about the data.So we select ten highest Eigen values from all the images.

2) Fisher's Linear Discriminant:
It can reduce the number of variable in the input by projecting data onto a possibly uncorrelated and low dimensional space.It reduces the number of features in the input to a manageable level.Some variable with information that is not related to facial expression can be excluded during the projection on low dimensional space.This helps the network to prevent from learning unwanted details in the input.The above property can improve the network classifier's performance and generalization.By minimizing within class variance and maximizing between class variance.The most famous example of dimensionality reduction is principal component analysis.This technique searches for direction in the data that have largest variance and subsequently project the data onto it.That removes some of the noisy directions.There are many issues with how many directions one needs to choose.It is a unsupervised technique [23].
When compare with Principle Component Analysis, Independent Component Analysis, and FLD gives maximum percentage of the output.It is best for classification, improves the performance and reduction technique.Fisher Linear Discriminant Analysis considers maximizing the following objective.
Where S B is the "between class scatter matrix" and S W is the "within class scatter matrix".Then we find two matrixes which contain specific information of the data.

3) Singular Value Decomposition:
The Singular Value Decomposition (SVD) is one of the most important tools of numerical signal processing.It plays an interesting, fundamental role in many different applications.SVD in digital applications provides a robust method of storing large images as smaller, more manageable square ones.This is accomplished by reproducing the original image with each succeeding nonzero singular value.To reduce storage size, by using fewer singular values [25].The SVD of a face image has good stability in which it defined that whenever a small perturbation is added to a face image, a large variance of its singular values (SV) didn't occur.Since it has stated that singular value represent algebraic properties of an image.Hence, SV features possess algebraic and geometric invariance as an instance [26].The theory of SVD states that any matrix A of size m x n can be factorized into a product of unitary matrices and a diagonal matrix, as the following [24].
The diagonal elements of ∑ are called the singular values of A and are usually ordered in descending manner.We use the orthogonal matrix U as the projection vectors.U and V matrices are inherently orthogonal in nature.These directions are encoded in U and V matrices.This method called as FLD+SVD method.In contrast to eigenvector calculations involved in conventional algorithms, the SVD has several advantages.Computationally efficient and it is robust under noise conditions.
Using the formula in mat lab, we can find the svd of sw. using this value we can find Phi 1 vector and Phi2 vector.
We concatenate the results so as to convert high dimensional data into two dimensional data.First of all, we read all the images (four persons with three different situations for three basic emotions).After reading the image, correlation of the images can be finding.Then we find Eigen vectors, Eigen values for all the images in order to get original data.After, Using FLD and SVD feature selection is done.Now we get the graph for all emotions.Figure 4 and figure 6 describe the different emotional graph and their corresponding values.

D. Neural networks
A neural network have been used in the field of image processing, it provides an optimistic result in terms of quality of outcome and ease of implementation.Neural network proved itself to be invaluable in applications where a function based model or parametric approach to information Processing is difficult to formulate.The description of neural network can be summarized as a collection of units that are connected in some pattern to allow communication between the units.These units are referred as neurons or nodes generally.The output signals feed to other units along the connections which known as weight.The weights usually excite or inhibit the signal that is being communicated.One of the specialty of neural networks is that the hidden units factors.The function of the hidden units or hidden cells or also called hidden neurons is to intervene between the external input and the network output.The network which implemented neural network in it actually has the ability to extract higher order statistics by adding one or more hidden layers [27].
Hence, this characteristic is particularly valuable when the size of the input layer is large, specifically in the face recognition field [28].

E. Radial Basis Function Network
It's a two layer, Hybrid feed forward learning network.It is fully connected network.This is becoming an increasing popular neural network with diverse applications and is probably the main rival to the multi layered perceptron.Much of the inspiration for RBF network has come from traditional statistical pattern classification techniques.It's mainly used as classification tool.It's used by broom head and Lowe in 1988.
Each hidden neuron has a symmetric radial basis function as an Activation Function.The purpose of the hidden neurons is to cluster the input data and reduce dimensionality.Train input data in order to minimize the sum of square errors and find the optimal weights between hidden neurons and output nodes.These optimal weights can classify effectively the test data into correct classes.Figure 5 describes the basic architecture of RBF network.

F. Training Phase
We present the network with training examples, which consist of a pattern of activities for the input units together with the desired pattern of activities for the output units.We determine how closely the actual output of the network matches the desired output.We change the weight of each connection so that the network produces a better approximation of the desired output.The input to the training phase is a collection of images showing human faces.These images are also called as Face images.These face images are then passed through a feature extraction step.In the feature extraction step key attributes of the images are computed and stored as a vector called feature vector.We get phi 1 vector and phi2 vector.These feature vectors define or represent the most important properties observed in the face image.Highest Eigen values are chosen.
There are two advantages of this step.First, the size of the data is reduced from the entire image to only a few selected important features.Second, the selection of features gives more structured information than just basic pixel values of the images.Thus the feature vectors can be considered as the minimal set that is adequate to represent the face image.The training could be done on face images showing a selected class of emotion or the entire set of emotions.If the training is done for selected class of emotions then the model is build for each class of emotion.Hence in this case three basic emotions mentioned earlier a separate will be build for each of three emotions.The input images for a particular model will be only the images that show the corresponding emotion.If the training is desired for building a single model for the entire set of emotions then the entire set of face images is used for the training space.In our case after the training the radial basis functional network will get the separate values for each emotion.
Now training has been done.We conclude that the value for Angry Emotion is nearly 0.1, the value for Disgusted Emotion is nearly 0.2 and the value for Happy Emotion is nearly 0.3.Figure 7 shows the approximate results for the three basic

G. Testing phase
This phase can be performed to measure the classification rate.The inputs to this phase are the models that were build during training phase and the test images for which the emotions are to be recognized.Here again only the face region is used as rest of the image do not contribute information about the emotion.In a typical real time scenario the input image would be detected face image from an earlier face detection phase.
The first step here again would be a feature extraction phase where the key features from the face image are extracted.The extraction method must be same as the one used in the training phase.The output of this step is the feature vector of the face image that would then be subjected to a testing step.In the testing step the feature vector is tested against the models built during the training phase.The output of the testing step is a score that indicated the emotion that is detected by the model.This score is usually in the form of distance or probability and it defines which model was best suited for the feature vector extracted in the previous step.In the testing step there are two possible ways that can be employed.The first possibility is in the case when one model was built per class of emotion.Here the feature vector is tested against all the models and their scores then define which model was the most suited one.The second possibility is the case when only one model was built for the entire set and a single score defines the possible emotion detected.During testing a simple image the approach can correctly classify it as the correct emotion expressed.In another case the approach can also wrongly classifies a sample image as the correct emotion expressed.These cased constitute false positives.It could also be the case that the approach wrongly classifies a sample images as incorrect emotion expressed.These cased constitute false negatives.

VI. CONCLUSION
In this paper we have presented an approach to expression recognition in the static images.This emotion analysis system implemented using FLD, SVD for feature selection and RBF    network for classification.This paper is designed to recognize emotional expression in human faces using the average values calculated from the training samples.We evaluated that the system was able to identify the images and evaluate the expressions accurately from the images

VII. FUTURE WORK
In this paper we classify the emotional expression for three basic emotions.This gives accurate performance.In future we may include other expression also.We proceed with Video images also.

Figure 1 .
Figure 1.Angry expression for different situations.

Figure 2 .
Figure 2. Disgust expression in different situations

Figure 3 .
Figure 3. Happy expression in different situations

Figure 4 .
Figure 4. Above Graph for different emotions * for Angry emotion, + For disgust emotion, ^ For happy emotion

Figure 5 .Figure 6 .
Figure 5. Graph has been plotted using above measurement

Figure 7 .
Figure 7. Shows the approximate results for three basic emotions.

Figure 9 .
Figure 9. Shows the result for disgust image.

Figure 10 .
Figure 10.Shows result for angry image.

Figure 8 .
Figure 8. Shows the result for happy image