Face Recognition in Uncontrolled Environment

This paper presents a novel method of facial image representation for face recognition in uncontrolled environment. It is named as augmented local binary patterns (A-LBP) that works on both, uniform and non-uniform patterns. It replaces the central non-uniform pattern with a majority value of the neighbouring uniform patterns obtained after processing all neighbouring non-uniform patterns. These patterns are finally combined with the neighbouring uniform patterns, in order to extract discriminatory information from the local descriptors. The experimental results indicate the vitality of the proposed method on particular face datasets, where the images are prone to extreme variations of illumination. Keywords—Face recognition, A-LBP, descriptor, distance metrics, area under curve, decidability index


I. INTRODUCTION
Face recognition is one of the popular biological characteristic that is universally accepted for personal identification.In order to automate face recognition process, the biometric researchers have devised numerous methods [1]- [4].Typically, a face recognition system consists of facial image acquisition and its processing, that includes image normalization, face detection and their alignment, data representation for extraction of relevant features.Extracted facial features finally classify the facial images of the individuals.The automated process of face image is shown in Figure 1.
The applications of automatic face recognition are diverse, in a variety of civil spheres ranging from public checking systems to stringent border crossing.In these domain of applications, the face recognition challenges are partially addressed.The face recognition methods that are performing strictly in controlled environments are principal component analysis [1], linear discriminant analysis [2], Fisherfaces [3], independent component analysis [5], and many more.In some other application areas including visual inspection, remote sensing, biometrics and motion analysis, the environments are not controlled.Therefore, there is a need to devise an efficient method that is correctly recognising the individuals from their uncontrolled facial images.
In literature, the methods that work in uncontrolled environments are mainly based on texture representations.The local features-based and multi-biometric based frameworks for the face recognition have achieved much greater attention in the biometric field [7]- [9].The local features-based and multibiometric based frameworks are less sensitive to variations, such as pose and illumination than the traditional techniques.In uncontrolled environments, the local binary pattern (LBP) is one of the most popular approaches for face recognition.The intention behind using the LBP operator for facial image Fig. 1.Example of a typical face recognition system [6].representation is that: (i) The faces can be visualised as a composition of various micro-patterns and (ii) Because of its insensitiveness to variations of pose and illumination too.The final description of a face image is obtained by combining these micro-patterns [10].
The major challenges of LBP approaches are insensitiveness to monotonic transformation of gray-scale, while they are still susceptible by the variation of illumination that generate the non-monotonic gray-scale changes.LBP may not work properly on the areas of the constant gray-levels because of the thresholding schemes of the operator [11].
The remainder of the paper is structured as follows: Section II describes the review works.Section III describes the technique of facial image representation for face recognition framework.Section IV presents the experimental setup requires for computation.Evaluation of the proposed method and their comparison with LBP method are presented in Section V. Finally, the conclusions are outlined in the last Section.

A. Frameworks of Local Binary Patterns
Ojala et al. have introduced LBP operator for the study of texture of gray-scale images.It is an efficient technique for texture representation [4].It was encouraged by Ahonen et al. in recognition of human faces [11], [12].The intention behind using the LBP operator for facial description is that the faces can be considered as a composition of the various micropatterns.And, it is also found that the LBP is insensitive to the variation of (e.g., small changes of illumination and rotation).The description of the facial image is obtained by combining these micro-patterns [6].
A LBP operator which labels the pixels of an image with decimal numbers and also encodes the local structure around each pixels as illustrated in Figure 2.Each pixel is compared with its eight neighbours in a 3×3 neighbourhood by subtracting the central pixel's value as a threshold.The www.ijacsa.thesai.orgresulting non-negative values are encoded with 1 and the others with 0, as mentioned in Equation (1).A binary number is obtained by concatenating all these binary codes in a clockwise direction starting from the top-left corner and placed from left to right.Its decimal value is then computed, and used for the labelling perspective.The derived binary numbers are called LBP.The working framework of LBP operator is illustrated as follows: where P runs over 8 neighbours of the central pixel and R is the radius of the neighbourhood, g c and g p are the gray values of the central pixel and its neighbour pixels, the function s(z) is defined as thresholding function.After the LBP pattern of each pixel is identified, a histogram is computed to represent the texture image: f (x, y) = 1, x = y 0, otherwise where K is the maximal LBP pattern.The value U of a LBP pattern is defined as the number of spatial transitions in a pattern A pattern is called uniform pattern, if it has at most two bit wise transitions from 0 to 1 or vice-versa when traversed circularly.For instance, strings 00000000 (0 transition), 11000011 (2 transitions) and 11000001 (2 transitions) are uniform patterns, whereas the strings 10100001 (4 transitions) and 01010100 (6 transitions) are non-uniform patterns.Ojala et al. observed that the only 90% patterns are viable from the entire patterns when traversed in (8, 1) neighbourhood, whereas 70% are viable in the (16, 2) neighbourhood [11].Therefore, some of the information is lost by assigning all non-uniform patterns to a single bin in histogram computation.Since, there is only 58 patterns that are uniform among 256 possible patterns of size 8-bits each.This leads towards space saving while computing the LBP histograms, e.g., there are total p(p -1) + 2 uniform patterns for p bits of sampling point.A final histogram is computed by concatenating all these sub histograms that to be compared with the template histograms thereafter.

B. Variants of Local Binary patterns
In order to improve the performance of a LBP operator in the sphere of uncontrolled environments, the numerous LBP variants have been proposed in the literature are as follows: Zhang et al. proposed multi-block LBP that uses the mean pixel value of the image block to replace the other pixel values and that are finally used in creation of LBP operator [13].Hong and Wang computed Hamming distance based LBP for noise reduction from the facial images [14].Shu and Shing, developed an elongated LBP that considered the elliptical sampling for capturing anisotropic structures of the facial images [15].Ahonen and Pietikäinen developed two fuzzy membership functions rather than a single threshold to achieve the robustness in local descriptors i.e., soft local binary patterns [16].The combination of information that is received from LBP and Gabor methods has been proposed by Tan and Triggs to improve the performance [17].In addition to that, the local ternary patterns (LTP) was initiated by Tan and Triggs, to strengthen the insignificant information [18].LTP requires a parameter called a threshold constant, which defines a tolerance for similarity between different graylevels allowing robustness to noise.However, LTP and LBP representations are still limited because of its non variable quantization.
Wolf et al. developed the three-patch LBP approach that obtained by comparing the values of the three patches resulting a single bit value in the code assigned to each pixels.Furthermore, the four-patch LBP has also been suggested to compare two center symmetric patches in the inner ring with two other center symmetric patches presented in the outer ring [19].Liao et al. proposed dominant LBP (DLBP) which makes use of the most frequently occurred patterns of LBP to improve the recognition results.In addition to that, the Gabor-based features is used to supplement to the features of the DLBP [20].Heikkila et al. proposed center-symmetric local binary pattern that replaces the gradient operator which is based on fusion of strengths of SIFT and LBP operators [21].Werghi et al. has proposed a framework called mesh-LBP that is used to extract local binary patterns from a 2D mesh-manifold [22].They utilize advantages of mesh-LBP for the task of 3D texture classification of triangular-mesh surfaces collected from public datasets [23].Their results have shown an improved performance over LBP.Some of the above approaches have focused on how to make accountable the non uniform patterns for facial representations up to certain extent.Therefore, it inspires us to visualise on the non-uniform patterns too.The article concern is to make them accountable (transformed form of non uniform patterns) and combined with the existing uniform patterns thereafter, so that the discerning information can be extracted from image texture.This effort may contribute a significant role in texture classification.In order to improve performance towards uncontrolled environments, the paper proposes a new approach for facial image representations.www.ijacsa.thesai.org

III. FRAMEWORKS OF AUGMENTED LOCAL BINARY PATTERNS
This section proposes a novel method of face recognition under variable conditions is called augmented local binary patterns (A-LBP).Earlier works on LBP method have not much focused on the utilization of non-uniform patterns.Some work already have treated them as an unwanted information that are discarded during the texture representation whereas very few work used them in integration with uniform patterns as a separate bin [11], [12].The proposed approach considers the non-uniform patterns along with uniform patterns and extract the discriminatory information available to them.Thus, they prove their usefulness in distinguishing the facial patterns.The non-uniform patterns are used in combination with the neighbouring uniform patterns and extract the discriminatory information from the local descriptors.A typical schematic diagram of a proposed A-LBP face recognition method is illustrated in Figure 3.
The proposed approach uses a segment-based texture to compute the histogram that results a image signature.More formally, it replaces all non-uniform patterns with the majority value of neighbouring uniform patterns instead of putting nonuniform patterns into 59 th bin as in LBP.To do this, a kernel of size 3×3 is taken and moved on the entire image (e.g., a kernel of 3 × 3 consists a set of 9-elements and can be represented as p = {p c , p 0 , p 1 , p 2 , p 3 , p 4 , p 5 , p 6 , p 7 }, where p c represents the intensity value of the central pixel and p i (0 ≤ i ≤ 7) represents the intensity value of the neighbouring pixels).In this process, the central pixel value, (p c ) is replaced with a majority value (i.e., m = majority(p i )) of a set in case of nonuniformity (i.e., p c / ∈ U ) of central pixel.This set contains 8closet neighbours of the central pixel, wherein all non-uniform patterns are replaced with a value of 255 i.e., p i = 255.Here, 255 is a highest uniform value.Working framework of the proposed approach is illustrated as in Algorithm.
The look-up table containing decimal values of 8-bit uniform patterns are U = {0, 1, 2, 3,4,6,7,8,12,14,15,16,24,28,30,31,32,48,56,60,62,63,64,96,112,120,124,126,127,128,129,131,135,143,159,191,192,193,195,199,207,223,224,225,227,231,239,240,241,243,247,248,249,251,252,253, 254, 255} [24].The uniformity and non-uniformity of the patterns are verified from constructed look-up table which contains only a set of uniform patterns.During computations it has been found that, there are only 58 uniform patterns out of 256 patterns in 8-bit image.The values of uniform patterns in look-up table are computed, harnessing the stated principle's of the uniformity and the non-uniformity patterns (as described in Section II-A).The filtering process of non-uniform pattern to uniform pattern used in the proposed approach is demonstrated in Figure 4.
Algorithm : Computing steps of the A-LBP approach Input: Original image Filtering procedure: 1) Check the uniformity of the central pixel's value p c .
2) If p c is uniform, then go to step 1 with next p c .
3) Otherwise, construct a set N 8 containing 8-closet neighbours of p c .4) Replace all non-uniform patterns in N 8 with a uniform pattern, i.e., 255.5) Assign majority value of N 8 to p c .

A. Complexity Analysis
For analysing the complexity of the proposed method, an image is of size n × n and the kernel size is m × m.There are k distinct elements that has to be compared {1, 2,..., k} e.g., in this case the value of k is 58.Since, it has to processed each and every pixel of the image as a central pixel of the kernel, hence total number of comparisons is n × n i.e., n 2 .The complexity has been analysed in two different ways: (a) If the central pixel value matches to any value of look-up table, then it is k.n 2 .O(1), where O(1) time is required to compare the central pixel, k, the total no of times comparison happens for an element and n 2 , the total no of elements.Thus, time complexity of A-LBP method in best case is O(k.n 2 ).(b) If central pixel does not find any match with the values of the look-up table, then the following operations are taken place: (i) Process the remaining (m 2 −1) elements by matching them with the k elements.Thus, time requires = O(m 2 − 1).k, (ii) For finding the maximum occurrence in the (m 2 −1) elements, time requires = O(m 2 − 1), and (iii) Now, replace the element with maximum occurrence with the central pixel, the time requires = O(1).Thus, time required in worst case by A-LBP method which is [O (1).k+O(m 2 −1).k+O(m 2 −1)+O(1)]n 2 www.ijacsa.thesai.org⇒ O(km 2 .n 2 ).If m n, then worst case time can be reduced to best case time i.e., O(kn 2 ).
Similarly, space requirement of the proposed method is given as follows: (i) Space requires for storing k elements is O(k), (ii) Space requires for storing the n × n image, O(n 2 ), (iii) Space requires for storing the kernel size of m × m, O(m 2 ), and (iv) The space requires to store [0, 255] value of gray-scale image is O(256) = O(1).Therefore, total worst case space complexity of A-LBP method is O(k + n 2 + m 2 ).

A. Face Datasets
This section briefs the characteristics of the challenging face datasets used in this experiment.Different face datasets available publicly are used for testing of LBP and the proposed face recognition (A-LBP) methods.These face datasets are, AT & T-ORL (ORL) [25], extended Yale B (EYB) [26], Yale A (YA) [27], and Labeled Faces in the Wild (LFW) [28].The concern of selecting these face datasets is to test the robustness of the proposed method for recognizing individuals on the presence of the different variations in the dimensions e.g., pose (p), illumination (i), facial expression (e), eye glasses and occlusion (o), in their facial images.A total of 3435 images are used to recognize 133 distinct individuals from these face datasets.During processing, image samples of each face datasets are down sized by some heuristic to reduced the computational complexity.The new sample size is represented in Table I.The system is trained for each face datasets separately, whereas the test image is selected randomly from the sample set of each individual, and then the performance is measured.

B. Performance Measures
In this experiment, the paper computes the performance of the identification systems using following metrics that include-decidability index, receiver operating characteristic (ROC) curve, and area under curve (AUC).
Decidability index is a metric used to define the separation between genuine and impostor scores distributions [29].The decidability index DI can be computed as follows: where µ G and µ I be the mean of the genuine and impostor distributions respectively, whereas σ G and σ I be the standard deviations of the above scores.
The ROC curve is a measure of performance of classification that plots the genuine positive rate (GPR) against the false positive rate (FPR), which is shown in Figure 5.The area under the ROC curve (AUC) is used to compare the performance of different methods.It is a two-dimensional visualisation of ROC curve set to assess classifiers performance.The simplest way is to compute the area under the ROC curve which is part of the area of the unit square.Consequently the value of AUC will always satisfies the following inequalities 0 ≤ AU C ≤ 1, if it computes it on normalized match scores [30].It is assume that, if the AUC is close to 1, then it can be indicated that the system performance is better; otherwise, the system performance may or may not be better.
Classification performance of the proposed face recognition method is evaluated using the chi-square (CS) distance measure.Whose description is illustrated in [24].It is also been evaluated using the new suggested distance metric i.e., Bray Curtis dissimilarity (BCD) metric for the computation of distance of peer face images, for the soundness of the A-LBP face recognition method, [31]- [33].

V. EXPERIMENTAL RESULTS
The face recognition accuracy of the proposed method, A-LBP is compared with LBP using CS distance and BCD distance metrics too, under different face datasets.The experimental results indicate that A-LBP performs better than the LBP on most of the face datasets.For ORL dataset, A-LBP achieved 95% recognition accuracy in both distance metrics, whereas LBP reports an accuracy of 92.50% and 94.52%, respectively.Similar trends are also observed for EYB, YA and LFW datasets.For EYB dataset, proposed method performs better than LBP, such as the accuracy values are reported to 81.22% and 86.45%, respectively; whereas the LBP reports to www.ijacsa.thesai.org74.11% and 81.83%, respectively.For YA dataset, proposed method reported better accuracy value of 73.33% and 71.90%, respectively in comparison to LBP accuracy value of 61.19% and 60.00%, respectively.
Similarly, in case of LFW face dataset, the reported values of A-LBP are 65.00% and 67.37%, whereas LBP reports 65.00% and 65.29%, respectively for CS and BCD distance metrics.This proves that the A-LBP face recognition method is robust against the severe illumination variation, and certain extent to p, eye glasses, and o which can be seen from the Table I.Similarly, the comparative performance of each dataset using two different distance measures are illustrated in Table II.That is an alternative way to visualise the comparative performance of the proposed approach.
The ROC curve for ORL dataset is plotted and shown in Figure 5(a).It indicates that the GPR is found highest for the proposed A-LBP method and reported value 78%; when FPR is actually zero.As FPR value increases, the GPR value also increases.For example, the GPR value is found 93% for LBP and 96% for A-LBP at the value of 5% FPR.The GPR value www.ijacsa.thesai.org is found maximum 100% at 32% of FPR.
Similarly, the results of EYB dataset is visualised in Figure 5(b).It indicates that the GPR is found highest for A-LBP method and reported value of 32%; when the FPR is actually zero.As FPR value increases, the GPR value also increases for all methods.For example, the GPR is found 62% for LBP and 82% for A-LBP at 20% FPR.The GPR is found maximum 100% at 83% of FPR for LBP and 78% of FPR for A-LBP.The A-LBP method achieves better recognition accuracy, because it is insensitive to changes such as illumination.
The ROC curve for YA dataset is plotted and shown in Figure 5(c).It indicates that the GPR is found highest for A-LBP method and reported value of 20%; when the FPR is actually zero.As FPR value increases, the GPR value also increases for all methods.For example, the GPR is found 50% for LBP and 69% for A-LBP at 20% of the FPR.The GPR is found maximum 100% at 90% of FPR for LBP and 82% of FPR for A-LBP.The A-LBP method achieves better recognition accuracy, due to its insensitiveness to the changes of illumination.
Similarly, the results of LFW face dataset is also visualised in Figure 5(d), which indicates that the GPR is found higher in A-LBP method and reported value of 15% at zero FPR.As FPR increases, the GPR value will also increase, for both methods, respectively.For example, the GPR found 35% and 41% for LBP, 39% and 50% for A-LBP at 6% and 16% of the FPR, respectively.The GPR is found maximum 100% for both methods at 98% of FPR.A-LBP method shows the marginal recognition accuracy over the LBP on LFW dataset.It shows a moderate improvement over LBP method.It is because of the proportion of non-uniform patterns is comparatively larger than the uniform patterns.As an effect, the feature descriptor of A-LBP becomes brighter that may loss the some discriminatory information.
In order to evaluate the performance of reported results for ORL dataset, AUC of A-LBP and LBP are resulted as 0.962782 and 0.962654, respectively.These values for extended Yale B dataset are achieved 0.884814 and 0.840937, respectively; while on Yale A dataset the computed values are 0.736571 and 0.694004, respectively.Finally, on LFW dataset AUC of A-LBP and LBP are resulted as 0.743165 and 0.728369, respectively.In most of the cases, A-LBP shows better AUC than LBP.Performance of A-LBP and LBP are given in Table II.
DI is a critical metric that is used to find out separation between genuine and impostor scores distributions.The DI of A-LBP and LBP on ORL dataset are found 3.1501 and 3.0752, respectively.The DI on extended Yale B dataset are found 2.1048 and 1.9643, respectively; while on Yale A dataset it has achieved values are 0.9323 and 0.6942, respectively.Finally on the LFW dataset, the reported values of DI for A-LBP and LBP are 0.8712 and 0.8463, respectively.In all the cases, on the basis of DI values, it can be concluded that the A-LBP face recognition method perform better than LBP, which can be verified from the results depicted in Table II.

VI. CONCLUSION
The paper has presented a novel method of facial image representation for face recognition in uncontrolled environ-ments.This method is called as augmented local binary pattern (A-LBP).It works on the combination of principles of locality of uniform and non-uniform patterns.It can be used with combination of neighbouring uniform patterns to extract discriminatory information from the local descriptor.
Comparative results analysis of face recognition methods has been performed, using following publicly available face datasets, such as AT & T-ORL, extended Yale B, Yale A, and Labeled Faces in the Wild.It has been computed using chisquare distance and Bray Curtis dissimilarity measures too.Proposed A-LBP method has efficiently recognised faces from their wild face datasets.The experimental results have indicated that the performance of the A-LBP method has improved substantially with respect to LBP on different experimented datasets, particularly when there is extreme variations in the illumination of facial images in the dataset (e.g., extended Yale B and Yale A).
In future, more exploration can be made to use the nonuniform patterns in addition to uniform patterns.The majority criterion to be experimented on the values of (8, 2) and (16,2) neighbourhoods in the direction of the robustness of the A-LBP frameworks.It could be further extended to include region-wise weights based on the probability of occurrences of the non-uniform patterns in particular regions.

Fig. 3 .
Fig. 3.A typical schematic diagram of A-LBP face recognition system.

Fig. 4 .
Fig. 4. (a) Neighbouring non-uniform patterns are replaced with highest uniform pattern i.e., 255, (b) Central non-uniform pattern 36 is replaced with majority value of 32, and (c) The next central pattern 64 is found to be uniform, so, it remains unchanged.

Fig. 5 .
Fig. 5. Receiver operating characteristic curves showing the comparative performance of LBP and A-LBP face recognition methods under chi square distance and Bray Curtis dissimilarity metrics on various publicly available face datasets: (a) ORL, (b) EYB, (c) YA, and (d) LFW.

TABLE I .
CHALLENGING FACE DATASETS USED IN THIS EXPERIMENT.

TABLE II .
A COMPARISON OF RECOGNITION PERFORMANCE BETWEEN LBP AND A-LBP FACE RECOGNITION METHODS ON DIFFERENT FACE DATASETS UNDER VARIOUS DISTANCE MEASURES.