A Novel Permutation Based Approach for Effective and Efficient Representation of Face Images under Varying Illuminations

Paramount importance for an automated face recognition system is the ability to enhance discriminatory power with a low-dimensional feature representation. Keeping this as a focal point, we present a novel approach for face recognition by formulating the problem of face tagging in terms of permutation. Using a fundamental concept that, dominant pixels of a person will remain dominant under varying illuminations, we develop a Permutation Matrix (PM) based approach for representing face images. The proposed method is extensively evaluated on several benchmark databases under different exemplary evaluation protocols reported in the literature. Experimental results and comparative study with state-of-the-art methods suggest that the proposed approach provides a better representation of face, thereby achieving higher efficacy and lower error rates.


I. INTRODUCTION
Facial Recognition, one of the biometric techniques is a form of computer vision that uses faces to identify a person or verify a person's claimed identity.The main attraction for Face recognition research over other biometric techniques is that it is nonintrusive in nature.This has paved path to several applications like security, access control, surveillance system, human computer interface, recognition in galleries and consumer devices.Significant amount of work is carried out by Research Scientists and Engineers in the field of Face recognition and is still growing exponentially.In spite of these efforts, studies reveal [1,2] that there is no single state of art face recognition system.The main reason quoted is that applications in this area have varying requirements and constraints.Therefore there has been always growing interest for developing newer computational algorithms that can serve as a model for human face recognition function.
Recognition of faces primarily involves collection of descriptive measurements known as feature vectors extracted from each training set of images.These can be facial components, such as eyes, nose, and mouth and facial outline [30].Feature extraction is performed to provide effective information that is useful for distinguishing between faces of different persons and to be stable with respect to the geometrical and photometrical variations.
Face Recognition has been tackled in two modes, one focusing on generic face recognition algorithms and other on specific face recognition algorithms.Generic face recognition algorithms should work well with databases having complex conditions like combination of pose, illumination and expressions variations.Some of the example methods experimented in this direction are Singular Value Decomposition (SVD), Linear Descriminant Analysis (LDA), Locality Preserving Projections (LPP), Hidden Markov models (HMM), Gaussian Mixture Model (GMM) [3 -17].These systems work on extracting face manifold in a subspace.On the other hand, specific face recognition method requires preprocessing of database to meet a specific condition, keeping other conditions invariant.Spherical representations have been used for modeling illumination variations or both illumination and pose variations in face images [18].Mian et al. [19] have handled the expression problem using a fusion scheme in which three kinds of methods, spherical face representations, Automatic preprocessing of 3D Spherical Face Representation (SFR), Scale-Invariant Feature Transform (SIFT)-based face data matching and a modified iterative closest point(ICP) have been combined to achieve the final result.Their results have showed the potential of appearance-based methods for solving the expression problem [20].The models for illumination range from highly specular objects such as mirrors to models for matte objects.Most objects belong to the latter category, which is described by a Lambertian reflectance model.Early shape from shading approaches assumed a constant albedo field.But they have shown to be violating [21] at locations such as eyes and edges of mouth.For the human face, the Lambertian reflectance model with a varying albedo field has been shown to provide a reasonable approximation.
Pose variation is essentially a correspondence problem.With an assumption of Lambertian reflectance model, dense correspondences across poses are available with a rank-1 constraint implied.Recovering a 3D model from 2D images is a difficult task.Two types of approaches have been around: [22][23][24][25] one using model-based and the other using the imagebased approach.Explicit [26,27] knowledge of prior 3D models have been used in Model-based approaches.Imagebased approaches have not used prior 3D models.In general, model-based approaches register the 2D face [28] image to 3D models that are given beforehand.www.ijacsa.thesai.orgII.PERMUTATION MATRIX -AN OVERVIEW PM is a square binary matrix that has exactly one entry 1 in each row and each column and 0's elsewhere.Each such matrix represents a specific permutation of m elements and, when used to multiply another matrix, can produce that permutation in the rows or columns of the other matrix [33].
PM is obtained by reordering the rows of an identity matrix I.A typical PM looks like It is obtained from 4X4 Identity matrix by cyclic permutation of the columns C1->C2->C3->C4->C1.
In LU factorization problem the PM has been very useful.In case of LU factorization without pivoting, A = LU, where L unit lower triangular, U upper triangular.This may not always exist even if A is nonsingular.In the case of LU factorization with row pivoting, A = PLU, where P is the PM, L is the unit lower triangular and U is the upper triangular matrices.This exists for all nonsingular A's.
The product PA is the matrix A with its rows reordered in exactly the way as the rows of I were reordered to produce P. Well written software for solving systems of linear equations chooses the pivot elements carefully to ameliorate the bad effects of round off error.The natural order of the rows gives way to an ordering that pays attention to the magnitudes of the elements available as pivots.There is a necessity to choose strong pivot elements to avoid growth of round off errors.Applying P reorders or permutes the rows of A so that the reordered matrix has a factorization

A. Introduction
The method proposed in this paper is based on generation of a PM.It captures the dominant values in the image matrix thus suppressing the less dominant values.The idea is that, between two samples of the same person images relativity is maintained in dominant places.Even if the variations intensity values do occur, it is usually found that the dominant values continue to be dominating.This is found after an exploration is done on the data set at hand.The data sets have several kinds of variations and in spite of these; variations found at leading positions are not much.The algorithm scans the columns for the leading and non-leading elements and assigns weights for them differentially.The PM thus generated is binarized.The advantage with such a matrix is that it is sparse in nature.

The mathematical representation is given below:
Let the input square matrix A of order n be represented as A= The notation A (i, j) denotes the element located i th row and jth column of A.

B. Sequence Generation
Let P={p1,….,pk}be the set of permutation matrices corresponding to the gallery image set G. Each image in the gallery set is a square matrix with a resolution of nXn and k is the number of images in the gallery set.For each p € P, generate sequence vectors S={s1,….sk},where each s € S is a sequence vector generated from non-zero entries of PM of the image.The value s generated is the max (column of p) for each column excluding earlier considered row and column values.Similarly, the permutation matrices and the sequences are generated for the set Q, the probe set.Let the set PQ={pq1,….pql}represent the sequence set of the probe image after quantizing the probe with application of permutation.Finally the tagged images consisting of the permutation sequences are compared.
The computation of similarity score Si(n) is carried out as follows: Given any probe q  Q , it is first mapped it onto the feature space.Formally, we have x = F(q)  Rd , where x is the feature vector extracted from the probe p through the feature extraction function F(), and Rd is the d-dimensional feature space.Similarly the feature vector of every image gi in gallery G is extracted through F().Finally, the probe set Q is scored against the gallery G by comparing similarity scores Si(n) based on F(Q) for each qi  Q and gn  G.We assume that smaller similarity score implies closer match.Thus, the objective of a permutation based face recognition task is to determine the identity of q, i.e., ID(q), where ID(q) ∈ ID{g1, . . .,gK } using the concept of PM.
For each PQ find the distance between gallery's permutation sequence and the probe sequence and the result is represented as set R={r1….rk}.We then compute d= min(R), the min distance between the probe and gallery.
If the d is below the threshold , the probe is accepted as a match otherwise marked as unknown.The image score is not only checked against the top score but also tested for other ranks where rank L represents the image match in first L scores.The one which yields few permutations will be marked as the matched one.The limiting case for the permutation in most cases is zero.The block diagram in

D. Computational Complexity
The number of comparisons required for generating PM is: ) ( Where n is the order of the P matrix The reduction in number of comparisons per image using PM when compared to n2 algorithms is From a computational complexity point of view, the PM method has a saving in cost of efficiency by a factor of 49.5% for a matrix size of 100X100.Higher the resolution of the image, higher will be the efficiency.The permutation sequence generated from PM has size n.The advantage of this method is that the sequence generation of an image doesn't need the knowledge of database images and is produced on the fly.

A. Experimental Setup
Extensive experiments were carried out to investigate the efficacy of our proposed approach for face recognition.Essentially, six benchmark databases namely, ORL [35], Face 95 [37], Face96 [37], Yale [34], Pose [36] and Grimace [37] have been utilized for experimentations.Fig. 2 shows the selected representative subjects in the six databases.These databases incorporate several deviations from the ideal conditions, including pose, illumination, occlusion, and gesture.
Several standard evaluation protocols reported in the face recognition literature including statistical metrics, CMC curve and ROC plots have been adopted.We compare our algorithm with four state-of-the-art baseline techniques for face recognition namely SVD [29,3], LDA [6], LPP [31] and ICA [32].It is appropriate to indicate that the developed approach has been shown to perform well for the cases of severe illumination and expression with little change in pose, scale and rotation.The following table 1 gives a summary of these databases.Organization of database is one of the important steps in experimentation setup.In this direction, the databases are organized into two groups, gallery and probe.Thus, all experiments were performed with two experimentation protocols.The first protocol consisted of one mage per subject in the gallery with two different resolutions and the second protocol was similar to first except that, it contained five images per subject in the gallery set. .There was no overlap between the gallery and probe sets.Few subjects were used for unknown set.The size of gallery and probe for each of the databases under different experimental configurations is tabulated below.Original images were normalized and were resized to 100X100 pixels and 180X180 pixels for the all databases.Further all images were formatted with 256 gray levels per pixel.

B. Features Extraction
Feature extraction is carried out using four predominant existing approaches namely SVD, LDA, LPP, ICA and our newly proposed PM approach.The features are extracted from the gallery set and probe set.The face representations obtained from these techniques are illustrated below:  SVD: Aims at identifying a lower dimensional space maximizing the variance among the data.give information on the data that is otherwise hidden in the large data set.

C. Recognition Accuracy
After having extracted features from the gallery images and probe images, recognition is possible with relatively little computational effort by comparing sequence vectors.The difference between probe sequence and gallery image sequence is found and the number of zeros is counted.More the number of zeros, closer is the match.This gives the similarity measure in our case.The experiments were carried out in Matlab 7.5 on a 32 bit Core 2 Duo, 2 GHz processor, with 8 GB RAM.The recognition rates we have been able to obtain with the different methods including PM are reported in table 3 to 6.Consistent results have been obtained using PM.The relative advantage of PM based over other techniques is quite evident from the recognition accuracy, error rates and other trivial metrics.The proposed PM approach copes well with the problem of illumination and expressions in the presence of small pose variations, achieving high recognition rates of 95.22%.The PM approach, however, shows slight degradation in recognition rates by 2% to 3% for the severe pose variations.It is closer to SVD in terms of accuracy and substantially performs better than LDA, LPP and ICA.PM method is simpler, has higher recognition speed and smaller memory requirements.A key success of PM approach is the unique way of extracting relevant information.More precisely, it exploits only dominant features.

D. Variation Of Gallery Set
Table 3 to 6 shows the results of varying the number of images per subject used in the gallery set.It is evident that LDA and LPP are more sensitive to the particular choice of the gallery set.In other words, both LDA and LPP suffer from Small Sample Size (SSS) problem which makes them sometimes perform poorer.Apart from this LPP's performance is sensitive to the values for the neighborhood size and the similarity matrix.Thus, PM based approach like SVD do not suffer from SSS problem.

E. Train and Test Time Requirements
In this experiment, the time requirements of each method are discussed.Fig. 10 and 11 gives the runtime for training and testing phase for various methods on Face 95 and ORL databases respectively.It can be seen that PM, SVD and LDA have minimal time requirements in terms of training and testing.However, time requirements for LPP method are slightly higher because of its computational complexity.

A. CMC and ROC Plots
To measure respectively the performance of the proposed method for verification and identification tasks, we present both ROC curves and CMC curves for the results obtained in this paper.
The CMC curve for SVD, LDA, LPP, ICA and PM based face recognition on Grimace database is plotted in figure 12 (a) and the corresponding match score for rank 1 to 17 are tabulated in the table 8. Based on the CMC curve and tabular values, it is evident that PM has faster convergence with 94% at Rank 1 and 100% accuracy at Rank 9. Similarly, the CMC plot for the five methods on Face 95 is illustrated in figure 12 (b).The corresponding match score is tabulated in the table 9.The results obtained evaluate here, the PM converged with 100% at rank 9. We plot CMC and tabulate results obtained on ORL database against these methods in figure 12 (c) and table 10 respectively.Here, the convergence is still faster with full accuracy at Rank 5. Figure 12 (d) and (e) illustrates the CMCs plotted against the POSE and YALE databases with respect to the five methods.Ranks scores for POSE and YALE have been tabulated in table 11 and 12 respectively.The convergence rate with 100% accuracy is always faster with PM method which is evident from the test conducted on several databases.The ROC plots for different methods are depicted in the Fig. 13.The area under ROC is statistically greater than 0.5 for PM, SVD, LDA and some cases for LPP.The blue circle indicates the cut-off point for best Sensitivity and Specificity.The cut-off point for PM varies from 0.9 to 1.0 which is an indicator of effectiveness of the method used.In all of the test cases the plot is well above the guess line and reaching to almost top left corner as desired on some databases.The cutoff point indicates the algorithm's strength in identifying true and false positives.

B. Contingency Table
Face Recognition is a pattern classification problem, in which the outcomes are labeled either as positive (p) or negative (n).There are four possible outcomes namely, if the outcome from a classification is p and the actual value is also p, then it is called a true positive (TP); however if the actual value is n then it is said to be a false positive (FP).Conversely, a true negative (TN) has occurred when both the classification outcome and the actual value are n, and false negative (FN) is when the classification outcome is n while the actual value is p.We define two experiments from P positive instances and N negative instances on Face95 and ORL databases.The four outcomes are depicted in a 2×2 contingency table or confusion matrix, as shown in Fig. 14.Typically, a good biometric system must have low error rate, FPR and should have higher TPR, TNR, PPV, NVP, ACC.From the contingency table (Fig. 14), various metrics (Table 13 and Fig. 15), we can infer that the proposed technique satisfies these requirements.Apart from these metrics, many performance evaluation metrics like F-Score and Gain have been stated as standard metrics in many literatures.Table 14 depicts the score that we have been able to achieve for these metrics.

VI. CONCLUSION AND FUTURE ENHANCEMENTS
In summary, PM based approach seems to offer satisfactory results for face recognition under varying illumination, expression and pose to certain extent.The results obtained have been verified on larger and benchmark databases.A more important problem that we did consider here is the reliable rejection of images of faces not contained in the data bases.There are many possible ways to improve the robustness of our scheme, for instance, by expanding the set of faces images in the gallery with all possible face variations.Considerable comparative analysis with the state of-the-art algorithms clearly reflects the relative potency of the proposed approach.The proposed PM approach reveals a number of interesting outcomes namely, faster convergence rate of recognition, reduction in feature vector size and saving in time complexity for generation and transmission of feature vector.Considering its efficacy at low-subspace dimensionality and its relative simplicity, the PM based approach is a highly effective subspace modeling technique for surveillance systems.The error rate is well below 0.1 which indicate the robustness of the algorithm proposed and is quite satisfactory.
Future work is suggested towards working on local permutation matrices instead of global holistic one.Yet another dimension would be to look at newer ways of generating permutations.The research can also be directed towards assigning differential weights to image matrix.
Fig 1 depicts the overall process.The proposed method has been compared against SVD, LDA, LPP and ICA techniques that are being used for face recognition.C. Pseudocode colMax = [] #Our 1xn feature vector for i = 0 to n: #Number of iterations maxEntry = -1 for r = i to n: #row for c = i to n: #column if maxEntry < A[r][c]: #Get index of column with biggest entry.maxEntry = A[r][c] colMax[i] = c return colMax[] #resulting feature vector for image, A. www.ijacsa.thesai.org

Fig. 1 .
Fig.1.Block Diagram of the proposed method

Fig. 3 .
Fig.3.Sample Eigen Faces  LPP: Aims at finding linear transformation-preserving local structure information of original data.

Fig. 4 .
Fig.4.Sample Laplacian Faces LDA: Objective is to identify a lower dimensional space minimizing the interclass similarity while maximizing the intra-class similarity simultaneously.

Fig. 5 .
Fig.5.Sample Fisher Faces  ICA: A function from an m-dimensional space to an ndimensional space such that the transformed variables

Fig. 6 .
Fig.6.Sample ICA Faces  Permutation: Aims at identifying dominant intensity values which are likely to be dominant even under variations in two samples of images.

Fig. 10 .
Fig.10.Test and Train Time for Different Methods on Face 95 Database

Fig. 11 .
Fig.11.Test and Train Time for Different Methods on ORL Database V. PERFORMANCE EVALUATION

TABLE I .
Summary of Benchmark Database

TABLE II .
Size of gallery, probe and unknown sets for different databases with five images per person in Gallery

TABLE IV .
Recognition rates (%) of SVD, LDA, LPP, ICA and PM based methods on query images of 180 X 180 resolutions and with 1 image per subject in gallery

TABLE V .
Recognition rates (%) of SVD, LDA, LPP, ICA and PM based methods on query images of 100 X 100 resolutions and with 5 images per subject in gallery

TABLE VIII .
TOP 17Rank Scores of PM, SVD, LDA, LPP and ICA on Grimace database

TABLE XII
True Positive vs. False Positive Rate of SVD, LDA, LPP, ICA and PM

TABLE XIII .
Various Biometric Metrics Computed for PM on Face95 and ORL databases Fig.15.Graph indicating various metrics for PM on Face 95 and ORL Databases.

TABLE XIV .
Trivial Biometric Metrics Computed for PM on Face95 and ORL databases