Pakistan Sign Language Detection using PCA and KNN

Every society has a large majority group of disable people. The technology is developing day by day but no significant developments are undertaken for the improvement of these people. Sign language is an efficient mean of information exchange with special people, such as Deaf and Dumb people, they communicate with each other through sign language, but it become difficult when they communicate to outer world so sign language is used for this purpose. Different research has been done for this in America, Indonesia and India, but not much work done in Pakistan. In this research paper, author introduce a system for recognizing Pakistan Sign Language (PSL) including the alphabet to facilitate communication between special people and normal. This system capture input through webcam without making use of any additional hardware, then using segmentation approach we separate hand from the background and extract required feature from image using Principal Component Analysis (PCA) and then finally classifies the gesture feature by utilizing K Nearest Neighbors (KNN). This research will fill the communication gap between the deaf and normal people of the Pakistan country. Keywords—Deaf and dumb; hand gesture recognition; knearest neighbors; Pakistan sign language; principal component analysis; Urdu alphabets


I. INTRODUCTION
Every society has a large maniority group of disable people.The technology is developing day by day but no significant developments are undertaken for the improvement of these people.Sign Language is an efficient mean of information exchange with special people like Deaf and Dumb, they communicate with each other through sign language, but it become difficult when they communicate to outer world so Sign Language is used for this purpose.
Different research has been done for this in America, Indonesia and India because every country has its own sign language even it's also vary from region to region.So Pakistan also has its own sign language for communication with special people but not much work has been done in Pakistan.For PSL detection a system is required.
In this paper we introduce standard Pakistan sign language (PSL) detection System through hand gestures which is implemented in MATLAB.The main contribution of this research is to design system for standard Pakistan Sign Language for Urdu Alphabets using PCA and k-Nearest Neighbors algorithm.PCA is used to reduce data dimensions.
PCA takes high dimension data and convert it into lower dimension data.It finds a new set of variables, which is smaller than the original set [7].In this research main focus will be on accuracy then the speed.
This system capture input image through webcam without making use of any additional hardware, or gloves which is being used by earlier sign detection system, then use different image processing technique for feature extraction and finally identify the gesture and display letter against that gesture using some classification method.The complete working flow of purposed system has been shown in Fig. 1.The prototype designed during this research will be introduced in "Govt.Special Education Centre, Jaranwala" having strength more then 150.
The paper is divide in the following sections.Section 2 talks about proposed model for Pakistan sign language Urdu alphabet identification based on hand gesture recognition using neural networks.Section 3 consist of the results obtained by experiment on different input.Section 4 deals with Conclusion and future.

II. METHODOLOGY
The basic idea behind the proposed model is to identify PSL Urdu alphabets depending on hand gesture captured by webcam.The work flow of proposed system is shown in Fig. 2. www.ijacsa.thesai.org

A. Image Preprocessing
Fig. 3 shows steps involve in image processing.In first step image is provided as input.In order to take input from the webcam a system is implemented using MATLAB.Then this image goes to image pre-processing steps in which segmentation and image conversion is performed, the RGB image is converted into grayscale image i.e.BW image for processing, because grayscale is more preferable.The function rgb2hsv is being used for conversion [2].Then skin region is extracted by calculating skin probability, after that thresholded region is computed.Threshold is simplest way of segmentation in which conversion of grey scale image to binary image is performed.In computer vision, segmentation or image segmentation is a process of partitioning an image into multiple segments (sets of pixels).Segmentation can also be defined as a process of separating the required region from its background.In PSL as hand gesture is used, so segmentation involve separating hand region from the background.After performing Segmentation, the hand region is extracted as white colour and other regions of image are assigned black colour.Results of skin thresholding during prepressing step has been shown in Fig. 4.

B. Feature Extraction
The binary images obtained after performing some processing like Skin extraction are used to extract the features using Principal Component Analysis.Feature extraction is a process in which dimension of image is reduced.As we are working on gesture the no of frames are too large to processed, so input of image pre-processing step is reduced here.Different feature extraction procedures available such as HOG [4], PCA [5], Fourier De-scripters [3], etc.
In our case we will make use of PCA algorithm for feature extraction which will return output in the form of feature weight Matrix.Step 5: We find the weights of the images in this transformed space with respect to the Eigen Vectors Vi.This is calculated as: Usually Euclidean distance is used as the distance metric in KNN.In our experiment, classification is performed by finding the nearest Euclidean distance of the transformed weights of the test image.Distance using Euclidean function is calculated using following equation:  For displaying URDU words we implemented URDU Optical character recognition system, which make use of Unicode for this purpose, such as Unicode of ‫ب‬ is "0628" [8] as shown in Fig. 7.

III. IMPLEMENTATION AND INTERPRETATION OF EXPERIMENTAL RESULTS
The efficiency of the PSL detection system is evaluated by testing both training and testing data set.Standard PSL have been shown in Fig. 8.The System has been implemented in MATLAB R2015b using PCA and KNN algorithm.The database being used in Pakistan sign language detection system is self-created because it's not available.Database of PSL alphabets is created by videos which are captured by web cam.We have used 30 image per alphabet so our database contain 38x30= 1140 images, as there are 38 Urdu alphabets.Two data set are being used one for testing and other for training.There are 140 images in Testing dataset and 1000 images in Training dataset.These images are taken from different distance and from different angle for checking system performance and its efficiency.Testing database is used to test system with possible inputs, it's also used to test to evaluate the performance of Training dataset.For the performance evaluation, the system also has been tested multiple times on testing and training data set by more than 5 user.The proposed also system has been tested by different user using webcam having different skin type, color and gender, for test database data has been collected from people which belongs to different age.
The purposed prototype has been deployed in "Govt.Special Education Centre, Jaranwala" for testing, and it is tested nearabout 50 special children of different age , having different skin color.www.ijacsa.thesai.orgAfter testing system fom different cases the avegare accuracy is 85% , but for some Urdu alphabet the accuracy was 100% like " ‫","الف‬ ‫ب‬ " but we get low accuracy rate when dealing with Urdu alphabet like ‫غ"‬ ".As some of the Pakistan signs are almost similar it was very difficult to identify such as ‫غ"‬ " and ‫ف"‬ " shown in Fig. 9 below.

IV. CONCLUSION AND FUTURE WORK
In this research paper, Pakistan Sign language (PSL) detection system for Urdu alphabets has been developed using MATLAB.The aim of this system is to help disabled persons living in Pakistan to communication with other.The hand gesture were recognized by using PCA and KNN classifier giving 85% accuracy rate for test data set which contain more than 1000 images and 80% when tested through webcam.
This system can further be carried out for English alphabet for single and double handed of PSL and will also improve accuracy rate and also will introduce voice fature for text to help blind one.A mobile base application for this purpose will also be designed because now a days the use of mobile phone is increasing rapidly.

1 )
Principal component analysis (PCA) PCA is used to reduce data dimensions.PCA takes high dimension data and convert it into lower dimension data.It finds a new set of variables, which is smaller than the original set.It's maintain sample important information when extracting relevant and required information from dataset.It's useful for compression reduction and classification of data [6].PCA algorithm steps in PSL detection: Step1: Obtain images I1, I2, Im images for training.Step2: subtract the mean Ai = Γi -Ψ Step 3: Compute Covariance matrix as C = ATA.This is of size M x M. Step 4: We find the best Eigen vectors from this covariance matrix R. The Eigen vectors Ui we find from this correspond to the M best Eigen vectors Vi of the C matrix.And it can be shown that these two Eigen vectors are related by Vi = A * Ui.These are the Principal Components of the Image Matrix A. Fig. 5 shows the Eigen vectors extracted using PCA of URDU alphabet.


Using this distance calculation and KNN classifier Urdu alphabet for the give sign is displayed.Fig.6shows the output after KNN classifier.