Arabic Alphabet and Numbers Sign Language Recognition

This paper introduces an Arabic Alphabet and Numbers Sign Language Recognition (ArANSLR). It facilitates the communication between the deaf and normal people by recognizing the alphabet and numbers signs of Arabic sign language to text or speech. To achieve this target, the system able to visually recognize gestures from hand image input. The proposed algorithm uses hand geometry and the different shape of a hand in each sign for classifying letters shape by using Hidden Markov Model (HMM). Experiments on real-world datasets showed that the proposed algorithm for Arabic alphabet and numbers sign language recognition is suitability and reliability compared with other competitive algorithms. The experiment results show that the increasing of the gesture recognition rate depends on the increasing of the number of zones by dividing the rectangle surrounding the hand. Keywords—hand gestures; hand geometry; Sign language recognition; image analysis; and HMM


INTRODUCTION
The main problem in the language of deaf people makes it difficult to translate thoughts and feelings into words and phrases understandable and aware. The normal people translate ideas into words audible, but the deaf people translate ideas into visual signs through the hands movement.
Over the years used the deaf and dumb signs among themselves. It became the different sign language of each community in the world. These signs are only one thing for the deaf and dumb to communicate with each other and the outside world to them [1]. There has been a growing interest in the recognition of human hand movements. Normally, there is no problem when deaf persons communicate with each other by using their common sign language. The problem appears when a Deaf people want to communicate with a non-deaf people. Usually, both will be disgruntled in a very short time [2].
A Sign language is different from country to another country; the researchers attempt to unify the sign language in each country separately have been carried out such as Jordan, Saudi Arabia, and Egypt to help persons of the deaf and dumb for each country. The researchers are working on hand gestures in different sign languages as the Australian Sign Language (Auslan) [3], the Chinese Sign Language (CSL) [4], the American Sign Language (ASL) [5], and the Dutch Sign Language. The Arabic Sign Language has not received attention in researchers [7].
All Previous researches on sign languages depend on glove or vision based methods [6]. In the glove based method, the user wears special devices, like special gloves or markers, the system related with data on the hand shape and motion. While in the vision method, the system recognizes the gestures by using image processing techniques without putting any limitation on the user [7].
The work [2] created an automatic translation system for gestures of manual alphabets in the Arabic sign language recognition. It does not rely on using any visual markings or gloves. The extracted features depend on two stages, featurevector-creation stage and edge detection stage. It used multilayer perceptron (MLP) classifier and minimum distance classifier (MDC) to detect 15 characters only of 28 characters.
The research work in [7] investigated appearance-based features for the deaf person-vision-based on sign language recognition. It does not depend on a segmentation of the input images, and he used the image as a feature. The system used a combination of features including PCA, hand trajectory, hand position, and hand velocity. The grey scale image with a reduced frame size 195x165 pixels and downscaling to 32x32 pixels used rwth-boston-104 database.
A system of the recognition and translation of the numbers was designed [8]. The system is consist of four main phases; pre-processing phase, feature extraction phase, interpolation phase and the classification phase. The extracted features are scale invariant and make the system more flexible. The experimental result revealed that the system was able to recognize a representing numbers from one to nine based on the minimum Euclidean distance between the numbers.
The research work in [9] introduced new two features for American Sign Language recognition: those are kurtosis position and principal component analysis PCA. Principal component analysis is used as a descriptor to provide a www.ijacsa.thesai.org measure for hand orientation and hand configuration. PCA has been used before in sign language as a dimensionality reduction. As a local feature for measuring edges and reflecting the position, Kurtosis position is used. It used motion chain code that represents the movement of hand as a feature. The system input is a sign from RWTH-BOSTON-50 database, and the recognition error rate of the output is 10.90%.
A system for the recognition and translation of the Arabic letters was designed [10] . The system depends on the inner circle position on the hand contour and divides the rectangle surrounding by the hand shape into 16 zones. The extracted features are scale invariant. Experiments revealed that the system was able to recognize Arabic letters based on the hand geometry. The experiment results shown that the different signs gesture recognition rate of Arabic alphabet for were 81.6 %.
The research work in [11] used Adaptive Neuro-Fuzzy Inference system (ANFIS). The system used 30 Arabic sign language alphabets visually. The recognition rate of the system was 93.55%. The research work in [12] built an ArSL system and measures the performance of ArSL data collected. The system based on Polynomial Classifiers. It collected a 30 letter of ArSL. It collected the data by using gloves marked with six different colours at different regions as shown in Fig. 1 [12]. The recognition rate is 93.41 % This paper is organizes as follows. Section two explains HMM classifier. Section three presents the proposed system. Section four shows the experimental data. Section five explains the experimental results. Section six presents the conclusions.

II. HMM CLASSIFIER
HMM is used as a classifier for speech [13] and used in sign language recognition systems. In HMM-based approaches, the information of each sign is modelled by a different HMM. The model that gives the highest likelihood is selected as the best model and the test sign is classified as the sign of that model [14]. It consists of a set of N states where the transition from each state to another state. It is denoted by Eq. 1:

A. Skin Detection and Background Removal
The algorithm uses skin detection [15]. The algorithm adopts skin colour detection as the first step. Due to color space transform, YCbCr is faster than other approaches  [16,17]. The algorithm calculates the average luminance of the input image as given in Eq.4. ∑ Where is normalized to the range {0 to 255}, where i, j are the indices of the pixel in the image.
According to , the algorithm can calculate the compensated image by the following equations Eq.5 and Eq.6 [15]: It should be noted that the algorithm compensates the colour of and to reduce computation. Due to chrominance ( ) which can well represent human skin, the algorithm only consider factor for colour space transform to reduce the computation. is defined as follows Eq. 7 [17]: Cr=0.5R'− 0.419G' − 0.081B (7) Accordingly, the human skin binary matrix can be obtained as follows: Where "0" is the white point and "1" is the black point. The algorithm implements a filtration by a 5 × 5 mask.
First, the algorithm segments Sij into 5×5 blocks, and calculate show many white points in a block. Then, every point of a 5 × 5 block is set to white point when the number of white points is greater than half the number of total points. Otherwise, if the number of black points is more than a half, this 5 × 5 block is modified to a complete black block, as shown in Fig. 3 [16].

B. Face and Hand Isolating
The algorithm tracks the objects in each image. The algorithm neglected the small objects, and then detects the largest objects as hands and the face. The algorithm isolates the hand and face as in Fig. 4. After detecting the skin colour and removing background the position of the face and hands can be isolate and detected as Fig.5.  Figure 4 shows the detected skin with background removal. The image contains a right hand and a face. The algorithm detects the hand and a face by the position and shape of each. Fig.5 shows isolating the face and hands, then isolate the right hand to detect the letter.

C. Observation Detection and HMM
The proposed algorithm divides the rectangle surrounding by the hand shape in Fig. 5 into 16 zones as in Fig. 6.  Step 1: Divide the rectangle surrounding by the hand into 16 zones as Fig. 6 and Fig. 7.
Step 2: Count the number of white pixel in each zone.
Step 3: Sort the zone numbers in ascending order depends into a number of white pixels in each zone.
Step 4: The observation vector of the letter is a vector of a sorted zone numbers, numbers between 1 and 16.
Step 5: Train the HMM for each letter ( ) to maximize ( )   Row 3 16 15 14 13 Row 4 (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 6, No. 11, 2015 213 | P a g e www.ijacsa.thesai.org V. EXPERIMENTAL RESULT In this research, the HMM is applied on the ArASLRDB with 29 Arabic alphabet sign language. The recognition system is tested when dividing the rectangle surrounding by the hand shape in Fig. 5 into 4 , 9, 16, and 25 zones.
 At 16 zones: the recognition rate changes with changing the number of states until arrive to 100 % at 19 states, as shown in Fig.9.
 At 4 and 9 zones: the recognition rate cannot arrive to 100%, as shown in Fig. 9.
 At 4 zones: the recognition rate is very poor and cannot exceed 40%.
 At 9 zones: the recognition rate cannot exceed 97% whatever increases the number of states.
 At 25 zones: the recognition rate changes with changing the number of states until arriving 100 % at 18 states, as shown in Fig.9.
 The average time to execute the proposed algorithm to arrive of 100% recognition rate is shown in table 2. Finally, the best number of zones=16 with 19 states to recognition Arabic alphabet of sign language. The algorithm can achieve to 100% recognition rate with increase the zone number more than or equal 16 zones but more time is required.
As shown in table 3, Reference [2] used minimum distance classifier (MDC) and also used multilayer perceptron (MLP) classifier to detect 15 characters only of letters with recognition rate 91.7 % and 83.7 % respectively. Reference [11] recognized Arabic letters based on the hand geometry and the recognition rate of Arabic alphabet for different signs was 81.6 %. This system can reach a 100 % recognition rate with increasing number of zones and number of states. Reference [13] used Gloves marked with six different colour, the system used polynomial classifiers to recognize 30 letters with recognition rate of 93.41 %. Reference [12] did not use gloves and used ANFIS to recognize 30 letters by recognition rate of 93.55 %.

VI. CONCLUSIONS
In this paper, a new feature is used to recognize the Arabic Alphabet sign language via HMM. The proposed system is demonstrated experimentally. The phases of the proposed algorithm consists of skin detection, background exclusion, face and hands extraction, feature extraction, and also classification using Hidden Markov Model (HMM). The proposed algorithm isolates the hand from the image to recognize the letter. The proposed algorithm divides the rectangle surrounding by the hand shape into zones. The best number of zones is 16 zones. The observation of HMM is created by sorting zone numbers in ascending order depending on the number of white pixels in each zone. Experimental results show that the proposed algorithm achieves 100% recognition rate with minimum execution time at 16 zones with 19 states.