Communicator for Hearing-Impaired Persons using Pakistan Sign Language ( PSL )

Communication with a hearing-impaired individual is a big challenge for a normal person. Hearingimpaired people uses hand gesture language (sign language) to communicate with each other, which is not easy to understand by a normal person because he/she is not trained to understand sign language. This communication gap between a hearing-impaired and a normal person created big problem for hearing-impaired individuals during their shopping, hospitalization, at their schools and homes. Especially in case of emergency, it is very difficult to understand the statement of a hearing-impaired one’s who uses sign language. In the last few years researchers and developers from all over the world presented different ideas and works to solve this problem but no such solution is available to resolve this issue and can create two-way communication between hearing-impaired and normal persons. This paper presented a detail description about a two-way communication system based on Pakistan Sign Language (PSL). This duplex system is developed through conversion from the text in simple English into hand gestures and vice versa. However, conversion from hand gestures is available not only in text but also with voice providing more convenience to normal person. Main objective is to facilitate a large population and making hearingimpaired persons, the vital part of our civilization. A normal person can enter the text (sentence) in application, after the checking of spelling and grammar, the text is divided into tokens and sub-tokens. A token is a gesture against each word of the text while sub-tokens are the gestures of each character of the words. The combination of tokens created the gestures of text. On the other hand when gestures were input in to the application, using image processing technique, the nature of hand gesture were recognized and converted into corresponding text or voice. KeywordsCommunicator; hearing-impaired; Pakistan Sign Language (PSL); hand gesture; special person; token


I. INTRODUCTION
Deaf person (hearing-impaired) uses hand gestures as a basic language (sign language) for the purpose of communication with normal-hearing persons.Normally it is difficult to understand this sign language for hearing-persons without proper training and it creates a big gap between hearing-impaired and normal-hearing persons.The proposed application is a dual mode application that can be used as an easy and proper communication between them.This duplex system is developed through conversion from the text in simple English into hand gestures and vice versa.However, conversion from hand gestures is available not only in text but also with voice providing more convenience to normal person.Main objective is to facilitate a large population and making special persons, the integral part of the society.The system "communicator" is based on Pakistan Sign Language (PSL).In this application a normal person can enter the text (sentences) in application, after the checking of spelling and grammar, the text is divided into tokens and sub-tokens.A token is a gesture against each word of the text while subtokens are the gestures of each character of the words.The combination of tokens created the gestures of text.On the other hand, when gestures were input in to the application, using image processing technique the nature of hand gesture were recognized and converted into corresponding text or voice.
According to R&D report of Pakistan (2012) the estimated population size of Pakistan is approximately 180.7 million [1].Due to this very highly rate of population growth in Pakistan there are several issues and health is one of the most important areas of concern [2], [3].Another fact which is published in R&D report is high number of hearing-impaired individuals in urban and rural areas of Pakistan [4], [5] as mentioned in Table I.
According to given published data the province Punjab having large number of hearing-impaired persons as compare to other provinces (Fig. 1).The main purpose to present this work is to facilitate hearing-impaired persons [6] and reduce the communication gap between hearing-impaired and normal individuals of the society [7].According to published work, it was recorded that minimum duration to identify a gestures is 6-20 fps [8].
It is a fact of society that hearing-impaired persons not feeling comfort and cannot socialize among normal persons even with other persons of family as well.According to published work [9] around 300 million persons are hearingimpaired in the world and mostly are not well-trained with sign language and it creates a big gap of communication [4].Authors in [10] also discussed the issues related to understanding of sign language.In [11], [12], some work related to electronic device, which can be used as an interpreter between normal and hearing-impaired persons.
Most of previous works were single mode that can translate hand gestures into text, but the proposed work is a dual mode, desktop based application.

II. SIGN LANGUAGE
Each country of the world having its own gesture language (Sign language) and therefore there are many development and research work have been reported in this area of research [12]- [14].This Sign language is different in different country and based on certain gestures [15].The gestures patterns are based on different arrangement of hand and fingers.In this proposed work these gestures were recognized first and then converted into certain text which can be understood by normal persons who cannot understand sign language.

III. LITERATURE REVIEW
In [16]- [18] authors discussed the importance and need of gesture based communication.They also discussed the way to improve the techniques.In [19]- [22], authors discussed the association of facial behavior with gestures, conversion of gestures into text for understanding, efficiency and accuracy of gesture recognition and web based application for distance learning and communication.

IV. PAKISTAN SIGN LANGUAGE (PSL) AND COMMUNICATOR
In Pakistan there are number of hearing-impaired institutions where PSL uses as a standard language for hearing-impaired persons.PSL is a combination of gestures patterns consist of alphabets, words and sentences [15].PSL is based on single and double handed gestures.PSL is used for the purpose of communication among hearing-impaired individuals and now it can be used as an interpreter between hearing-impaired and normal hearing persons.The presented work is a dual mode interpreter.It can convert PSL into text and for hearing-impaired persons text can be converted into gestures as well.PSL deals with both English and Urdu versions but proposed work is related with English conversion.Using communicator it might be possible that hearing-impaired and normal individuals can communicate with each other"s without any hesitation.Fig. 2 showed basic alphabets symbols of Pakistan Sign Language.
The communicator is a desktop application; a hearingimpaired person can input gestures using PSL into the system, which is converted into text/voice for normal hearing persons who did not understand language of deaf persons.On the other hand a normal hearing person can input text or voice into the system, which is translate into gestures according to PSL and easily can understand by hearing-impaired individuals.

V. METHODOLOGY
The methodology of the work is divided into two phases.In first phase authors discussed the way to translate test to gesture conversion in application and in second phase gestures to text or voice conversion has been discussed.

A. Text to Gestures Conversions
Text or voice can be input by the normal hearing person, which is converted into gestures using designed application.

VI. GESTURES TO TEXT/VOICE COMMUNICATION
Using Leap motion controller hand gestures of hearingimpaired persons can be input into the system.Leap motion controller provide hand gesture coordinate values which helps to developed algorithm to recognized gestures and converted into related text or voice as an output.The designed system is easy to portable and more accurate to recognized hand gestures.Fig. 5 describes the way conversion between gestures (input) to text (output) and the key stages and functionalities are shown in Fig. 6.

VII. KEY MODULES OF SOFTWARE
In this developed system hand gestures and text/voice can be input into the system.In half portion of screen text is mentioned and in remaining part of screen corresponding gestures are shown.In this application a user need to register in the system to use and record gestures.An admin panel is used to add, update and delete ant gestures, text or audio file (Fig. 9).

VIII. RESULTS
Table I shows the results of communicator and rate of acceptance of alphabets.To achieve high accuracy all test runs were repeated 10 times for 100 individuals (deaf-persons).Fig. 10 shows the graphical view of Table II.
Table III shows the results of gestures and text conversions.

IX. CONCLUSION
The developed application was tested on 100 deaf individuals in Image Processing Research Lab (IPRL) at Usman Institute of Technology.Each testing repeated 10 times for the purpose of accuracy.It is observed that the system is working with high accuracy in term of recognized gestures and text/voice conversion.The proposed system is a desktop based application, which is using as an interpreter between normal hearing and hearing-impaired person.In next phase of development the authors of this research work are working to develop an android based application, which will be easy to use.
Convert www.ijacsa.thesai.orgBefore conversion application, first check spelling and grammar of the provided input text.grammar check application created token and sub-token against each character and words to create proper sentences.Code:Gestures "B & How" is mentioned in Fig.3.

Fig. 4
Fig. 4 describes the flow-diagram of text to gesture conversion.

Fig. 7
Fig. 7 and 8 describes the key phases and system flow.

TABLE III .
RESULTS OF TEXT AND GESTURES CONVERSIONS