Repository of Static and Dynamic Signs

Gesture-based communication is on the rise in Human Computer Interaction. Advancement in the form of smart phones has made it possible to introduce a new kind of communication. Gesture-based interfaces are increasingly getting popular to communicate at public places. These interfaces are another effective communication medium for deaf and dumb. Gestures help conveying inner thoughts of these physically disabled people to others thus eliminating the need of a gesture translator. These gestures are stored in data sets so we need to work on developing an efficient dataset. Many datasets have been developed for languages like American sign language; however, for other sign languages, like Pakistani Sign Language (PSL), there has not been done much work. This paper presents technique for storing datasets for static and dynamic signs for any sign language other than British Sign Language or American Sign Language. For American and British Sign Languages many datasets are available publicly. However, other regional languages lack public availability of data sets. Pakistan Sign Language has been taken as a case study more than 5000 gestures have been collected and they will be made part of a public database as a part of this research. The research is first initiative towards building Universal Sign Language as every region has a different Sign Language. The second focus of the research is to explore methodologies where a signer communicates without any constraint like Data gloves or a particular background. Thirdly, the paper proposes use of spelling based gestures for easier communication. So the dataset design suggested is not affected by constraints of any kind. Keywords—Data gloves; feature extraction; human computer interaction; image segmentation; object recognition


I. INTRODUCTION
According to the reports by World Health Organization around 360 million people around the world are deaf.Noise pollution might affect a huge number of young people who are in 12-35 year age range.The sufferers use hearing aids and sign language; and other forms of educational and social support.This number surpasses 1.25 million deaf children in Pakistan, yet very few attend school.
Whenever there is a community of humans, a sign language gets evolved.Many deaf mostly learn sign language from their parents.Usually 90% of the parents of deaf are normal hearing parents.These are the children who get formal sign language education.6,909 spoken and 138 sign language are known so far [1], [2].
Several surveys have been done to address the topics related to datasets.These surveys have covered vision based human-activity recognition to detect abnormality in video streaming, full body activity recognition, action recognition pose estimation and tracking, hand gestures for human computer interaction and the sign language itself etc. Databases for gesture recognition require natural gesture set, size of set and recording of data set with multiple sensors.
However, most algorithms are dependent either on typical background, or use of sensors or wearing of special kind of gloves or dresses.Sometimes use of Kinect and sometimes some special type of camera is used to capture input.The proposed dataset uses ordinary cameras of mobile or laptop, to capture input.The videos and images in dataset also come from ordinary cameras.This results in tradeoff between efficiency of the system whereas at the same time giving flexibility to the system.
First of all, we start with evolution of sign language, then we discuss the guidelines for a good data set.Materials and methods sections provides details how the static and dynamic signs will be stored.Results section discusses how the data set can facilitate on research on static and dynamic signs.

II. EVOLUTION OF SIGN LANGUAGE
Whenever there is a community of humans, a sign language gets evolved.There are very few deaf in any society.Their source of learning sign language is their parents.Usually 90% of the parents of deaf are normal hearing parents.
Sign language is just like natural languages, they meet all social and mental functions of spoken languages.However their medium of communication differs entirely from vocal auditory spoken languages.
Here we take Pakistan Sign Language (PSL) as a case study for this paper.The same approach can be applied to sign languages other than English Language based sign languages.
We find traces of PSL initiated by Syed Iftikhar Ali [13].The dictionary of PSL created by him contained 750 signs.This was first step in this direction.2 nd contribution was by "Anjuman-e-Behbood-e-Samat-e-Atfal" (ABSA) for the development of PSL.This group worked on standardization of PSL.A special authority named "National Institute of Special Education" (NISE) runs special education centers.An NGO www.ijacsa.thesai.orgnamed "Pakistan Association of the Deaf" (PAD) has realized the need of a standardized sign language.Their effort resulted in a book and a learning tool on Grammar for PSL.
First ICT assisted learning tool was funded by USA.It was started in 2002 by Sabahat.They have provided lessons for learning and a CD containing practice for the sign language.This project targeted to improve learning of PSL among deaf.Along with development of sign language, people have also provided their effort for learning.A very recent advancement is the development of categorical PSL learning resource.The researchers at PSL Deaf Reach Program have designed 5000 videos, a mobile app and a book containing 1000 PSL signs.Boltay Haath is another project related to PSL.However this project is dependent on use of data gloves, it uses statistical template matching.The accuracy of the system is from 70 -80%.Sumaira has proposed another system using data gloves but this time the use of colored ones is proposed by Sumaira et al.She uses fuzzy classifiers to identify gestures [14], [15].
Ahmed et al has developed PSLIM.PSLIM takes voice input and convert it into sign/gesture of PSL.They have built several interfaces.Their product accuracy is almost 78%.They have added to the PSL vocabulary.They have also improved speech recognizer along with Translator component.[14].
Asif Ali suggested a system which uses Haar"s Algorithm and takes input in both forms text and image of sign and convert it into other form without using any special camera.Khan et al has introduced techniques for deaf to teach them programming and Farooq et al. has introduced DWT based technique using ordinary RGB camera with an accuracy of about 86%.Their system can process the changes in hand pose also but only for static and single hand gestures.Sami et al [9] has proposed another technique using computer vision.They have used cross correlation with an accuracy of 75% [14].

III. GUIDELINES
Before creating a dataset we must keep few guidelines in our mind.Usually our end user is either a researcher who uses data set to test his/her algorithms, or a developer who uses this data to produce solution and to test it before using it in an actual real life situation.The dataset creation involves selection of tasks, requirements by the algorithm, types of gestures to be covered and the classes in the data set if any [16].
Creating a dataset is also a complex task which involves many hours of work.The researcher creating a dataset should always keep in mind the possibility of releasing the dataset publicly at the end of his work.It should provide all possible support for image analysis and segmentation.Indeed the time spent to record a dataset may quickly become very long and the dataset could be valuable to other researchers [20].Fig. 1 shows us images of basic alphabets from Pakistan Sign Language.Fig. 2 shows signs of numerics from 0-9.The data set should have Careful design covering all the desired characteristics and recording conditions.The data set is build using an appropriate framework.Acquisition of data is quite time consuming.We should acquire data with highest possible quality.Next step is to verify the data using well known algorithms.Once tested for public use it can be released in lower quality.Lastly a good documentation and description of the dataset is important for a public release of the dataset elaborating the data acquisition in detail [17]- [19].

IV. MATERIALS AND METHODS
Urdu has been written in Urdu font as well as Roman Urdu.Initially British introduced the concept of Roman Urdu but now it is commonly used for text-messaging and Internet services.A person from Islamabad chats with another in Delhi on the Internet only in Roman Urdū.They both speak the same language but with different scripts.In addition to this Oversees Pakistanis, mostly their kids also use Roman Urdu [17].The following Table 1 tells us about some Urdu language alphabets and their Roman Urdu substitutes.www.ijacsa.thesai.orgThis research work has been inspired by the work of Deaf Reach Program in Pakistan [22].There are two ways to use dynamic gestures in all sign languages.

1) Word based dynamic gestures(existing) 2) Spelling based dynamic gestures (Proposed)
All over the world, the dynamic gestures are word based i.e. every word has its own gesture.These gestures may use one or two hand.It means a language that has very rich vocabulary, will require more effort to learn its sign language.Fig. 3 shows word based gesture for Joota (shoes).
Alphabets make meaningful words and sentences [21].The proposed spelling based gestures are more closer to computer based processing.They may also use one hand or two hand gestures, but here they are using one hand gestures only.Although communication through these gestures will be slow, but these dynamic words are easy to learn as they are using existing static gestures.Fig. 4 and 5 show spelling based dynamic gesture for Joota (shoes) and Khatt (letter) respectively.This paper presents slightly different technique for storing the videos for continuous signs in PSL.The data set has been stored in MS SQL.For this initially a data repository has been used.The structure of the repository is as shown in Table 3: In Fig. 3, few entries have been added to show how videos and images are stored in the repository.Urdu words are selected from different categories.‫بلی‬ (cat) is stored as BILLI.‫تا‬ (dog) is stored as KUTTA.However there is still clash of few words e.g.‫آم‬ (Mango) ‫عام‬ ( Ordinary) both have been spelled as aam.As a starting point, Urdu words have been selected from Categories mentioned below:  Common adverbs and adjectives, public places, electronic appliances, Buildings, educational institutes, offices, clothes, computer grammar edible items, families and their norms, law and order, Agriculture, professions Sports, etc. www.ijacsa.thesai.orgUrdu is then coded in Roman Urdu.While converting these words in roman Urdu, all the rules for pronouncing the Urdu words are followed.A few resources like Urduworld.comprovide facility to carry on English to Urdu, Urdu script to English and from Roman Urdu to English translation [2].
When we will switch to any other sign language like Chinese Sign Language, we will use Roman Chinese in place of the attribute Roman Urdu.Urdu Script attribute will be replaced with Chinese Script attribute in the repository.

V. RESULTS AND DISCUSSION
Alphabets are stored in the form of pictures.Words are stored in the form of videos which are converted to images using any appropriate tool whether it is an online tool or a program segment written in MATLAB or any other visual language.
To present words using spelling based gestures, videos have been recorded using ordinary mobile or laptop camera.For alphabets images have been captured.Only one hand gestures have been used.Fig. 5 shows selected frames from the video of spelling based gesture of the word "shoe" while Fig. 6 shows selected frames from video of the word "letter" in Pakistan Sign Language.Simplicity of this technique is that no special gloves, clothes, or any other electronic gadgetry is required.This makes system even more flexible.

VI. CONCLUSION
This paper proposes a simple method to store static and dynamic gestures.Many groups are working on PSL static and dynamic gestures.Words have been chosen for all walks of life.Emphasis is to create a touch free interface.Future work may include making this dataset publically available.Some datasets are available but they have not been following any algorithm so they cannot be used in interaction with computer based systems very efficiently.
Few guidelines have also been provided about the main facts researchers should keep in mind when selecting or creating datasets for research.Future work includes design of a Universal Sign Language and replacement of word based gestures with spelling based gestures.

TABLE I .
SOME ALPHABETS OF URDU LANGUAGE

Table 2
lists few Urdu language words, their English and Transliteration of the Urdu language words.

TABLE II .
URDU LANGUAGE WORDS WITH ENGLISH TRANSLATION

TABLE III .
REPOSITORY STRUCTURE