A Survey on using Neural Network based Algorithms for Hand Written Digit Recognition

The detection and recognition of handwritten content is the process of converting non-intelligent information such as images into machine edit-able text. This research domain has become an active research area due to vast applications in a number of fields such as handwritten filing of forms or documents in banks, exam form filled by students, users’ authentication applications. Generally, the handwritten content recognition process consists of four steps: data preprocessing, segmentation, the feature extraction and selection, application of supervised learning algorithms. In this paper, a detailed survey of existing techniques used for Hand Written Digit Recognition(HWDR) is carried out. This review is novel as it is focused on HWDR and also it only discusses the application of Neural Network (NN) and its modified algorithms. We discuss an overview of NN and different algorithms which have been adopted from NN. In addition, this research study presents a detailed survey of the use of NN and its variants for digit recognition. Each existing work, we elaborate its steps, novelty, use of dataset and advantages and limitations as well. Moreover, we present a Scientometric analysis of HWDR which presents top journals and sources of research content in this research domain. We also present research challenges and potential future work. Keyword—Neural network; digit recognition; segmentation; supervised learning; image classification; computer vision


I. INTRODUCTION
The huge increase in the data volume generated by the use of scientific instruments, handwritten documents, the use of the social media and channels has brought forth research challenges.These research challenges have attracted the researchers in several research domains and new research fields have emerged as well.The huge bulk of the data, due to the free content generation facility by the social web users, is in structured as well as unstructured format.The progress in data storage, its processing has given rise to various data related technologies such as sentiment analysis [1,2], finding top influential users [3,4]diverse and multilingual data analysis [5,6]and recognition of handwritten content [7] Computer vision is an active research domain.It is serving the humanity in diverse fields such as video capture cards, scanners, sound cards, webcam and digital cameras.In the computer vision field, the researchers detect the objects from the given input, which are in the form of images and videos and apply the relevant function for recognition of various target objects.The supervised learning approach is applied to this research task which deals in various steps including data input, data pre-processing, the feature extraction, the feature selection using proper algorithms, the data selection based on data split or cross validation, application of various supervised learning algorithms and performance evaluation.Hand Written Digit Recognition (HWDR) is one of the main research problems of image processing, machine learning and computer vision.A digit can be different in style, shape, orientation and size etc. from the same letter written by another person.These variations make it difficult to recognize the digits automatically with the help of machine learning algorithms.HWDR, as a research domain, may be defined as a procedure of detecting and recognizing any digit from the given image and after processing on it, convert the digit into such form that is readable and editable for the machine.A typical HWDR system consists of different stages.First is the image acquisition, which is a process of obtaining input image either by taking a picture by using a camera or by drawing on the piece of paper and then scanned that paper or by creating drawings using a digital pen.Second step is the pre-processing which is a process by which we enhance the quality of the image there are certain things in the www.ijacsa.thesai.orgenvironment that may damage the quality of the obtained image.Preprocessing is a way by which we enhance the quality of the image by applying certain operations on it like filtering, normalizing, thinning, etc.These operations may vary according to our needs and according to the further uses of the image.Third steps are called segmentation which is used to decompose the image into meaningful subparts.It is a process by which we separate the objects from one another that are present in the picture.So, we can say that segmentation is the process by which we divide the image into more subparts and each subpart represent the presence of an object on it.This process simplifies the image for further processing.The next steps are features extraction which is applied for considering and ranking characteristics of the objects present in the subparts of the image this is helpful for differentiating between more than one objects.Characteristic obtained made further processing like classification easier.There are several techniques that are used for feature extraction.The next step is the classification which is applied for prediction which the object belongs to which class .In this phase, we define more than one classes, then, according to the characteristics of the object we assign them a class, for example, we have two classes human and cat now whenever an object with human characteristic is arrived, we will assign it to the class human and whenever an object with cat characteristics is arrived.We will assign it to the cat class on the basis of its characteristics.Several classifiers are used for classification purpose.This research paper focuses on various methods used for feature extraction and neural network methods for the HWDR and also shares the pros and cons of each technique.
The rest of the paper is divided as follows: Section II discusses the details or neural network, Section III reviews the contemporary relevant literature.domain,Section IV discusses the research methodology and Section V shares the potential research challenges and future directions before concluding the paper in the next section.

II. AN OVERVIEW OF NEURAL NETWORK AND ITS VARIATIONS
Neural Network is also known Artificial Neural Network (ANN).ANN consists of a number of processing elements which process the given data by their external inputs.ANN is organized into a number of layers which are produced from a number of nodes.Patterns are shown with the help of an input layer.The input layer connects with the hidden layers where the actual processing takes place.Then the hidden layers connect with the output layer which presents the required output in the network.There are different types of ANN algorithms, few are discussed as follows:

A. Feed Forward Neural Network (FFNN)
FFNN is the basic type of artificial neural network.In this type of NN, nodes do not make a cycle.First an input node is connected to the hidden node and the data goes only in the forward direction in the network.An example of FFNN is shown in Figure 1.

B. Deep Feed Forward Neural Network (DFF)
The Deep Neural Network is introduced in the emerging new field of the machine learning, which is known as the deep learning.DFF is a type of FFNN, but includes a number of the hidden layers for information processing.At every step, an input layer transforms the given data into more abstract forms.The data is processed using a number of hidden layers.DFF applies on a number of fields such as computer vision, speech recognition, Natural Language Processing (NLP), etc.An example of DFF is shown in Figure 2.

C. Recurrent Neural Network (RNN)
RNN is also a type of ANN in which nodes make use of the directed graph.RNN utilizes its internal memory to proceed with input data.Recurrent refers to two vast classes of networks having the same structure.These two classes show dynamic behavior.One is the infinite impulse whereas another is the finite impulse.The finite class uses a directed acyclic graph and infinite class uses directed cyclic graph.RNN ia applied on handwriting recognition and speech recognition.An example of RNN is shown in Figure 3.

D. Extreme Learning Machine (ELM)
ELM is a feed forward NN used for classification, regression, clustering, and sparse approximation.ELM consists of many hidden layers that need to be tuned for data processing.The hidden layers are not changed and can be taken from their ancestor nodes.The output weight of the hidden nodes is acquired in a single step.These weights define the ratio of learning of hidden layers.An example of ELM is shown in Figure 4.

E. Convolutional Neural Network (CNN)
CNN is one of the popular classes of Neural Network.CNN also has the shift invariants.CNN is based on their shared weights.CNN is a variation of multilayer perceptron.CNN needs less processing compared to other image classification algorithms.CNN is independent of human knowledge and used for processing of visual images.

F. Deep Convolutional Network (DCN)
DCN is a very popular class of NN.They use convolutional cells and kernels.The kernel process input data and make groups of layers to simply it.DCN is used for image recognition and visualize sub images.An example of DCN is shown in Figure 5.  Pinki and Pooja [1] described a Neural Network method for the offline HWDR.In this method, the authors used conventional features with the back propagation network.For training, the classification accuracy was 100%, but recognition accuracy was 91.2%, which shows that the model was overfitted.This research work concludes that the accuracy can be improved with the addition of more features.Hayder Naser Khraibet Al-Behadili [8] has compared the performance of the different machine learning approaches such as K-nearest neighbor (KNN), Neural Network(NN) and Decision Tree(DT) which are used for the HWDR by using the MNIST dataset which is the widely used dataset in the relevant litearature.In these approaches, the Neural Network algorithm is more precise in determining the accurate results.Neural network approaches and feature extraction techniques for handwritten digits recognition.The main objective is to achieve accuracy and high recognition rate [9].The performance of two Artificial Neural Network Models Feed Forward Neural Network (FFNN) and Recurrent Neural network(RNN) for the HWDR by using the MNIST dataset, is compared by the researchers in which the output shows that the RNN is better in the right recognition of the digits.Neural Network method for the HWDR is built in which, the researchers used binarization and skeletonized binary pixel of the different were used as a input for the multilayer perceptron network [10].The classification and recognition result were better by using the existing features.The authors concluded that they have reached the computer to the human brain by the use of digit recognition.In a final discussion, they argued that the neural network is much better than other techniques used for digits recognition [11].Moreover, this research study is regarded as a guide for character recognition research area.It is trusted this definite talk is be gainful understanding into different ideas included, and help additionally progresses in the region [12].
According to another research study [13], different techniques are available to recognize hand written documents.These algorithms can be used to recognize many more languages like hindi, Punjabi, etc. fuzzification with back propagation algorithm can be added to improve the efficiency and correctness of the algorithm.Furthermore, the performance of three different algorithms for HWDR is compared by using the SVM for the classification www.ijacsa.thesai.orgpurpose and applied on the MNIST dataset.These three algorithm given as BFSS( Binary Fish School Search) , BPSO(Binary Particle Swarm Optimization) and 3rd was ABACO (Advanced Binary Ant Colony Optimization) [14].
We find another neural network based method [15] which proposed the neural based system and surf feature extraction strategy for character recognition.The proposed approach enhanced a procedure for character recognition utilizing neural network and the the results were promising. .Backward Propogation based Neural Network(BPNN) and surf feature strategy have been utilized.Ismail et al [16] approach proposed arrangement method that depends on the MLP under other benchmarking datasets.Two arrangements of trials were led for testing the adequacy of the proposed procedure.The principal approach acknowledges highlightsas the data sources and the second incorporates four highlights as information sources.According to the authors, the imperative region of research is multi-lingual character acknowledgment framework.The work of Optical Character Recognition (OCR) and proposal of the frameworks for it in various applications remains a dynamic area of research [17].
According to another research study [18], a content investigation pipeline comprising of four phases: optical character acknowledgment, sentence limit identification, tokenization, and grammatical feature labeling.Utilizing a formal algorithmic model for assessing the execution of multiarrange forms, the authors introduced trial comes about analyzing the effect of delegate OCR blunders on later stages in the pipeline.
In a recent research study, Pink et al [19] present a different feature extraction method along with different classifiers.To achieve high recognition of numeral text, this can be done in Classification and recognition process.The outcome of this proposed work, the neural network is proved to be the best option to recognize handwritten digits as compared to other algorithms.Muhammad et al [24] used a simplistic approach for recognition of handwritten digits using artificial neural networks.The knowledge base of the proposed system has been proven to be updated by feeding it new fonts of digits or feeding different patterns of earlier digits.Juan et al [25], applied neural networks to hand writing character recognition and they show improved results.According to the authros, Convolutional Neural Networks and Deep Neural Networks allow reaching an optimal performance measured on the MNIST Database which is the widely used data set in the HWDR.The MNIST Database contains 70,000 digitized handwritten numerals distributed in as many as ten classes.In this research study, the dataset has been divided into 60,000 images for training purposes, and the remaining 10,000 are considered asthe test set which is used to measure the accuracy of learning of the proposed model during the training phase.
The Arabic digits are usually used all over the world.We also find research work which used variations of Arabic numbers or different number sets.Alwzwazy1 et al [20] used a new challenging digit Arabic dataset which was prepared from different study levels of schools.The authors realized the lack of large data sets, thus they prepared a large dataset by paying vast effort for distributing and collecting digit forms over hundreds of primary, high, college students.Sara Sabour et al [21] utilized to conceal Markov models with Gaussian blends as yield dispersions.These models were difficult to learn; however, the results were promising.Edgar et al [22] compared the performance of CCN with his own model named as MNIST Model.The MNIST model works very efficiently as compared to CCN, but the authors shared their intent that in the future there could be an improvement in this model using pose matrix.
Meenu Alex et al [23] proposed a framework that can perceive characters from old corrupted archives like authoritative reports and change over it into machine editable shape keeping in mind the end goal to save them.The extreme exactness decrease can be seen from first stage to the second stage.In this paper, the authors proposed a strategy for the acknowledgment of Malayalam transcribed characters utilizing a blend of unique classifiers.Li Deng [24] uses MNIST dataset and computed a statis classification model for the domain area that save the problem of pre-processing.The work is novel in this regards as this uses a new dataset and work well in speech recognition research area.Zahangir et al [25] developed a new model for HWDR recognition.This model is based on deep neural network techniques.It used the dataset named CMATERdb.This model can give results more precisely and accurately by using other deep learning techniques.
Abdelhadi et al [26] proposed a neural systems for manually written digit acknowledgment from the MINST database.. McDonnell et al [27] proposed their framework on the basis of SLFN and achieved as good accuracy as the deep training via the help of RLM-type layer network.The authors share that the most accurate network based model is a combination of several non-iterative learning methods to define a projection from the input space to a hidden layer.While the intuitive elegance of deep networks is hard to deny, and the economy of structure of multilayer networks over single layer networks is proven, the authors argue that the speed of training and ease of use of ELM-type single layer networks makes them a pragmatic first choice for many realworld machine learning applications.The authors have demonstrated the computational and performance based advantages based on many use of different NN based algorithms.The performance of different models is compared in detail in Table 1.www.ijacsa.thesai.org

A. Other Advanced Algorithms of NN
There are some other methods that are used in this research.Sakshica, and Gupta [28] compared the performance of three different Neural Network approaches for the HWDR.These approaches are Single layer perceptron model, Hopfield Neural Network and back propagation algorithms.According to the result, the back propagation is more efficient, accurate and fast in digit recognition process.Another research study [29] proposed the RBF Neural Network uses decoupled Kalman filter training method for recognition of handwritten digits of different fonts.The offered method is tested on handwritten digits from 0 to 9 of 25 different fonts.It is found that this method has a very high success level in recognizing the handwritten digits.Mishra and Singh [30] proposed a slope for extraction method and grouping assignment is dealt with utilizing Radial Basis Function NN and backengendering neural system.In this research study, the authrors used another consolidated approach of highlight extraction and neural system which increment the exactness of the proposed framework.The exactness of back-proliferation neural system is 83.66% and spiral premise work is 98.26% which is superior to other existing strategies.According to Gaurav Jain et al [31], a classification algorithm to recognize www.ijacsa.thesai.orghandwritten digits (0to 9).Different classifiers and combination methods evaluate the caveats are included in handwritten digit recognition.The classifier based research have significance in handwritten digit recognition and use such as online handwriting recognition on computer tablets, etc.People write the same digit in many different ways.
SmallNORB is a well-known dataset for developing new shape-recognition models precisely, because it lacks many of the additional features of images.The authors implement an efficient version and test much novel models on a larger dataset.Zamani et al [32] proposed RF technique performed practically identical to the best in class strategies.They also plan to explore preprocessing stages that close the hole on RFs and the best in class techniques on this dataset.A proficient Persian manually written digit acknowledgment strategy in view of arbitrary timberland and convolutional neural system has also been presented.Broad examinations with different baselines performed on the Hoda dataset provided promising results.Viragkumar et al [33] exhibited quick, proficient simulated neural system for manually written digit acknowledgment on the GPU to diminish preparing time with PTM (Parallel Training Method).In any case, if the information dataset is bigger than GPU based parallelization is reasonable to diminish preparing the datasets.the authors inferred back proliferation calculation on GPU based parallelization ought to be favored by and large with contrasted with CPU based program.Wong et al [34] analyzed the execution of multi-layer feedforward systems with monotonic and periodical actuation capacities when connected to writing by hand digit acknowledgment.From the recreation comes about, the system with sinusoidal enactment work beats the ones with monotonic actuation work.Calderon et al [35] proposed a convolutional neural networks have been applied for handwritten digit recognition.CNNs were modified by the use of Gabor letters, which are known as good feature extractors.Furthermore, the modification of Gabor alters parameters, improvements of committee machines and development of better learning algorithms and network topologies can be studied.Ajay [36] focused on getting better accuracy rate for classification.Now, the branch of computer vision mainly depends on deep learning features like convolutional neural networks.Many other methods like angent Distance Classifier have been developed using LeNet architecture.The comparison of other models is shown in Table 2.

IV. SCIENTOMETRICS MAPPING AND INVESTIGATION
Scientometric and Bibliometric analysis deals in the study in which research related data analysis is carried out.One of the novel approach is that we present investigative system which presents both computational and manual undertakings.In this paper, we likewise played out a computational investigation of the information and processed diverse pointers as characterized in standard Scientometrics papers.We present analysis using notations such as TP denotes Total Papers, TC denotes Total Citations, ACPP denotes Average Citations per Paper, RGR denotes Relative Growth Rate, DT denotes Doubling Time and ICP denotes International Collaborative Papers.The computational examination utilizing Scientometric approach went for distinguishing year-wise research yield on HWDR using NN, rate of development, and establishment showed conveyance of distributions, global community oriented paper occasions, top production sources, etc.We have then performed content investigation of all exploration papers to distinguish real catch phrases happening in them and their event blasts.The significant subject catchphrases are chosen and a topical thickness plot is additionally produced for the exploration yield information got.We present the vital scientometric markers processed through computational examination of the information.The subsections underneath display points of interest of different markers registered and tables and figures representing the resultant qualities.
As a matter of first importance, the quantity of distributed papers on Handwritten digit is estimated using the Neural system strategies for every one of the years 2010 to 2017.Table 3 demonstrates the quantity of distributed papers in HWDR on a year-wise plot.We can watch that this bend has been pretty much level till 2011, after which there is a lofty ascent.From 2015 to 2017, the quantity of distributed papers has diminished by around five times.
We have additionally registered the relative development rate (RGR) and multiplying time (DT) for HWDR distribution information acquired.While, RGR is a measure meaning the rate of development regarding time, the parameter DT measures the time required for the quantity of distributions in a specific year to end up twofold.The parameters RGR and DT are characterized as takes after as shown in eq (1) (1) Where, T1 and T2 are two chronological time periods and N1 and N2 T2.In our case, the values are calculated on an annually, hence accordingly re-written as: are number of publications at time periods T1 and (T2 − T1) can be taken as 1 and equations can be given as shown in Eq (2) and Eq (3) (2) (3) We have processed both these parameters for a HWDR look into production information.Table 3 presents the registered qualities for RGR, Mean RGR, DT and Mean DT for HWDR explore distributions amid the period 2010-2017.We can watch that the RGR in 2012 (0.46) is twofold to that of the incentive in 2013 (0.28).We can likewise watch that as RGR builds, DT will be lessening and the other way around.The mean RGR and mean DT are likewise ascertained for the entire time frame and we can see from the table that these qualities are 0.24 and 7.00, separately.
The table shows four distinct pointers, in particular TP (Total Papers), TC (Total Citations), ACPP (Average Citations Per Paper) and h-file for the for HWDR look into yield starting from different establishments.Here, it would be worth saying that ACPP esteem is characterized as given in Eq (4) The h -list metric measures both the efficiency and effect of the distributed work of a researcher or a researcher.It is characterized as: a researcher has list h, if h of his/her N p papers have at any rate h references each, and the other (N ph) papers have at most h references each (Hirsch, 2005).The h -file can be figured for people, establishments, diaries and so forth.We can watch that most creating establishments are generally referred to.For instance, the Microsoft Research and Circuit &system Intel inquire about lab contribute a most astounding number of research papers with their association, address additionally have accomplished the most astounding number of references.
We have contemplated the HWDR's examination production information to locate the significant distribution sources (essentially diaries) where HWDR's exploration work has accounted for amid 2010-2017 period.We figured the aggregate number of research articles distributed in every one of the unmistakable diary names found in the downloaded information.We have likewise ascertained TC, ACPP and hfile esteems for every one of the diaries for the HWDR's examination distribution information.
We have likewise considered the examination distribution on HWDR information to locate the most gainful and most referred to creators.We display a rundown of 15 most gainful creators in Table 4.
The results show that the pattern recognition is the top journal according to the proposed framework.Moreover, the top 12 journals include the "Information Sciences", "Neural Networks" and other journals.The value of TP for Pattern recognition is 6 while the TC score is 161.64 and the value of ACPP is 26.94.This means that average citations per paper for the Pattern recognition journal is 26.94.Similarly, the "Computer Vision and Image Understanding" is also among the top journal in the field as with the total number of 4 papers in the journal, the total citation counts are 129 that means the average citation per paper are 32.Whereas the "Journals of Visual Communication and Image Representation" have a total number of 100 citations that means average citation per paper for 4 papers are 25.These relevant research topics in different journals shows that there are as low as 3.27 average citation per paper and as high as 32.47 citations per paper while among them the average citations among all the journals is 12.This means the number of citations are very high.

V. RESEARCH CHALLENGES AND FUTURE DIRECTIONS
The latest trend has witnessed the use of the latest techniques in hand written digit in diverse languages such as Persian language and Arabic language.The existing work of the Persian language target to detect and recognize the personal language handwritten digit using a combined feature metho [37].A one good potential research work may be to detect the Persian language hand written detection using the diverse neural network based algorithms such as Convolutional neural network.The CNN has already been used for HWDR in the Arabic language [38]This research work already combines the Restricted Boltzmann and CNN and achieves better results.Another potential work can be used the other advanced neural network based algorithms such as Recurrent Neural Netowrk (RNN) and You Only Look Once (YOLO).
The use of NN based deep learning algorithms has also been applied for HWDR of the Bangla language [25].The comparative analysis of deep learning approaches for the Bangla language is a potential future work.
We find diverse work related to HWDR related to Asian languages, but to the best of our knowledge, there is no HWDR related to Urdu handwritten digit recognition.A good potential work can be to use the latest neural network based algorithms for Urdu HWDR.This may be good research work as Urdu Digits are different from Arabic and other digits and www.ijacsa.thesai.orgalso people use the Arabic and Urdu Digits in a same document as well, the difference is shown in the Table 5.It is also interesting to plan a research study about detection of handwritten digit written with right hand or left hand.This may be interesting if such studies can distinguish the writers as these and similar research studies may be helpful in criminal detection and recognition as well finding the authentic owner of a handwritten document.In addition, detection of the gender of the writer of handwritten digits may also be interesting as well which may also be beneficial in recognition of the actual writer in case of ambiguity or in the course of finding the actual writer.

VI. CONCLUSION
This paper explores the details of past work done in the field of HWDR using NN and its variations.The basic as well as advanced NN algorithms have been applied in the relevant literature to detect and recognize the manually handwritten digits in various forms.The diversity of the human nature and write up of digits in diverse forms present a challenging research problem which has become an active research domain.This survey is unique in the sense that it is limited to HWDR only while a number of surveys are related to OCR.In addition, it is also limited to the use of NN algorithms for HWDR.

Fi g . 2 . 2 A
Typical Architecture of Deep Feed Forward Neural Network.

Fig. 4 .
Fig. 4. A Typical Architecture of Extreme Learning Machine.

5 3
https://towardsdatascience.com/the-mostly-complete-chart-of-neuralnetworks-explained-3fb6f23674644 https://towardsdatascience.com/the-mostly-complete-chart-of-neuralnetworks-explained-3fb6f2367464 5 https://towardsdatascience.com/the-mostly-complete-chart-of-neuralnetworks-explained-3fb6f2367464G.Multilayer Perceptron (MLP)MLP is a category of feed forward NN.MLP contains three layers.The input node is called the neuron which utilizes the nonlinear activation function.MLP uses back propagation technique for training.MLP techniques are widely used for diverse research applications such as speech recognition, character recognition and image recognition.III.APPLICATION OF NEURAL NETWORK ALGORITHMS FOR HWDRThis Section reviews the existing research studies which are only related to the use of Neural Network and its variations for HWDR.We have classified the existing work into two Sections: first which focuses on the application of basic Neural Network methods while the second sub-section provides the details of the latest NN techniques for HWDR.

TABLE I .
A COMPARATIVE ANALYSIS OF RESEARCH WORK FOR HWDR USING NN

TABLE IV .
LIST OF TOP JOURNAL SOURCES FOR RELEVANT RESEARCH PUBLICATIONS

TABLE V .
A COMPARISON OF ARABIC AND URDU NUMBERS