Handwriting Recognition using Artificial Intelligence Neural Network and Image Processing

—Due to increased usage of digital technologies in all sectors and in almost all day to day activities to store and pass information, Handwriting character recognition has become a popular subject of research. Handwriting remains relevant, but people still want to have Handwriting copies converted into electronic copies that can be communicated and stored electronically. Handwriting character recognition refers to the computer's ability to detect and interpret intelligible Handwriting input from Handwriting sources such as touch screens, photographs, paper documents, and other sources. Handwriting characters remain complex since different individuals have different handwriting styles. This paper aims to report the development of a Handwriting character recognition system that will be used to read students and lectures Handwriting notes. The development is based on an artificial neural network, which is a field of study in artificial intelligence. Different techniques and methods are used to develop a Handwriting character recognition system. However, few of them focus on neural networks. The use of neural networks for recognizing Handwriting characters is more efficient and robust compared with other computing techniques. The paper also outlines the methodology, design, and architecture of the Handwriting character recognition system and testing and results of the system development. The aim is to demonstrate the effectiveness of neural networks for Handwriting character recognition.


I. INTRODUCTION
Handwriting digits and character recognitions have become increasingly important in today's digitized world due to their practical applications in various day to day activities. It can be proven by the fact that in recent years, different recognition systems have been developed or proposed to be used in different fields where high classification efficiency is needed. Systems that are used to recognize Handwriting letters, characters, and digits help people to solve more complex tasks that otherwise would be time-consuming and costly. A good example is the use of automatic processing systems used in banks to process bank cheques. Without automated bank cheque processing systems, the bank would be required to employ many employees who may not be as efficient as the computerized processing system. The handwriting recognition systems can be inspired by biological neural networks, which allow humans and animals to learn and model non-linear and complex relationships [1,2]. That means they can be developed from the artificial neural network [4]. The human brain allows individuals to recognize different Handwriting objects such as digits, letters, and characters [5]. However, humans are biased, meaning they can choose to interpret Handwriting letters and digits differently [8]. Computerized systems, on the other hand, are unbiased and can do very challenging tasks that may require humans to use a lot of their energy and time to do similar tasks. There is a need to understand how human-read underwriting [10]. The human visual system is primarily involved whenever individuals are reading Handwriting characters, letters, words, or digits. It seems effortless whenever one is reading handwriting, but it is not as easy as people believe. A human can make sense of what they see based on what their brains have been taught, although everything is done unconsciously. A human may not appreciate how difficult it is to solve handwriting. The challenge of visual pattern recognition is only apparent to develop a computer system to read handwriting [6,17]. The artificial neural networks approach is considered as the best way to develop systems for recognizing handwriting. Neural networks help to simulate how the human brain works when reading handwriting in a more simplified form. It allows machines to match and even exceed human capabilities at reading handwriting. Humans have different handwriting styles, some of which are difficult to read. Besides, reading handwriting may be time-consuming and tedious, especially when individuals are required to read several Handwriting documents by different individuals [25]. A neural network is the most appropriate for the proposed system due to its ability to derive meaning from complex data and detect trends from data that are not easy to identify by either other human techniques or human [23]. The main aim of this paper is to develop a model that will be used to read Handwriting digits, • How can the performance of Handwriting recognition systems be improved using artificial neural networks?
C. Target Group This paper will be targeting university students and instructors who want to convert their Handwriting notes and papers into electronic format. Despite the increased adoption of digital technology in institutions of higher education, handwriting remains part of students' and instructors' daily lives. Students take Handwriting notes while listening to their lectures and take notes while reading from different sources. Some also note down their thoughts, plans, and ideas on their notes. Likewise, lecturers have Handwriting notes that they would want to communicate to students. Hence, this paper will be targeting students and lecturers to develop a system that will allow them to convert their Handwriting works into electronic works that can be stored and communicated electronically. The central assumption of this paper is that students and lecturers need to have copies of their works that are stored electronically in their personal computers. Further, handwriting with pen and paper cannot be entirely replaced by digital technology.

II. THEORETICAL BACKGROUND
Handwriting character recognition is one of the research fields in computer vision, artificial intelligence, and pattern recognition [3,9]. A computer application that performs handwriting recognition can be argued to have the ability to acquire and detecting characters in pictures, paper documents, and other sources and convert them into electronic format or machine-encoded form. The system may obtain Handwriting sources from a piece of paper through optical scanning or intelligent word recognition. Also, the system may be designed to detect the movement of the pen tip on the screen. In other words, handwriting recognition may involve a system detecting movements of a pen tip on the screen to get a clue of the characters being written [7]. Handwriting recognition can be classified into two: offline recognition and online recognition. Offline handwriting recognition involved the extraction of text or characters from an image to have letter codes that can be used within a computer [15]. It involves obtaining digital data from a static representation of handwriting. A system is provided with a Handwriting document to read and convert the handwriting to a digital format. Online handwriting recognition, on the other hand, involved automatic detection or conversion of characters as they are written on the specialized screen [28,35]. In this case, the system sensors movement of pen-tip to detect characters and words. Different methods and techniques are used to ensure that computer systems can read characters from Handwriting images and documents [32,26]. Among the existing techniques that are used to model, and train Handwriting character recognition include neural network, Hidden Markov Model (HMM), Machine Learning, and Support Vector Machine, to mention a few. This paper focuses on artificial intelligence networks, machine learning, Hidden Markov Model, and the Support Vector Machine.

A. Artificial Intelligence
The idea of reading Handwriting characters, digits, and words by computer systems can be argued to be an imitation of a human being. In other words, such a system can be argued that they use artificial intelligence to read handwriting from images or any Handwriting source [11]. Artificial intelligence refers to intelligence that is demonstrated by machines [13]. The term is used to describe computer or machines that can mimic "cognitive" functions that are associated with the human mind. Artificial intelligence allows the machine to learn from experience, adjust to new data (inputs), and perform tasks that can be performed by humans [12,20]. Branches of artificial intelligence include machine learning, neuron network, and deep learning.

B. Machine Learning
Machine learning technology is inspired by psychology and biology that focus on learning from a set of data. The central assumption is that machines can learn to perform given tasks by learning from data [21]. A machine learning model is provided with training data that is specific to the given problem domain and the solution to each instance of the problem. That way, the model learns how to solve certain problems based on learning [14]. Fig. 1 shows a simple demonstration of the machine learning model used in the handwriting recognition system. The model takes an image that has a Handwriting digit and determines the specific digit based on the learning data.

C. Artificial Neural Network (ANN)
Artificial Neural Network (ANN) refers to information processing paradigm or computing systems that are inspired by biological neural networks that constitute the human brain [18]. The systems are not identical to the biological neural systems, but they are designed to process information the same way the human brain and animal brain process information [27]. The networks are composed of many interconnected neurons working in unison to achieve specific goals [37]. Just like the human brain, ANN learns from example. Hence, an ANN can be configured for an application, such as data classification or character recognition through the learning process. The learning process involves adjusting the system to a connection [24]. The artificial neural network comprises a network of multiple simple processors, each with a small amount of local memory [39]. The processors (units) are linked together by unidirectional communication channels and operates only on local data and input their get through their links.

D. Biological Neuron and ANN
As indicated earlier, artificial neural networks are inspired by the biological neural system. Hence, learning how biological neurons works can help to understand the artificial neural network [16]. The human body's neural system consists of three stages: neural network, receptors, and effectors as shown in Fig. 2. The first phase is the receptor which receives stimuli from the external or internal environment and then passes the information to neurons [14,16]. The second phase involves the processing of information by the neural network to make a proper decision of output. The third and final stage involves translation of the electrical impulses into responses to the external environment.  An artificial neural network can be argued to be a simplified imitation of the central nervous system. The structural constituents of human brains known as neurons perform computations such as logical inference, cognition, and pattern recognition, to mention a [19,38].
The neuron models are shown in Fig. 3 and 4; however, does not do anything different that cannot be done by conventional computers. In other words, it is just a simple representation of a neural network system that does not do much different from what a traditional computer can do. The Fig. 5 presents a more complicated model (McCulloch and Pitts Model) which is different from the previous model since it has inputs that are "weighted" [17]. That means each input has a different effect on decision making. The weight of an input can be described as the number which when multiplied with the input, it results in weighted input [39]. The results are then added together, and if they exceed the certain predetermined threshold value, the neuron fires, else, the neuron does not fire [29,33].   neuron can adapt to different situations by changing its weights and/or threshold [17]. Various algorithms can be used to make neurons to "adapt," with Delta rule and the back-error propagation being the most used algorithms.

E. Deep Neural Network
The neural network has layers of units where each layer takes some value from the previous layer. That way, systems that are based on neural networks can compute inputs to get the needed output [29]. The same way neurons pass signals around the brain, and values are passed from one unit in an artificial neural network to another to perform the required computation and get new value as output [17]. The united are layers, forming a system that starts from the layers used for imputing to layer that is used to provide the output. The layers that are found between the input and output layers are called the hidden layer. The hidden layers refer to a deep neural network that is used for computation of the values inputted in the input layer. The term "deep" is used to refer to the hidden layers of the neural network [25] as shown in Fig. 6. In Handwriting character recognition systems, the deep neural network is involved in learning the characters to be recognized from Handwriting images [33]. With enough training data, the deep neural network can be able to perform any function that a neural network is supposed to do. It is only possible if the neural network has enough hidden layers, although the smaller deep neural network is more computationally efficient than a more extensive deep neural network [19].

F. Hidden Markov Models (HMM)
Hidden Markov Model (HMM) has been used in many handwriting recognition systems as a primary modeling component. It is essential to examine the theoretical background of this model to have a clear understanding of how 139 | P a g e www.ijacsa.thesai.org (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 11, No. 7, 2020 handwriting recognition systems work [31]. HMM is a statistical Markov model that is used in a system that is supposed to assume the Markov process [40]. It can be considered as the most straightforward dynamic Bayesian network. Hidden Markov Models are class pf probabilistic graphical models used for predicting a sequence of hidden variables from a set of observed variables [15]. For instance, these types of models can be used to predict the weather based on the types of people's clothing. The weather, in this case, is the hidden variable while the people's clothes are what has been observed (known) [40]. In the same way, HMMs have successfully been implemented in the speech recognition, and character recognition since the models can help systems to predict unknown from the observed [23]. The fact that handwriting can be a statistical model is the main reason HMM can be argued to be one of the most preferred models in the development of Handwriting character recognition systems [30].

G. Support Vector Machine
Handwriting recognition can be considered as a problem of supervised learning and classification from a discriminative classifier point of view, with this assumption, Support Vector Machine which a discriminative classifier is considered as one of the models that can be effective in developing handwriting recognition systems [34]. Like a neural network, a support vector machine is a subset of machine learning [36]. The support vector machine refers to a supervised learning model that is dependent on learning algorithms for classification and regression analysis. A support vector machine can be considered as a computational algorithm that finds out a hyperplane or a line in a multidimensional space that separate classes. The separation between two or more linear classes can be achieved by any hyperplane [2,17]. This method is known as linear classification. However, several hyperplanes can be used to classify the same set of data, as shown in Fig. 7. A support vector machine is an approach where the main aim is to find the best separation hyperplane.
The comparison of all approaches is shown in Table I below.  HMM is a statistical Markov model which is used in a system that is supposed to assume the Markov process -Strong statistical foundation [31].
-It allows a flexible generalization of sequence profiles [40] -Have many unstructured parameters.
-Algorithms are expensive in terms of computational time and memory [15] -Training requires repeated iterations, and this can be time-consuming [19] Machine Learning Machine learning-powered systems rely on patterns and inference instead of explicit instruction to read text and characters [21] -No human intervention needed [27].
-Allows continuous improvement [19] -Requires massive data to train [21] -Expensive in terms of time and resources [27] -High-error susceptibility [31] Neural Network A neural network can be considered as a large parallel computing system comprising of many interconnected nodes.
-Ability to work with incomplete knowledge -Parallel processing capability -Ability to make machine learning Different training may damage the capability of the system Overreliance on hardware [22] Support Vector Machines (SVM) Classifies the data using a hyperplane Unlike neural networks, SVM approach relies on learning examples and structural behavior [23]. Has better generalization due to structural risk minimization It is difficult to select a "good" kernel function Difficult to understand and interpret It is hard to visualize the impact of SVM models.

IV. DESIGN AND ARCHITECTURE
This section discusses the design and architecture of the proposed handwritten character recognition system that will be using the neural network approach. The proposed system comprises input pre-processing, CNN, and output sections as shown in Fig. 8. The explanation of the architecture is provided below:

A. Neural Network Arthictecture
As indicated earlier, the HRS systems are most efficient when they are based on neural networks. Hence, there is a need to understand the neural network architecture. The neural network architecture refers to the elements that are connected to make a network that is used for handwriting recognition. The human brain works loosely to inspire neural networks. It is based on the idea of how neurons pass signals around the human brain to process input into an output [16]. Several units are layers to form a network and arrange from the ones that are responsible for receiving input to the layer that is responsible for output values. Between the output and input level layers, there is a hidden layer that is involved in much of processing. Different neural network architectures can be used to provide different results from the input images of handwriting. It is because architectures are based on different parameters, data, and duration of training. Fig. 9 shows a clear visualized of architecture used to recognize handwriting from images. The "X" shows the input while "Y" represents the output.
The size of a deep neural network layer is dependent on the work that the system is supposed to do. However, in most cases, more computational efficient smaller hidden layers can be developed to achieve the same task as one that can be achieved with an exponentially large deep neural network [35]. The deep neural network is supposed to memorize the training data to be able to recognize handwriting. Hence, deep neural networks are commonly used in optical character recognition systems.

B. Convolutional Neural Network
The system will use the convolutional neural network (CNN), which class of deep neural networks that are used for character recognition from images. Fig. 10 shows an underlying architecture of CNN that will be used in the OCR system. The architecture shows different types of layers, with the first layer being the input layer and the last layer being the output layer. The second later is called the convolutional layer and is followed by pooling layers and convolutional layers. The description of the CNN architecture is as follows: 1) Input layer: The input layer is used to feed the system with the image with the handwriting. The layer can be colored image (RGB values) or grayscale. It can have dimension W*H*W, depending on the input image. The W*H refers to the width and height of the image, while D refers to the depth of the image.
2) Convolution layer: The convolution layer is the building block of the whole network. Most of the computational work that is required to recognize characters from the input is done in this layer (Aggarwal, 2018). The layer consists of a set of learnable filters known as parameters of the convolution layer.
3) Pooling layer: The pooling layers are found between the convolutional layers in the CNN architecture. They are responsible for progressively reduce the spatial size of computational work in the network. They help to streamline the underlying computation. They do so by reducing the dimension of the input data by combing the outputs of the neuron clusters. They operate independently. That way, the system can achieve the intended outputs.

4) Fully connected layer:
Neurons in a fully connected layer are fully connected to all activations in the prevision layer. Hence, this layer, activations, can be computed with matrix multiplication. Based on the architecture, a system can have multiple fully connected layers. In summary, CNN can be used to achieve a solution to every pattern recognition issue. The architecture demonstrated above shows how OCR systems using neural networks can read handwriting. The convolutional networks work in the hierarchy and can be used to solve complex structures found in handwriting inputs. Humans inspire the whole idea can recognize writing objects and process what they see in their brains.  Vol. 11, No. 7, 2020 V. METHODOLOGY The current OCR system will consist of five phases. The phases are image acquisition and digitization, preprocessing, segmentation, feature extraction, and recognition. Fig. 11 shows the methodology that will be used to read handwriting.

A. Image Acquisition and Digitization
The image acquisition step involves acquiring an input image that contains handwriting. The image, in this case, should be in specific formats such as PNG and JPEG. The image is acquired through a digital camera, scanner, or any other suitable input device. The digitization step, on the other hand, involves converting the input paper into electronic format [20]. The conversion is achieved by first scanning the original document and representing it in the form of an image that can be stored on a computer. The digital image is essential for the pre-processing phase.

B. Preprocessing
Preprocessing is the second phase of OCR after the digital image has been made as shown in Fig. 12. The digitized image is pre-processed to remove noise, and then it is checked for skewing. Preprocessing is essential for developing data that that are easy for optical character recognition systems. The main objective of pre-processing is to remove the background noise, enhance the region of interest in the image, and make a clear difference between foreground and background.

1) Image enhancement techniques:
To modify attributes of the image to make it more suitable and to improve the quality of the image by reducing noise, increasing contrast, image blurring, and providing more details. Hence, to process an image so that result is more suitable than the original image and providing better input for automated image processing techniques.
2) Noise removal: Addictive noises of different types can contaminate images. Hence there is a need to remove noise to improve the quality of the image.
3) Binarization: This method is used to transform the grayscale image and converting it to black and white, substantially reducing the information contained within the image from different shapes of gray into a binary image.

4) Normalization:
This process in image processing that changes the range of pixel intensity values. Its common purpose of converting an input image into a range of pixel values that are more familiar to the senses. Normalization involves converting images into a standard size.

5) Skew correction, thinning:
This is one of the first operations to be applied to scanned documents when converting data to digital format. This process helps to get a single-pixel width to allow easy character recognition.
Preprocessing for handwriting characters of current approach is shown in Fig. 13.

C. Segmentation
Segmentation can be argued to be the most critical process in character recognition techniques. Segmentation of images is done in the testing stage only. It checks for any error point inclusion by checking all points against the average distance between segmentation points incomplete image. The process involves separating individual characters from an image, as shown in Fig. 14. The process results in multiple segments of the image known as super pixels. The main aim of segmentation is to simplify the representation of an image into something that can be analyzed easily. Hence it has a positive impact on the recognition rate of the script.

D. Feature Extraction
In this phase, features of the image are extracted and are defined based on the following attributes: height of the character, numbers of horizontal lines, widths of the character, number of circles, pixels, position of different features and number of vertically oriented arcs, to mention a few.

E. Recognition
In this phase, the neural network is used for classification and recognition of the characters from the image. The most neural networks that are used by optical character recognition systems are the multiplayer perception (MLP) and Korhonen's Self Organizing Map.

VI. TESTING
This section presents the testing and results of the OCR system. The testing subsection will present the indications on the correctness and functionality of the OCR system. The aim is to provide relevant information that will be useful to critical users. The information is about the quality of the system. The results subsection, on the other hand, highlights indications that the system was successfully implemented.

A. Unit Testing
Unit testing was used to test individual units of the system. The testing focused on the following: image acquisition and digitization, preprocessing, segmentation, feature extraction, and recognition modules. Unit testing was vital since it was the first testing effort performed on code. The testing helped to identify bugs in the code and made it easy to fix the code. Early detection of bugs in code is the most effective way of ensuring that the right system is developed. It helps to avoid the costs of fixing a faulty system later in the development process. Developers make sure that every unit is functioning as expected.
The following list of functions of unit testing that were tested: • Select the scanned input image of Handwriting document/images • Applying pre-processing • Apply segmentation • Apply feature extraction • Extract digital character

B. Integration Testing
Individual units of the systems were combined and tested as a group or unit. Input models tested in unit testing were targeted in this type of testing. The main aim was to expose faults in the interaction of the integrated units. The integration testing mainly focused on interfaces and the flow of data between integrated system units. Different from unit testing, integration links were given more priority. The main benefits of integration testing include: • Making it easy to integrate different system module.
• Allowing faster development and increases developers' confidence.
• The testing is easy to conduct.
• Helps to test the system for real-work cases.
• It makes it easy to discover issues such as a broken database.

C. Validation Testing
Validation testing was conducted to determine whether the development process meets specified requirements. Validation testing has the following benefits: • The testing will help to identify defects in the system.
• Validation testing is essential since it helps to understand the functionality of the system better.
• The testing ensures that the right system is developed based on the specified requirements.

D. GUI Testing
GUI testing was conducted to ensure that the system's graphical user interface meets the specified specification and is user-friendly. GUI testing has the following benefits: • The testing helped to identify regression errors.
• The testing helped to reduce the margin of errors.
• Helped to increase the efficiency of handwriting character recognition.
• The testing helped to ensure that GUI is user-friendly.
VII. RESULTS AND DISCUSSION The OCR system was used to recognize Handwriting characters and digits. As indicated earlier, it was implemented using a neural network. The following were the expectations of the system: • The system will have the capacity to show single word recognition. It will be an indication that the training was done correctly.
• The system will show more than one-word recognition (sentence recognition). It will also be an indication that the system was well trained. The recognition should be at least 99.9 percent accurate.
• The system will show characters that it was not able to recognize well or characters that were not well trained. The system will recognize special characters and digits.

A. Dataset and Feature Selection
The dataset has sample Handwriting digits for evaluating machine learning models on the problem of Handwriting digit recognition. It contains 21,000 testing and 21,000 training of Handwriting digits from (0 to 9). Each of the digits is standardized and cantered in a grayscale (0 -255) images with a size of 28x28 pixel. In each of the images consists of 784 pixels that represent the structures of the digits. A sample of dataset is shown in Fig. 15.

B. Digits Recognition
The decision tree classification model was used to train more than 42,000 datasets. The dataset was split into two halves; half of the datasets for the training set and the remaining half for training sets. The following steps were performed for the classification on Handwriting digits dataset: • Load the datasets of Kaggle Handwriting digits for classification.
• Split the datasets into two sets; one for training and other for testing.
• The recognizer was trained to predict that given an image of Handwriting digits.
• Test the accuracy of the classifier as shown in figure.
The python code for digits recognition is shown in Fig. 16, and OCR result in Fig. 17. Fig. 18 shows the list of model testing after training the machine learning model on the dataset. It further shows that some of the digits were not recognized by the machine learning model. The machine learning model was trained with a dataset that contains 42,000 rows and 720 columns, which the result shows 83.4% accuracy. The digit images pixels are used as features vector and decision tree as classifiers. Moreover, data repository is used for training and testing the datasets so, the result shows that the decision tree classifier is effective in recognition of Handwriting digits. The accuracy of decision tree classifier is shown in Fig. 19 and 20. The digit 1 was recognized with the highest accuracy of 93.73 whereas digit 0 was having least accuracy of 83.56.

VIII. CONCLUSION
The main aim of this research was to develop a system that will help in the classification and recognition of Handwriting characters and digits. Recognition of characters and digits is vital in today's digitized world, especially in organizations that deal with Handwriting documents that they need to analyze using computer systems. Systems that are used for classification and recognition of handwriting help organizations and individuals to solve complex tasks. The current system used neural networks to process and read handwriting characters and digits. The system benefited from Convolution Neural Networks (CNN) with the help of training data that allowed easy recognition of characters and digits. Like the human visual system, CNN allowed the OCR system to be more sensitive to different features of objects. That way, it was easy to classify and recognize different Handwriting characters and digits based on the training data stored in the system's database. The phases of handwriting recognition included image acquisition, digitization, preprocessing, segmentation, feature extraction, and recognition. The system was tested using unit testing, integration testing, GUI testing, and validation testing. The final system satisfied the specified requirements of accuracy as well as recognition. The work of the current research can be extended for character recognition in other languages. It can be used to convert books, newspapers, handwritten notes, and newspapers into digital text format using machine learning models used by the current research.