A High Performance System for the Diagnosis of Headache via Hybrid Machine Learning Model

Headache has been a major concern for patients, medical doctors, clinics and hospitals over the years due to several factors. Headache is categorized into two major types: (1) Primary Headache, which can be tension, cluster or migraine, and (2) Secondary Headache where further medical evaluation must be considered. This work presents a high performance Headache Prediction Support System (HPSS). HPSS provides preliminary guidance for patients, medical students and even clinicians for initial headache diagnosis. The mechanism of HPSS is based on a hybrid machine learning model. First, 19 selected attributes (questions) were chosen carefully by medical specialists according to the most recent International Classification of Headache Disorders (ICHD-3) criteria. Then, a questionnaire was prepared to confidentially collect data from real patients under the supervision of specialized clinicians at different hospitals in Jordan. Later, a hybrid solution consisting of clustering and classification was employed to emphasize the diagnosis results obtained by clinicians and to predict headache type for new patients respectively. Twenty-six (26) different classification algorithms were applied on 614 patients’ records. The highest accuracy was obtained by integrating K-Means and Random Forest with a migraine accuracy of 99.1% and an overall accuracy of 93%. Our web-based interface was developed over the hybrid model to enable patients and clinicians to use our system in the most convenient way. This work provides a comparative study of different headache diagnosis systems via 9 different performance metrics. Our hybrid model shows a great potential for highly accurate headache prediction. HPSS was used by different patients, medical students, and clinicians with a very positive feedback. This work evaluates and ranks the impact of headache symptoms on headache diagnosis from a machine learning perspective. This can help medical experts for further headache criteria improvements. Keywords—High performance computing; Clinical Decision Support System (CDSS); machine learning; primary and secondary headache; performance analysis and improvement; headache diagnosis; open medical application


I. INTRODUCTION
Headache is a common community physical discomfort, which has a negative impact on people's life especially in terms of work productivity and social relations. Headache is a pain in the various parts of head, which is categorized into two major types: primary headache and secondary headache. Primary headache consists of three main types: • Tension-Type headache (TTH) [1]: it is a very common, mild to moderate head pain, which often feels like a tight band around the head but it can also be intense. Its causes are not understood very well.
• Cluster headache [2]: it is one of the most painful headache types. Patients may wake up at night because of the intensity in one side of head and/or around one eye.
• Migraine headache [3]: it is a recurring severe headache, which usually affects one side of the head accompanied with nausea, visual disturbances, and sound and light intolerance.
In contrast, secondary headache [4] is caused by or occurred secondarily to a long list of other conditions. The most common of which is medication-overuse headache. This type requires further medical examinations for better diagnosis.
Patients can buy over-the-counter headache pain medicines, which might be harmful in some cases. On the other hand, the increase of the numbers of patients put significant pressure on clinicians and healthcare facilities. This pressure may lead to unexpected medical errors. There is an increased risk of depression that may affect patients suffering from severe headaches. Moreover, some types of headache may cause silent death under certain circumstances because of different reasons such as lack of healthcare, wrong diagnosis, or getting large doses of painkillers. Regrettably, the process of headache diagnosis is not trivial because of the similarity in all headache types' symptoms. The short time spent by doctors in hospitals on each patient's case may arise medical errors [5] because of the large number of visiting patients. According to [6], [7], headache is one of the main reasons for medical consultation in the primary care units and neurological clinics. Furthermore, non-specialist people cannot detect the difference between headache symptoms, which could put their lives at risk. In some cases, such as headaches caused by high blood pressure or low blood sugar, painkiller is useless. A big challenge facing the healthcare industry is the quality of service. Quality of service means diagnosing the diseases correctly while providing effective treatment to patients. Celik et al. [5] showed that poor diagnosis can lead to disastrous unacceptable consequences. Therefore, it is essential for research scientists to move towards more efficient diagnosis methods for the sake of better healthcare. It is necessary to find solutions to reduce the bad consequences of headache by increasing patient's awareness of his health.
The revolution of computer-based systems and technologies has led to the development of decision support systems (DSS), which can give useful assistance in many fields. The Clinical Decision Support System (CDSS) [8] is an interactive software that helps specialists and others in the medical field to make correct decisions. CDSS has been successful in improving the decision of healthcare institutions and clinicians. Dong et al. [6] showed many existing evidences indicating that DSSs, mostly the guideline-based CDSSs, can be super useful in improving medical decision making.
The main motivation of this paper lies in overcoming some critical limitations in existing systems such as missing some headache types, lacking medical-patient tool, low prediction accuracy, using many attributes, lacking analysis of headache symptoms, etc. In order to overcome these limitations, we developed a user-friendly, safe and accurate high performance web-based Headache Prediction Support System (HPSS) focusing on the main headache types while reducing the number of classification attributes. HPSS is based on machine learning and relies on measurement attributes that can safely distinguish between the aforementioned main headache types. HPSS can be used by patients, medical students, and even clinicians for preliminary guidance with less time and effort, while obtaining high accuracy.
The major contributions of this work are listed below: • Reduce the number of selected attributes, compared with the previous studies, through which we can accurately classify the four main headache types (Migraine, Tension Type, Cluster, and Secondary Headache), while maintaining high prediction accuracy.
• Collect a sufficient number of headache patients' records to create a sample that can be used to train a headache classifier.
• Employ a hybrid machine learning algorithm that consists of clustering and classification to train a classifier, while using 26 different classification algorithms.
• Create a Headache Prediction Support System (HPSS), which integrates a web-based user interface with a machine learning prediction model, so that HPSS can be easily used by patients, medical students and even clinicians as a preliminary guidance for better headache diagnosis.
• Test HPSS by allowing patients and clinicians to use it and check its usefulness and accuracy.
The rest of the paper is organized as follows: Section II describes the related research work in this area. Section III presents the methodology that was employed to build our Headache Prediction Support System (HPSS). Section IV explains the components of HPSS and demonstrates its user interface. Section V gives a comparison study of the different available headache diagnosis systems. Section VI provides concluding remarks and highlights future directions.

II. RELATED WORK
Computerized CDSS has become an excellent choice to manage headaches because of its usefulness in solving complex medical problems, reducing medical errors [9], as well as improving patient-clinician cooperation [10]. The development of CDSS for the diagnosis of headache has long been a significant research topic. Most existing headache DSSs are based on the International Classification of Headache Disorders (ICHD) criteria [11], [6].
Maizels et al. [12] developed an online Clinical Headache Assessment Tool (CHAT) with a focus on migraine headache. However, the tool targeted patients not clinicians. Yin et al. [13] developed a computerized headache guideline method using the SAGE (Standards-based sharable Active Guideline Environment) [14] module in addition to developing a DSS for the diagnosis of headache disease. Dong et al. [6] improved the previous work by developing a headache CDSS based on ICHD-3 beta and validating it via 543 headache patients. Although the CDSS gave high accuracy ratios in the diagnosis of some types of headaches, it still depended on the modeling method of SAGE, where the computerized clinical guideline representation module of headache diagnosis cannot be directly executed by computer. In the aforementioned two studies, the SAGE module was used to summarize the logical comparison expression in headache diagnosis. In contrast, this work depends on machine learning algorithms to diagnose main headache types. Krawczyk et al. [15] proposed a DSS for automatic classification of primary headaches by combining classification models with feature selection algorithms to better differentiate between TTH and migraine with aura or without aura. They used 6 machine learning algorithms for testing. The focus was on migraine, tension-type headache, and other headaches. The research was conducted on 1022 employees of both sexes between the ages of 20-65 via a questionnaire consisting of questions based on the ICHD-2 criteria . They found that headache is present in 579 (56.65%) of the employees, while the accuracy of classification was 81.02%. In comparison, our model is based on the latest ICHD-3 criteria [6] and was tested against 26 different classification algorithms while using 9 different attribute selection methods. Moreover, we developed an open-source and user-friendly web application that can be used by both patients and clinicians for better early diagnosis.
In case of reasoning, Yin et al. [16] developed a hybrid CDSS for primary headache disorder diagnosis by combining both rule-based reasoning (RBR) and case-based reasoning (CBR) models in order to simulate a headache specialist's thinking process. Their proposed approach achieved 95.24% for a probable migraine, 95% for probable tension-type headache, and 80% for probable cluster headache. In contrast, our work employs a hybrid model to diagnose migraine, tension-headache, cluster headache, in addition to secondary headache, while maintaining a better migraine accuracy of 99%. Yin et al. exploited their hybrid reasoning model by proposing a new CDSS based on CBR to solve the similarities between a probable migraine (PM) and probable tension-type headache (PTTH) [17]. In this research, 74 symptoms were selected for diagnosis, while the accuracy of PTTH was low 77%. In contrast, this work uses 19 symptoms only to diagnose migraine, tension-headache, cluster headache, and secondary headache, while maintaining high prediction accuracy.
Aljaaf et al. [18] created a synthetic dataset based on (ICHD-2), which means that they filled the dataset with the ideal symptoms for each instance. Their data set consists of 900 instances and 8 attributes, 66.67% of them are for patients diagnosed with primary headache disorders (migraine with and (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 11, No. 5, 2020 without aura, tension type headache and cluster headache), while the others instances is for normal persons. Although they use 8 attributes, some of these attributes consist of multiple symptoms depending on the occurrence or nonoccurrence of all these symptoms together. This means that they actually have more than 8 real attributes. The study tested seven machine learning algorithms. The best obtained accuracy was 96.67%. In comparison, we collect our dataset by asking real patients to fill a headache questionnaire based on what they feel. In addition, from a medical point of view, we excluded migraine with aura to keep our application safe to be used by patients as migraine with aura has similar symptoms to stroke. Celik et al. [19] presented a website expert system that evaluates the classification accuracy of the ant colony optimization (ACO) algorithm for the primary headaches diagnoses via a questionnaire. This questionnaire consists of 40 attributes and is based on the ICHD-2 criteria and was filled by 850 headache patients under the supervision of a neurologist. The system was developed using MySQL database and PHP programming language. The overall accuracy was 96.9412%. The main disadvantage of this system was the large number of questions (40) that must be answered by patients, which affects the easiness of its usage. In addition, Celik et al. [5] created a summary table that consists of different proposed techniques for headache prediction such as [15], [20], and [21].

III. METHODOLOGY
While it is possible to consult the International Classification of Headache Disorders, 3rd edition (ICHD-3) [4] for headache classification, the complexity of headache subtypes and diagnostic criteria may confuse even neurological specialists who are not familiar with the ICHD criteria. HPSS was also developed to be used by patients to increase their health awareness. Hence, we relied on machine-learning for constructing our Headache prediction Support System (HPSS). We briefly explain the main challenges encountered while constructing our machine learning model. We then describe the machine learning steps that were performed to build a reliable decision support system that can conveniently be used by patients and doctors in the form of web application.

A. Challenges in Machine Learning
Machine learning can be used to solve various problems. However, each problem is associated with some challenges and our problem is not an exception. In the following, we list the main encountered challenges: • Sufficient training data had to be collected for accurate prediction while maintaining high levels of confidentiality. Communicating with real patients was done under the supervision of specialized clinicians after getting the required official approvals.
• The search space we initially had consists of many types of headache as well as many features (symptoms) according to ICHD-3. We reduced the number of symptoms by consulting specialized clinicians and employing different feature selection techniques while obtaining high accuracy. From a medical point of view, we only focused on the main headache types to maintain safety as HPSS can be used by patients.

B. Construction of Headache Prediction Support System (HPSS)
The core of our high performance web-based HPSS is based on machine learning techniques that include feature selection, data pre-processing, clustering, and classification algorithms. The basic idea of our approach is to train a machine classifier with a set of relevant records collected from real patients suffering from different types of headaches. Fig. 1 demonstrates the machine learning steps in HPSS. 1) Attribute selection: The first step for constructing HPSS was to study the features that can be used to classify the main primary headache types (Migraine, Cluster Headache, Tension-Type Headache), while also distinguishing secondary headache. We relied on ICHD-3 criteria to extract HPSS attributes. The attributes express the symptoms that the patient feels when a headache occurs. Our main goal in this step was to reduce the attributes domain (number of attributes) in ICHD criteria while maintaining accurate classification. For this reason, we looked at the problem from two points of view: machine learning and medical. We first used 9 different attribute selection methods implemented in Weka tool [22] (CfsSubsetEval, GainRatioAttributeEval, InfoGainAttributeEval, OneRAttributeEval, Principal components, ReliefFAttribu-teEval, SymmetricalUncertAttributeEval, WrapperSubsetEval, and CorrelationAttributeEval) to extract the highest ranked attributes that can categorize headache symptoms into one of the four chosen headache types. We applied these methods on a synthetic data set. Most methods showed that the attributes rank is convergent in importance. We then consulted medical specialists to strengthen our attribute selection findings and ensure the correctness and safety of using these attributes in HPSS for headache diagnosis and prediction. We ended up having 19 attributes (headache symptoms).
These attributes were used to create a questionnaire. The ministry of health in Jordan gave us the approval, upon checking the questionnaire, so that we can ask patients diagnosed with headache in public hospitals to fill it under the supervision of specialized clinicians. We collected a sample consisting of 614 records for patients (199 males and 415 females) with different ages. This sample was supervised, which means that each patient's record was assigned a headache type (migraine, cluster, tension, or secondary) according to the diagnosis of his/her clinician. We then performed pre-processing on our sample. This step will be discussed in the next section. Table I shows a sorted average rank for the 19 chosen attributes obtained by applying the aforementioned 9 different attribute selections on our new real data set. As can be seen, previous similar symptoms, Phonophobia, and Photophobia were the attributes with the highest ranks. Gender was found to have the least impact. We used the results of this table to validate our chosen attributes and check the impact of eliminating some attributes with low ranks on the overall accuracy. We show these results later in this paper. 2) Data Pre-processing: Data preprocessing was applied on our 614 records sample collected from patients to extract a clean data set. Fig. 2 describes the data preprocessing steps. Data cleaning was the first step in which we corrected the unconscionable data. In data normalization, the attributes' values were scaled down. In data transformations, we converted the data set format to Attribute-Relation File Format (ARFF) to ensure that all attributes contain legal values. Later, some missing values, such as gender, were filled. Finally, data integration and noise identification were performed to create a clean real data set, which was later used as a training set for classification.

3) Clustering and Classification:
We employed a hybrid model that integrates K-Means clustering with Random Forest classifier. The initial diagnosis was done by clinicians. The clustering phase of this model was exploited to emphasize the initial medical diagnosis and confirm the efficiency of chosen attributes in headache prediction from a computerized point of view. K-Means results remarkably matched the initial medical diagnosis. However, for the few unmatched predictions, we had further discussions with our specialized clinicians and modified the data accordingly. We later passed K-Means clustering output to the classification phase.
Classification is a two-step process, where the first is to train a classifier (training phase) while the second is to use the trained classifier for classifying unknown instances (prediction phase). We applied 26 different classification algorithms, with the default number of folds (10), on our supervised real data set generated from the previous phases. These algorithms differ in terms of speed, accuracy, scalability, robustness, and interpretability. We chose the most accurate classifier in the prediction phase of HPSS. Table II demonstrates the results of the classifiers.
Depending on the results from Table II, the RandomForest gave the highest accuracy among all classifiers with (92.7%) by applying the default number of folds (10) and the number of tree was (500). Usually, the higher the number of trees the better to learn the data but adding large number of trees can slow down the training process. The time spent to build the model for the 614 records was (0.45) seconds. The correctly classified instances were 569, while 45 were incorrectly classified. The number of folds means that the classifier divides the data into (10) sections (folds). The classifier uses nine folds for training data and one fold for testing data. This process is repeated in each classifier stage. Table III illustrates the performance of the Random Forest algorithm through its confusion Matrix.
The first row in the confusion matrix indicates that 329 instances out of the 332 that were actually classified as migraine were correctly predicted. The second row indicates that 162 out of 187 instances were correctly predicted as secondary. Third row indicates that 46 instances out of 56 were correctly predicted as tension, and the fourth row indicates that 32 instances out of 39 were correctly predicted as cluster.
The recall values refer to the recall rate of the classifier in each class. The precision values in the confusion matrix refer to the precision of the classifier at predicting each class. The number of instances of each class significantly affects the accuracy of its prediction. We collected 332 instances of migraine, 187 instances of secondary, 39 instances of tension and 56 instances of cluster. We treated a small number of patients suffering from tension-type headache, although it is the most common type of headache. This is due to the fact that patients who suffer from tension-type headache rarely visit clinics. Moreover, cluster headache is considered a very rare headache to happen, so we got few patients suffering from cluster headache. The accuracy should notably improve by increasing the number of patients diagnosed with cluster and tension type headaches. Our last step in the training phase was to test our classifier while reducing the number of attributes. This reduction was performed through different stages by eliminating some low rank attributes at each stage. Table IV shows the accuracy after each stage. We used this experiment to check the impact of eliminating the low rank attributes on the overall accuracy. As can be obtained, while reducing the number of attributes, we were still able to obtain high accuracy. Moreover, while eliminating more attributes, the accuracy decreased because it becomes harder for the machine to classify patients.

IV. HPSS GRAPHICAL USER INTERFACE (GUI)
In order to enable patients, medical students or even specialists to use our system in a convenient way, we developed our system as a web-based software called Headache Prediction Support System (HPSS). HPSS was programmed with Java Server Page (JSP). We used bootstrap to build a responsive mobile website, while using Hypertext Markup Language (HTML), Cascade Style Sheet (CSS) and JavaScript (JS) for the front end. The integrated development environment (IDE) was eclipse EE.
(IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 11, No. 5, 2020 A. HPSS Work Mechanism Fig. 3 demonstrates the work mechanism of HPSS in details. The user is first asked to fill an online questionnaire. The answers are saved in an ARFF file as a testing data. This file is then tested against our training set using Random Forest classification. Based on this testing, the system displays the predicted headache type based on the questionnaire's answers and gives a guidance message to the user to take the required action. HPSS also provides useful tips and links about the different types of headache. The prediction page, seen in Fig. 7, consists of some check-box questions that represent the 19 chosen headache symptoms. When the user checks a specific box, this means that he/she has this particular symptom. Then, when the user completes choosing the symptoms he/she has, he/she must click on the apply button. The results are saved in an ARFF file. The Random Forest algorithm reads the user symptoms and tests them against the existing training set. The headache type is then displayed to the user on the screen depending on the classification results as a message on the top of the screen. This message can be one of the following based on the predicted headache type: 1) You most likely have Cluster Headache. Please check with your healthcare provider. If your age is above 50 and/or you have these symptoms for the first time, it is highly recommended to check with your clinician for further evaluation.
2) You most likely have a Tension-Type Headache. Please check with your healthcare provider. If your age is above 50 and/or you have these symptoms for the first time, it is highly recommended to check with your healthcare provider for further evaluation.
3) You most likely have Migraine. Please check with your healthcare provider. If your age is above 50 and/or you have these symptoms for the first time, it is highly recommended to check with your healthcare provider for further evaluation. 4) You most likely have Secondary Headache. Please check with your healthcare provider to find out the cause of your headache to receive appropriate treatment.
We used the tooltip class in CSS to show a brief description about every symptom when the user moves the curser over the corresponding checkbox. A prediction message example is shown in Fig. 8. The Headache Types Page is shown in Fig. 5. This page has four clickable photos, where each one of them explains one particular headache type classified in HPSS. The explanation gives an overview about the headache type, its symptoms, triggers, causes, and preventive measures.
The frequently asked questions page, shown in Fig. 6, contains some general questions/answers related to headaches. A specialized clinician was consulted to create these pages. Finally, the (About Us) page, seen in Fig. 9, explains the purpose of HPSS and its main concepts. It also gives contact information in the case of any queries about the usage of HPSS. There have been many studies targeting headache diagnosis. In this comparative study, we mainly focused on the most recent and related systems, which provide clear prediction  Table V. Table V highlights the usage of these systems via 9 different metrics. The missing values in the table were either unexplained or unclear in some of these systems.
In contrast, our model is based on a hybrid solution that consists of clustering and classification. It was tested with 26 different classification algorithms while using 9 different attribute selection methods. We relied on the latest headache ICHD-3 criteria to extract HPSS attributes. We reduced the attribute's domain to 19 attributes while predicting different types of primary headache in addition to secondary headache with high accuracy of 99.1% for migraine prediction and overall accuracy of 93%. HPSS provides a web-based system for patients, medical students, and clinicians for better early diagnosis. Patients can safely use HPSS due to its helpful advices and guidance without giving any prescriptions.
Our study also ranks headache attributes, as shown in Table  I. This can assist clinicians and medical experts to better diagnose headaches while saving much time and effort. This can also help for further headache criteria improvements.

VI. CONCLUSION AND FUTURE WORK
This paper presented our experiences in developing a hybrid, reliable, and web-based prediction support system for the diagnosis of primary and secondary headaches according to the latest IHCD-3 criteria. This work aimed to improve the process of headache diagnosis via machine learning model accompanied with graphical user interface called HPSS. HPSS targets different end-users that include patients, medical students and clinicians for initial safe headache diagnosis. HPSS provides an easy-to-use interface for predicting both primary and secondary headaches with an overall accuracy of 93% among 614 real patients with 19 attributes only, while obtaining a migraine accuracy of 99.1%. HPSS showed that a hybrid machine learning model that combines unsupervised with supervised learning can improve headache prediction accuracy compared with a singular model consisting of classification only. HPSS was tested by different patients, medical students, and clinicians. All of them gave us a very positive feedback about its convenient interface, usefulness, and accuracy.

A. Future Work
We believe that examining more patients' records should lead to further accuracy improvement. The scope of prediction can also be extended to cover other subtypes of headache.