A Review of Feature Selection Algorithms in Sentiment Analysis for Drug Reviews

Social media data contain various sources of big data that include data on drugs, diagnosis, treatments, diseases, and indications. Sentiment analysis (SA) is a technology that analyses text-based data using machine learning techniques and Natural Language Processing to interpret and classify emotions in the subjective language. Data sources in the medical domain may exist in the form of clinical documents, nurse’s letter, drug reviews, MedBlogs, and Slashdot interviews. It is important to analyse and evaluate these types of data sources to identify positive or negative values that could ensure the well-being of the users or patients being treated. Sentiment analysis technology can be used in the medical domain to help identify either positive or negative issues. This approach helps to improve the quality of health services offered to consumers. This paper will be reviewing feature selection algorithms, sentiment classifications, and standard measurements that are used to measure the performance of these techniques in previous studies. The combination of feature extraction techniques based on Natural Language Processing with Machine Learning techniques as a feature selection technique can reduce the size of features, while selecting relevant features can improve the performance of sentiment classifications. This study will also describe the use of metaheuristic algorithms as a feature selection algorithm in sentiment analysis that can help achieve higher accuracy for optimal subset selection tasks. This review paper has also identified previous studies that applied metaheuristics algorithm as a feature selection algorithm in the medical domain, especially studies that used drug review data. Keywords—Sentiment analyis; drug reviews; feature selection;


I. INTRODUCTION
Sentiment analysis (SA) or opinion mining is a field that analyses opinions, comments, expressions, and views on different entities, such as products, services, organisations, and individuals. It is subjective in sentiment analysis to analyse each comment to identify the types of sentiment polarity, as either positive, negative, or neutral. According to [1], sentiment analysis is widely implemented in the domains of products, restaurants, movies, etc. However, this technique is not widely used in the medical domain, which could probably be due to privacy and ethical issues [1].
The current widespread use of social media has allowed users the freedom to speak their mind by giving their opinions or views in various aspects, such as medical quality, services they received, the effectiveness and side effects of drugs, and medical costs. Users would use social media platforms as a place for them to express their dissatisfaction or satisfaction with the goods or services provided. According to [2], medical documents are classified into six types, namely, nurse's letter, radiological report, discharge summary, drug reviews, Medblogs, and slashdot interviews.
This study has specifically focused on drug reviews, namely, users' comments on drugs in terms of effectiveness, side effects, symptoms, facilities, and the value of the drugs. The main problem in sentiment classification is that features extracted from user comments often contain data that are redundant, irrelevant, or even misleading [3], [4].
According to [5], there are three levels of SA, namely, feature, sentence, and document. The focus of this study has been on feature, which is to identify features embedded in customers' comments, as either positive, negative, or neutral. This study has also identified previous studies that were conducted on the medical domain, which were specifically related to drug reviews. This study has focused on identifying feature selection techniques and techniques for classifying sentiments in customers' comments on drug use. Several combinations of keywords were used ("feature selection + sentiment analysis + drug review or drug) during the search process in standard databases, such as Elsevier, ACM, Google Scholar, Science Direct, Elsevier, SpringerLink, Scopus, Taylor & Francis; and IEEE Xplore.
II. BACKGROUND RESEARCH According to [2], drug reviews refer to comments on drugs, which are related to their effectiveness, side effects, convenience, and value. User comments can help other users find the best businesses, destinations, or services by sharing opinions and ratings on these drugs.
It is important to analyse these drug reviews and identify users' views or opinions on these drugs, whether they are good or vice versa. The results of this analysis could contribute insights related to the health field to the stakeholders of this field [1]. Apart from that, the results of this sentiment analysis could also help the community understand the effects of drugs on human health. This paper will describe drug reviews, sentiment analysis and feature selection in detail.

A. Drug Review
According to [6], drug reviews consist of posts in social media, where patients express their experiences and opinions about treatments or medicines. According to [7], the Pharmaceutical Care Network Europe (PCNE) defined medication review as a structured evaluation of a patient's medicines, with the aim of optimising medicine usage and improving health outcomes, in terms of drug-related problems and recommended interventions. As reported by [2], a drug review can be defined as a user's personal perceptions on several drug-related categories, including effectiveness, side effects, convenience, and value. According to [8], a drug review is a patient-written review on various drugs based on their experiences and preference. This kind of review provides a lot of information that can lead to accurate decisions about public health and drug safety.

B. Sentiment Analysis
Sentiment analysis (SA), also referred to as opinion mining, is the area of research that analyses the perceptions, thoughts, opinions, evaluations, behaviour, and emotions of people on anything, for example, products, services, organisations, people, concerns, activities, topics, and their attributes [5]. SA is the process of evaluating a word or a sentence based on their sentiment. Any opinion or emotion expressed in the form of a text would contain a negative, positive, or neutral element [9]. As stated by [2], SA could be used to gather information on the effectiveness of a treatment or medication from social media and health records. According to [6], drug manufacturers could also benefit from SA, particularly in pharmacovigilance, as particular adverse effects of a drug can be found more easily from public repositories or social media posts. A drug review may contain a high proportion of sentiment terms formed from personal impressions and feelings [2]. Therefore, SA can be used to collect useful information that can assist in making accurate decisions on public health and drug safety.

C. Feature Selection
Features are topics or keywords found in users' comments. A feature can be a topic that is being discussed or things that users made comments on. An example of a user's comment sentence: 'This camera is very good': the feature in this sentence is 'camera' and the word sentiment is 'good'. Various definitions of feature selection have been provided by previous studies [10 -14]. Based on studies by [4,9,15], it is important to produce an optimal feature subset by reducing feature size to increase classification accuracy. In conclusion, feature selection is a process of selecting and identifying features that are not redundant and relevant to reduce the size of feature dimension and improve the accuracy of sentiment classification. Therefore, this study aimed to identify feature selection techniques used in previous studies to select features in drug review datasets.

III. A REVIEW FEATURE SELECTION ALGORITHMS USED IN SENTIMENT ANALAYSIS FOR DRUG REVIEWS
The world of social media is full of people who would make various comments, either positive or negative. Social media is full of information regarding users' preferences and experiences when using products or services. This type of information should be utilised by identifying valuable insights in such comments using artificial intelligence technologies, such as sentiment analysis. Big-sized and high-dimensional data are a major problem that can decrease the accuracy of classification performance and complicate the process of obtaining an optimal feature subset. Feature selection in SA is an important step to produce an optimal feature subset [14], without having to change the original meaning of the feature. This study will identify feature selection methods, feature extraction, sentiment classification, data sets, and evaluation standards that are being used to measure the performance of the methods used.
NLP concepts, such as part of speech tagging, n-gram, content words, and function words have also been used to extract features from tweet data [16]. The Penguin Search Optimization (PeSOA) algorithm [16] was also used as a feature selection technique to select optimal features based on the keywords of drugs and cancer in tweet data. The K-Nearest Neighbour (KNN), Naïve Bayes (NB), and Support Vector Machine (SVM) methods were used through MATLAB simulation software to classify the tweet data. The performance metrics used were processing time, accuracy, precision, recall, and F-Measure to measure the performance value of each proposed method. Based on the combined feature selection techniques, which consisted of PeSOA and three classification methods, namely, PeSOA-KNN, PeSOA-NB, and PESOA-SVM, it was found that the combination of PeSOA-SVM was able to produce high accuracy, precision, recall, and F-Measure values compared to the other combinations. Similarly, PeSOA-SVM required less processing time to complete the classification process compared to other combinations. This increased performance was due to the ability of the combined SVM and PeSOA to classify larger data sizes from the search process from multiple dimensions. Their study had only focused on comments that contain the keyword drug, regardless of the type and effect of the drug. According to [1], the Bag of Words (BoW) technique or the term frequency-inverse document frequency (TF-IDF) technique were used to extract important words in a document. Once the keywords have been extracted from the document, the next process was to select an optimal feature subset using the Fuzzy-Rough Quick Reduct (FRQR) technique. By using BoW to determine the value of a feature, the feature selection process was able to significantly reduce the generated feature space. FRQR was able to select 43 optimal features from the 903 original features using the forward search strategy. Meanwhile, 56 optimal features were selected using the backward search. These two resultant feature subsets were tested using four classification methods, namely, the Ripper, Naive Bayes, Random Forest, and Decision Tree. The performance of these methods was measured based on training accuracy, performance of running independent hold-out test, and the time required to build the model. The experimental results showed that the FRQR technique was able to increase sentiment accuracy, as well as reduce the complexity of feature space, and the classification of run-time overheads.
According to [17], machine learning methods are insufficient to address the complex grammatical relationships between words in clauses. Their study applied a linguistic approach to overcome weaknesses in machine learning approaches. The advantage of using a linguistic approach is that this method can determine sophisticated rules for dealing 127 | P a g e www.ijacsa.thesai.org with various grammatical relationships between words in sentences or clauses. New rules based on linguistics can also be added to the system at certain levels. A comparison between SVM (a machine learning method) and linguistic approach was conducted using a dataset obtained from DrugLib.com. Experimental results showed that the linguistic approach was more effective compared to the SVM method. However, several problems have been identified based on the error analysis. This situation showed that the proposed linguistic approach required improvement. Satisfaction with drug use was analysed based on drug reviews from www.askapatient.com [18]. Several experimental analyses were conducted on the performance of Probabilistic Neural Network (PNN) and Radial Base Function Neural Networks (RFN) using two different datasets, namely, cymbalta and depo-provera. The results showed that the Neural Network approach surpassed the SVM method in terms of precision, recall, and F-Score values. The RFN method showed a higher performance value compared to the PNN method.
In their research [19] used the Probabilistic Aspect Mining Model (PAMM), which is a method to identify the relationship between features and class labels. Due to the unique features of PAMM, it focuses on finding features related to one class only rather than simultaneously finding features for all classes in each implementation. Apart from finding features, it also has properties that can be distinguished by the class. This means PAMM can be used to differentiate between classes, which help reduce the likelihood of features being formed from mixing different class concepts. Thus, the identified features would be easier to construe. Researchers have argued that this method can avoid features that have been identified as having contents mixed from different classes. Better and more specific features can be identified by focusing on the tasks in one class. This approach is also different from the intuitive approach, whereby reviews were grouped first according to their class label and followed by features for each group. The proposed model used all reviews when finding features that were specific to the target class. This approach helped to distinguish reviews from different classes.
Various sentiment categories for consumer review on drugs have been identified for the introduction to Adverse Drug Reactions (ADRs) [20]. The Weakly Supervised Model (WSM) was introduced using data labelled as weak to pretrain model parameters. Then, WSM was combined with the Convolutional Neural Network (CNN) and the Bidirectional Long Short-term Memory (Bi-LSTM) to produce another model, known as the WSM-CNN-LSTM to implement the sentiment classification process. The experiments showed that the proposed model was able to improve ADR recognition based on accuracy, precision, and F-Score values compared to other models.
According to [8], two deep fusion models have been proposed based on the three-way decision theory to analyse drug reviews. The first fusion model was known as the 3-way fusion of one deep model with traditional models (3W1DT). In 3W1DT, each classic algorithm is combined with a deep learning method separately. For example, Naïve Bayes (NB) was combined with Gated Recurrent Unit (GRU), Convolutional Neural Network (CNN), and Three-Way Convolutional Recurrent Neural Network (3CRNN), and known as GRU-NB, CNN-NB, and 3CRNN-NB, respectively. The second combination models were known as the 3-way fusion of three deep models with traditional models (3W3DT) to improve the performance of the deep learning methods. This second model combined three learning algorithms, namely, GRU, CNN, and 3CRNN with traditional algorithms, which were NB, Decision Tree (DT), Random Forest (RF), and K-Nearest Neighbour (KNN). These combinations were known as 3W3DT-NB, 3W3DT-DT, 3W3DT-RF, and 3W3DT-KNN. Data sets from Drugs.com were used to test these two models. The 3W1DT and 3W3DT methods showed better results compared to the stand-alone traditional and deep learning methods. Meanwhile, a comparison between 3W1DT and 3W3DT showed that 3W3DT was able to produce higher accuracy and F1-Score values compared to 3W1DT. The study [8] had also intended to apply a metaheuristic feature selection technique and evolutionary algorithm to improve the performance of the proposed fusion models in the future.
In their study, [21] implemented two feature extraction methods, namely, Word Embedding and Position Encoding in Vector Representation to extract features from drug review datasets. The obtained features were tested using four sentiment classification methods, namely, NB, SVM, RF, and Radial Basis Function Network (RBFN). They compared the sentiment classification of the original SentiWordNet (SWN) lexicon with the medical domain-based SentiWordNet lexicon (Med-SWN). Experimental results showed the effectiveness of the proposed method in the feature selection process. Meanwhile, an assessment on the performance of sentiment classification has proven that the features extracted from Med-SWN outweighed those from SWN.
Based on the summary in Table I, the use of metaheuristic techniques as part of feature selection techniques is still in its infancy. Therefore, further research must be conducted to prove that metaheuristic techniques are able to produce optimal feature subsets and help improve the performance of sentiment classification accuracy. The use of metaheuristic feature selection techniques was suggested by [8] to improve the performance of sentiment classification accuracy. However, this situation depends on the data training sets. Tests based on domains could also play an important role in each study.
128 | P a g e www.ijacsa.thesai.org Accuracy, precision, recall, and F-Measure [17] Not mentioned in paper. Not mentioned in paper. Rule-based Linguistic Precision, recall, accuracy, and F1-score [18] Not mentioned in paper. Not mentioned in paper.
Probabilistic neural network (PNN), and radial basis function neural networks (RFN) Precision, recall, and F1-Score [19] The specific type of feature extraction was not mentioned.
Not mentioned in paper. Probabilistic aspect mining model (PAMM) Mean Pointwise Mutual Information (PMI) and accuracy [20] Not mentioned in paper. Not mentioned in paper.
Weakly supervised model (WSM), convolutional neural network (CNN), and bidirectional long short-term memory (Bi-LSTM) Accuracy, precision, and F1-Score [21] Word embedding and position encoding in vector representation Not mentioned in paper. Naive Bayes, SVM, RF, and RBFN Precision, recall, and F-Score

IV. A SURVEY OF FEATURE SELECTION USING METAHEURISTIC ALGORITHMS IN SENTIMENT ANALYSIS
This section will briefly present feature selection techniques that use metaheuristic algorithms in sentiment analysis. Metaheuristic techniques can solve various problems with satisfactory solutions in a reasonable time. According to [22], metaheuristic techniques have been used for over 20 years in numerous applications. Most applications that use this technique demonstrated efficiency and effectiveness for solving large and complex problems.
These techniques are a high-level strategy and iteration generation process, which can guide the process of exploring the search space using different techniques. Metaheuristic techniques may include ant colony optimization (ACO), artificial immune system (AIS), bee colony, genetic algorithm (GA), particle swarm optimization (PSO), and genetic programming [22], [23]. According to the study by [23], metaheuristic characteristics are as follows: 1) A strategy that provides guidance in the search process.
2) Able to effectively explore the search space and find the optimal solution; and.
3) A simple local search procedure for complex learning processes.
Metaheuristic techniques have been used as feature selection techniques by [4], [24], [25], and [26]. Table II lists several studies in other domains that similarly used metaheuristic algorithms, such as particle swarm optimization, ant colony optimization, hybrid cuckoo search, and artificial bee colony that have been proven to show good results based on precision, recall, F-measure or accuracy values. 129 | P a g e www.ijacsa.thesai.org Next, the search for research papers on feature selection using metaheuristic algorithms that use drug review data was based on the following combinations of keywords: 1) ("Feature selection + sentiment analysis + metaheuristic + drug review); 2) ("Feature selection + sentiment analysis + optimization + drug review); and 3) ("Feature selection + sentiment analysis + swarm intelligence + drug review).
Searches in benchmark databases, such as ACM, IEEE Xplore, Elsevier, SpringerLink, Scopus, Google Scholar Taylor & Francis; and Science Direct showed no results. However, when the keyword combination has no 'sentiment analysis' and 'drug review', several papers were found containing the following keywords: 1) ("Feature selection + swarm intelligence + medical); and 2) ("Feature selection + swarm intelligence + health).
Brief descriptions on each paper are given in the following section. In their work, [34] studied feature selection techniques for the classification of medical datasets based on Particle Swarm Optimisation (PSO). Their research was focused on multivariate filter and wrapper approaches, combined with PSO using medical dataset. PSO was used as a filter and CFS was used as a fitness function. They also proposed using the wrapper approaches with PSO on five classifiers, namely, decision tree, Naïve Bayes, Bayesian, radial basis function, and k-nearest neighbour to increase classification accuracy. This method had been tested for feature selection classification on three medical datasets, which were the breast cancer dataset, the Statlog (Heart) dataset, and the dermatology datasets. A comparison was performed between the proposed approaches with the feature selection algorithm based on genetic approach. The results showed that the PSO_CFS filter was able to improve classification accuracy, while the proposed wrapper approaches with PSO showed the best classification accuracy. However, two studies had identified that GA_CFS is more reliable than the proposed method, which would be when KNN and RBF classifiers were applied to the to Statlog (Heart) datasets.
Confidence-based and cost effective feature selection (CCFS) methods were proposed using binary PSO on UCI lung cancer dataset [35]. The results showed that the proposed algorithm demonstrated effectiveness in terms of accuracy and cost of feature selection. Additionally, [36] applied the Binary Quantum-Behaved Particle Swarm Optimisation (BQBPSO) algorithm as a feature selection technique for selecting optimum feature subsets for a microarray dataset that contains five types of data set, namely, Leukaemia, Prostate, Colon, Lung, and Lymphoma. The BQBPSO showed more significant results in terms of accuracy and optimal feature subset compared to two comparison algorithms, namely, Binary Particle Swarm Optimization (BPSO) and Genetic Algorithm (GA).
An ontology-based two-stage approach to medical text classification, with feature selection using particle swarm optimization research was conducted by [37]. They developed a two-stage methodology to analyse domain principles and identify which concepts are discriminatory to a classification problem. This research used a set of clinical text, known as the 2010 Informatics for Integrating Biology and Bedside (i2b2) dataset. This dataset must go through an ontology-based feature extraction during the first stage. The MetaMap tool was then used to send the document to Unified Medical Language System (UMLS) to extract all features with meaningful phrases. A simple idea was applied in the concept section set and finally, a tf-idf measure was used to transform the feature into a vector. In the second stage, PSO was used to further remove redundant and unwanted features. To test the accuracy of the suggested method, five classifiers were used, namely, Naive Bayes (NB), Linear Support Vector Machine (LSVM), K-Nearest Neighbour (KNN), Decision Tree (DT), and Logistic Regression (LR). The results showed that the two-stage approach was able to extract meaningful features, reduce the number of features, and improve classification accuracy.
Based on the summary of previous studies in Table III, metaheuristic algorithms have been used as a feature selection algorithm in the medical or healthcare domain. However, these experiments had only included other disease datasets, such as breast cancer, lung cancer, and leukaemia. Experiments using drug review data were not found in this literature review.
Therefore, further research should be conducted using drug review datasets as research data to implement the use of metaheuristic algorithms. Additionally, studies should be conducted to identify metaheuristic algorithms that would be appropriate for drug review datasets. The literature review in this research paper was conducted in three parts. The first part was to identify which feature selection algorithms were used for drug review data in 130 | P a g e www.ijacsa.thesai.org previous studies. Table I shows the results of the first search, whereby natural language processing, machine learning, and metaheuristic algorithms were used. However, several studies did not state the type of feature selection techniques used in their study. Table I also shows that the use of metaheuristic algorithms as a feature selection technique is still lacking in the domain of drug reviews. Next, this study identified the use of metaheuristic algorithms as a feature selection technique in sentiment analysis in general. The results are summarised into Table II, which shows several studies in different domains using metaheuristic algorithms as a feature selection technique. Table II also shows excellent experimental results based on the measured values. Next, this study searched for previous research papers in a list of standard databases that applied metaheuristic algorithms as a feature selection technique in the drug review domain. However, no matches were found. Then, the keyword combinations were changed, which were "feature selection + swarm intelligence + medical" or "feature selection + swarm intelligence + healthcare". Several research papers have been found using these keywords, as listed in Table III. Table II and Table III show that metaheuristic algorithms can be used as a feature selection technique in the domains of movies, customer reviews, tourism, medical, and healthcare, with excellent experimentation results. SA plays an important role in the decision-making process. Health-based organisations or services would have to make decisions on the use of drugs, side effects or services provided based on user comments. Numerous approaches can be used in SA. Metaheuristic-based feature selection techniques can assist in the selection of optimal features, with higher accuracy. The literature review has shown that research to implement metaheuristic algorithms as feature selection in the medical domain have great potential which would benefit from further studies on drug review data. Researchers also need to identify the advantages and disadvantages of metaheuristic algorithms that would be used as feature selection algorithms in further studies. More studies are needed to identify previous studies that applied metaheuristic techniques. More experiments should be conducted to identify metaheuristic techniques that are appropriate for future drug review data.