Comparison of Classifiers and Statistical Analysis for EEG Signals Used in Brain Computer Interface Motor Task Paradigm

Using the EEG Motor Movement/Imagery database there is proposed an off-line analysis for a brain computer interface (BCI) paradigm. The purpose of the quantitative research is to compare classifier in order to determinate which of them has highest rates of classification. The power spectral density method is used to evaluate the (de)synchronizations that appear on Mu rhythm. The features extracted from EEG signals are classified using linear discriminant classifier (LDA), quadratic classifier (QDA) and classifier based on Mahalanobis distance (MD). The differences between LDA, QDA and MD are small, but the superiority of QDA was sustained by analysis of variance (ANOVA). Keywords—brain computer interface; electroencephalogram; event related (de)synchronization


INTRODUCTION
Brain computer interface (BCI) facilitates a direct communication between brain and an external device.The system -hardware and software -enables humans to interact with their surroundings without involvement of peripheral nerves and muscles, by using control signals generated by brain activity [1].The interface enhances the possibility of communication for people with severe neuromuscular and motor disabilities.The variety of BCI applications includes: environmental control, locomotion, entertainment and multimedia.The artificial intelligence system recognize a certain set of patterns in brain signals following the stages: signal acquisition, preprocessing, feature extraction, classification and the control interface.Different methods such as electroencephalogram (EEG), magnetoencephalogram (MEG), positron emission tomography (PET), single photon emission computed tomography (SPECT) are used in measuring and studying the brain activity.The EEG is the most convenient method used in BCI systems: because it is non-invasive, it has relative low costs, the real-time analysis may be performed and can be used in a portable device.EEG based BCIs use a set of sensors that pick up the EEG signals from different brain areas.
Preparing a movement or imagining movement can cause changes in the sensorimotor rhythms (SMR).The SMR refer to oscillations recorded in brain activity concentrated in certain frequency bands [3].
The event-related desynchronizations (ERD) are changes that appear while executing or imagining the movement.ERD starts when the subject begins to imagine a movement and manifests itself as a power decrease in Mu rhythm band.After that, a different phenomenon occurs, event-related synchronization (ERS) -an increase in power when the subject stops executing or imagining a movement.
The phenomenon of ERD/ERS related to motor imagery is stronger for the contralateral hemisphere and weaker in the ipsilateral hemisphere.
In the section II a presentation of the database is completed, how the features are extracted and how the statistical methods are applied.The paper ends with a conclusion and some recommendations based on our results (section III).
It contains recordings from 109 subjects, who executed real or imagined tasks.The EEG signals were recorded using International System 10-20 with 64 electrodes placed on the scalp.Subjects 43, 84, 88, 89, 92 and 100 were excluded because the contained recordings are not reliable for further processing.We have considered only FC1, FC2, FC3, FC4, C1, C2, C3, C4, CP1, CP2, CP3, CP4 electrodes, reported in the literature for enhancing Mu desynchronization.Every subject performed 14 experimental tasks: 2 runs of 1 minute for relaxation (one with eyes closed and one with eyes open) and 4 runs of 2 minutes for each of the following tasks: opening and closing left/right fist when a target appears on the screen followed by relaxation, imagining opening and closing left/right fist, opening and closing both fists, imagining opening and closing SC CEMENTLAB SRL, www.c-labortech.comwww.ijarai.thesai.orgboth fists.In order to implement the proposed methods, there were used the first two sets described above.
Each signal is coded as follows: T0 corresponds to the resting period, T1 corresponds to movement/movement imagery left wrist, T2 corresponds to onset of motion (real or imagined) of the right wrist.The EEG signals are sampled at 160 Hz.There are three trials for wrist movement (named 3, 7, 11) and other three for wrist movement imagery (named 4, 8, 12).

B. Data Processing
Signals loaded from database are filtered with a 8-12 Hz band pass filter corresponding to the Mu rhythm frequency range.No artifact rejection or corrections were performed.We selected segments from the EEG signals (2 s after the stimulus appearance) according to annotation for each mental task (T2, T1) extracting the information corresponding to right/left wrist movement.For the relaxation period (T0) sequences of 2 s following right/left wrist movement are extracted.
The most widely used methods for EEG signal feature extractions are based on frequency analysis, for example discrete Fourier transform (DFT) or power spectral density (PSD).We use a method based on PSD to find the desynchronization during movement.Power spectral densities were calculated for all the useful mentioned channels and for all trials 3, 7, 11 which correspond to right/left wrist movement.The average of these trials was calculated using pwelch function from Matlab with a Hanning window [5].The same procedure was applied both for computing the PSD during the movement period, denoted by PSD MOVEMENT and for the relaxation period which comes after right wrist movement and left wrist movement respectively, denoted by PSD REST .The resulted value, denoted by ERD, is used to assess the desynchronization/synchronization which appears on the pair of electrodes during right or left wrist movement. (1) The feature vector was formed from each pair of electrodes on the left/right hemisphere in the following way: ERD calculated for right wrist movement for the signal recorded from left hemisphere ( FC1, FC3, C1, C3, CP1 or CP3), ERD calculated for left wrist movement for the paired electrode from left hemisphere ( FC2, FC4, C2, C4, CP2 or CP4), ERD calculated for left wrist movement for the electrode from left hemisphere, ERD calculated for left wrist movement for the electrode from right hemisphere.

C. Classifiers
Linear discriminant analysis (LDA) is one of the most popular classification algorithms for BCI application, and has been successfully used in a large number of BCI systems such as motor imagery based BCI, P300 speller and steady state visual evoked potentials based BCI [6].In essence, LDA linearly transforms data from a high dimensional space to a low dimensional space, and finally the decision is made in the low dimensional space, thus the definition of the decision boundary plays an important role on the recognition performance [7].
Linear classifier is suitable for offline and online BCIs.Moreover, LDA, is simple to use and provides satisfactory results whether we are referring to a large or small databases.
Quadratic discriminant analysis (QDA) is closely related to LDA, where it is assumed that the measurements from each class are normally distributed.QDA makes no assumption that the covariance of each of the classes are identical [8].Although it is not reported and used as much as linear classifier in BCI systems, the quadratic classifier reported satisfactory and encouraging results.
Mahalanobis distance (MD) is a statistical distance function.In mathematical terms, the Mahalanobis distance is equal to the Euclidean distance when the covariance matrix is the unit matrix.The use of the Mahalanobis distance removes several of the limitation of linear classifiers based on Euclidean metric, since it automatically account for the scaling of the coordinate axes, as well as for the correlation between the different considered features [9].Mahalanobis classifier is simple but at the same time robust and leads to good results, as shown in [10].Despite its good performance, it is still rarely used in the literature on brain computer interfaces.
Classifiers LDA, QDA and MD were used for all six pairs of electrodes.The steps described above were also accomplished for the trials corresponding to right/left imagery of wrist movement.The classification error obtained for the test set was surveyed for all the subjects, for movement/ imagery of movement, pair of electrodes and classifier.

D. Statistical Analysis
A two-way Analysis of Variance (ANOVA) was performed using Statistical Package for the Social Sciences (SPSS) [10] on the error values obtained on movement/movement imagery.The first main factor was CLASSIFIERS with levels LDA, QDA, and MD, while the second main factor was ELECTRODES with levels FC1-FC2, FC3-FC4, C1-C2, C3-C4, CP1-CP2, CP3-CP4.The Levene test was used for testing homogeneity of variances and the Tukey's test was used as post-hoc test at the 2% level of significance.

III. RESULTS
Table 1 shows the means of error rates for all subjects, classifiers and pair of electrodes.For movement, the smallest error of 11,31% was obtained with the quadratic classifier for FC3-FC4, while the largest error 17,62% was obtained with the classifier based on Mahalanobis distance for C3-C4.
For the imaginary of movement there were attained the following errors: the lowest, 12,82%, with the quadratic classifier for FC1 -FC2 and 19,85%, the highest value with classifier LDA for C3-C4.High errors could be explained due to the imperfect contact of the electrodes on the scalp or as the Mu rhythm could not be developed as specified in [11].
On 58% of 103 analyzed subjects the smallest classification errors were obtained for wrist movement, 27% of subjects were able to perform better the imposed task for imagining motor movement than movement.Small errors were achieved for movement as well as imagining the movement for 15% of subjects.
The errors obtained after applying the quadratic classifier were better than those obtained using linear classifier and classifier based on Mahalanobis distance, both for real and imagined motor task.Differences between results for LDA classifier and MD classifier are very small (Fig. 1).We have selected the subjects who attained, even with the quadratic classifier, low classification rates when the EEG signal was passed through a 8-12 Hz band-pass filter.We suppose that these subjects could elicit low or high Beta rhythm instead of Mu rhythm.So, for these subjects only the data were filtered with a 13-18 Hz eighth order Butterworth band pass filter.Filtering on 13-18 Hz was performed on subjects 2, 21, 36, 42, 54, 64, 74, 78, 82, 87, 102, 106.The errors achieved for subject 54 are shown in Table 2.For subject 54, as he attained the best results, the errors decrease significantly if the filter is on 13-18 Hz instead on 8-12 Hz.We can conclude that subject 54 achieved low Beta rhythm.Beta rhythm was also developed by subjects 21, 74, 82, 102.The results show that frequency band other than that of the Mu range may contain useful information.Notable changes on errors were not found for the other seven subjects on 13-18 Hz or 19-26 Hz.In conclusion these subjects could not attain low Beta rhythm or high Beta rhythm.The ANOVA demonstrated that the use of different classifiers improves the error values.The p-value obtained for CLASSIFIERS was 0.001 for movement and 0.002 for movement imagery.On ELECTRODES the p-value was 0.001 for both tasks.
No interactions between CLASSIFIERS x ELECTRODES was found (p=0.815 for movement and p=0.649 for movement imagery).
Since differences were found in between classifiers, we performed the post-hoc statistical analysis comparison (Turkey test) to determine which classifier is the best.The tests shown that the classification rates obtained for movement/movement imagery with quadratic classifier are higher than those obtained with LDA and MD.At the α = 0.02 significance level, there is not enough evidence to conclude that the used electrodes have a significant interaction effect on errors obtained for both tasks.
In Table 4 are depicted the differences between p values obtained with different post hoc tests for classifiers LDA, QDA and DM.Differences between p values obtained with Tukey and Scheffe test are small.The ratio for movement is 0.836/0.850and for movement imagery 0.965/0.968.Thus, we are confident 98% that with classifier QDA we attained small error that when using LDA and MD classifier.The use of LDA and MD will yield to higher errors since they are similar (Table 4).Although Scheffe procedure is most popular due its conservatism and flexibility, leads to type II errors.Tukey procedure is used mostly for means comparison and leads to type I erors.In [12] and [11], using the same dataset relevant results were reported, but also some drawbacks (there are unknown the timing between runs, the age of the subjects or if there are right or left handed subjects).
In another work [13] the classification results were reported only for 30 subjects and were not applied statistical tests.
In most papers regarding BCI research, the classification is performed using a single classifier.A recent trend involves using several classifiers.The combination of multiple classifiers has the advantage of obtaining lower classification errors [6].

IV. CONCLUSIONS
Using power spectral density on the EEG signals contained in EEG Motor/ Movement Imagery Dataset we have studied if desynchronizations appear in the frequency band 8-12 Hz.The classifiers LDA, QDA and MD applied on feature vector were used to determine the classification errors for all six pairs of electrodes.
The results of classification errors vary from subject to subject.The differences among classifiers as LDA and MD are small and reasonable results were attained considering the large database (103 subjects were tested).The used method showed the best performance for the QDA classifier.The performances could be altered because some subjects cannot concentrate well in performing each task.Sometimes they can be absent-minded, ocular or muscles artifacts can occur or they may not have the capacity to imagine movement.Movement and imagining involves sustained mental effort.Also it is important to notice that the recorders contained in database were made on healthy subjects.As some studies revealed that the people who suffer different disabilities can develop Mu rhythm better than the healthy ones, it is possible to get higher classification rates for these persons.Future work will be focused on a combination of classifiers used in this paper in order to reduce the classification error.

Fig. 1 .
Fig. 1.The error rate for movement and movement imagery using LDA, QDA and MD

TABLE II .
THE ERRORS FOR SUBJECT 54 WHEN THE SIGNALS ARE BAND PASS FILTERED ON 8-12HZ AND ON 13-18HZ

TABLE III .
THE ERRORS FOR SUBJECTS 2, 74, 87 ON TEST AND TRAINING DATA

TABLE IV .
CLASSIFIER COMPARISON WITH POST HOC TESTS SCHEFFE AND TUKEY