Mind-Reading System-A Cutting-Edge Technology

In this paper, we describe a human-computer interface (HCI) system that includes an enabler for controlling gadgets based on signal analysis of brain activities transmitted from the enabler to the gadgets. The enabler is insertable in a user’s ear and includes a recorder that records brain signals. A processing unit of the system, which is inserted in a gadget, commands the gadget based on decoding the recorded brain signals. The proposed device and system could facilitate a brainmachine interface to control the gadget from electroencephalography signals in the user’s brain. Keywords—Brain-machine interface; Bio-signal computer command; mind-reading device; human-computer interface


I. INTRODUCTION
HCI has been primarily implemented by monitoring direct manipulation of devices such as mice, keyboards, pens, touch surfaces, etc. However, as digital information becomes more integrated into everyday life, situations arise where it may be inconvenient to use hands to directly manipulate a gadget. For example, a driver might find it useful to interact with a vehicle navigation system without removing hands from the steering wheel. Further, a person in a meeting may wish to invisibly interact with a communication device. Accordingly, in the past few years there have been significant activities in the field of hands-free human-machine interface [1]. It is predicted that the future of HCI is moving toward compact and convenient hands-free devices.
Notably, in a recent report [2], IBM has predicted that at least in the next five years, mind-reading technologies for controlling gadgets would be available in the communication market. In the IBM report it is predicted that "if you just need to think about calling someone, it happens…or you can control the cursor on a computer screen just by thinking about where you want to move it." Accordingly, there is a need to make such enablers that could capture, analyze, process, and transfer the brain signals, and command a gadget based on the instructions that a user has in mind. This paper discusses an enabler that is insertable in a user's ear to record an electroencephalography in the brain as brain signals while the user imagines various commands for controlling a gadget. The ear could provide a relatively inconspicuous location. Indeed, ear is known as a site where brain wave activity is detectable. Certain areas of the ear, such as the area of the ear canal have proven to be better locations for detecting brain wave activity. Particularly, the area of the upper part of the ear, called the triangular fossa has high brain wave activity, especially near the skull. It is considered that the thinness of the skull at this area could facilitate higher reading of the brain wave activities. The proposed enabler of this paper could transmit, for example wirelessly, the brain signals to a processing unit inserted in the gadget. The processing unit decodes the received brain signals by a pattern recognition technique. Based on the decoded brain signals, the processing unit could control applications that are installed in the gadget. The details of the device and system that could facilitate such brain-machine interface are discussed in this paper. This paper addresses the current technologies in mind-reading systems, the deficiencies and limits of the existing technologies, along with possible solutions to have a practical device for braincomputer interaction, and the future plans to achieve such cutting-edge technology.

II. CURRENT STATUS OF TECHNOLOGY
Traditional human-computer interfaces are limited since they require a human to physically interact with a device, such as pressing a button by a finger. In one of the most recent attempts to address this problem, speech processing devices have been considered for voice activation. However, voice activation technology suffers from many use-related limitations, including poor operation in noisy environments, inappropriateness in public places, difficulty of use by those with speech and hearing problems, and issues to capture and recognize different and not previously stored patterns of accents and languages. Further, attempts have been made to use head and eye movement schemes to move a cursor around on a computer screen. Such methods are limited in functionality and require additional measures to provide a reliable control interface [3]. It is noted that the direction of HCI is moving toward hands-free brain-computer interface with a fast pace [4]. Among promising technologies in this field, Nokia Corporation (hereinafter "Nokia") recently has proposed a system for providing a hierarchical approach to command-control tasks using a brain-computer interface [5]. This system includes a hierarchical multi-level decision tree structure that applies internal nodes and leaf nodes, in which the decision tree structure represents a task. The system performs navigating, using information derived from detected mental states of a user, through levels of the decision tree structure to reach a leaf node for achieving the task. The navigating includes selecting, using the information derived from the detected mental states of the user, between attribute values associated with the internal nodes of the decision tree structure to communicate with a device, including a name dialing or a command/control task. However, Nokia's device suffers from the complexity of the system, which requires a noticeable space that could not be accommodated in a compact unit to be carried by the user or inserted in the gadget. Further, in Nokia's system there is a possibility of a limited understanding of the user's brain and its electrical activities, since the accuracy of a mind signal detection could be degraded as the number of mind states increases. For example, it is unclear if this system can recognize a series of words that the user may think to implement a task. Another system recently proposed by Koninklijke Philips Electronics N.V. (hereinafter "Philips") [6] includes creating a user profile for use in a brain-computer interface that includes conducting a training exercise, measuring the user's brain signals during the training exercise, mapping specific signals of the user's brain signals to predefine mental task descriptions, and creating a user profile including the user's brain signals mapped to the mental task descriptions. The created user profile can be used in a method of creating the brain-computer interface for the user to conduct an application.
Further, Philips' system includes accessing the profile of the user that includes the user's brain signals mapped to mental task descriptions, accessing an application profile that includes properties of the application, matching mental task descriptions from the user profile to a respective property from the application profile, and creating a brain-computer interface. However, one of the problems that could arise from the Philips' system is that before applying the brain-computer interface by the user, a significant training is required. The users need to learn how to modulate their brain activities to generate proper electrophysiological signals. Further, the Philips' system needs to log many signals of the user, and then design a model or extract features. The electroencephalogram signals, however, are non-stationary, differ from subject to subject, and are very noisy. Indeed, the signal variability and the noises could distort the performance of Philips' device, and for such systems, a tedious and time-consuming training process is needed for learning the specific characteristic of the brain signals [7]. Yet in another system proposed by Microsoft Corporation [8] (hereinafter "Microsoft"), an HCI includes a wearable device having a plurality of sensor nodes, in which each sensor node includes electromyography sensors, a module for measuring muscle generated electrical signals using the sensors, a module for determining the electrical signals correspond to which ones of a user gestures, and a module for causing computing devices to automatically execute commands corresponding to the specific user gestures. The problem that could arise from Microsoft's system is the requirement of utilizing the plurality of sensors collectively in various areas of the user's body including chest, head, forehead, etc. This could be cumbersome, time consuming, and inconvenient for the user. Further, systems have been proposed for wireless electroencephalogram transmission, such as a system considered by Pedifutures, Inc. (hereinafter "Pedifutures") [9]. Although this system includes a device to transmit and receive electromyography data by radio frequency telemetry, it requires Manchester encoding which includes combining data with its associated clock in a single transmitted data stream. Manchester encoding is essential to obviate inherent frequency instability of the transmitter, which instability may result in impairment of the performance of the overall system. However, Manchester encoding does not provide error correction of transmitted signals and could reduce the effective data transmission rate, which would reduce data transmission efficiency and transmitted data integrity. Indeed, it is essential to a brain-computer interface system to produce and compare the brain response of a subject to audible stimuli. This would require accurate timing of the brain wave response to the stimuli and a high degree of transmitted data integrity. Pedifutures's system, however, does not provide an accurate timing and transmitted data integrity essential for an effective brain-computer interface system. Based on the systems and devices discussed above, considering that the HCI field is growing fast in the direction of compact and user-friendly interface devices, there is a need to provide a comfortable and compact device and system to collect mind signals, to transmit the recorded signals to a processing unit, and to process the transmitted signals into commands that could control a gadget.

III. FUTURE PLANS, LIMITATIONS
In view of the above deficiencies in the conventional devices, considering aforementioned predictions in the area of mind-reading technologies, it appears that the trend for the HCI is toward providing affordable and convenient enablers that could efficiently facilitate conveying the brain signals of a user to command various gadgets. This paper discusses an enabler for controlling a gadget based on signal analysis of brain activities transmitted from the enabler to the gadget in a system, which could overcome the issues set forth above in the conventional devices. Therefore, it is an advantage of the system disclosed in this paper to provide an improved humancomputer interface system, having many of the same capabilities as conventional input devices, but which is handsfree and does not require hand operated electromechanical controls, or microphone-based speech processing methods, and is easy to insert to provide comfort for a user of the enabler to enable easily controlling gadgets such as mobile phones, personal digital assistant devices, media players, etc.
With the proposed enabler and system of this paper, these gadgets can be controlled without a need for an additional hardware, particularly without additional electrodes outside the enabler. The enabler includes a recorder that is insertable in an outer ear area of the user. The recorder records electroencephalography signals generated in the brain. The recorded signals are transferred to a processing unit inserted in the gadget for converting the signals to command applications in the gadget. The proposed system is illustrated in Fig. 1, in which signals derived from the user's ear are used for decoding the brain activities to enable mental controlling of a gadget. As shown in the figure, in the proposed system, an HCI enabler is inserted in the ear of the user.
The enabler uses electroencephalography recordings from the canal of the external ear to obtain brain activities in a way that is used as a brain-computer interface using signals of complex cognitive. A recorder that is inserted in the enabler records the brain signals. The recorder has an electrode that is located at the entrance of the ear, and could be mounted with an earplug. Signals can be amplified and digitized for transmitting from the enabler.
The enabler wirelessly transmits the recorded brain signals to the processing unit that includes a decoder. A transmitting device installed in the enabler produces a radio frequency signal corresponding to voltages sensed by the recorder and transmits the radio frequency signal by radio frequency telemetry through a www.ijacsa.thesai.org transmitting antenna. The transmitting device could include the transmitting antenna, a transmitter, an amplifying device, a controller, and a power supply unit, such as a battery. The amplifying device could include an input amplifier and a bandpass filter. The amplifying device receives an electrode signal from the recorder. The electrode signal is a response to changes in the brain electrical activities of the user. The input amplifier could provide an initial gain to the electrode signal, and the bandpass filter could provide an additional gain to the electrode signal resulting in an output signal with an overall gain of much higher than the electrode signal. The controller is electrically connected to the bandpass filter. The output signal from the bandpass filter is inputted to the controller.
The controller provides signal conditioning to the output signal to provide telemetry transmission.
Such signal conditioning includes analog to digital conversion. The controller also controls the transmitter channel frequency thereby controlling the frequency of the radio frequency signal to be transmitted. A receiving device in the processing unit, through a receiving antenna, receives the radio frequency signal generated and transmitted by the transmitting device. The receiving device produces a data output corresponding to the received radio frequency signal. The receiving device could include any radio frequency receiving means with a plurality of channels. A desired channel is selected via processor control of the frequency of an oscillator. The receiving device could employ a frequency shift keyed demodulation format. The receiving could also include a microcontroller incorporated to program the oscillator. The receiving device outputs data and error correction bits to the microcontroller which removes error correction bits and outputs corrected data as the data output to an operator interface. The data output corresponds to the radio frequency signal received by the receiving device. The receiving device outputs the data output to the operator interface. The operator interface could include software which allows automatic synchronization of the stimulus with the data output.
The decoder decodes the data output using a common algorithm such as pattern classifier. By evaluating frequencies in a wide range from theta to gamma brain signals recorded by the recorder, complex cognitive signals are decodable and are used for controlling the gadget. The processing unit converts the decoded signals to command signals for running an application inserted in the gadget. The above-mentioned pattern classifier can utilize conventional algorithms that apply classifier-directed pattern recognition techniques to identify and measure specific changes in each input signal and derive an index of the relative strength of the change [3]. In one method, a rule-based hierarchical database structure, describes the relevant features within each signal and a weighting function for each feature. A self-learning heuristic algorithm governs the use and reweighting criteria for each feature, maintains the database of the feature indexes, and regulates feedback from a Feedback Control Interface. The output vectors are sent through cascades of classifiers, which select the most appropriate combination of the features necessary to generate a control signal to match an application in the gadget. Calibration, training, and feedback adjustment could be performed at the classifier stage prior to characterization of the control signal to match the control interface requirements. In sum, the proposed method to implement the enabler of this paper could include receiving a signal indicative of a mental activity of a user, decoding the signal, using pattern recognition to identify and quantify a change in the signal, classifying the signal according to a response index to obtain a classified signal, comparing the classified signal to data in a response cache to identify a response that corresponds to the classified signal, and delivering a command to the gadget to implement the response.
Other pattern recognizing algorithms such as wavelet, Hilbert, Fourier or other transformation can also be applied to single trails of the electroencephalograph from the recorder to perform pattern recognition. After a program for executing the above method is loaded into a memory of the processing unit, the program enables the processing unit to carry out the method for controlling the gadget, in which the processing unit coordinates with a recording electrode, and the detector for detecting signals of an electroencephalogram is placed in the canal of the outer ear of the user. Accordingly, the brain activities are decoded from the signals of the electroencephalogram and the gadget is controlled based on the results of the decoding.

IV. RESULTS AND DISCUSSION
The proposed system can provide a brain-user interface that utilizes miniaturized lightweight acquisition devices and computing electronics, and applies signal processing methods to acquire and measure mind data under real-time conditions. It is noted that in conventional devices one of the issues associated with implementing the brain signals is that it can be relatively difficult for a user to control the brain activities [10]. Alpha, beta, and gamma brain waves are readily accessible for sensing with sensors that sense electroencephalography and can be separated into subgroups based on frequency properties. However, for most individuals it is very difficult to influence activity of selected subgroups of brain waves, particularly in a time-controlled manner. Indeed, timing of signals is critical for the most control functions. In order to justify that the proposed enabler and system of this paper could overcome the aforementioned deficiencies of the conventional systems, and could provide an www.ijacsa.thesai.org efficient method to convey the brain activities to command signals for controlling a gadget, it is noted that a similar in-ear enabler has been shown being enable to detect oscillatory brain responses to complex cognitive challenges [11]. In-ear electroencephalography recording is sensitive to the changes in brain activities in response to complex cognitive demands.
The above-mentioned study shows using a working memory task in which participants have to keep an image, illustrating a natural scene in mind over a predetermined period. The onset of the scene image causes a change in theta power, which can be measured from the in-ear device. The activity measured at the device is sensitive to the experimental manipulation as to whether performance in a given trial is rewarded, such that an increase of theta power becomes stronger when performance in the trial is rewarded. In the above-mentioned study, individuals were presented with images of natural scenes that either depicted indoor or outdoor sceneries. The decoding was performed with a Multivariate Pattern Classifier (MVPC) algorithm using routines, in which neural network topology was defined by an input layer, which includes each of the frequency features of the electroencephalography recorded from the in-ear detector, a hidden layer including units, and an output layer, defined by two units, one for each of the indoor and outdoor categoryspecific patterns. The target patterns were for an indoor scene and an outdoor scene. Neural network training was always stopped after certain iterations. Figure 2 shows the accuracy of decoding from electroencephalography activity recorded with the in-ear electroencephalography detector depending on whether a subject is currently looking at and memorizing an indoor or an outdoor scene.
Based on the results of the above study depicted in Fig. 2, in-ear electroencephalography recordings and the frequency decomposition of these electroencephalography recordings are suitable to decode complex and highly cognitive brain activities. In the study illustrated in this figure, single-subject indoor and outdoor MVPCs were computed separately every 80 ms from −36 ms prior to 764 ms after sample onset during encoding. X axis labels time points where the MVPC was trained and tested. Plots represent subjects' mean MVPC accuracy at sample encoding for control (Cont; black line), nonconfigural (N-Conf; blue line), and configural (Conf; red line) conditions. MVPC results showed correct classification of sample pictures into indoor and outdoor categories from 200-300 ms onward. Figure 2 demonstrates (A) Trial structure of two variants of a blocked DMS working memory task, one with and other without associative configural maintenancedemands and a control task without maintenance requirements. (B) Behavioral performance at probe for each experimental condition. Working memory performance was better in the nonconfigural than the configural condition [paired t test: t(7) = 4.02, p = 0.005] and accuracy in control and configural was similar [paired t test: t(7) = 0.8, p = 0.45], showing that the two conditions were equated for difficulty. * p < 0.05; ns: p >0.4. (C) Single-subject indoor and outdoor MVPCs were computed separately every 80 ms from −36 ms prior to 764 ms after sample onset during encoding. Fig. 2. Results of pattern recognizing for indoor and outdoor scenes in an inear recording brain signals presented in reference [11] X axis labels time points where the MVPC was trained and tested. Plots represent subjects' mean MVPC accuracy at sample encoding for control (Cont; black line), (N-Conf; blue line), and configural (Conf; red line) conditions. MVPC results showed correct classification of sample pictures into indoor and outdoor categories from 200-300 ms onward. The statistical threshold for correct MVPC classification was set at p < 0.04 and at p < 0.002 after correcting for multiple comparisons. Error bars denote standard error of the mean (SEM) in (B) and (C) [11].
The results demonstrated above indicate that the in-ear electroencephalography recordings are suitable for constructing a brain-computer interface. Based on the results shown above, it is reasonable to conclude that, considering the above-mentioned system is able to distinguish brain thoughts, such as thinking about indoor versus imagining outdoor, an inear brain-computer interface could decode complex mental www.ijacsa.thesai.org thoughts and allow individuals to use more complicated thoughts to create a variety of cognitive commands to control gadgets.
In a separate study [12], mobile terminals have been demonstrated to illustrate measurements of biological signal, such as brain waves, in which the mobile terminal includes electrodes for body grounding as reference electrodes, and electrodes for differential amplification that are installed at a terminal body or a part of an outer surface of the terminal body and contact with the skin of a user of the mobile terminal.
Further, in another study [13], a device controls an electronic or computer system by a fluid flow to provide a pressure current input to a computer system, in which the device includes a sensor that detects the pressure current input provided by a user and converts the pressure current input into an electric signal, a signal processing unit to process the electric signal, and a flexible arm to secure the device in an input position, such as the chin area of the user, to detect the pressure current input provided by the user.
Yet, in another study [14], it has shown that brain waves of an individual can control a cursor on a monitor or other communication devices. Such communication via mu rhythms can be optimized in real time and by the stored data. The mu rhythms are detected, filtered, and amplified, when a person faces display screens. Electrodes are placed in approximately 64 locations in the scalp of the individual, for monitoring the mu rhythms of the alpha band and other electroencephalography rhythms. An ear lobe is used as an electrode reference. The electrode signals are fed to a channel differential amplifier to amplify the difference between the 64 channels and a reference electrode. The signals are digitized and filtered. The resulting signals are periodically submitted to a frequency analysis. The powers n specific frequency bands of signals are the independent variables for equations used to control a movement of a cursor. Each variable includes the amplitude at a specific frequency in a specific signal. Two control channels are provided for horizontal and vertical movements. Each control channel is defined by an equation. The intercept of each equation corresponds to an average of the value of the independent variables that are stored or recorded from the previous performance of the individual. The cursor could move several times per second in accordance with the scanning frequency. Movement of the cursor on the screen can control other processes and devices by invoking programmed commands displayed on the screen.
Further, in another device [15], a system for monitoring brain waves includes a detection electrode that detects brain waves and is located on a part of the ear. The detection electrode generates a brain wave data signal. A reference electrode detects a reference signal and generates a reference data signal. A monitor receives the brain wave data signal and the reference data signal. The detection electrode and the reference electrode form an electrode pair, and the monitor processes the data reference signal and the brain wave data signal. Accordingly, the above studies indicate that the proposed system and device in this paper for enabling mind reading based on brain signals to control a gadget is feasible and could be a potential candidate for the future HCI devices. Indeed, each of the studies cited in this section justify that the proposed enabler of this paper could be practical, in which the enabler receives a signal indicative of a mental activity of a user, and a processing unit decodes the signal, using pattern recognition to identify and quantify a change in the signal, classifies the signal according to a response index to obtain a classified signal, compares the classified signal to data in a response cache to identify a response that corresponds to the classified signal, and delivers a command to implement the response to a gadget.
One of the unique features of the proposed enabler of this paper is the compactness of the system, since the proposed system can be partly inserted in the ear of the user and the encoding and signal analysis part of the system is performed in the gadget that receives the brain signals to command the gadget. It is noted that some of the distinct features of the proposed system and enabler of this paper might be disclosed in the above studies and the prior art discussed in section 2 of this paper. However, the proposed system and enabler of this paper have novel features as a whole, which were not collectively considered in the cited references. These novel features that could distinguish the proposed system and enabler from the conventional art, among many other features, include easy installation of the enabler in the user's ear to provide convenience wireless communication to adapt to the fast growing technologies of the mobile devices, decoding of the data signals in the processing unit placed in the gadget to reduce the workload, space, and weight of the enabler, and a proper pattern recognition and signal analysis system that could measure mind data under real-time conditions. Accordingly, the propose enabler of this paper to control the gadgets based on the user's mind activities could provide a compact, convenient, and hands-free device, which appears to be the trend in the future of the communication devices, as noted in the aforementioned sections of this report. In additional to command traditional uses of gadgets in daily life, there are several other possible applications for the mindreading enabler presented in this paper.
In the area of medical devices, for example, for paralyzed people, or individuals with language impairment the in-ear brain-computer interface enabler can serve as a portable and easy to use device for communication through mind to control devices such as telephones, electric wheelchairs, etc. through complex mental signals. Further, in the field of leisure activities, the proposed in-ear brain-computer interface of this paper can be used in gaming, in which users can learn to mentally control games.

V. CONCLUSIONS
In conclusion, based on the discussion presented in this paper, it appears that the trend in the future of the HCI devices and systems is moving toward providing systems and devices that could efficiently convey the brain signals to command gadgets, while a user is thinking about commanding the gadgets.
Recently, among researchers in industry and academia, several attempts have been made to enhance the brain-reading www.ijacsa.thesai.org interface technologies. However, as set forth in this paper, each of these devices and systems suffers from deficiencies that refrains the field from achieving maturity. Indeed, it appears that much more research is needed to achieve to a point of commercializing these systems and devices that could be affordable and comfortable for the users. One of the major obstacles in this journey appears to be in the area of pattern recognition of the mind signals, considering the limited understanding of a user's brain and its electrical activities, since the accuracy of a mind signal detection could be degraded as the number of mind states increases, such as when the user thinks about a series of words to implement a task. In this paper a system was proposed that includes an enabler for controlling gadgets based on signal analysis of brain activities transmitted from the enabler to the gadget. The enabler could be inserted in the user's ear and includes a recorder that records brain signals. A processing unit of the system commands the device based on decoding the recorded brain signals. The proposed enabler could provide a compact, convenient, and hands-free device to facilitate a brain-machine interface to control the gadget from electroencephalography signals in the user's brain.