Temporal Attention Networks for Real-Time Multimodal Emotion Recognition from EEG and fNIRS Signals

Abstract: Emotion recognition is critical in the development of real-time mental health care and individualized cognitive behavior. Current strategies to recognize cognitive emotions frequently fail to capture complex time dependencies and multimodal physiological reactions, leading to sub-optimal performance and inaccurate generalization. To overcome such shortcomings, the proposed study suggests TCADNet, a new deep learning model that integrates Temporal Convolutional Networks (TCN), attention-based feature weighting, and GAN-based data augmentation to achieve a high recognition rate of the emotional states through EEG and fNIRS recordings. The model utilizes the TCNs to extract both short-term and long-term temporal trends, and the attention mechanism emphasizes salient parts that bring about emotions, which improves interpretability. Moreover, a Deep Convolutional GAN creates artificial signals of unrepresented emotion classes, eliminating data imbalance and enhancing generalization. The TCADNet model is coded in Python on the TensorFlow/Keras system, and its key components are preprocessing, time modeling, attention weighting, data augmentation, and last classification by SoftMax layers. Experimental outcomes indicate that TCADNet has high recognition performance, with overall recognition, accuracy, precision, and recall, and F1-scores of over 98, which is higher than conventional CNN, LSTM, and separate TCN models. The suggested methodology can be useful to researchers, clinicians, and mental health professionals as it allows them to monitor cognitive and emotional conditions in real-time with a reliable, decipherable, and scalable instrument and provides an opportunity to detect and respond to the issue promptly and implement a tailored intervention plan in educational or health-related settings.

Vinod Waiker, Anne Marie D. Pahiwon, Ankush Mehta, Gadug Sudhamsu, Pavithra M, K. Kiran Kumar, B Kiran Bala and Osama R. Shahin. “Temporal Attention Networks for Real-Time Multimodal Emotion Recognition from EEG and fNIRS Signals”. International Journal of Advanced Computer Science and Applications (IJACSA) 17.2 (2026). http://dx.doi.org/10.14569/IJACSA.2026.0170267

@article{Waiker2026,
title = {Temporal Attention Networks for Real-Time Multimodal Emotion Recognition from EEG and fNIRS Signals},
journal = {International Journal of Advanced Computer Science and Applications},
doi = {10.14569/IJACSA.2026.0170267},
url = {http://dx.doi.org/10.14569/IJACSA.2026.0170267},
year = {2026},
publisher = {The Science and Information Organization},
volume = {17},
number = {2},
author = {Vinod Waiker and Anne Marie D. Pahiwon and Ankush Mehta and Gadug Sudhamsu and Pavithra M and K. Kiran Kumar and B Kiran Bala and Osama R. Shahin}
}