Abstract: This study presents a robust multimodal emotion analysis model aimed at improving emotion prediction in film and television communication. Addressing challenges in modal fusion and data association, the model integrates a Transformer-based framework with multi-task learning to capture emotional associations and temporal features across various modalities. It overcomes the limitations of single-modal labels by incorporating multi-task learning, and is tested on the Cmumosi dataset using both classification and regression tasks. The model achieves strong performance, with an average absolute error of 0.70, a Pearson correlation coefficient of 0.82, and an accuracy of 47.1% in a seven-class task. In a two-class task, it achieves an accuracy and F1 score of 88.4%. Predictions for specific video segments are highly consistent with actual labels, with predicted scores of 2.15 and 1.4. This research offers a new approach to multimodal emotion analysis, providing valuable insights for film and television content creation and setting the foundation for further advancements in this area.

Wan Fan, “Enhancing Emotion Prediction in Multimedia Content Through Multi-Task Learning” International Journal of Advanced Computer Science and Applications(IJACSA), 16(2), 2025. http://dx.doi.org/10.14569/IJACSA.2025.01602118

@article{Fan2025,
title = {Enhancing Emotion Prediction in Multimedia Content Through Multi-Task Learning},
journal = {International Journal of Advanced Computer Science and Applications},
doi = {10.14569/IJACSA.2025.01602118},
url = {http://dx.doi.org/10.14569/IJACSA.2025.01602118},
year = {2025},
publisher = {The Science and Information Organization},
volume = {16},
number = {2},
author = {Wan Fan}
}