Cross-Modal Fine-Grained Interaction Fusion in Fake News Detection

The popularity of social media has significantly increased the speed and scope of news dissemination, making the emergence and spread of fake news easier. Current fake news detection methods often ignore the correlation between text and images, leading to insufficient modal interaction and fusion. To address these issues, a cross-modal fine-grained interaction and fusion model for fake news detection is proposed. Specifically, this study addresses the correlation problem between text and image modalities by designing an interaction similarity domain. It extracts features of text word weight distribution using an attention mechanism network, guides the features of different regions of the image, and calculates the local similarity between the two. This approach analyzes positive and negative correlations between modalities at a fine-grained level, thereby strengthening the intermodal connection. Additionally, to tackle the problem of insufficient fusion of semantic feature vectors between text and images, this paper designs a fusion network that employs improved encoding and decoding using a Transformer for inter-modal information fusion, achieving the final multimodal feature representation. Experimental results show that our proposed method achieves excellent performance on WeiboA and Twitter, with accuracies of 88.2% and 89%, respectively, outperforming the benchmark model in several evaluation metrics.

@article{Che2024,
  title     = {Cross-Modal Fine-Grained Interaction Fusion in Fake News Detection},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {15},
  number    = {5},
  year      = {2024},
  publisher = {The Science and Information Organization},
  author    = {Zhanbin Che and GuangBo Cui},
  doi       = {10.14569/IJACSA.2024.0150596},
  url       = {https://doi.org/10.14569/IJACSA.2024.0150596}
}