Interpreting Multimodal Fake News Detection Models: An Experimental Study of Performance Factors and Modality Contributions

The widespread dissemination of multimodal mis-information requires models that can reason across textual and visual content while remaining interpretable. However, many existing multimodal fusion approaches implicitly assume uniform modality reliability, providing limited transparency into modality contributions. This study introduces TweFuse-W, a lightweight multimodal framework for fine-grained fake-news detection that reframes multimodal fusion as a modality reliability estimation problem, rather than merely merging modalities or explicitly modeling their interactions. TweFuse-W integrates BERTweet-based textual representations with Swin Transformer visual features using a sample-conditioned, learnable weighted-sum gate operating at the modality level, producing global reliability weights without cross-attention overhead. By explicitly param-eterizing modality contributions during inference, the proposed approach provides intrinsic interpretability. Experiments on the six-class Fakeddit dataset show that TweFuse-W achieves a macro-F1 score of 0.838, outperforming simple concatenation (macro-F1 = 0.820). Analysis of the learned modality weights confirms meaningful interpretability, with textual representations dominating in Satire, Misleading, False Connection, and Imposter Content (αT = 0.57–0.62), while visual cues exert greater influence in Manipulated Content (αV = 0.51). Overall, these findings demonstrate that adaptive modality weighting enhances both predictive performance and model transparency, serving as a lightweight and interpretable complementary fusion strategy for multimodal fake-news detection.

@article{Eldien2026,
  title     = {Interpreting Multimodal Fake News Detection Models: An Experimental Study of Performance Factors and Modality Contributions},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {17},
  number    = {1},
  year      = {2026},
  publisher = {The Science and Information Organization},
  author    = {Noha A. Saad Eldien and Wael H. Gomaa and Khaled T. Wassif and Hanaa Bayomi},
  doi       = {10.14569/IJACSA.2026.0170186},
  url       = {https://doi.org/10.14569/IJACSA.2026.0170186}
}