A Soft and Hard Mixture-of-Experts Approach for Improved ADR Extraction from Patient-Generated Narratives

Traditional single-architecture neural models, in-cluding monolithic transformer-based and sequence-to-sequence architectures, often struggle to extract Adverse Drug Reactions (ADRs) from patient-generated health narratives due to informal language, high linguistic variability, and complex relationships among drugs, diseases, and adverse events. Although Mixture-of-Experts (MoE) architectures have demonstrated strong performance across various Natural Language Processing (NLP) tasks, their effectiveness for ADR extraction from unstructured patient narratives remains largely unexplored. This study investigates the application of MoE architectures, specifically Soft MoE and Hard MoE, for ADR extraction from patient-generated content. The task is formulated as a sequence-to-sequence generation problem and evaluated on the PsyTAR dataset using both strict and relaxed evaluation metrics. Experimental results demonstrate that Soft MoE consistently outperforms Hard MoE, achieving a relaxed F1-score of 80.40% compared to 79.40%. These findings highlight the critical role of expert-routing strategies in capturing linguistic variability in patient narratives and establish MoE architectures as a competitive and reliable approach for automated ADR extraction in biomedical text mining and pharmacovigilance applications.

@article{Elbiach2025,
  title     = {A Soft and Hard Mixture-of-Experts Approach for Improved ADR Extraction from Patient-Generated Narratives},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {16},
  number    = {12},
  year      = {2025},
  publisher = {The Science and Information Organization},
  author    = {Oumayma Elbiach and Hanane Grissette and El Habib Nfaoui},
  doi       = {10.14569/IJACSA.2025.01612129},
  url       = {https://doi.org/10.14569/IJACSA.2025.01612129}
}