Towards Two-Step Fine-Tuned Abstractive Summarization for Low-Resource Language Using Transformer T5

This study explores the potential of two-step fine-tuning for abstractive summarization in a low-resource language, focusing on Indonesian. Leveraging the Transformer-T5 model, the research investigates the impact of transfer learning across two tasks: machine translation and text summarization. Four configurations were evaluated, ranging from zero-shot to two-step fine-tuned models. The evaluation, conducted using the ROUGE metric, shows that the two-step fine-tuned model (T5-MT-SUM) achieved the best performance, with ROUGE-1: 0.7126, ROUGE- 2: 0.6416, and ROUGE-L: 0.6816, outperforming all baselines. These findings demonstrate the effectiveness of task transfer-ability in improving abstractive summarization performance for low-resource languages like Indonesian. This study provides a pathway for advancing natural language processing (NLP) in low-resource language through two-step transfer learning.

@article{Nasution2025,
  title     = {Towards Two-Step Fine-Tuned Abstractive Summarization for Low-Resource Language Using Transformer T5},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {16},
  number    = {2},
  year      = {2025},
  publisher = {The Science and Information Organization},
  author    = {Salhazan Nasution and Ridi Ferdiana and Rudy Hartanto},
  doi       = {10.14569/IJACSA.2025.01602120},
  url       = {https://doi.org/10.14569/IJACSA.2025.01602120}
}