Abstract: This study explores the potential of two-step fine-tuning for abstractive summarization in a low-resource language, focusing on Indonesian. Leveraging the Transformer-T5 model, the research investigates the impact of transfer learning across two tasks: machine translation and text summarization. Four configurations were evaluated, ranging from zero-shot to two-step fine-tuned models. The evaluation, conducted using the ROUGE metric, shows that the two-step fine-tuned model (T5-MT-SUM) achieved the best performance, with ROUGE-1: 0.7126, ROUGE- 2: 0.6416, and ROUGE-L: 0.6816, outperforming all baselines. These findings demonstrate the effectiveness of task transfer-ability in improving abstractive summarization performance for low-resource languages like Indonesian. This study provides a pathway for advancing natural language processing (NLP) in low-resource language through two-step transfer learning.

Salhazan Nasution, Ridi Ferdiana and Rudy Hartanto, “Towards Two-Step Fine-Tuned Abstractive Summarization for Low-Resource Language Using Transformer T5” International Journal of Advanced Computer Science and Applications(IJACSA), 16(2), 2025. http://dx.doi.org/10.14569/IJACSA.2025.01602120

@article{Nasution2025,
title = {Towards Two-Step Fine-Tuned Abstractive Summarization for Low-Resource Language Using Transformer T5},
journal = {International Journal of Advanced Computer Science and Applications},
doi = {10.14569/IJACSA.2025.01602120},
url = {http://dx.doi.org/10.14569/IJACSA.2025.01602120},
year = {2025},
publisher = {The Science and Information Organization},
volume = {16},
number = {2},
author = {Salhazan Nasution and Ridi Ferdiana and Rudy Hartanto}
}