Arabic Image Captioning: The Effect of Text Pre-processing on the Attention Weights and the BLEU-N Scores

Image captioning using deep neural networks has recently gained increasing attention, mostly for English langue, with only few studies in other languages. Good image captioning model is required to automatically generate sensible, syntactically and semantically correct captions, which in turn requires good models for both computer vision and natural language processing. The process is more challenging in case of data scarcity, and languages with complex morphological structures like the Arabic language. This was the reason why only limited number of studies have been published for Arabic image captioning, compared to those of English language. In this paper, an efficient deep learning model for Arabic image captioning has been proposed. In addition, the effect of using different text pre-processing methods on the obtained BLEU-N scores and the quality of generated images, as well as the attention mechanism behavior were investigated. Furthermore, the “THUMB” framework to assess the quality of the generated captions is used -for the first time- for Arabic captions’ evaluation. As shown in the results, a BLEU-4 score of 27.12, has been achieved, which is the highest obtained results so far, for Arabic image captioning. In addition, the best THUMB scores were obtained, compared to previously published results on common images.

@article{Lasheen2022,
  title     = {Arabic Image Captioning: The Effect of Text Pre-processing on the Attention Weights and the BLEU-N Scores},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {13},
  number    = {7},
  year      = {2022},
  publisher = {The Science and Information Organization},
  author    = {Moaz T. Lasheen and Nahla H. Barakat},
  doi       = {10.14569/IJACSA.2022.0130751},
  url       = {https://doi.org/10.14569/IJACSA.2022.0130751}
}