Deep Learning based, a New Model for Video Captioning

Visually impaired individuals face many difficulties in their daily lives. In this study, a video captioning system has been developed for visually impaired individuals to analyze the events through real-time images and express them in meaningful sentences. It is aimed to better understand the problems experienced by visually impaired individuals in their daily lives. For this reason, the opinions and suggestions of the disabled individuals within the Altınokta Blind Association (Turkish organization of blind people) have been collected to produce more realistic solutions to their problems. In this study, MSVD which consists of 1970 YouTube clips has been used as training dataset. First, all clips have been muted so that the sounds of the clips have not been used in the sentence extraction process. The CNN and LSTM architectures have been used to create sentence and experimental results have been compared using BLEU 4, ROUGE-L and CIDEr and METEOR.

@article{Özer2020,
  title     = {Deep Learning based, a New Model for Video Captioning},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {11},
  number    = {3},
  year      = {2020},
  publisher = {The Science and Information Organization},
  author    = {Elif Güsta Özer and Ilteber Nur Karapinar and Sena Basbug and Sümeyye Turan and Anil Utku and M. Ali Akcayol},
  doi       = {10.14569/IJACSA.2020.0110365},
  url       = {https://doi.org/10.14569/IJACSA.2020.0110365}
}