Stacking Deep-Learning Model, Stories and Drawing Properties for Automatic Scene Generation

Text-image mapping is of great interest to the scientific community, especially for educational purposes. It helps young learners, mainly those with learning difficulties, to better understand the content of stories. In this paper, we propose to capture the teacher’s experience in manually building relevant scenes for animal behavior stories. This manual work, which consists of a pair of texts and a set of elementary images, is fed into a Long Short-Term Memory (LSTM) followed by a Conditional Random Field (CRF) that aims to associate the relevant words in the text with their corresponding elementary image while preserving the drawing properties. This association is then used for scene construction. Several experiments were con-ducted to show how better the constructed scenes convey textual information than the scenes constructed from the competitor’s models.

@article{Elloumi2023,
  title     = {Stacking Deep-Learning Model, Stories and Drawing Properties for Automatic Scene Generation},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {14},
  number    = {1},
  year      = {2023},
  publisher = {The Science and Information Organization},
  author    = {Samir Elloumi and Nzamba Bignoumba},
  doi       = {10.14569/IJACSA.2023.0140197},
  url       = {https://doi.org/10.14569/IJACSA.2023.0140197}
}