Text-to-Image Generation Method Based on Object Enhancement and Attention Maps

In the task of text-to-image generation, common issues such as missing objects in the generated images often arise due to the model's insufficient learning of multi-object category information and the lack of consistency between the text prompts and the generated image contents. To address these challenges, this paper proposes a novel text-to-image generation approach based on object enhancement and attention maps. First, a new object enhancement strategy is introduced to improve the model’s capacity to capture object-level features. The core idea is to generate difficult samples by processing the object mask maps of tokens, followed by dynamic weighting of the attention map using latent image embeddings. Second, to enhance the consistency between the text prompts and the generated image contents, we enforce similarity constraints between the cross-attention maps and the attention-weighted mask feature maps, penalizing inconsistencies through a loss function. Experimental results demonstrate that the Stable Diffusion v1.4 model, optimized using the proposed method, achieves significant improvements on the COCO instance dataset and the ADE20K instance dataset. Specifically, the MG metrics are improved by an average of 12.36% and 6.55%, respectively, compared to state-of-the-art models. Furthermore, the FID metrics show a 0.84% improvement over the state-of-the-art model on the COCO instance validation set.

@article{Huang2025,
  title     = {Text-to-Image Generation Method Based on Object Enhancement and Attention Maps},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {16},
  number    = {1},
  year      = {2025},
  publisher = {The Science and Information Organization},
  author    = {Yongsen Huang and Xiaodong Cai and Yuefan An},
  doi       = {10.14569/IJACSA.2025.0160193},
  url       = {https://doi.org/10.14569/IJACSA.2025.0160193}
}