HCC: A Hierarchical Chart Captioning Model for Enhanced Accessibility of Chart Data for Visually Impaired Users

In educational settings, charts and graphs are commonly used to convey complex information in a simple and understandable manner. However, these visual representations often present accessibility challenges regarding Accessibility for Visually Impaired users, as they cannot be directly interpreted by screen readers without proper alternative text. This pa-per proposes a novel hierarchical captioning model (HCC: Hierarchical Chart Captioning) designed to facilitate effective Chart Interpretation. The model utilizes spatial token features to generate captions at multiple levels, each offering varying degrees of detail and abstraction, mimicking human cognitive processing. Three hierarchical levels are developed: Level 1 offers basic and factual descriptions, Level 2 presents more detailed information, and Level 3 provides intuitive interpretations and inferences. By integrating a fine-tuned Transformer Models, this approach ensures efficient caption generation and supports user-selectable caption lengths. The model’s effectiveness is evaluated through user surveys involving 20 instructors, confirming that Level 2 captions provide the most comprehensible descriptions. Experimental results demonstrate that the proposed method outperforms existing captioning approaches, improving both the efficiency and accessibility of educational materials for visually impaired students. These findings highlight the potential of hierarchical learning models to create more inclusive and accessible educational experiences.

@article{Song2025,
  title     = {HCC: A Hierarchical Chart Captioning Model for Enhanced Accessibility of Chart Data for Visually Impaired Users},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {16},
  number    = {12},
  year      = {2025},
  publisher = {The Science and Information Organization},
  author    = {Yoojeong Song and Kanghyeon Seo and Svetlana Kim and Joo Hyun Park},
  doi       = {10.14569/IJACSA.2025.01612117},
  url       = {https://doi.org/10.14569/IJACSA.2025.01612117}
}