HeritageLM: Culturally-Aware Multimodal Language Modeling with Memory-Enhanced Cross-Dialect Adaptation

The HeritageLM system solves the acute problem of language loss by proposing a multimodal language model that brings into Generative Memory the cultural context. Manual documentation and linguistic archiving are the traditional ways of preserving dialects, which may not be effective in preserving the phonetic variety and other cultural peculiarities of the face of an endangered dialect. The current NLP models, such as BERT and GPT, are not effective to produce dialectal content because they do not have exposure to under-resourced and historically rich language varieties. These shortcomings are mitigated by training Cultural Contextual Embeddings (CCE), Generative Memory Augmentation (GMA), and Cross-Dialect Contrastive Transfer Learning (CDCP) using reinforcement learning with Cultural Rewards (RLCR). It is a step-by-step process that builds a Multimodal Cultural Knowledge Graph (MCKG), matches dialect embeddings in contrastive learning, and retrieves culturally relevant information in the generation process. The model was trained on the Indian Languages Audio Dataset of Kaggle, which also included phonetic variations of ten languages with preprocessing steps of text-to-speech analysis, phonetic annotation, and semantic tagging. HeritageLM, which was implemented in Python, scored above 98 in its BLEU, ROUGE-L, phonetic accuracy, and cultural embedding, showing that it can effectively generate linguistically accurate, phonetically accurate, and culturally authentic results. These outcomes are a major step towards the resurrection of dying dialects and maintaining their distinct cultural background.

@article{Ramana2025,
  title     = {HeritageLM: Culturally-Aware Multimodal Language Modeling with Memory-Enhanced Cross-Dialect Adaptation},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {16},
  number    = {11},
  year      = {2025},
  publisher = {The Science and Information Organization},
  author    = {Pasupuleti Venkata Ramana and K. K. Sunalini and A. Swathi and R. Lakshmi and Raman Kumar and Elangovan Muniyandy and Khaled Bedair},
  doi       = {10.14569/IJACSA.2025.0161160},
  url       = {https://doi.org/10.14569/IJACSA.2025.0161160}
}