Leveraging Intelligent Speech Training to Elevate Phonetic Accuracy and Prosodic Fluency in English Learners

The successful teaching of pronunciation, as well as prosody, is the significant challenge that still remains to the English as Foreign Learning (EFL) students. Traditional pedagogical theories tend to focus on segmental phoneme accuracy but ignore suprasegmental components (stress or rhythm and intonation) which are natural and intelligible speech components. The currently available systems of computer-assisted pronunciation training (CAPT) are useful, but limited by the fact that they are based on limited acoustic models and incomplete coverage of prosodic characteristics, leading to less than optimal accuracy and limited pedagogical suitability. To overcome these shortcomings, the current paper proposes Attention-Guided Cross-Lingual Self-Supervised Learning (AG-CLSSL), a new model that is both able to combine phoneme-level representations of XLS-R (wav2vec2-large-xlsr-53) and prosodic representations of the pitch, energy, and duration through a Phoneme-Prosody Cross-Attention Fusion (PP-CAF) process. This conglomeration allows the joint and context specific representation of the speech that is further refined by the multi-task Transformer-based scoring model to jointly assess the accuracy of pronunciation, the consistency of the prosody and the general intelligibility. The framework is implemented in Python, with support of PyTorch and Hugging Face Transformers and is trained on an evaluated corpus of EFL learner speech (n=100) with a variety of L1 backgrounds, including Mandarin, Hindi, and Spanish. Experimental assessments indicate significant performance improvement with 55.4% decrease in Phoneme Error rate, 52.0 percent decrease in Word Error rate, 43.3 percent increase in Stress Placement Accuracy and 34.9 percent increase in Pitch Alignment Score. The total acoustic similarity to native speech went up by 36.1, which demonstrates the ability of AG-CLSSL to progress articulatory accuracy as well as the naturalness of prosody and provide interpretable and attention-directed information on scalable AI-based pronunciation and prosody training.

@article{Khapekar2025,
  title     = {Leveraging Intelligent Speech Training to Elevate Phonetic Accuracy and Prosodic Fluency in English Learners},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {16},
  number    = {11},
  year      = {2025},
  publisher = {The Science and Information Organization},
  author    = {Amit Khapekar and Nidhi Mishra and Vijaya Lakshmi Mandava and T K Rama Krishna Rao and Bhuvaneswari Pagidipati and Prasad Devarasetty and Elangovan Muniyandy},
  doi       = {10.14569/IJACSA.2025.0161175},
  url       = {https://doi.org/10.14569/IJACSA.2025.0161175}
}