Rubric-Relational Discourse Modeling with Counterfactual Explainability for Multi-Trait Automated Essay Scoring

Abstract: Automated Essay Scoring (AES) systems often rely on holistic prediction and show weak alignment with rubric-based human evaluation. Existing deep learning approaches achieve moderate agreement but struggle to model discourse coherence and provide trait-faithful explanations. This study proposes a rubric-aware and discourse-faithful essay scoring framework that integrates contextual embeddings with sentence-level discourse modeling and rubric-specific attention. The framework generates both holistic and trait-level scores, while enabling counterfactual explanation of scoring decisions. Experiments conducted on the Learning Agency Lab – Automated Essay Scoring 2.0 dataset show that the proposed model achieves a Quadratic Weighted Kappa (QWK) of 0.86, Root Mean Square Error (RMSE) of 1.41, and Mean Absolute Error (MAE) of 1.12, outperforming CNN-LSTM, BERT-LSTM, and DeBERTa baselines. QWK evaluates ordinal agreement, while RMSE and MAE measure numerical prediction error. Trait-level performance reaches F1-scores of 0.89 for Content and 0.87 for Grammar, indicating strong rubric alignment. The proposed framework improves scoring reliability, interpretability, and consistency with human grading practices. It is suitable for large-scale educational assessment, formative feedback systems, and intelligent tutoring applications, offering a scalable and explainable solution for multi-trait essay evaluation.

N. Sreedevi, M. Madhusudhan Rao, Sridevi Dasam, Roopa Traisa, Jasgurpreet Singh Chohan, V. Saranya and Ahmed I. Taloba. “Rubric-Relational Discourse Modeling with Counterfactual Explainability for Multi-Trait Automated Essay Scoring”. International Journal of Advanced Computer Science and Applications (IJACSA) 17.2 (2026). http://dx.doi.org/10.14569/IJACSA.2026.0170242

@article{Sreedevi2026,
title = {Rubric-Relational Discourse Modeling with Counterfactual Explainability for Multi-Trait Automated Essay Scoring},
journal = {International Journal of Advanced Computer Science and Applications},
doi = {10.14569/IJACSA.2026.0170242},
url = {http://dx.doi.org/10.14569/IJACSA.2026.0170242},
year = {2026},
publisher = {The Science and Information Organization},
volume = {17},
number = {2},
author = {N. Sreedevi and M. Madhusudhan Rao and Sridevi Dasam and Roopa Traisa and Jasgurpreet Singh Chohan and V. Saranya and Ahmed I. Taloba}
}