Predicting Multiclass Java Code Readability: A Comparative Study of Machine Learning Algorithms

The classification of program code readability has traditionally focused on two target classes: readable and unreadable. Recently, it has evolved into a multiclass classification task in three categories: readable, neutral, and unreadable. Most of the existing approaches rely on deep learning. This study investigated the multiclass classification of Java code readability using four feature metric datasets and 14 supervised machine learning algorithms. The dataset comprises 200 labeled Java function declarations. Readability features were extracted using Scalabrino’s tool, generating three datasets: Scalabrino, Buse-Weimer, a combined set (Dall), and a fourth (Dcorr) via feature selection based on interfeature correlation. Each model underwent hyperparameter tuning via a Randomized Search and was evaluated through 30 iterations of a five-fold cross-validation. Scaling techniques (MinMax, Standard, Robust, and None) were also compared. The best performance, with an average accuracy of 61.1% and minimal overfitting, was achieved by Random Forest with MinMax scaling on Dcorr. Feature importance analysis using permutation methods identified 22 key metrics related to comments: code complexity, syntax, naming, token usage, and density. Despite its moderate accuracy, the findings offer valuable insights and highlight essential features for advancing code readability research.

@article{Susanto2025,
  title     = {Predicting Multiclass Java Code Readability: A Comparative Study of Machine Learning Algorithms},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {16},
  number    = {4},
  year      = {2025},
  publisher = {The Science and Information Organization},
  author    = {Budi Susanto and Ridi Ferdiana and Teguh Bharata Adji},
  doi       = {10.14569/IJACSA.2025.01604102},
  url       = {https://doi.org/10.14569/IJACSA.2025.01604102}
}