Comparative Evaluation of Traditional and Transformer-Based Models for Risk-Level Classification of Uzbek Telegram Messages

The rapid growth of Telegram-based communication has increased the dissemination of harmful and risky content, particularly in low-resource languages such as Uzbek. This study investigates the automatic classification of Uzbek Telegram messages according to risk level using both traditional machine learning and transformer-based models. A dataset consisting of 10,000 real Telegram messages was collected and manually annotated into two classes: Safe and Dangerous. To improve data quality and consistency, preprocessing techniques including URL removal, emoji normalization, stop-word filtering, and script unification were applied. The study compares the performance of TF-IDF + Logistic Regression, FastText, mBERT, and XLM-RoBERTa for harmful content detection in Uzbek Telegram texts. Experimental results show that transformer-based models significantly outperform traditional approaches. Among all evaluated models, XLM-RoBERTa achieved the highest performance, with an Accuracy of 91.2%, a precision of 90.8%, a recall of 91.5%, and an F1-score of 91.1%, while mBERT achieved an Accuracy of 84.9% and an F1-score of 84.6%. The results demonstrate the effectiveness of contextual transformer architectures for identifying harmful content in low-resource language environments. The findings confirm that transformer-based models provide a reliable solution for automatic risk-level classification of Uzbek social media texts and can support practical applications in content moderation, information security, and social media monitoring systems.

@article{Qoyliyeva2026,
  title     = {Comparative Evaluation of Traditional and Transformer-Based Models for Risk-Level Classification of Uzbek Telegram Messages},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {17},
  number    = {6},
  year      = {2026},
  publisher = {The Science and Information Organization},
  author    = {Feruzakhon A. Qoyliyeva and Ozod J.Babomuradov and Akmal A. Savurbayev},
  doi       = {10.14569/IJACSA.2026.0170651},
  url       = {https://doi.org/10.14569/IJACSA.2026.0170651}
}