Abstract: The current era is characterized by technological advancement and innovation, which affect various sectors. Numerous remarkable and alluring computer programs and applications have surfaced, including ones that aim to replicate human behavior. A chatbot is an example of an Artificial Intelligence (AI) computer program that uses natural language to mimic human conversations in voice or content. Even though a lack of Arabic chatbots, most of these chatbots use Modern Standard Arabic instead of Arabic dialects. This research presents the development and evaluation of a chatbot designed to respond to academic inquiries from university students using the Western Saudi dialect. A traditional Support Vector Machine (SVM) baseline model was first implemented to establish a reference point for performance. Subsequently, a fine-tuned version of Generative Pre-trained Transformer (GPT) 3.5-Turbo-0125 was developed using a culturally specific system prompt to enhance the model’s understanding of regional language and academic contexts. Evaluation was conducted through a multi-dimensional framework combining human assessments, BERTScore semantic similarity measurements, and GPT-4-based automatic judging. With human assessors determining that 85% of GPT-3.5's replies to 132 messages of test data were appropriate, the transformer-based model clearly outperformed the SVM baseline, which had an accuracy of 42.86% on 20 messages of test data. These findings highlight the importance of cultural and contextual fine-tuning in building effective conversational agents for dialectal Arabic communities. The research contributes to the growing field of localized AI by demonstrating how advanced language models can be adapted to serve specialized linguistic and academic needs.

Maimounah Alhujaili and Ruqayya Abdulrahman, “Fine-Tuning OpenAI GPT Chatbot in Western Saudi Dialect: A Case Study of Taibah University” International Journal of Advanced Computer Science and Applications(IJACSA), 16(6), 2025. http://dx.doi.org/10.14569/IJACSA.2025.0160632

@article{Alhujaili2025,
title = {Fine-Tuning OpenAI GPT Chatbot in Western Saudi Dialect: A Case Study of Taibah University},
journal = {International Journal of Advanced Computer Science and Applications},
doi = {10.14569/IJACSA.2025.0160632},
url = {http://dx.doi.org/10.14569/IJACSA.2025.0160632},
year = {2025},
publisher = {The Science and Information Organization},
volume = {16},
number = {6},
author = {Maimounah Alhujaili and Ruqayya Abdulrahman}
}