Evaluating AI-Based Veterinary Symptom-Assessment Chatbots: A Review-Informed Framework for Safety, Triage, Usability, and Owner-Style Symptom Reporting

In the medical field, AI has made its mark, and almost every medical profession has experienced the development of a chatbot to check symptoms and provide instructions to the owners at an early stage. However, determining the safety and reliability of these tools will be challenging since pets are unable to self-report their symptoms, and the input for the chatbot will often be from the pet owner's perception. This may make it difficult to interpret symptoms, assess urgency, and communicate safety advice. This research proposes and adopts a systematic framework for a controlled pet health evaluation scenario to assess an AI-based veterinary symptom assessment Chatbot. The framework evaluates the chatbot's reactions in four areas – diagnostic accuracy, triage behaviour, safety communication, and usability. It is compared with the benchmark results set by experts for controlled owner-style scenarios, and a detailed analysis of the failures of this chatbot’s response is conducted using an interaction-stage error taxonomy. The framework was applied to one veterinary chatbot using 13 controlled owner-style scenarios. Each scenario was evaluated once as a single-turn interaction. The findings showed variation across the four evaluation dimensions, particularly in triage behaviour and safety communication. These findings should be interpreted as evidence from this controlled test setting and should not be generalised to all veterinary chatbots or to real-world clinical use. The study demonstrates how owner-style symptom input, expert benchmarking, multidimensional scoring, and interaction-stage error analysis can be combined within a veterinary-focused evaluation framework.

@article{Chawla2026,
  title     = {Evaluating AI-Based Veterinary Symptom-Assessment Chatbots: A Review-Informed Framework for Safety, Triage, Usability, and Owner-Style Symptom Reporting},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {17},
  number    = {8},
  year      = {2026},
  publisher = {The Science and Information Organization},
  author    = {Manisha Chawla and Abeer Alsadoon and Ahmed Fadhil and Mohammed Ameen and Arafat Abdulgader Mohammed Elhag and Areej Althubaity and Ahmed Hamza Osman},
  doi       = {10.14569/IJACSA.2026.01708100},
  url       = {https://doi.org/10.14569/IJACSA.2026.01708100}
}