Corpus for Test, Compare and Enhance Arabic Root Extraction Algorithms

Abstract: Many studies have focused recently on building, evaluating and comparing Arabic root extracting algorithm. The main challenges facing root extraction algorithms are the absence of standard data set for testing, comparing and enhancing different Arabic root extraction algorithms. In addition, the absence of complete lists of roots prefixes suffixes and patterns. In this paper, we describe the development of a new corpus driven from traditional Arabic dictionaries “mu’jams”. The goal is to use the corpus, as a new gold standard data set for testing, comparing and enhancing different Arabic root extraction algorithms. This data set covers all types of words and all roots. It contains each word and its root as a pair to avoid the consultation of a human expert needed to verify the correct roots of words used in the testing or comparing process. We describe the individual phases of the corpus construction, i.e. normalisation, reading derivation words and roots as a pair, and reading each root and its definition part. We have automatically extracted (12000) roots, (430) prefixes, (320) suffixes, (4320) patterns, and (720,000) word-root pair. Konja’s and Garside Arabic root extraction algorithm was tested on this corpus; the accuracy was (63%), then we test it after supplying it with our lists of roots prefixes suffixes and patterns, the accuracy of it became 84%.

Nisrean Thalji, Nik Adilah Hanin, Yasmin Yacob and Sohair Al-Hakeem. “Corpus for Test, Compare and Enhance Arabic Root Extraction Algorithms”. International Journal of Advanced Computer Science and Applications (IJACSA) 8.5 (2017). http://dx.doi.org/10.14569/IJACSA.2017.080529

@article{Thalji2017,
title = {Corpus for Test, Compare and Enhance Arabic Root Extraction Algorithms},
journal = {International Journal of Advanced Computer Science and Applications},
doi = {10.14569/IJACSA.2017.080529},
url = {http://dx.doi.org/10.14569/IJACSA.2017.080529},
year = {2017},
publisher = {The Science and Information Organization},
volume = {8},
number = {5},
author = {Nisrean Thalji and Nik Adilah Hanin and Yasmin Yacob and Sohair Al-Hakeem}
}