Dynamic Programming Method Applied in Vietnamese Word Segmentation Based on Mutual Information among Syllables

Abstract: Vietnamese word segmentation is an important step in Vietnamese natural language processing such as text categorization, text summary, and automated machine translation. The problem with Vietnamese word segmentation is complicated because Vietnamese words are not always separated by a space. One word can include one or more syllables depending on the context. This paper proposes a method for Vietnamese word segmentation based on the mutual information among the syllables combined with dynamic programming. With this method, we can achieve an accuracy rate of about 90% with a raw text corpus.

Nguyen Thi Uyen and Tran Xuan Sang. “Dynamic Programming Method Applied in Vietnamese Word Segmentation Based on Mutual Information among Syllables”. International Journal of Advanced Research in Artificial Intelligence (IJARAI) 3.9 (2014). http://dx.doi.org/10.14569/IJARAI.2014.030904

@article{Uyen2014,
title = {Dynamic Programming Method Applied in Vietnamese Word Segmentation Based on Mutual Information among Syllables},
journal = {International Journal of Advanced Research in Artificial Intelligence},
doi = {10.14569/IJARAI.2014.030904},
url = {http://dx.doi.org/10.14569/IJARAI.2014.030904},
year = {2014},
publisher = {The Science and Information Organization},
volume = {3},
number = {9},
author = {Nguyen Thi Uyen and Tran Xuan Sang}
}