Cross-Language Plagiarism Detection using Word Embedding and Inverse Document Frequency (IDF)

The purpose of cross-language textual similarity detection is to approximate the similarity of two textual units in different languages. This paper embeds the distributed representation of words in cross-language textual similarity detection using word embedding and IDF. The paper introduces a novel cross-language plagiarism detection approach constructed with the distributed representation of words in sentences. To improve the textual similarity of the approach, a novel method is used called CL-CTS-CBOW. Consequently, adding the syntax feature to the approach is improved by a novel method called CL-WES. Afterward, the approach is improved by the IDF weighting method. The corpora used in this study are four Arabic-English corpora, specifically books, Wikipedia, EAPCOUNT, and MultiUN, which have more than 10,017,106 sentences and uses with supported parallel and comparable assemblages. The proposed method in this paper combines different methods to confirm their complementarity. In the experiment, the proposed system obtains 88% English-Arabic similarity detection at the word level and 82.75% at the sentence level with various corpora.

@article{Aljuaid2020,
  title     = {Cross-Language Plagiarism Detection using Word Embedding and Inverse Document Frequency (IDF)},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {11},
  number    = {2},
  year      = {2020},
  publisher = {The Science and Information Organization},
  author    = {Hanan Aljuaid},
  doi       = {10.14569/IJACSA.2020.0110231},
  url       = {https://doi.org/10.14569/IJACSA.2020.0110231}
}