Retrieval-Augmented Generation Approach: Document Question Answering using Large Language Model

This study introduces the Retrieval Augmented Generation (RAG) method to improve Question-Answering (QA) systems by addressing document processing in Natural Language Processing problems. It represents the latest breakthrough in applying RAG to document question and answer applications, overcoming previous QA system obstacles. RAG combines search techniques in vector store and text generation mechanism developed by Large Language Models, offering a time-efficient alternative to manual reading limitations. The research evaluates RAG's that use Generative Pre-trained Transformer 3.5 or GPT-3.5-turbo from the ChatGPT model and its impact on document data processing, comparing it with other applications. This research also provides datasets to test the capabilities of the QA document system. The proposed dataset and Stanford Question Answering Dataset (SQuAD) are used for performance testing. The study contributes theoretically by advancing methodologies and knowledge representation, supporting benchmarking in research communities. Results highlight RAG's superiority: achieving a precision of 0.74 in Recall-Oriented Understudy for Gisting Evaluation (ROUGE) testing, outperforming others at 0.5; obtaining an F1 score of 0.88 in BERTScore, surpassing other QA apps at 0.81; attaining a precision of 0.28 in Bilingual Evaluation Understudy (BLEU) testing, surpassing others with a precision of 0.09; and scoring 0.33 in Jaccard Similarity, outshining others at 0.04. These findings underscore RAG's efficiency and competitiveness, promising a positive impact on various industrial sectors through advanced Artificial Intelligence (AI) technology.

@article{Muludi2024,
  title     = {Retrieval-Augmented Generation Approach: Document Question Answering using Large Language Model},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {15},
  number    = {3},
  year      = {2024},
  publisher = {The Science and Information Organization},
  author    = {Kurnia Muludi and Kaira Milani Fitria and Joko Triloka and Sutedi},
  doi       = {10.14569/IJACSA.2024.0150379},
  url       = {https://doi.org/10.14569/IJACSA.2024.0150379}
}