Optimizing Text Summarization with Sentence Clustering and Natural Language Processing

Text summarization is an important task in natural language processing (NLP), with significant implications for information retrieval and content management. Traditional summarization methods often struggle with issues like redundancy, loss of key information, and inability to capture the underlying semantic structure of the text. This paper addresses these challenges by presenting an advanced approach to extractive summarization, which integrates clustering-based sentence selection with the BART model. The proposed method tackles the problem of redundancy by using Term Frequency-Inverse Document Frequency (TF-IDF) for feature extraction, followed by K-means clustering to group similar sentences. This clustering step is designed to reduce redundancy by ensuring that each cluster represents a distinct theme or topic. Representative sentences are then selected from these clusters based on their cosine similarity to a user query, which helps in retaining the most relevant information. These selected sentences are then fed into the BART model to generate the final abstractive summary. This combination of extractive and abstractive techniques addresses the common problem of information loss, ensuring that the summary is both comprehensive and coherent. The approach is evaluated using the CNN/DailyMail and XSum datasets, which are widely recognized benchmarks in the summarization domain. Results assessed through ROUGE metrics demonstrate that the proposed model substantially improves summarization quality compared to existing benchmarks.

@article{Edress2024,
  title     = {Optimizing Text Summarization with Sentence Clustering and Natural Language Processing},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {15},
  number    = {10},
  year      = {2024},
  publisher = {The Science and Information Organization},
  author    = {Zahir Edress and Yasin Ortakci},
  doi       = {10.14569/IJACSA.2024.01510115},
  url       = {https://doi.org/10.14569/IJACSA.2024.01510115}
}