Automatic Generation of Comparison Charts of Similar GitHub Repositories from Readme Files

GitHub is a widely used platform for hosting open-source projects, with over 420 million repositories, promoting code sharing and reusability. However, with this tremendous number of repositories, finding a desirable repository based on user needs takes time and effort, especially as the number of candidate repositories increases. A user search can result in thousands of matching results, whereas GitHub shows only basic information about each repository. Therefore, users evaluate repositories’ applicability to their needs by inspecting the documentation of each repository. This paper discusses how comparison charts of similar repositories can be automatically generated to assist users in finding the desirable repository, reducing the time required to inspect their readme files. First, we implement an unsupervised, keyword-driven classifier based on the Lbl2TransformerVec algorithm to classify relevant content of GitHub readme files. The classifier was trained on a dataset of readme files collected from Java, JavaScript, C#, and C++ repositories. The classifier is evaluated against a different dataset of readme files obtained from Python repositories. Evaluation results indicate an F1 score of 0.75. Then, we incorporate rule-based adjustments to enhance classification results by 13%. Finally, the unique features, similarities, and limitations are automatically extracted from readme files to generate comparison charts using Large Language Models (LLMs).

@article{Albassam2024,
  title     = {Automatic Generation of Comparison Charts of Similar GitHub Repositories from Readme Files},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {15},
  number    = {11},
  year      = {2024},
  publisher = {The Science and Information Organization},
  author    = {Emad Albassam},
  doi       = {10.14569/IJACSA.2024.01511110},
  url       = {https://doi.org/10.14569/IJACSA.2024.01511110}
}