Holistic Evaluation Framework for Automated Bug Triage Systems: Integration of Developer Performance

Bug Triage is an important aspect of Open Source Software Development. Automated Bug Triage system is essential to reduce the cost and effort incurred by manual Bug Triage. At present, the metrics that are available in the literature to evaluate the Automated Bug Triage System are only recommendation centric. These metrics address only the correctness and coverage of the Automated Bug Triage System. Thus, there is a need for user-centric evaluation of the Bug Triage System. The two types of metrics to evaluate the Automated Bug Triage System include Recommendation Metrics and User Metrics. There is a need to corroborate the results produced by the Recommendation Metrics with User Metrics. To this end, this paper furnishes a Holistic Evaluation Framework for Bug Triage System by integrating the developer performance into the evaluation framework. The Automated Bug Triage System is also to retrieve a set of developers for resolving a bug. Hence, this paper proposes Key Performance Indicators (KPI) for appraising a developer’s effectiveness in contribution towards the resolution of the bug. By applying the KPIs on the retrieved set of developers, the Bug Triage System can be evaluated quantitatively.

@article{Akila2016,
  title     = {Holistic Evaluation Framework for Automated Bug Triage Systems: Integration of Developer Performance},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {7},
  number    = {4},
  year      = {2016},
  publisher = {The Science and Information Organization},
  author    = {Dr. V. Akila and Dr.V.Govindasamy},
  doi       = {10.14569/IJACSA.2016.070429},
  url       = {https://doi.org/10.14569/IJACSA.2016.070429}
}