Improving Performance and Accuracy in Decision Trees: A Literature Survey on Impurity Functions

Abstract: Decision tree algorithms have remained some of the most popular tools for supervised learning tasks. This has been because of their comprehensibility, malleability, and robustness to accommodate varying types of data. Nevertheless, their accuracy and performance are dependent on the use of impurity measures to guide the process of tree splits. The standard impurity measures, such as the Gini impurity, Entropy, and Classification Error, tend to face problems as the complexity, imbalance, and noise levels are raised. This often gives rise to overfitting. An examination of the disadvantages and present research efforts on impurity-based optimization of decision trees, as well as the study of new paradigms such as the use of Rényi Entropy, the utilization of the Tsallis Entropy, combinations of impurity functions, complexity-aware tree splits, and tailored impurity function augmentation, makes it clear that there are efforts underway to improve the accuracy, robustness, and comprehensibility, as well as the processing complexity, for the tasks of decision tree algorithms.

Abed Alsulami, Reda Khalifa and Wajdi Alghamdi. “Improving Performance and Accuracy in Decision Trees: A Literature Survey on Impurity Functions”. International Journal of Advanced Computer Science and Applications (IJACSA) 17.4 (2026). http://dx.doi.org/10.14569/IJACSA.2026.0170495

@article{Alsulami2026,
title = {Improving Performance and Accuracy in Decision Trees: A Literature Survey on Impurity Functions},
journal = {International Journal of Advanced Computer Science and Applications},
doi = {10.14569/IJACSA.2026.0170495},
url = {http://dx.doi.org/10.14569/IJACSA.2026.0170495},
year = {2026},
publisher = {The Science and Information Organization},
volume = {17},
number = {4},
author = {Abed Alsulami and Reda Khalifa and Wajdi Alghamdi}
}