Iterative Partition Optimization: A Novel Approach for Feature Selection in NIR Spectroscopy

Machine learning for near-infrared (NIR) spectroscopy requires effective feature selection to address high dimensionality and multicollinearity. This study proposes Iterative Partition Optimization (IPO), a framework integrating Model Population Analysis with Weighted Binary Matrix Sampling through segment-wise optimization: partitioning spectra into segments, isolating one active segment while freezing others, and using adaptive weighted sampling that learns from best-performing sub-models. Validation across four diverse NIR datasets (n=54-523 samples, 100-700 wavelengths) demonstrates IPO’s consistent performance improvement over conventional methods. For agricultural products (soy flour, wheat kernels), IPO achieved lower RMSECV while reducing wavelengths. In chemical analysis (diesel fuels, manure), the method maintained high prediction accuracy (RPD>3.0) using less than half the original variables. Notably in multi-component manure analysis, IPO improved predictions across seven chemical properties (N, NH4, P2O5, CaO, MgO, K2O, DM) while reducing spectral variables, consistently outperforming MCUVE in both accuracy and wavelength selection efficiency. These results establish IPO as an effective wavelength selection method for NIR spectroscopy, addressing multicollinearity while preserving spectral interpretation through optimized interval selection.

@article{Hoang2025,
  title     = {Iterative Partition Optimization: A Novel Approach for Feature Selection in NIR Spectroscopy},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {16},
  number    = {11},
  year      = {2025},
  publisher = {The Science and Information Organization},
  author    = {Phuong Nguyen Thi Hoang and Thinh Ngo Hung and Tuong Nguyen Huy and Hieu Nguyen Van},
  doi       = {10.14569/IJACSA.2025.0161182},
  url       = {https://doi.org/10.14569/IJACSA.2025.0161182}
}