Residual DDPG Control with Error-Aware Reward Rescaling for Active Suspension Under Unseen Road Conditions

Abstract: This study investigates a hybrid residual control framework combining Deep Deterministic Policy Gradient (DDPG) and a Proportional–Integral–Derivative (PID) based correction module for active suspension (AS) systems, aiming to improve ride performance and generalization under complex road excitations. The DDPG controller is trained on sinewave inputs, while the PID module compensates for residual errors to enhance robustness. To further guide policy optimization, an error-aware reward rescaling strategy is introduced during training, adaptively shaping the reward signal based on acceleration deviation. The controller is tested under five typical road conditions. These include sinewave inputs and step inputs, and ISO 8608 Level B random profiles. Simulation results show that the residual DDPG (RDDPG) controller works better than both DDPG alone and the PID controller. It reduces vertical acceleration RMS by 50.35% under a 0.05 m sinewave input. This shows that using reinforcement learning (RL) with fast correction and reward adjustment is a useful and stable way to control AS in different driving conditions.

Zien Zhang, Abdul Hadi Abd Rahman and Noraishikin Zulkarnain. “Residual DDPG Control with Error-Aware Reward Rescaling for Active Suspension Under Unseen Road Conditions”. International Journal of Advanced Computer Science and Applications (IJACSA) 16.8 (2025). http://dx.doi.org/10.14569/IJACSA.2025.0160823

@article{Zhang2025,
title = {Residual DDPG Control with Error-Aware Reward Rescaling for Active Suspension Under Unseen Road Conditions},
journal = {International Journal of Advanced Computer Science and Applications},
doi = {10.14569/IJACSA.2025.0160823},
url = {http://dx.doi.org/10.14569/IJACSA.2025.0160823},
year = {2025},
publisher = {The Science and Information Organization},
volume = {16},
number = {8},
author = {Zien Zhang and Abdul Hadi Abd Rahman and Noraishikin Zulkarnain}
}