Achieving Long-Term Autonomy: A Self-Correcting Deep Reinforcement Learning Agent for Edge IoT Using Digital Twin-Based Drift Compensation

Ensuring long-term autonomy in Edge AI systems remains one of the most persistent challenges in environmental monitoring and biorisk management. Over time, the degradation of low-cost sensors—particularly sensor drift—leads to cumulative measurement errors, distorted state perception, and catastrophic decision failures in Deep Reinforcement Learning (DRL) agents. This paper proposes a novel Self-Correcting Deep Reinforcement Learning (SCDRL) framework that enables robust, long-term autonomy through in-loop drift compensation. The proposed Self-Correcting Agent (SCA) integrates a dual-input architecture combining (i) the local, drifted sensor reading and (ii) a stable reference prediction from a macro-scale Digital Twin (DT). By learning to correlate both signals, the agent implicitly estimates and neutralizes sensor bias in real time, achieving self-calibration without human intervention. To validate this approach, a nine-year simulation of autonomous water management was conducted using real-world hourly climate data from Arequipa, Peru. Results show that a conventional “blind” DRL agent suffers complete performance collapse as drift accumulates, whereas the proposed SCA maintains stable operation indefinitely. Quantitatively, the SCA achieved a 722% higher cumulative reward (415,662 vs. 57,556) and a 53% reduction in plant stress (RMSE 0.2238 vs. 0.4762). These findings establish a validated blueprint for fault-tolerant Edge AI, demonstrating that the fusion of local sensing with digital twin predictions enables self-calibrating agents capable of sustained, reliable autonomy in real-world, resource-constrained environments.

@article{Monroy2025,
  title     = {Achieving Long-Term Autonomy: A Self-Correcting Deep Reinforcement Learning Agent for Edge IoT Using Digital Twin-Based Drift Compensation},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {16},
  number    = {12},
  year      = {2025},
  publisher = {The Science and Information Organization},
  author    = {Jhon Monroy and Miguel Paco and Miguel Portella and Geral Basurco and Jeymi Valdivia and Fiorela Jara and Guido Anco},
  doi       = {10.14569/IJACSA.2025.01612130},
  url       = {https://doi.org/10.14569/IJACSA.2025.01612130}
}