Automatic Flipper Control for Crawler Type Rescue Robot using Reinforcement Learning

In recent years, many natural disasters have occurred, and rescue robots have been used to gather information at disaster sites. Rescue robots, particularly crawler type rescue robots are operated through remote control by their operators via wireless communication or wired. However, certain robots have been known to not return owing to tipping over or disconnection of communication wires caused by missed operations. Therefore, studies have focused on automatic control of rescue robots. Adapting the rescue robot for uneven terrain or unexpected obstacle shape to travel in autonomous control situation is challenging. It requires complete autonomous control as well as partial control of the rescue robot, which necessitates assistance for teleoperation. This study proposed automatic flipper control of rescue robots using reinforcement learning for stepping over steps. The proposed method involved designing of the learning environment, reward setting, and system configuration for reinforcement learning. The input data for the training data were coarse-grained information using a distance sensor, gyro sensor, and GPS coordinates information. Reinforcement learning was performed through a physical simulation within an environment wherein the shape of a step changed once every 100 episodes. The robot’s compensation was designed to reduce the impact on the robot’s body by changing the robot’s attitude angle. The learned knowledge, which is contained action-value function, was reused to verify that the flipper could be automatically controlled by the operator when the rescue robot is operated as moving along a direction remotely, and that the robot could step over steps.

@article{Kono2024,
  title     = {Automatic Flipper Control for Crawler Type Rescue Robot using Reinforcement Learning},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {15},
  number    = {6},
  year      = {2024},
  publisher = {The Science and Information Organization},
  author    = {Hitoshi Kono and Sadaharu Isayama and Fukuro Koshiji and Kaori Watanabe and Hidekazu Suzuki},
  doi       = {10.14569/IJACSA.2024.01506147},
  url       = {https://doi.org/10.14569/IJACSA.2024.01506147}
}