We model a whole arm manipulation task about holding as a deep reinforcement learning problem in order to provide a behavior that can directly respond to external perturbation and target motion. To improve the performance of deep learning in robotics application, we propose a new state as the input of the networks, the topology representation. This state allows transferring the learned policy to various shapes, sizes and poses because they are the same in topology space. Compared to RGB image state or pose coordinates state, it can better describe the interaction state between the robot and environments. Besides, there is no reality gap between simulation and reality, which means the policy trained in simulator can be directly transferred to real world.

The video is available on https://youtu.be/Al-QZl-WGlw