Sim2Real Manipulation on Unknown Objects with Tactile-based Reinforcement Learning

Entong Su,Chengzhe Jia,Yuzhe Qin,Wenxuan Zhou,Annabella Macaluso,Binghao Huang,Xiaolong Wang,Entong Su,Chengzhe Jia,Yuzhe Qin,Wenxuan Zhou,Annabella Macaluso,Binghao Huang,Xiaolong Wang

Using tactile sensors for manipulation remains one of the most challenging problems in robotics. At the heart of these challenges is generalization: How can we train a tactile-based policy that can manipulate unseen and diverse objects? In this paper, we propose to perform Reinforcement Learning with only visual tactile sensing inputs on diverse objects in a physical simulator. By training with di...