Toward Sim-to-Real Directional Semantic Grasping

Shariq Iqbal,Jonathan Tremblay,Andy Campbell,Kirby Leung,Thang To,Jia Cheng,Erik Leitch,Duncan McKay,Stan Birchfield,Shariq Iqbal,Jonathan Tremblay,Andy Campbell,Kirby Leung,Thang To,Jia Cheng,Erik Leitch,Duncan McKay,Stan Birchfield

We address the problem of directional semantic grasping, that is, grasping a specific object from a specific direction. We approach the problem using deep reinforcement learning via a double deep Q-network (DDQN) that learns to map downsampled RGB input images from a wrist-mounted camera to Q-values, which are then translated into Cartesian robot control commands via the cross-entropy method (CEM)...