Learning Neuro-symbolic Programs for Language Guided Robot Manipulation

Namasivayam K,Himanshu Singh,Vishal Bindal,Arnav Tuli,Vishwajeet Agrawal,Rahul Jain,Parag Singla,Rohan Paul,Namasivayam K,Himanshu Singh,Vishal Bindal,Arnav Tuli,Vishwajeet Agrawal,Rahul Jain,Parag Singla,Rohan Paul

Given a natural language instruction and an input scene, our goal is to train a model to output a manipulation program that can be executed by the robot. Prior approaches for this task possess one of the following limitations: (i) rely on hand-coded symbols for concepts limiting generalization beyond those seen during training [1] (ii) infer action sequences from instructions but require dense sub...