Learning by Watching: Physical Imitation of Manipulation Skills from Human Videos

Haoyu Xiong,Quanzhou Li,Yun-Chun Chen,Homanga Bharadhwaj,Samarth Sinha,Animesh Garg,Haoyu Xiong,Quanzhou Li,Yun-Chun Chen,Homanga Bharadhwaj,Samarth Sinha,Animesh Garg

Learning from visual data opens the potential to accrue a large range of manipulation behaviors by leveraging human demonstrations without specifying each of them mathe-matically, but rather through natural task specification. In this paper, we present Learning by Watching (LbW), an algorithmic framework for policy learning through imitation from a single video specifying the task. The key insight...