The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark Towards Physically Realistic Embodied AI

Chuang Gan,Siyuan Zhou,Jeremy Schwartz,Seth Alter,Abhishek Bhandwaldar,Dan Gutfreund,Daniel L.K. Yamins,James J. DiCarlo,Josh McDermott,Antonio Torralba,Joshua B. Tenenbaum,Chuang Gan,Siyuan Zhou,Jeremy Schwartz,Seth Alter,Abhishek Bhandwaldar,Dan Gutfreund,Daniel L.K. Yamins,James J. DiCarlo,Josh McDermott,Antonio Torralba,Joshua B. Tenenbaum

We introduce a visually-guided task-and-motion planning benchmark, which we call the ThreeDWorld Trans-port Challenge. In this challenge, an embodied agent is spawned randomly in a simulated physical home environment and required to transport a small set of objects scattered around the house with containers. We build this benchmark challenge using the ThreeDWorld simulation: a virtual 3D environme...