Sonicverse: A Multisensory Simulation Platform for Embodied Household Agents that See and Hear

Ruohan Gao,Hao Li,Gokul Dharan,Zhuzhu Wang,Chengshu Li,Fei Xia,Silvio Savarese,Li Fei-Fei,Jiajun Wu,Ruohan Gao,Hao Li,Gokul Dharan,Zhuzhu Wang,Chengshu Li,Fei Xia,Silvio Savarese,Li Fei-Fei,Jiajun Wu

Developing embodied agents in simulation has been a key research topic in recent years. Exciting new tasks, algorithms, and benchmarks have been developed in various simulators. However, most of them assume deaf agents in silent environments, while we humans perceive the world with multiple senses. We introduce Sonicverse, a multisensory simulation platform with integrated audio-visual simulation ...