Impact Makes a Sound and Sound Makes an Impact: Sound Guides Representations and Explorations

Xufeng Zhao,Cornelius Weber,Muhammad Burhan Hafez,Stefan Wermter,Xufeng Zhao,Cornelius Weber,Muhammad Burhan Hafez,Stefan Wermter

Sound is one of the most informative and abundant modalities in the real world while being robust to sense without contacts by small and cheap sensors that can be placed on mobile devices. Although deep learning is capable of extracting information from multiple sensory inputs, there has been little use of sound for the control and learning of robotic actions. For unsupervised reinforcement learni...