Robot Sound Interpretation: Combining Sight and Sound in Learning-Based Control
Peixin Chang,Shuijing Liu,Haonan Chen,Katherine Driggs-Campbell,Peixin Chang,Shuijing Liu,Haonan Chen,Katherine Driggs-Campbell
We explore the interpretation of sound for robot decision making, inspired by human speech comprehension. While previous methods separate sound processing unit and robot controller, we propose an end-to-end deep neural network which directly interprets sound commands for visual-based decision making. The network is trained using reinforcement learning with auxiliary losses on the sight and sound n...
