Multi-view Fusion for Multi-level Robotic Scene Understanding

Yunzhi Lin,Jonathan Tremblay,Stephen Tyree,Patricio A. Vela,Stan Birchfield,Yunzhi Lin,Jonathan Tremblay,Stephen Tyree,Patricio A. Vela,Stan Birchfield

We present a system for multi-level scene awareness for robotic manipulation. Given a sequence of camera-inhand RGB images, the system calculates three types of information: 1) a point cloud representation of all the surfaces in the scene, for the purpose of obstacle avoidance. 2) the rough pose of unknown objects from categories corresponding to primitive shapes (e.g., cuboids and cylinders), and...