Unsupervised Learning of Dense Optical Flow, Depth and Egomotion with Event-Based Sensors

Chengxi Ye,Anton Mitrokhin,Cornelia Fermüller,James A. Yorke,Yiannis Aloimonos,Chengxi Ye,Anton Mitrokhin,Cornelia Fermüller,James A. Yorke,Yiannis Aloimonos

We present an unsupervised learning pipeline for dense depth, optical flow and egomotion estimation for autonomous driving applications, using the event-based output of the Dynamic Vision Sensor (DVS) as input. The backbone of our pipeline is a bioinspired encoder-decoder neural network architecture - ECN. To train the pipeline, we introduce a covariance normalization technique which resembles the...