DMotion: Robotic Visuomotor Control with Unsupervised Forward Model Learned from Videos

Haoqi Yuan,Ruihai Wu,Andrew Zhao,Haipeng Zhang,Zihan Ding,Hao Dong,Haoqi Yuan,Ruihai Wu,Andrew Zhao,Haipeng Zhang,Zihan Ding,Hao Dong

Learning an accurate model of the environment is essential for model-based control tasks. Existing methods in robotic visuomotor control usually learn from data with heavily labelled actions, object entities or locations, which can be demanding in many cases. To cope with this limitation, we propose a method, dubbed DMotion, that trains a forward model from video data only, via disentangling the m...