RTVS: A Lightweight Differentiable MPC Framework for Real-Time Visual Servoing

M. Nomaan Qureshi,Pushkal Katara,Abhinav Gupta,Harit Pandya,Y V S Harish,AadilMehdi Sanchawala,Gourav Kumar,Brojeshwar Bhowmick,K. Madhava Krishna,M. Nomaan Qureshi,Pushkal Katara,Abhinav Gupta,Harit Pandya,Y V S Harish,AadilMehdi Sanchawala,Gourav Kumar,Brojeshwar Bhowmick,K. Madhava Krishna

Recent data-driven approaches to visual servoing have shown improved performances over classical methods due to precise feature matching and depth estimation. Some recent servoing approaches use a model predictive control (MPC) framework which generalise well to novel environments and are capable of incorporating dynamic constraints, but are computationally intractable in real-time, making it diff...