Knowledge Transfer in Multi-Task Deep Reinforcement Learning for Continuous Control
Zhiyuan Xu,Kun Wu,Zhengping Che,Jian Tang,Jieping Ye
While Deep Reinforcement Learning (DRL) has emerged as a promising approachto many complex tasks, it remains challenging to train a single DRL agent that iscapable of undertaking multiple different continuous control tasks. In this paper,we present a Knowledge Transfer based Multi-task Deep Reinforcement Learningframework (KTM-DRL) for continuous control, which enables a single DRL agentto achieve expert-level performance in multiple different tasks by learning fromtask-specific teachers. In KTM-DRL, the multi-task agent first leverages an offlineknowledge transfer algorithm designed particularly for the actor-critic architectureto quickly learn a control policy from the experience of task-specific teachers, andthen it employs an online learning algorithm to further improve itself by learningfrom new online transition samples under the guidance of those teachers. Weperform a comprehensive empirical study with two commonly-used benchmarks inthe MuJoCo continuous control task suite. The experimental results well justifythe effectiveness of KTM-DRL and its knowledge transfer and online learningalgorithms, as well as its superiority over the state-of-the-art by a large margin.
