The Dormant Neuron Phenomenon in Deep Reinforcement Learning
Ghada Sokar,u00a0Rishabh Agarwal,u00a0Pablo Samuel Castro,u00a0Utku Evci
In this work we identify the dormant neuron phenomenon in deep reinforcement learning, where an agentu2019s network suffers from an increasing number of inactive neurons, thereby affecting network expressivity. We demonstrate the presence of this phenomenon across a variety of algorithms and environments, and highlight its effect on learning. To address this issue, we propose a simple and effective method (ReDo) that Recycles Dormant neurons throughout training. Our experiments demonstrate that ReDo maintains the expressive power of networks by reducing the number of dormant neurons and results in improved performance.
