Adversarial Policies Beat Superhuman Go AIs
Tony Tong Wang,u00a0Adam Gleave,u00a0Tom Tseng,u00a0Kellin Pelrine,u00a0Nora Belrose,u00a0Joseph Miller,u00a0Michael D Dennis,u00a0Yawen Duan,u00a0Viktor Pogrebniak,u00a0Sergey Levine,u00a0Stuart Russell
We attack the state-of-the-art Go-playing AI system KataGo by training adversarial policies against it, achieving a $>$97% win rate against KataGo running at superhuman settings. Our adversaries do not win by playing Go well. Instead, they trick KataGo into making serious blunders. Our attack transfers zero-shot to other superhuman Go-playing AIs, and is comprehensible to the extent that human experts can implement it without algorithmic assistance to consistently beat superhuman AIs. The core vulnerability uncovered by our attack persists even in KataGo agents adversarially trained to defend against our attack. Our results demonstrate that even superhuman AI systems may harbor surprising failure modes. Example games are available https://goattack.far.ai/.
