Title: Coordination as Inference in Multi-Agent Reinforcement Learning
Speaker: 苏开乐教授，澳大利亚格里菲斯大学（Griffith University）
Time: 7月3日14:00-15:00
Venue: 人工智能研究院311会议室（老行政楼北楼3楼）
Contents of report:
The Centralized Training and Decentralized Execution (CTDE) paradigm, where a centralized critic is allowed to access global information during the training phase while maintaining the learned policies executed with only local information in a decentralized way, has achieved great progress in recent years. Despite the progress, CTDE may suffer from the issue of Centralized-Decentralized Mismatch (CDM): the suboptimalityof one agent’s policy can exacerbate policy learning of other agents through the centralized joint critic. In contrast to centralized learning, the cooperative model that most closely resembles the way humans cooperate in nature is fully decentralized, i.e. Independent Learning (IL). However, there are still two issues that need to be addressed before agents coordinate through IL: (1) how agents are aware of the presence of other agents, and (2) how to coordinate with other agents to improve joint policy under IL. In this paper, we propose an inference-based coordinated MARL method: Deep Motor System (DMS). DMS first presents the idea of individual intention inference where agents are allowed to disentangle other agents from their environment. Secondly, causal inference was introduced to enhance coordination by reasoning each agent’s effect on others’behavior. The proposed model was extensively experimented on a series of Multi-Agent MuJoCo and StarCraftII tasks. Results show that the proposed method outperforms independent learning algorithms and the coordination behavior among agents can be learned even without the CTDE paradigm compared to the state-of-the-art baselines including IPPO and HAPPO.
Introduction to the Speaker:
苏开乐，博士，现任澳大利亚格里菲斯大学教授，清华大学逻辑研究中心兼职教授，南京信息工程大学人工智能学院名誉院长。主要研究领域为人工智能逻辑与算法。曾任中山大学（1999-2007）、北京大学（2007-2014）教授和博导。2007年获得了国家自然科学基金“杰出青年基金”项目，2009年获得澳大利亚国家基金研究会ARC Future Fellow。获得AiML 2002（法国图卢兹）最佳论文奖、SAT Challenge 2012: Best Sequential Solver（Random Track）奖。近年来，共发表了中国计算学会（CCF）推荐A类论文30多篇，包括人工智能顶级学术会议AAAI（15篇）、IJCAI（10篇）、顶级期刊Artificial Intelligence、Information and Computation、IEEE Trans. on Computers和IEEE Trans. on Software Engineering等。
