New Memo: Establishing Foundational Principles and Thresholds for Multi-Agent AI Governance

Advanced AI agents are increasingly deployed across digital environments, and will soon creating complex interactions between systems developed and operated by different actors. These multi-agent settings introduce system-level risks that current governance and risk management frameworks are not equipped to contend with.

‍

Our new memo, ‘Establishing Foundational Principles and Thresholds for Multi-Agent AI Governance’, explores practical steps for addressing these emerging risks.

‍

The memo draws on discussions from a workshop convened by the Cooperative AI Foundation and the Brookings Institution at the 2026 International Association for Safe & Ethical AI Conference in Paris. It examines some of the foundational challenges facing multi-agent AI governance, including the need for shared terminology, better evaluation methods, and clearer mechanisms for oversight and accountability.

‍

It sets out five areas for action:

‍

• Identify: Develop standardised agent IDs, model registries and infrastructure to support attribution and accountability.
• Evaluate: Develop evaluations and sandboxes that test how agents behave when interacting with other systems, including risks such as conflict, collusion and adversarial influence.
• Monitor: Establish continuous monitoring and automated containment mechanisms that can respond to emerging risks at the speed of AI systems.
• Report: Create mechanisms for identifying and reporting incidents before they develop into wider systemic failures.
• Incentivise: Use tools such as liability frameworks, insurance and procurement standards to encourage responsible behaviour and cooperation across the AI ecosystem.

‍

The memo also considers lessons from adjacent domains, including financial markets, autonomous vehicles and legal systems, and highlights the importance of collaboration between AI developers, deployers, evaluators, policymakers, regulators, and other actors across the AI lifecycle.

‍

The workshop brought together experts from governance, policy, industry, ethics, and law. We are grateful to Elham Tabassi, Joel Z. Leibo, Julia Smakman, Kevin Baum, Luis Aranda, Merve Hickok, Patricia Paskov, Trish Shaw, and Zhijing Jin for their contributions and thoughtful comments on earlier drafts of the memo.

‍

Read the memo: Establishing Foundational Principles and Thresholds for Multi-Agent AI Governance

‍

Join our forthcoming seminar with IASEAI which builds on this work: When AI Agents Meet: A Blueprint for Addressing Multi-Agent AI Governance

‍