Updates in Cooperative AI

As LLM-based multi-agent systems coordinate through free-form language, a critical safety concern emerges: what happens when individual agents form coalitions and collude to pursue secondary goals at the expense of the joint objective? In this talk, Sahar will present Colosseum, a framework for auditing collusive behaviour in multi-agent settings. Colosseum grounds agent cooperation through Distributed Constraint Optimisation Problems (DCOPs) and measures collusion via regret relative to the cooperative optimum, enabling rigorous evaluation across different objectives, persuasion tactics, and network topologies. Their audit reveals that most out-of-the-box models are susceptible to collusion when a secret communication channel is available. They also uncover a surprising phenomenon – ‘collusion on paper’ – where agents plan to collude in text but ultimately select non-collusive actions, highlighting a gap between language-level intent and behavioural outcomes. Colosseum provides a principled new way to study collusion by jointly analysing communications and actions in rich yet verifiable environments.

We’re delighted to host this eighth seminar in our 'Updates in Cooperative AI' series. We're running these seminars monthly, and you're welcome to subscribe to our Google Calendar to stay up-to-date on all upcoming events.