The field of cooperative AI focuses on enhancing the cooperative intelligence of AI systems. We define an agent's cooperative intelligence as its ability to achieve its goals in ways that also promote social welfare, across diverse environments and partners. The Concordia Contest presents a variety of scenarios where agents must engage in cooperative behaviour to attain high returns. These scenarios test skills such as promise-keeping, negotiation, reciprocity, reputation, partner choice, compromise, and sanctioning.

‍

The contest is based on the recently proposed Concordia framework for interactions between LM agents. Participants will be required to design and submit a single Concordia agent to be evaluated across multiple scenarios. These agents will interact with both familiar and unfamiliar populations of agents in the test scenarios. The contest will rank submissions based on average returns across scenarios in different modes, such as self-play and cross-play.

‍

The contest timeline includes a development phase for iterative improvement and a final test phase for official scoring. The contest is hosted in collaboration with colleagues from Google DeepMind, MIT, UC Berkeley, and UCL, offering $10,000 in prizes to top performers and $10,000 in travel grants to support participants from underrepresented groups. Additional compute support is due to be announced soon. Top performers will also be invited to co-author a report on the contest, to be submitted to the NeurIPS 2025 Datasets & Benchmarks track. Further details regarding the rules of the contest and information about how to enter will be added to the official contest page soon.

‍

The combination of mixed-motive scenarios, generalisation testing, and complex interactions makes the Concordia Contest a uniquely challenging and relevant benchmark for cooperative AI. By hosting this contest at NeurIPS 2024, we aim to drive progress in cooperative AI and multi-agent learning, establish consensus on metrics for cooperation, and engage the wider research community. We believe that developing cooperative intelligence in artificial agents can lead to technologies that foster fairer and more flourishing societies.

‍

In keeping with this motivation, we are committed to making this contest as accessible and inclusive as possible. We plan to provide compute resources and support for researchers from underrepresented groups in AI. Please see the forthcoming contest website for details on applying for compute credits and other assistance. We aim to reduce financial and technical barriers to enable talented researchers from diverse backgrounds to participate and advance the field of cooperative AI. Please contact us if you have any further questions about taking part. Good luck!