Foundation

The Cooperative AI Foundation (CAIF) is a new charitable entity, backed by a $15 million philanthropic commitment from Polaris Ventures. CAIF's mission is to support research that will improve the cooperative intelligence of advanced AI for the benefit of all. For updates and funding opportunities, please sign up to the Cooperative AI mailing list below.

Board of Trustees

Allan Dafoe
Eric Horvitz
Gillian Hadfield
Dario Amodei
Ruairí Donnelly

Mission

Importance, Neglectedness, and Tractability


Scientific progress is often hard to predict. Great research is frequently driven by intrinsic curiosity, and it can be difficult to say in advance what the most important research directions for a field will turn out to be. On the other hand, we think that there are features which make some research directions more promising than others. Following an increasing number of philanthropic organisations (see here, for example), CAIF is guided by the importance, neglectedness, and tractability of potential activities, while maintaining an awareness of the value of curiosity and serendipity for great science.

Importance. Is this research area likely to make an important contribution to the cooperative intelligence of advanced AI systems for the benefit of all?

Neglectedness. Is this work likely to be done anyway?

Tractability. Does the research area lend itself especially well to making progress?

While tractability is a common dimension along which to assess potential research, our emphasis on the qualities of importance and neglectedness differ somewhat from their standard interpretations in academia. For instance, a result may be viewed as particularly important within the researcher’s own field, but to warrant the highest prioritisation such work should also be likely to help us build more cooperative AI systems, now and in the future. Likewise, in many fields there may be multiple research groups knowingly racing to solve the same problem, but from the perspective of counterfactual impact this may not be the kind of research that’s essential to prioritise, because it’s likely to happen with or without our support.

Supporting research that will improve the cooperative intelligence of advanced AI for the benefit of all

Differential Progress


In the course of building AI systems to be more cooperatively intelligent, those systems might also gain capabilities that could be used to harm others rather than contribute to improvements in social welfare. The alignment problem is one example of this: as AI systems become more generally intelligent, divergences in their goals and humans’ become more dangerous for humans. In the context of Cooperative AI, the ability to understand other agents can lead to improvements in cooperation, but also in deception and manipulation, and the same abilities that allow one to commit to honouring mutually beneficial agreements could also be used to commit to coercive threats.

With these risks in mind, CAIF is interested in supporting differential progress on cooperative intelligence. That is, we want to support research that leads to significant progress on cooperative capabilities – capabilities that lead to increases in social welfare in a wide range of environments – relative to progress on capabilities that are dual-use (e.g., useful for deception, manipulation, disempowering other agents) and therefore may not robustly improve social welfare. This idea is discussed in further detail in a recent seminar from the New Directions in Cooperative AI series.

Improvements

Improvements that CAIF prioritises are counterfactual and long-term, i.e., those improvements over the next 10-20 years (or longer) that would have been unlikely without our support (see also the discussion of 'Neglectedness' further below).

Advanced AI

Advanced AI systems include not only the present day state-of-the-art, but the kinds of powerful AI systems we can expect to see in the next 10-20 years, and the networks of humans and organisations in which they are embedded.

The Benefit of All

The benefit of all is our fundamental concern, and highlights the fact that not all advances in Cooperative AI may be beneficial for everyone; we must take into account different perspectives and values.

Cooperative Intelligence

Cooperative intelligence refers to the skills required for promoting cooperation between humans, machines, or organisations, though further research is required to fully conceptualise and define these skills.

Supporting Research

Supporting research includes standard academic grantmaking, but also fostering research in other ways, such as organising workshops and other events, supporting students, awarding prizes, and providing educational tools.

Activities

Intro text about our activities

01
Grantmaking

CAIF intends to use its philanthropic endowment to:

– Make grants to support Cooperative AI research, especially that which is important, tractable, and neglected. This includes work which helps to build up the infrastructure of the field, such as novel benchmark environments and metrics of cooperative success.

– Offer scholarships to promising young researchers intent on entering the field of Cooperative AI.Details on calls for proposals and applications forthcoming.

02
Workshops

In 2020, the first Cooperative AI workshop was organised at NeurIPS. CAIF intends to continue to organize workshops at major machine learning conferences, including IJCAI, AAAI, AAMAS, and NeurIPS.

03
Seminar series

CAIF will host a series of online seminars featuring scholars working on the frontier of Cooperative AI. Further details of our first seminar series, New Directions in Cooperative AI, and our call for seminar proposals can be found here.

04
Other activites

CAIF will explore additional ways of contributing to the growth of Cooperative AI, including administering prizes and hosting tournaments which encourage progress in our understanding of the cooperative intelligence of AI systems.

Foundation Activities

Announcements

Announcements

New Research Engineer

Staff Announcement

We are delighted to have Chandler Smith joining CAIF as a research engineer. Chandler will be leading on the development of a cooperative AI contest at NeurIPS 2024 and assisting with the building of cooperative AI benchmarks.

Learn more
Six New Advisors

Staff Announcement

CAIF is delighted to welcome on board six new advisors. Noam Brown, Jakob Foerster, Edward Hughes, Natasha Jaques, Kate Larson, and Joel Leibo will join Vincent Conitzer in helping shape CAIF's strategy, enabling the foundation to better serve the cooperative AI research community.

Learn more
New Managing Director

Staff Announcement

CAIF is delighted to announce that David Norman is joining us as our new Managing Director. David will lead CAIF alongside our Research Director Lewis Hammond as we grow the Foundation’s work and help build the nascent field of Cooperative AI and the research community at its heart.

Learn more
Over $1m in New Grants

Grant

CAIF has recently agreed several new grants, totalling over $1 million, to a number of leading cooperative AI researchers, on topics ranging from the role of intent in cooperation, to learning the preferences of multiple agents, to interactions between language models. More details will follow soon.

Learn more

Competitions

Grants

Competitions

Grants

Competitions

Grants

Maxime Et Quis Ipsum

Voluptatum molestiae aliquid neque dicta. Eaque rerum perspiciatis non quibusdam minima. Enim impedit aut neque ipsa dolorem ut praesentium consequatur alias. Cumque rem aut quia hic praesentium deserunt pariatur nemo dolorem. Non repudiandae est tenetur sint. A necessitatibus

Link
Et

Quae ipsum quidem tempore blanditiis ducimus. Nihil vel maxime iste possimus veniam est consectetur dolor. Tempora eligendi et voluptatem et aspernatur aut iste. Provident quo laudan

Link
Explicabo Distinctio

Nobis iusto consequatur sunt voluptatem est tempore sunt perspiciatis totam. Atque molestiae ullam quis autem eaque cumque culpa. Vel et modi qui id sit. Et qui nisi dolorem aut nam nemo. Sit laudantium ea non doloremque

Link
Doloremque Voluptatibus Vel Culpa

Quia aut ratione et iure delectus quae magnam eum laudantium. Impedit ut neque. Est voluptas aut incidunt quam. Ipsum molestias non sint ut autem. Est nam simil

Link
Qui Pariatur Beatae

Iste quas consequuntur sed qui qui incidunt esse vero cumque. Blanditiis illo optio. Eum iure dolore et quaerat dolorem aut ab rem repudiandae. Quis culpa in accusantium et iste at beatae non. Doloremque libero dolor consequatur corporis. Praesentium eos quia id nemo neque nobis maio

Link
Beatae

Omnis animi commodi ut consequuntur natus totam optio. Sapiente qui quis. Accusantium natus ut. T

Link
Maxime Et Quis Ipsum

Voluptatum molestiae aliquid neque dicta. Eaque rerum perspiciatis non quibusdam minima. Enim impedit aut neque ipsa dolorem ut praesentium consequatur alias. Cumque rem aut quia hic praesentium deserunt pariatur nemo dolorem. Non repudiandae est tenetur sint. A necessitatibus

Link
Et

Quae ipsum quidem tempore blanditiis ducimus. Nihil vel maxime iste possimus veniam est consectetur dolor. Tempora eligendi et voluptatem et aspernatur aut iste. Provident quo laudan

Link
Explicabo Distinctio

Nobis iusto consequatur sunt voluptatem est tempore sunt perspiciatis totam. Atque molestiae ullam quis autem eaque cumque culpa. Vel et modi qui id sit. Et qui nisi dolorem aut nam nemo. Sit laudantium ea non doloremque

Link

Competitions

Grants

Doloremque Voluptatibus Vel Culpa

Quia aut ratione et iure delectus quae magnam eum laudantium. Impedit ut neque. Est voluptas aut incidunt quam. Ipsum molestias non sint ut autem. Est nam simil

Link
Qui Pariatur Beatae

Iste quas consequuntur sed qui qui incidunt esse vero cumque. Blanditiis illo optio. Eum iure dolore et quaerat dolorem aut ab rem repudiandae. Quis culpa in accusantium et iste at beatae non. Doloremque libero dolor consequatur corporis. Praesentium eos quia id nemo neque nobis maio

Link
Beatae

Omnis animi commodi ut consequuntur natus totam optio. Sapiente qui quis. Accusantium natus ut. T

Link

How to apply for opportunities

Funding

Cooperative AI Research Grants

The Cooperative AI Foundation is seeking proposals for research projects in cooperative AI.

Learn more

Deadline: 18 January 2025 23:59 AoE

Jobs

No items found.

Closed applications

Cooperative AI PhD Fellowships

The Cooperative AI PhD Fellowship is designed to provide future and current PhD students in the field of cooperative AI financial support to achieve their full potential.

Deadline: 14 October 2024 23:59 AoE

Team

Jesse Clifton
Research Analyst
Michelle Virgo
People & Operations Manager
Lewis Hammond
Research Director
Cecilia Elena Tilli
Program Manager
David Norman
Managing Director
Marta Bieńkiewicz
Partnerships and Community Manager
Rebecca Eddington
Grants and Events Officer
Chandler Smith
Research Engineer

Trustees

‍Allan Dafoe
Senior Staff Research Scientist, DeepMind
President, Centre for the Governance of AI
Eric Horvitz
Chief Scientific Officer, Microsoft
‍Gillian Hadfield
Director, Schwartz Reisman Institute for Technology and Society
Professor, University of Toronto
‍Dario Amodei
CEO, Anthropic
Ruairí Donnelly
President, Polaris Ventures

Advisors

Joel Leibo
Senior Staff Research Scientist, Google DeepMind
Noam Brown
Researcher, OpenAI
Jakob Foerster
Associate Professor, University of Oxford
Edward Hughes
Staff Research Engineer, Google DeepMind
Kate Larson
Professor, University of Waterloo
Natasha Jaques
Assistant Professor, University of Washington
Senior Research Scientist, Google DeepMind
Vincent Conitzer
Professor, Carnegie Mellon University
Professor, University of Oxford

Affiliates

Akbir Khan
Researcher, Anthropic
PhD Student, UCL