Foundation

As announced in a recent Nature commentary, the Cooperative AI Foundation (CAIF) is a new charitable entity, backed by a large philanthropic commitment. CAIF's mission is to support research that will improve the cooperative intelligence of advanced AI for the benefit of all. For updates and funding opportunities, please sign up to the Cooperative AI mailing list.

Board of Trustees

Allan Dafoe
Eric Horvitz
Gillian Hadfield
Dario Amodei
Ruairí Donnelly

Mission

Importance, Neglectedness, and Tractability


Scientific progress is often hard to predict. Great research is frequently driven by intrinsic curiosity, and it can be difficult to say in advance what the most important research directions for a field will turn out to be. On the other hand, we think that there are features which make some research directions more promising than others. Following an increasing number of philanthropic organisations (see here, for example), CAIF is guided by the importance, neglectedness, and tractability of potential activities, while maintaining an awareness of the value of curiosity and serendipity for great science.

Importance. Is this research area likely to make an important contribution to the cooperative intelligence of advanced AI systems for the benefit of all?

Neglectedness. Is this work likely to be done anyway?

Tractability. Does the research area lend itself especially well to making progress?

While tractability is a common dimension along which to assess potential research, our emphasis on the qualities of importance and neglectedness differ somewhat from their standard interpretations in academia. For instance, a result may be viewed as particularly important within the researcher’s own field, but to warrant the highest prioritisation such work should also be likely to help us build more cooperative AI systems, now and in the future. Likewise, in many fields there may be multiple research groups knowingly racing to solve the same problem, but from the perspective of counterfactual impact this may not be the kind of research that’s essential to prioritise, because it’s likely to happen with or without our support.

Supporting research that will improve the cooperative intelligence of advanced AI for the benefit of all

Differential Progress


In the course of building AI systems to be more cooperatively intelligent, those systems might also gain capabilities that could be used to harm others rather than contribute to improvements in social welfare. The alignment problem is one example of this: as AI systems become more generally intelligent, divergences in their goals and humans’ become more dangerous for humans. In the context of Cooperative AI, the ability to understand other agents can lead to improvements in cooperation, but also in deception and manipulation, and the same abilities that allow one to commit to honouring mutually beneficial agreements could also be used to commit to coercive threats.

With these risks in mind, CAIF is interested in supporting differential progress on cooperative intelligence. That is, we want to support research that leads to significant progress on cooperative capabilities – capabilities that lead to increases in social welfare in a wide range of environments – relative to progress on capabilities that are dual-use (e.g., useful for deception, manipulation, disempowering other agents) and therefore may not robustly improve social welfare. This idea is discussed in further detail in a recent seminar from the New Directions in Cooperative AI series.

Improvements

Improvements that CAIF prioritises are counterfactual and long-term, i.e., those improvements over the next 10-20 years (or longer) that would have been unlikely without our support (see also the discussion of 'Neglectedness' further below).

Advanced AI

Advanced AI systems include not only the present day state-of-the-art, but the kinds of powerful AI systems we can expect to see in the next 10-20 years, and the networks of humans and organisations in which they are embedded.

The Benefit of All

The benefit of all is our fundamental concern, and highlights the fact that not all advances in Cooperative AI may be beneficial for everyone; we must take into account different perspectives and values.

Cooperative Intelligence

Cooperative intelligence refers to the skills required for promoting cooperation between humans, machines, or organisations, though further research is required to fully conceptualise and define these skills.

Supporting Research

Supporting research includes standard academic grantmaking, but also fostering research in other ways, such as organising workshops and other events, supporting students, awarding prizes, and providing educational tools.

Activities

Intro text about our activities

01
Grantmaking

CAIF intends to use its philanthropic endowment to:

– Make grants to support Cooperative AI research, especially that which is important, tractable, and neglected. This includes work which helps to build up the infrastructure of the field, such as novel benchmark environments and metrics of cooperative success.

– Offer scholarships to promising young researchers intent on entering the field of Cooperative AI.Details on calls for proposals and applications forthcoming.

02
Workshops

In 2020, the first Cooperative AI workshop was organised at NeurIPS. CAIF intends to continue to organize workshops at major machine learning conferences, including IJCAI, AAAI, AAMAS, and NeurIPS.

03
Seminar series

CAIF will host a series of online seminars featuring scholars working on the frontier of Cooperative AI. Further details of our first seminar series, New Directions in Cooperative AI, and our call for seminar proposals can be found here.

04
Other activites

CAIF will explore additional ways of contributing to the growth of Cooperative AI, including administering prizes and hosting tournaments which encourage progress in our understanding of the cooperative intelligence of AI systems.

Foundation Activities

Announcements

Announcements

New Managing Director

Staff Announcement

CAIF is delighted to announce that David Norman is joining us as our new Managing Director. David will lead CAIF alongside our Research Director Lewis Hammond as we grow the Foundation’s work and help build the nascent field of Cooperative AI and the research community at its heart.

Learn more
Over $1m In New Grants

Grant

CAIF has recently agreed several new grants, totalling over $1 million, to a number of leading cooperative AI researchers, on topics ranging from the role of intent in cooperation, to learning the preferences of multiple agents, to interactions between language models. More details will follow soon.

Learn more
New Research Analyst

Staff Announcement

We delighted to have Akbir Kahn joining CAIF as a research analyst. Akbir will be leading on the development of a cooperative AI contest and building on his work in multi-agent reinforcement learning and language modelling.

Learn more
New Program Manager

Staff Announcement

CAIF is happy that Cecilia Tilli will be joining the foundation to lead on restructuring its grantmaking program and developing several new programs and initiatives to support junior and early career researchers.

Learn more

Competitions

Grants

Competitions

Grants

Competitions

Grants

Maxime Et Quis Ipsum

Voluptatum molestiae aliquid neque dicta. Eaque rerum perspiciatis non quibusdam minima. Enim impedit aut neque ipsa dolorem ut praesentium consequatur alias. Cumque rem aut quia hic praesentium deserunt pariatur nemo dolorem. Non repudiandae est tenetur sint. A necessitatibus

Link
Et

Quae ipsum quidem tempore blanditiis ducimus. Nihil vel maxime iste possimus veniam est consectetur dolor. Tempora eligendi et voluptatem et aspernatur aut iste. Provident quo laudan

Link
Explicabo Distinctio

Nobis iusto consequatur sunt voluptatem est tempore sunt perspiciatis totam. Atque molestiae ullam quis autem eaque cumque culpa. Vel et modi qui id sit. Et qui nisi dolorem aut nam nemo. Sit laudantium ea non doloremque

Link
Doloremque Voluptatibus Vel Culpa

Quia aut ratione et iure delectus quae magnam eum laudantium. Impedit ut neque. Est voluptas aut incidunt quam. Ipsum molestias non sint ut autem. Est nam simil

Link
Qui Pariatur Beatae

Iste quas consequuntur sed qui qui incidunt esse vero cumque. Blanditiis illo optio. Eum iure dolore et quaerat dolorem aut ab rem repudiandae. Quis culpa in accusantium et iste at beatae non. Doloremque libero dolor consequatur corporis. Praesentium eos quia id nemo neque nobis maio

Link
Beatae

Omnis animi commodi ut consequuntur natus totam optio. Sapiente qui quis. Accusantium natus ut. T

Link
Maxime Et Quis Ipsum

Voluptatum molestiae aliquid neque dicta. Eaque rerum perspiciatis non quibusdam minima. Enim impedit aut neque ipsa dolorem ut praesentium consequatur alias. Cumque rem aut quia hic praesentium deserunt pariatur nemo dolorem. Non repudiandae est tenetur sint. A necessitatibus

Link
Et

Quae ipsum quidem tempore blanditiis ducimus. Nihil vel maxime iste possimus veniam est consectetur dolor. Tempora eligendi et voluptatem et aspernatur aut iste. Provident quo laudan

Link
Explicabo Distinctio

Nobis iusto consequatur sunt voluptatem est tempore sunt perspiciatis totam. Atque molestiae ullam quis autem eaque cumque culpa. Vel et modi qui id sit. Et qui nisi dolorem aut nam nemo. Sit laudantium ea non doloremque

Link

Competitions

Grants

Doloremque Voluptatibus Vel Culpa

Quia aut ratione et iure delectus quae magnam eum laudantium. Impedit ut neque. Est voluptas aut incidunt quam. Ipsum molestias non sint ut autem. Est nam simil

Link
Qui Pariatur Beatae

Iste quas consequuntur sed qui qui incidunt esse vero cumque. Blanditiis illo optio. Eum iure dolore et quaerat dolorem aut ab rem repudiandae. Quis culpa in accusantium et iste at beatae non. Doloremque libero dolor consequatur corporis. Praesentium eos quia id nemo neque nobis maio

Link
Beatae

Omnis animi commodi ut consequuntur natus totam optio. Sapiente qui quis. Accusantium natus ut. T

Link

How to apply for opportunities

Funding

Cooperative AI Research Grants

The Cooperative AI Foundation is seeking proposals for research projects in Cooperative AI. Anyone is eligible to apply, and we welcome applications from disciplines outside of computer science. This call will remain continuously open throughout 2024, and will have four deadlines after which applications will be processed: January 14, March 17, July 28, and October 13.

Learn more

Deadline: 17 March 2024

Jobs

Partnerships & Community Manager

We’re looking for a Partnerships & Community Manager to raise our profile, expand our influence and spark new projects. You’ll host high-profile events, own our external communications, build new partnerships and lead on the promotion of our learning resources. You’ll share our belief that diverse voices around the table are essential to creating benefit for all. You’ll be able to hold your own whether you’re presenting, negotiating, managing outcomes, facilitating heated debates, or discussing technical concepts. You’ll be excited to learn about the field of cooperative AI, plan new activities and figure out what works!

Learn more

Application Deadline: 05 January 2024
Program Administrator

We’re looking for a Program Administrator to join the Cooperative AI Foundation, and support our work to improve the safety of future AI. We have a research grant program in place and have run a number of successful events. We want to do much more and have a Cooperative AI Fellowship program and an online course in the pipeline. This role will enable us to significantly increase the pace of our delivery, and will provide the successful candidate with opportunities for development and progression.

Learn more

Application Deadline: 18 December 2023

Closed applications

No items found.

Staff

Jesse Clifton
Research Analyst
Michelle Virgo
People & Operations Manager
Akbir Khan
Research Analyst
Lewis Hammond
Research Director
Cecilia Elena Tilli
Program Manager
Rebecca Eddington
Program Administrator
David Norman
Managing Director

Trustees

‍Allan Dafoe
Senior Staff Research Scientist, DeepMind
President, Centre for the Governance of AI
Eric Horvitz
Chief Scientific Officer, Microsoft
‍Gillian Hadfield
Director, Schwartz Reisman Institute for Technology and Society
Professor, University of Toronto
‍Dario Amodei
CEO, Anthropic
Ruairí Donnelly
President, Polaris Ventures

Advisors

Vincent Conitzer
Professor, Carnegie Mellon University
Professor, University of Oxford