New Directions in Cooperative AI

As part of its mission, the Cooperative AI Foundation (CAIF) is hosting a fortnightly seminar series on New Directions in Cooperative AI, in which we invite leading thinkers to offer their vision for research on cooperative AI. Unlike typical academic talks, these seminars will be explicitly agenda-setting, describing a line of work that many researchers could pursue and that CAIF could support. If you are interested in submitting a proposal for a seminar, we invite you to apply here. Proposals will be evaluated on a rolling basis, will receive notification of acceptance, revision, or rejection within one month, and – in recognition of the additional burden of preparing such a talk – will also receive a $5000 honorarium if successful.

We are excited to announce that our first seminar will be given by Vincent Conitzer (Duke University, University of Oxford), who will soon be starting the Foundations of Cooperative AI Lab (FOCAL) at Carnegie Mellon University, with support from CAIF. We will also be joined by Edith Elkind (University of Oxford) and Joseph Halpern (Cornell University). Further details of upcoming talks can be found below and you can also subscribe to our public calendar of events via Google, or by adding this ICS URL to your calendar application.

AI Agents May Cooperate Better If They Don’t Resemble Us

Speaker: Vincent Conitzer (Duke University, University of Oxford)
Discussants: Edith Elkind (University of Oxford) and Joseph Halpern (Cornell University)
Time: 15:00-16:00 GMT on Thursday 20th January 2022
Calendar Event: Google, ICS file
Zoom Meeting: Link (Please sign up to the mailing list here to receive the passcode, or send us an email if you're having trouble accessing the event)

Abstract
AI systems control an ever growing part of our world. As a result, they will increasingly interact with each other directly, with little or no potential for human mediation. If each system stubbornly pursues its own objectives, this runs the risk of familiar game-theoretic tragedies – along the lines of the Tragedy of the Commons, the Prisoner’s Dilemma, or even the Traveler’s Dilemma – in which outcomes are reached that are far worse for every party than what could have been achieved cooperatively.

However, AI agents can be designed in ways that make them fundamentally unlike strategic human agents. This approach is often overlooked, as we are usually inspired by our own human condition in the design of AI agents. But I will argue that this approach has the potential to avoid the above tragedies in new ways. The price to pay for this, for us as researchers, is that many of our intuitions about game and decision theory, and even belief formation, start to fall short. I will discuss how foundational research from the philosophy and game theory literatures provides a good starting point for pursuing this approach.

This talk covers joint work with Caspar Oesterheld, Scott Emmons, Andrew Critch, Stuart Russell, Abram Demski, Yuan Deng, and Catherine Moon.

Bio
Vincent Conitzer is the Kimberly J. Jenkins Distinguished University Professor of New Technologies and Professor of Computer Science, Professor of Economics, and Professor of Philosophy at Duke University. He is also Head of Technical AI Engagement at the Institute for Ethics in AI, and Professor of Computer Science and Philosophy, at the University of Oxford. He received Ph.D. (2006) and M.S. (2003) degrees in Computer Science from Carnegie Mellon University, and an A.B. (2001) degree in Applied Mathematics from Harvard University. Conitzer works on artificial intelligence (AI). Much of his work has focused on AI and game theory, for example designing algorithms for the optimal strategic placement of defensive resources. More recently, he has started to work on AI and ethics: how should we determine the objectives that AI systems pursue, when these objectives have complex effects on various stakeholders?

Conitzer has received the 2021 ACM/SIGAI Autonomous Agents Research Award, the Social Choice and Welfare Prize, a Presidential Early Career Award for Scientists and Engineers (PECASE), the IJCAI Computers and Thought Award, an NSF CAREER award, the inaugural Victor Lesser dissertation award, an honorable mention for the ACM dissertation award, and several awards for papers and service at the AAAI and AAMAS conferences. He has also been named a Guggenheim Fellow, a Sloan Fellow, a Kavli Fellow, a Bass Fellow, an ACM Fellow, a AAAI Fellow, and one of AI's Ten to Watch. He has served as program and/or general chair of the AAAI, AAMAS, AIES, COMSOC, and EC conferences. Conitzer and Preston McAfee were the founding Editors-in-Chief of the ACM Transactions on Economics and Computation (TEAC).

Recently, Conitzer has joined the Cooperative AI Foundation as an advisor, and has announced that he will be moving to Carnegie Mellon University in order to start a new lab, FOCAL (the Foundations of Cooperative AI Lab). The lab’s goal is to create foundations of game theory appropriate for advanced, autonomous AI agents – with a focus on achieving cooperation.