6 AI for Human Cooperation

Thus far, we have been focusing on the need for AI to be able to handle cooperation problems in order to manage the novel risks that arise from multi-agent interactions involving AI agents. However, the potential gains of having AI systems that can solve cooperation problems well are greater than that. As many of the greatest challenges that humanity faces are in fact cooperation problems, AI systems with cooperative capabilities could be invaluable to solving these.

AI for Social Planning

‍

The next resource is a well-known paper from this area of work: ‘The AI Economist’, which explores how reinforcement learning can be used to design effective tax policies. While reading this paper, note the connections to previous material on opponent shaping and adaptive mechanism design, if you engaged with that material. The paper describes their approach as “two-level deep RL”, meaning both the mechanism designer (that sets the taxation policy) and the agents (that participate in the taxed economy) are learning agents.

AI Tools for Public Deliberation and Preference Elicitation

‍

While the AI Economist aims to design effective tax policies under the assumption that policy objectives have been agreed upon, one of the central cooperation challenges in human societies is agreeing on such objectives—not just for economic policy but for all kinds of political decision-making. There have been several initiatives to use AI to facilitate collective deliberation and decision making, and we are going to review one notable example: Pol.is.

‍

We’ll first look at a non-technical article that provides some context for the implementation of digital solutions for improving collective deliberation and decision-making. Before diving into the details of technical solutions, it is important to recognise that there are other important challenges that relate more to the adoption and legitimacy of solutions than to technical features and capabilities.

We will now go more into technical detail on the Pol.is system mentioned in the previous article. While the first version of Pol.is came before large language models, this next resource is a paper on how modern AI systems can be used to improve it. The discussion section also applies quite broadly to the integration of LLMs and machine learning in deliberative systems.

Another notable project in this area is the Habermas machine that was developed by a team at Google Deepmind. It is named after the German philosopher Jürgen Habermas and his theory of communicative action. While Pol.is focuses on the challenge of eliciting and mapping opinions in large and diverse groups, the Habermas machine aims to help human groups reach consensus by generating new proposals that obtain wide agreement and leave groups less divided. The next optional resource is a talk by Christopher Summerfield presenting the Habermas machine in more detail.

AI Tools for Existential Security

‍

While human cooperation problems such as climate change or political deliberation predate the development of advanced AI systems, AI development also creates new challenges that makes the need for effective solutions more urgent. There is a growing overlap between the research communities that work on ‘AI for Democracy’ and those that focus on existential risk. The following resource is published by Forethought, a research nonprofit focused on how to navigate the transition to a world with superintelligent AI systems, and highlights coordination-enabling applications of AI as something that could mitigate existential risks.

If you are interested in exploring further work in this area, it is also worth looking into the work of the Collective Intelligence Project (CIP), an organisation focused on the research and development of collective intelligence capabilities: decision-making technologies, processes, and institutions that expand a group’s capacity to construct and cooperate towards shared goals.