3 Large and Complex Systems of Agents
Required Content 3hrs 45mins • All Content 4hrs 45mins
Complex systems theory is a field of research that developed in the latter half of the 20th century, aimed at understanding how emergent behaviours, large-scale patterns, and adaptability arise from interactions among many components — especially in domains like biology, physics, economics, and social systems.
Theory and methods from complex systems science can be an important complement to the more common game theory framings in cooperative AI.
One way of thinking about this is that interactions between few agents, but where each interaction is relatively high stakes, may be modelled better by game theoretic approaches, while complex systems theory might apply better to risks that arise from a very large number of (potentially individually low-stakes) interactions.
By the end of the section, you should be able to:
- Explain how complex systems theory complements game theory in cooperative AI.
- Identify situations where game-theoretic reasoning is more useful than a complex-systems framing, and vice versa.
- Identify concepts from complex systems science (e.g. feedback loops) in current and potential future multi-agent scenarios.
- Describe key risk factors in multi-agent AI systems stemming from complexity, including network effects, selection pressures, destabilising dynamics, and emergent agency.
Complex systems framings can be applied to AI and related safety issues on several levels, and the next resource outlines how this relates not only to multi-agent interactions but also to development of new capabilities or how feedback loops can influence the training of AI systems. The chapter ‘Lessons for AI governance’ is optional, but may be especially useful if you are interested in governance and regulation.
Tooltip Text
While complex systems theory can apply to AI safety on multiple levels, it is primarily the multi-agent interactions that are of interest for cooperative AI work. The next resource outlines some more concrete scenarios for what widespread deployment of AI agents could look like and how this can be understood as a complex, dynamic system.
Tooltip Text
‘Abstract’, ‘Introduction’, ‘Dynamics’ excluding ‘Opportunities and Challenges’
Tooltip Text
Consider the introduction to the paper 'Virtual Agent Economies'. Do you agree that “unless a change is made, our current trajectory points toward the accidental emergence of a vast, and likely permeable, sandbox economy”? Explain why.
Tooltip Text
Consider the example of social media algorithms for curating content. For each concept in complex systems science that you have been introduced to so far (e.g. feedback loops), consider whether or not the concept applies to some aspect of human's collective interaction with these algorithms and in what way.
Tooltip Text
Based on what we know from complex systems science, we could expect that emergent phenomena (behaviours, goals and capabilities) that are not present in individual AI agents could arise in multi-agent systems of agents. We do not yet have a good understanding of such phenomena, and Cooperative AI Foundation has highlighted specific research directions in this area that are perceived as particularly pressing to make progress on (the concepts mentioned here will be explained in more detail in the next external resource):
- Work on
destabilising dynamics, which could aim to answer questions about the conditions under which multi-agent systems involving AI have undesirable dynamics and how such phenomena can be monitored and stabilised. Such work may cover aspects such as how the number of agents, their objectives, and the features of their environment might precipitate undesirable dynamics. - Work on prevention of
correlated failures, that could arise due to similarities and shared vulnerabilities among agents in the multi-agent system. This could include work on the impact of AI agents learning from data generated by each other on shared vulnerabilities, correlated failure modes, and their ability to cooperate/collude. - Work on which
network structuresand interaction patterns lead to more robust or fragile networks of AI agents, and the development of tools for overseeing and controlling the dynamics and co-adaptation of networks of advanced AI agents. This might include ‘infrastructure for AI agents’ such as interaction protocols. - Theoretical and empirical work on establishing the conditions under which unexpected and undesirable goals and capabilities might emerge from multiple AI agents, how robust such phenomena are, and how quickly they can occur. Comparisons across specific scenarios could help to establish conditions under which these
emergent phenomenaare more likely, such as the degree of competition, complementarity of agents, access to particular resources, or task features.
In the previous section on Mixed-motive settings we included a section of the report Multi-Agent Risks from Advanced AI which focused on conflict dynamics between AI agents. We will now return to the same report but instead focus on selected sections of the third chapter which outlines different risk factors, many of which use framings grounded in complex systems science.
Tooltip Text
3.2 Network effects, 3.3 Selection Pressures, 3.4 Destabilizing dynamics, 3.6 Emergent Agency
Tooltip Text
Take a look at this article 'Introducing the Global Pulse Indicators'. Do you think the data collected could be useful for predicting emergent behaviour in human-AI systems? Why, and if so, what type of behaviour?
Tooltip Text
Complex systems and dynamic networks can seem like quite abstract concepts if you are new to it. The following optional piece of content is an interactive lesson on some important foundational concepts in network theory that can be helpful to make it more concrete. The context is set firmly in human interactions and dynamics, but there are still many useful concepts to extract.
Tooltip Text
Consider the simulations you saw in 'Wisdom and / or madness of the crowds'. What situations or dynamics among AI and human agents do you think simulations like that could help us understand?
Tooltip Text
Consider an example multi-agent catastrophic failure that exhibits some complex systems effect e.g. the 2010 flash crash. Propose 3 monitoring and infrastructural interventions that could have limited the damage if they had been in place.
Tooltip Text
Game theory classifies multi-agent scenarios into three main types: mixed-motive, fully cooperative, and fully competitive. When modelling a particular real-world system using game theory, the scenario type is often assumed to be static. What are some real-world examples of scenarios whose type might not remain static, and what interventions or events could cause these to shift from one type to another?
Tooltip Text
