Fellows' Spotlight

Reinforcement Learning (RL) is increasingly applied to social, data-limited environments where agents must interact with others without the luxury of online experimentation. However, current evaluations fail to capture this complexity: standard offline RL benchmarks are non-social. To bridge this gap, we introduce Molten Pot, a benchmark for offline mixed-motive social RL that spans five substrates from DeepMind's Melting Pot environment and roughly a terabyte of trajectory data. By testing agents across three levels of social complexity – ranging from static populations to zero-shot generalization across social shifts like role changes and reciprocity thresholds – we reveal a significant ‘social robustness gap’. Our findings show that while existing algorithms may achieve high returns, they often fail when social dynamics shift, establishing offline social evaluation as a critical and distinct challenge for the RL community.

We’re delighted to host this first seminar in our 'Fellows' Spotlight' series. Subscribe to our Google Calendar to stay up-to-date on all upcoming events.