Research

Background

There are two important aspects to decision-making that I find especially fascinating:

Uncertainty in our beliefs. It weaves together the main components of decision-making theory:
- Uncertainty about how to efficiently represent the world
  - How do we abstract the world in a way that minimally distorts our ability to choose actions to achieve a goal?
- Uncertainty about the value of actions we take
  - How does the uncertainty in our beliefs about the world transform into uncertainty about how valuable are the actions we can take in it?
- Uncertainty about what goals we should pursue
  - How do we form and select goals to pursue (ultimately defining what value means for us)?
Why do we sometimes make the effort to reduce these kinds of uncertainty, while other times we simply account for it, or even neglect it outright, even at our peril? While we are typically frugal with cognitive effort, we are also often simply suboptimal with respect to instrumental reward. Our decision utility extends to cognitive/informational and affective/social value. We also misinterpret or are misguided and misinformed by our social environment. We are a social species after all.
Social factors, personal values, and sense of identity. They can play a prominent role in the decision-making process within an individual, but also collectively within a society. Many of the pressing societal problems of our time, from polarization to climate change inaction, rest on how we form beliefs while consuming information and dialoguing in social venues, increasingly curated by algorithms, on a range of scales from peer-to-peer extending to large, online communities. The form of the information exchanged in these fundamentally social interactions and the form of the mediums over which this exchange occurs can play a role in the collective decisions we make as a society by biasing how individuals decide what to believe.

These are in my mind as I dig around at the various intersections of

social/cognitive psychology & behavioural sciences,
public policy & opinion research, and
computational social science

in pursuit of tractable, high-impact problems.

Ideal products for me are reproducible scientific analysis and efficient algorithms that provide insight about and scale-up, respectively, human decision-making for social well-being. A local (i.e. individual) and global (i.e. societal) approach to understanding social systems highlights two kinds of phenomena:

inference and decision-making in the behaviour of individuals and small groups, and
complex system dynamics of large numbers of interacting learning agents.

Note that the "interacting" in the latter point can be non-local (e.g. via social media). I am excited to take part in the modern trend of pushing the theory of games & machine learning towards contact with data to better understand real-world multi-agent systems (e.g. DIMACS workshop series on Bridging Game Theory and Machine Learning for Multi-party Decision Making).

Topic: Large-scale data science and simulations of society

Motivation: better accounting for the effects of the social milieu in which individuals come to believe who they are and what is important to them, my motivation is that we can better align the algorithms that structure our collective discourse, as well as develop prosocial, assistive technologies in that space. I organized a RLDM workshop on social alignment in humans and machines on this topic. In this area, I collaborate with psychologists and social (e.g. political) scientists in order to tease apart the interplay of belief, identity, and decision-making bias.

Selected Projects

Using topic correlations inferred from open-ended responses to large scale surveys about climate change beliefs (here about a tax on carbon) to operationalize ideology in pursuit of better climate and policy communication
(Puelma Touzel et al. Environmental Data Science 2024).

Collaborating with the complex data lab, I made contributions to a large-scale analysis of social media data (submitted Yang et al. 2024).

The advent of large langage models has opened up a new and promising approach to studying complex social decision-making phenomena. Using Concordia (Deepmind's LLM-based simulation library for modelling social systems), I am co-leading a large push in the lab to address societal scale manipulation (Puelma Touzel et al. 2024). I lead our first simulation work (Puelma Touzel et al. NeurIPS workshop. 2024).

Topic: Efficient learning algorithms for environments with realistic properties

My theory work is motivated to advance how we design learning algorithms for use in more realistic settings. While funded under the Canadian Excellence Research Chair in autonomous AI held by Irina Rish (UdeM), I and co-authors mathematically analyzed the subtle dynamics that emerge in multi-agent, continual reinforcement learning settings. Our work frames the challenges in the design of robust algorithms in this setting going forward.

Example Projects

Understanding how, as decision-making problems scale, the time to learn about them grows according to the ways in which the agent and the environment interact.
Continual Learning In Environments With Polynomial Mixing Times
Riemer M*, Chandra Raparthy S*, Cases I, Subbaraj G, Puelma Touzel M, Rish I.
NeurIPS (2022).

The world is full of other behaving and learning agents. How can the complexity of what everyone else is doing be abstracted efficiently?
Summarizing Societies: Agent Abstraction in Multi-Agent Reinforcement Learning
Puelma Touzel M*, Me'marian A*, Bhati R, Riemer M, Rish I.
From Cells to Societies: Learning Across Scales Workshop, ICLR (2022).

Scalable Approaches to a theory of many minds
Puelma Touzel M*, Me'marian A*, Riemer M, Mircea A, Williams A, Ahlstrand E, Lehnert L, Bhati R, Dumas G, Rish I.
Agentic Markets, ICML (2024).

Side Topic: Statistical Physics of Learning

I have a long,standing, but side interest in the research formulating online learning using theoretical statistical physics. This arises mostly from the nice correspondence between Bayesian inference and equilibrium thermodynamics. The correspondence is deep, e.g. the Nishimori line of spin glass phases maps to the case when a decoder has exact knowledge of the communication channel. There are many other beautiful connections: sampling as optimization in the space of measures, Entropy regularization/MaxEnt RL. Extending the formalism of information engines from stochastic thermodynamics to machine learning algorithms is an area full of promise. In particular, a thermodynamics of the system's stochastic dynamics reveals information processing as buried in its entropy production. I talked about this in a Talk I gave on Stochastic Thermodynamics of learning to the Physics of Machine learning reading group at Mila. It seems the community is catching on (e.g. this work on speed limits to deep learning). Currently, applications are most learning/adaptation in molecular machinery, but it would be wonderful to advance these tools so that they could impact learning algorithms more broadly. I keep abreast of this field because it is beautiful and because I have a hunch its methods will solve an otherwise inscrutable learning problem that I will confront in my applied work.

Call for co-founders: Social Tech for Climate Crisis

If you have read it this far, you are interested! Let me pitch: While we always need to be discerning when evaluating any given idea to ensure it isn't greenwashed by tech solutionism, I do think AI could play a big role in useful transition tech—applications like optimizing measurement sensing, energy efficiency, and BioTech. Given my training and research experience, I'm specifically interested in applications specializing on the less prevalent, but I think equally important applications of AI to climate action in the social science and social technologies space. ClimateMind is a great example. If these interest you, get in touch!