Research

There are two important aspects to decision-making that I find especially fascinating:

Uncertainty in our beliefs. It weaves together the main components of decision-making theory:
- Uncertainty about how to efficiently represent the world
  - How do we abstract the world in a way that minimally distorts our ability to choose actions to achieve a goal?
- Uncertainty about the value of actions we take
  - How does the uncertainty in our beliefs about the world transform into uncertainty about how valuable are the actions we can take in it?
- Uncertainty about what goals we should pursue
  - How do we form and select goals to pursue (ultimately defining what value means for us)?
Why do we sometimes make the effort to reduce these kinds of uncertainty, while other times we simply account for it, or even neglect it outright, even at our peril? While we are typically frugal with cognitive effort, we are also often simply suboptimal with respect to instrumental reward. A decision utility that extends to cognitive/informational and affective/social value is one line of investigation. Another stems from acknowledging that we misinterpret or are misguided and misinformed by our social environment. We are a social species after all.
Social factors, personal values, and sense of identity. They can play a prominent role in the decision-making process within an individual, but also collectively within a society. Many of the pressing societal problems of our time, from polarization to climate change inaction, rest on how we form beliefs while consuming information and dialoguing in social venues, increasingly curated by algorithms, on a range of scales from peer-to-peer extending to large, online communities. The form of the information exchanged in these fundamentally social interactions and the form of the mediums over which this exchange occurs can play a role in the collective decisions we make as a society by biasing how individuals decide what to believe.

These are in my mind as I dig around at the various intersections of

social/cognitive psychology & behavioural sciences,
public policy & opinion research, and
computational social science

in pursuit of tractable, high-impact problems.

Ideal products for me are reproducible scientific analysis and efficient algorithms that provide insight about and scale-up, respectively, human decision-making for social well-being. A local and global approach to understanding social systems highlights two kinds of phenomena:

inference and decision-making in the behaviour of individuals and small groups, and
complex system dynamics of large numbers of interacting learning agents.

I am excited to take part in the modern trend of pushing the theory of games & machine learning towards contact with data to better understand real-world multi-agent systems (e.g. DIMACS workshop series on Bridging Game Theory and Machine Learning for Multi-party Decision Making).

Applications

In my applied work, I collaborate with psychologists and social scientists in order to tease apart the interplay of belief, identity, and decision-making bias. By better accounting for the effects of the social milieu in which individuals come to believe who they are and what is important to them, my motivation is that we can better align the algorithms that structure our collective discourse, as well as develop prosocial, assistive technologies in that space. I organized a RLDM workshop on social alignment in humans and machines on this topic.

Example Project

Using topic correlations inferred from open-ended responses to large scale surveys about climate change beliefs (here about a tax on carbon) to operationalize ideology in pursuit of better climate and policy communication.
[with E. Lachapelle (UdeM)]

Theory

My theory work is motivated to advance how we design learning algorithms for use in more realistic settings. I am currently funded under the Canadian Excellence Research Chair in autonomous AI held by Irina Rish (UdeM). With students in her group, we analyze the subtle dynamics that emerge in multi-agent, continual reinforcement learning settings. This controlled modelling sandbox helps us develop theory, as well as to contrast with complex social decision-making phenomena in natural systems.

Example Projects

Understanding how, as decision-making problems scale, the time to learn about them grows according to the ways in which the agent and the environment interact.
Continual Learning In Environments With Polynomial Mixing Times
Riemer M*, Chandra Raparthy S*, Cases I, Subbaraj G, Puelma Touzel M, Rish I.
NeurIPS (2022).

The world is full of other behaving and learning agents. How can the complexity of what everyone else is doing be abstracted efficiently?
Summarizing Societies: Agent Abstraction in Multi-Agent Reinforcement Learning
Puelma Touzel M*, Me'marian A*, Bhati R, Riemer M, Rish I.
From Cells to Societies: Learning Across Scales Workshop, ICLR (2022).

Statistical Physics

I have a long-standing interest in the research formulating online learning using theoretical statistical physics. This arises mostly from the nice correspondence between Bayesian inference and equilibrium thermodynamics. The correspondence is deep, e.g. the Nishimori line of spin glass phases maps to the case when a decoder has exact knowledge of the communication channel. There are many other beautiful connections: sampling as optimization in the space of measures, Entropy regularization/MaxEnt RL. Extending the formalism of information engines from stochastic thermodynamics to machine learning algorithms is an area full of promise. In particular, a thermodynamics of the system's stochastic dynamics reveals information processing as buried in its entropy production. I talked about this in a Talk I gave on Stochastic Thermodynamics of learning to the Physics of Machine learning reading group at Mila. It seems the community is catching on (e.g. this work on speed limits to deep learning). If this topic interests you, get in touch! Currently, applications are most in molecular machinery, but it would be wonderful to advance these tools so that they could impact learning algorithms more broadly.

AI Startups and Climate Change

The climate change start-up scene is exploding and AI is jumping on board. The main resources to start engaging this community are:

While we always need to be discerning when evaluating any given idea to ensure it isn't greenwashed by tech solutionism, I do think AI could play a big role in useful transition tech—applications like optimizing measurement sensing, energy efficiency, and BioTech. Given my training and research experience, I'm specifically interested in applications specializing on the less prevalent, but I think equally important applications of AI to climate action in the social science and social technologies space. If these interest you, get in touch!