Autonomous Patterns of Sensorimotor Activity


Much of our behaviour is habitual. We run or cook or think in a particular way not because it is necessarily the most optimal way to do so, but because that is the way that we have done these things in the past. For a long period of time, habits were seen as the most fundamental concept in the study of the mind. But more recently, enthusiasm for computationalist views of the mind has relegated habits to a relatively unimportant role of "offloading computational load." The idea here is that once you have figured out how to do something, why spend any of "your brain's computational power" figuring it out again?

But what is lost in this simplification of habits?
In this research project, we bring modern tools to study the older, richer concept of habit. Instead of seeing habits as atemporal learned correlations between stimulus and response, we investigate habits as patterns of behaviour that reinforce the mechanism(s) that produce those very patterns of behaviour.

Sensorimotor space and the IDSM

Our approach is at an intermediate "meso-" level, where instead of describing dynamics at a "micro"-level of neural-networks, or the "macro"-level of whole actions, we consider self-reinforcing, patterns of sensorimotor activity. By monitoring trajectories in sensorimotor space (see Figures 1 and 2), our innovated robot controller, the Iterant Deformable Sensorimotor Medium (IDSM) causes robots or virtual agents to repeat patterns of sensorimotor activity that it has previously experienced.

Certain patterns of sensorimotor activity facilitate the repetition of that behaviour more than others. For instance, walking around in a circle until you are where you started sets you up perfectly to repeat that behaviour, where as jumping out of an exploding airplane is an action that does not.

In our first explorations of the IDSM, we were intrigued to notice that even in the absence of a reward mechanism, the patterns of behaviour that form relate to the environment in "meaningful" or goal-like ways. For example, the central pane of Video 1 shows an overhead view of a robot moving around a two-dimensional environment. In the center of this world is a light. Over time, the robot demonstrates a number of repetative, self-reinforcing "habits." ---and interestingly, they all relate in some way to the light, in many cases causing the robot to circle around the light. Other experiments have shown similar results, even when the light is moving around the environment.

After some consideration, it becomes clear that these behaviours must relate to the light in a consistent and repeatable manner, as they are based upon repeated sensorimotor trajectories. The only way for a sensorimotor trajectory to be repeated is if the robots motion causes its sensory state to change in a way that is consistently associated with motor changes. Without any reward or additional selection mechanism, the robot falls into patterns of behaviour that are based upon --in a sense, built out of-- sensorimotor contingencies.

Next steps

There are many appealing avenues for further investigation. Briefly here, I will outline a few projects of interest. If you have other ideas, please get in touch.

Enumerating, categorizing and calculating the relative frequency of the set of possible self-reinforcing sensorimotor trajectories of minimalistic robots in minimalistic environments. Some patterns of activity can be repeated others cannot. Some can be repeated from many more initial conditions than others. Studying the simplest possible environments and robots, we are building a compendium of habits, and calculating the relative frequency of each category of behaviour. By understanding what habits form in the absence of additional constraints, we can develop a better understanding of how to implement additional selective mechanisms that increases the likelihood of target habits forming.

Investigating the coconstruction of language. By coupling the IDSM to a microphone and a sound-synthesizer, we will study what is necessary for the IDSM to create self-sustaining and self-reinforcing patterns of interaction with humans.

How do "hard-wired" reflexive behaviours bias habit formation in the IDSM? Using genetic algorithms, we will evolve reflexive behaviours that bias habit formation such that desired patterns of behaviour are generated.

Zarco M., Egbert M. D. Different Forms of Random Motor Activity Scaffold the Formation of Different Habits in a Simulated Robot (submitted to ALIFE2019).
Woolford F. M. , Egbert M. D. (2019) Behavioral Variety of a Node-Based Sensorimotor-to-Motor Map. Adaptive Behavior
Egbert MD, Canamero L. (2014) Habit-based Regulation of Essential Variables. In: Artificial Life 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems.; 2014.
Egbert MD, Barandiaran XE. (2014) Modelling habits as self-sustaining patterns of sensorimotor behavior. Frontiers in Human Neuroscience. 2014;8(590).

1D Robot
Figure 1. A motor with one motor and one sensor. This robot can just move forward or backward, causing it to move closer or farther away from the light in front of it. If it oscillates back and forth in front of the light, the states of its light sensor and its motor can be plotted in "sensorimotor space" (see Figure 2).
2D Sensorimotor Space
Figure 2. A two-dimensional sensorimotor space. The robot in Figure 1 has one motor and one sensor, so its sensorimotor space is two-dimensional. Every point in sensorimotor space corresponds to a single configuration of motors and sensors. If the robot in Figure 1 is oscillating back and forth in front of the light, its sensorimotortrajectory would look something like this, with the coloured Xs relating the spatial position of the robot to points in sensorimotor space.
Video 1. Sensorimotor space, physical-space, and essential variables in a simulated IDSM-controlled robot. The left pane shows a cross-section through the 5-dimensional sensorimotor space of the simulated robot. When the robot is performing a habit, repeating cycles are visible here. The central pane shows an overhead view of the Khepera-esque robot moving around a two-dimensional, periodic environment. When it is performing a habit, the robot demonstrates a repeated motif of behaviour, which causes it to circle, move toward or away from the light in a clearly patterned manner. The right pane is not relevant for the current purposes. See [Egbert MD, Canamero L 2014] for details.