Ernest 12

Ernest 12  categorizes the entities in its environment based on the possibilities of interaction that they afford, and adjusts its behavior to categories.

Top-left: Ernest in its environment. The "eye" (half-circle) takes the color of the entity that get Ernest's attention at any given time.

Top-right: Ernest's spatial memory. Interactions are localized in space, and Ernest updates their position as he moves. Entities are constructed where interactions overlap. Rectangles and trapezoids represent interactions, circles represent entities.

Bottom: activity trace. Bottom: interactions (rectangles and trapezoids) enacted on the left, in front, or on the right of Ernest. Middle: the motivational value of the enacted interaction represented as a bargraph (green when positive, red when negative). Top: the actions (half-circles (turn), triangles (try to step forward)) and the entities (blue and green circles) learned over time.

In this run, Ernest, learns the "bishop behavior" during the first 50 steps. On steps 78, we introduce two targets in a raw. The spatial memory shows that Ernest interacts with these two targets at the same time. Ernest's spatial memory (associated with its rudimentary attentional system) allows Ernest to focus on one target at a time.

On step 110, we introduce a "wall brick", and Ernest learns that this kind of entity affords the interaction "bumping". Subsequently, when we introduce a target, Ernest will preferably go towards the target than towards the wall brick because it has learned that the targets are edible.

Ernest 12 implements ECA, the Enactive Cognitive Architecture.
(Demo implemented with Ernest r439 and Vacuum r392)

Enactive Robot Learning

Olivier L. Georgeon, Christian Wolf, and Simon Gay 2013. An Enactive Approach to Autonomous Agent and Robot Learning.  IEEE Third Joint  International Conference on Development and Learning and on Epigenetic Robotics (EPIROB2013). Osaka, Japan. August 18-22 2013.

This paper constitutes a short introductory version of our ECA paper. It also presents the experiment of Ernest7 in an e-puck robot. 

Enactive Cognitive Architecture

Olivier L. Georgeon, James B. Marshall, and Riccardo Manzotti 2013. ECA: An enactivist cognitive architecture based on sensorimotor modeling. Biologically Inspired Cognitive Architectures, Volume 6, pp. 46-57, doi: 10.1016/j.bica.2013.05.006. Also presented at BICA2013.

This paper introduces a new way of modeling an agent interacting with an environment called an Enactive Markov Decision Process, inspired by the Theory of Enaction. It also describes Ernest's motivational principle in relation with the autotelic principle (Steels, 2004) and the optimal experience principle (Csikszentmihalyi, 1990). It introduces ECA, the Enactive Cognitive Architecture that drives Ernest 11, and it reports the Ernest 11.2 experiment.

Sensemaking emergence demonstration

Olivier L. Georgeon and James B. Marshall 2013. Demonstrating sensemaking emergence in artificial agents: A method and an example. International Journal of Machine Consciousness, 5(2), pp 131-144, doi: 10.1142/S1793843013500029.

This paper addresses the sensemaking demonstration problem : the problem of demonstrating that an agent gives meaning to or understands its experience. We present a methodology to produce empirical evidence to support or contradict the claim that an agent is capable of a rudimentary form of sensemaking, based on an analysis of the agent's behavior.

As an example, we report an analysis of Ernest's behavior in the Small Loop Problem and we conclude that Ernest is capable of a rudimentary form of sensemaking. This paper is an extended version of our previous paper presented at BICA2012.

Training Ernest 7

This video demonstrates that Ernest develops different behaviors depending on the experience he has during his youth. Here, we have two instances of Ernest: Ernest 1 (brown) is initially kept in the small loop and released on step 290. Ernest 2 (bleu) is confronted to the complex environment right from his birth.

Ernest 1 develops more sophisticated behaviors than Ernest 2 because he is trained to touch both of its sides when he faces a wall. Consequently, after being released, he has a more exploratory behavior than Ernest 2.

Ernest 2 learns to preferably turn to the right when he faces a wall. Consequently, he tends to keep spinning in limited areas of the environment. Ernest 2's learning is limited by the fact that the environment is initially too complex for him to notice sophisticated sequences that involve touching to both sides.

The importance of training is an interesting property of Ernest because it accounts for theories of developmental learning.


(Demo implemented with Ernest r296 and Vacuum r203)

Ernest's source code

Ernest's source code is available here with the instructions to use it. The cleaned-up and tested recommended revision is r296. This revision demonstrates the exact behavior of Ernest 7 reported in the Small Loop Experiment.

Interactional Motivation

 Olivier L. Georgeon, James B. Marshall, and Simon L. Gay 2012. Interactional Motivation in Artificial Systems: Between Extrinsic and Intrinsic Motivation.  In proceedings of the 2nd Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics (EPIROB 2012), San Diego, pp. 1-2.

This paper presents the notion of interactional motivation that drives Ernest, and compares it to reinforcement learning as it is traditionally implemented in Partially Observable Markov Decision Processes (POMDPs).

Ernest 11.5 constructs goals

Like Ernest 11.4, Ernest 11.5 can recognize objects by the possibilities of interaction that they afford. Additionally, Ernest 11.5 has a specific inborn taste for stepping on flowers.

Ernest 11.5 simulates different possible sequences of interactions in spatial memory before selecting the best sequence to enact. These simulations are represented in the bottom-right area of the video. Simulations that produce predictable results (due to information available in spatial memory) are represented with orange outlines. Simulations that produce unpredictable results (due to the lack of information in spatial memory) are represented in blue. The video shows that Ernest learns to simulate increasingly elaborated sequences of interactions as time goes on (see blue squares and triangles spreading in all directions around Ernest from step 253 on).

The high value associated with stepping on flowers favors simulations that lead to even more stepping on flowers. As a result, Ernest learns to make a u-turn to return to a flower when he passes one (see Ernest keeping stepping on the flower from step 260 on).

We find this experiment interesting because it illustrates how an inborn drive can give raise to an explicit goal. Ernest's inborn tendency to step on flowers makes Ernest identify flowers as an interesting goal to reach. Once this goal is recognized, Ernest performs a rudimentary problem-solving computation to reach it. Perhaps the skill to choose a desirable point in space and find a sequence of operations to reach this point underlays higher-level problem-solving skills.