Ernest 11.2 in the Small Loop Problem

This video shows Ernest 11.2's temporal memory (up-left) and spatial memory (right) in the Small Loop Problem (bottom-left).

Like Ernest 11.1, Ernest 11.2 keeps track of interactions in his spatial memory. Touch interactions are represented by squares, move by white triangles, bump by red triangles, turn by half-circles.

Interactions that are enacted at the same spatial place are bundled together to represent the phenomenon that afforded them. The Small Loop Problem offers two kinds of phenomena: empty squares and walls. In temporal memory, the construction of bundles is represented by gray rounded rectangles (upper part). In spatial memory, bundles are represented by gray circles that contain interactions. These circles are fading to represent spatial memory decay.

For example, on step 13, Ernest bundles together the interactions touch front wall and bump. On step 19, the fact that Ernest touches a wall activates the still-partial wall bundle in front of Ernest (right side of spatial memory). This activation reminded Ernest that moving forward would cause bumping, which made him refrain from trying to move forward.

Ernest 11.2's spatial mechanism helps him perform better than Ernest 7 in the Small Loop Problem. Ernest 11.2 does not, however, constitute a full solution to the Small Loop Problem because he has pre-implemented knowledge. The position of interactions relatively to Ernest's body were hard-wired in spatial memory. Additionally, the consequences that certain interactions have on spatial memory were pre-implemented: move translates spatial memory and turn rotates spatial memory. A full solution to the Small Loop Problem must eliminate such pre-implemented assumptions.

Designing Environment-Agnostic Agents

Designing Environment-Agnostic Agents. Olivier L. Georgeon, Ilias Sakellariou. In proceedings of ALA2012, Adaptive Learning Agents workshop, at  AAMAS 2012, 11th International Conference on Autonomous Agents and Multiagent Systems. 25-32. (June 4-5th 2012).

In this paper, we define the notion of environment-agnostic agents. We also presents Ernest in NetLogo.

The Small Loop Problem

We submit the Small Loop Problem to the community of artificial developmental cognition.

The Small Loop Problem consists of implementing an artificial agent that would "smartly" organize its behavior through autonomous interaction with the Small Loop Environment shown in this video.

See our Small Loop Platform in NetLogo for a comprehensive explanation and illustration of this problem.

Our work thus far has only partially solved this problem. This video shows that Ernest indeed manages to learn to perceive its environment and to organize its behavior as we have demonstrated previously in this blog. Ernest, however, still does not manage to capture spatial regularities that would be required to "smartly" handle the passage in the upper-right corner of the loop.

While this problem may seem simplistic, we believe that it captures fundamental questions about self-motivation and early-stage developmental cognition. We suspect that the solution requires making the agent capable of some form of rudimentary reflexivity.

Ernest at Innorobo

This presentation of Ernest was shown at the Innorobo symposium in Lyon. Here is a shorter video without the audio presentation.

Object persistence

This video shows how we expect Ernest to learn about objects in the world. To do so, Ernest needs a Spatial Memory that keeps track of the places where Ernest enacted interactions. We implemented a mechanism that emulates a vestibular system to allow Ernest to keep track of his displacements in space.

In Ernest's Spatial Memory, blue half-circles represent places where Ernest turned; blue triangles represent places where he moved forward; red circles represent places where he bumped; green circles represent places where he situates the origin of visual perception that "attracts" him. These symbols are shrinking over time to represent memory decay.

This video shows that Ernest constructs a rough outline of the object in Spatial Memory, and localizes possibilities of interactions relatively to this object.

The big question now is how to make Ernest exploit this knowledge and how to entangle this knowledge construction mechanism with Ernest's "intrinsic motivation".

(Demo implemented with Ernest r228 and Vacuum r166)

Ernest 11.1 Local Space Memory

This video shows the content of Ernestine's local space memory during the Ernest 11.1 experiment. As discussed previously, Ernest attributes "causes" of interactions to "phenomena" situated in the external environment.

Ernest 11.1 uses vision and touch to localize places in local space memory that "represent" external phenomena:
- Line segments are places that are constructed using vision.
- Circle arcs are places that are constructed using both touch and vision.

Additionally, Ernest 11.1 localizes places of interactions represented by small symbols:
- Small triangles are places where Ernest enacted a move forward interaction.
- Small half-circles are places where Ernest enacted a turn interaction.

Places of interaction are localized relatively to Ernest and also possibly relatively to external phenomena:
- Fading blue symbols (triangles and half-circles) behind Ernestine represent interactions localized relatively to Ernestine.
- Bi-color blue and yellow symbols represent eating interactions (while moving forward or while turning) localized relatively to fish.
- Bi-color green and red triangles represent bumping interactions (while trying to move forward) localized relatively to walls.
- Bi-color gray and pink symbols represent cuddling interactions (while moving forward or turning) localized relatively to Ernesto.

Over time, this video shows that phenomena "evoke" possibilities of interaction based on previous episodes of interaction (e.g., fish become associated with different manners of eating fish, etc.). In the future, we expect that such a spatio-temporal episodic memory will help Ernest perform mental simulations of possible courses of action. Neuro-physiological studies suggest that the hippocampus supports this kind of spatio-temporal episodic memory in the vertebrate's brain (e.g., Barthoz, 1997).

Reference
Berthoz A. (1997) Le sens du mouvement. Odile Jacob: Paris.

Ernest 11.1

Ernest 11.1 works similarly as Ernest 11.0. Ernest 11.1 , however, has the improved local space memory that is shown in the next post.

Note that, in this run, Ernestine becomes more prone to eating fish than cuddling with Ernesto, while Ernesto becomes more prone to cuddling with Ernestine than eating fish.

(Demo implemented with Ernest r225 and Vacuum r165)

Implementing spatial awareness

Implementing Spatial Awareness in an Environment-Agnostic Agent. Simon Gay, Olivier Georgeon, Jong Wook Kim. Presented at the 21st Annual Conference on Behavior Representation in Modeling and Simulation (BRIMS 2012). Amelia Island, Florida. March 12-15. 62-69.

This paper presents Ernest 10.4 and our more general work on endowing an autonomous agent with a sense of space. We introduce the notions of local space memory, persistence memory, and bundles. In particular, we argue that this initial spatial awareness lays the bases for the emergence of external causality. Ernest now attributes the cause of sensory stimulations to phenomena situated in the external environment.