The Small Loop Problem

We submit the Small Loop Problem to the community of artificial developmental cognition.

The Small Loop Problem consists of implementing an artificial agent that would "smartly" organize its behavior through autonomous interaction with the Small Loop Environment shown in this video.

See our Small Loop Platform in NetLogo for a comprehensive explanation and illustration of this problem.

Our work thus far has only partially solved this problem. This video shows that Ernest indeed manages to learn to perceive its environment and to organize its behavior as we have demonstrated previously in this blog. Ernest, however, still does not manage to capture spatial regularities that would be required to "smartly" handle the passage in the upper-right corner of the loop.

While this problem may seem simplistic, we believe that it captures fundamental questions about self-motivation and early-stage developmental cognition. We suspect that the solution requires making the agent capable of some form of rudimentary reflexivity.

Ernest at Innorobo

This presentation of Ernest was shown at the Innorobo symposium in Lyon. Here is a shorter video without the audio presentation.

Object persistence

This video shows how we expect Ernest to learn about objects in the world. To do so, Ernest needs a Spatial Memory that keeps track of the places where Ernest enacted interactions. We implemented a mechanism that emulates a vestibular system to allow Ernest to keep track of his displacements in space.

In Ernest's Spatial Memory, blue half-circles represent places where Ernest turned; blue triangles represent places where he moved forward; red circles represent places where he bumped; green circles represent places where he situates the origin of visual perception that "attracts" him. These symbols are shrinking over time to represent memory decay.

This video shows that Ernest constructs a rough outline of the object in Spatial Memory, and localizes possibilities of interactions relatively to this object.

The big question now is how to make Ernest exploit this knowledge and how to entangle this knowledge construction mechanism with Ernest's "intrinsic motivation".

(Demo implemented with Ernest r228 and Vacuum r166)

Ernest 11.1 Local Space Memory

This video shows the content of Ernestine's local space memory during the Ernest 11.1 experiment. As discussed previously, Ernest attributes "causes" of interactions to "phenomena" situated in the external environment.

Ernest 11.1 uses vision and touch to localize places in local space memory that "represent" external phenomena:
- Line segments are places that are constructed using vision.
- Circle arcs are places that are constructed using both touch and vision.

Additionally, Ernest 11.1 localizes places of interactions represented by small symbols:
- Small triangles are places where Ernest enacted a move forward interaction.
- Small half-circles are places where Ernest enacted a turn interaction.

Places of interaction are localized relatively to Ernest and also possibly relatively to external phenomena:
- Fading blue symbols (triangles and half-circles) behind Ernestine represent interactions localized relatively to Ernestine.
- Bi-color blue and yellow symbols represent eating interactions (while moving forward or while turning) localized relatively to fish.
- Bi-color green and red triangles represent bumping interactions (while trying to move forward) localized relatively to walls.
- Bi-color gray and pink symbols represent cuddling interactions (while moving forward or turning) localized relatively to Ernesto.

Over time, this video shows that phenomena "evoke" possibilities of interaction based on previous episodes of interaction (e.g., fish become associated with different manners of eating fish, etc.). In the future, we expect that such a spatio-temporal episodic memory will help Ernest perform mental simulations of possible courses of action. Neuro-physiological studies suggest that the hippocampus supports this kind of spatio-temporal episodic memory in the vertebrate's brain (e.g., Barthoz, 1997).

Reference
Berthoz A. (1997) Le sens du mouvement. Odile Jacob: Paris.

Ernest 11.1

Ernest 11.1 works similarly as Ernest 11.0. Ernest 11.1 , however, has the improved local space memory that is shown in the next post.

Note that, in this run, Ernestine becomes more prone to eating fish than cuddling with Ernesto, while Ernesto becomes more prone to cuddling with Ernestine than eating fish.

(Demo implemented with Ernest r225 and Vacuum r165)