Olivier Georgeon's research blog—also known as the story of little Ernest, the developmental agent.

Keywords: situated cognition, constructivist learning, intrinsic motivation, bottom-up self-programming, individuation, theory of enaction, developmental learning, artificial sense-making, biologically inspired cognitive architectures, agnostic agents (without ontological assumptions about the environment).

Friday, September 19, 2008

Other famous Ernests

Ernest Nagel was among the most important philosophers of science of the twentieth century. He has taken up reflexivity as an issue in science. (Ernest Nagel, Wikipedia; Reflexivity, Wikipedia)

The Ernest's legend

Internal states, in green, descibe what Ernest has in mind. Internal operators, in orange, discribe the operations Ernest mentally computes. Actions, in pink, describe what Ernest acts. Environment, in blue, describe what the environment responds.
Of course, that's a legend, Ernest has no mind. A good reference to explain that is Artificial intelligence meets natural stupidity, Drew McDermott (1976).

Thursday, September 18, 2008

Poor Ernest

video
So, let's put Ernest in an environment a bit more complex. Now, to get Yees, one must make alternatively A and B. That requires to add some "intelligence" in the environment as well.
Of course, it is catastrophic. Poor Ernest only has a one-round memory and has no capacity to recognize AB regularities.
If the problem of cognition is to find one's happiness by exploiting the environment's regularities, then there is still a big deal of work for Ernest...

Wednesday, September 17, 2008

Ernest, the smallest learning artificial agent in the world


video
Ernest can only do two things: A or B.
Ernest can only perceive two things: "Yee" or "o-o". That is good, because those are the only two things that his environment can exhibit.
Ernest loves Yees and hates o-os. Will he learn that, in this environment, to get Yees, one must do B?
The answer is in his activity trace. Play it with your speakers on, because Ernest speaks!!
So, Ernest finds his happiness by exploiting the simplest possible regularity in his environment: A makes o-o and B makes Yee.
But what happens if his environment exhibits more complex regularities? That is what we have to study next.

Monday, September 1, 2008

Review of Chaput H. H. 2004

This is a review of Harold H. Chaput's PhD dissertation. I wrote it because I was impressed by his learning theory.

The Constructivist Learning Architecture: A Model of Cognitive Development for Robust Autonomous Robots

Harold Chaput proposes a new learning architecture called CLA: Constructivist Learning Architecture.
He takes inspiration from Piaget's work about constructivist epistemology and his notion of scheme or schema. Piaget’s constructivist epistemology emphasis that knowledge is grounded in action. Piaget proposed the notion of scheme, or schema, as the basic element of knowledge. A schema embeds perception, action and expectations in a single temporal pattern of behavior. Piaget proposed a theory of learning based on a progressive construction of schemas from a basic sensorimotor level to the most abstract level.
The CLA is an implementation of this theory in computer. In this dissertation, Harold Chaput demonstrates how this implementation can account for infants’ natural learning. He also demonstrates how the CLA can be used to build an autonomous robot that actually performs artificial learning.

Concerning its implementation in a robot, this work is based on the previous work of Schema Mechanism (Drescher 1991). Harold Chaput refers to the Schema Mechanism as one of the best implementations of constructivist learning, and as the only known learning system to model constructivism as described by Piaget.
He gives a very precise description of it in section 5.1. He summarizes it as follows:

To summarize, the Schema Mechanism starts with a set of primitive items and primitive actions. It then explores the environment to create a set of sensorimotor schemas. These schemas form the basis of new synthetic items. They also are used in the creation of goal-directed composite actions. Using these techniques, an agent can build a hierarchy of items to describe its environment, and a hierarchy of sensorimotor schemas that are combined into a plan for achieving some goal.

However, Harold Chaput deplores that the Schema Mechanism faces insurmountable scale-up issues. These scale-up issues make it impossible to use for a robot in a real world situation.
So, Chaput presents the CLA as an alternative for implementing the Schema Mechanism. Basically, the schema construction that was made in a deterministic manner by the Schema Mechanism, is made in a probabilistic manner by the CLA. This probabilistic schema construction uses a clustering mechanism called SOM (Self-Organizing Maps) (Kohonen 1997).

SOM is an unsupervised learning system that maps input data into a feature coordinate system, or feature map. In the CLA's implementation, patterns of behavior constitute the SOM's input data. The SOM is used to cluster them into a set of schemas. Along training, the SOM will organize itself as a network of nodes, where each node represents a prototype vector of the input. Harold Chaput has chosen the SOM because of its neural plausibility. He notices that other methods for vector clustering could be used as well. As an alternative, he cites the Independent Component Analysis (ICA; Hyvärinen, Karhunen & Oja 2001).
CLA uses several layers of SOM. Each layer takes the level below it as an input. This hierarchical architecture allows the robot to learn increasingly abstract schemas.

Chaput shows how the CLA implements the same functionalities as the Schema Mechanism, but without facing the scale-up limitation.
He illustrates it in a very precise way in section 6.2. This section describes an experiment made with a simulator of a robot, involved in a foraging task. Chaput demonstrates how different levels of schemas are learned, from low-level sensorimotors schemas, to more abstract description of strategies. He demonstrates fallback mechanisms to lower-level schemas, when higher-level schemas are failing. He also demonstrates recovery mechanisms, when the robot is damaged.

Finally, he describes a mechanism of reinforcement learning with delayed feedback, that was implemented to provide the robot with goals. A high reinforcement value was attributed to the final schema that completes the foraging task. A mechanism of spreading reinforcement value along prerequisite schemas was implemented. This caused the robot to organize its behavior according to the task.

In conclusion, I can only regret that this work was only validated in a robot simulator, but not in a real robot in a real environment. Chaput is listing future works aiming at validating it in a real world environment. I imagine that could raise new difficulties, for example the problem that schemas might need to be more "continuous", in opposition to the "discrete symbolic schemas", on which this work was focusing. Then would come the problem of linking continuous schemas to discrete schemas. Anyway, I think this work raises facinating research opportunities.

Reference

Chaput, H. H. (2004). The Constructivist Learning Architecture: A Model of Cognitive Development for Robust Autonomous Robots. Unpublished doctoral dissertation, The University of Texas, Austin