Review of Chaput H. H. 2004

This is a review of Harold H. Chaput's PhD dissertation. I wrote it because I was impressed by his learning theory.

The Constructivist Learning Architecture: A Model of Cognitive Development for Robust Autonomous Robots

Harold Chaput proposes a new learning architecture called CLA: Constructivist Learning Architecture.
He takes inspiration from Piaget's work about constructivist epistemology and his notion of scheme or schema. Piaget’s constructivist epistemology emphasis that knowledge is grounded in action. Piaget proposed the notion of scheme, or schema, as the basic element of knowledge. A schema embeds perception, action and expectations in a single temporal pattern of behavior. Piaget proposed a theory of learning based on a progressive construction of schemas from a basic sensorimotor level to the most abstract level.
The CLA is an implementation of this theory in computer. In this dissertation, Harold Chaput demonstrates how this implementation can account for infants’ natural learning. He also demonstrates how the CLA can be used to build an autonomous robot that actually performs artificial learning.

Concerning its implementation in a robot, this work is based on the previous work of Schema Mechanism (Drescher 1991). Harold Chaput refers to the Schema Mechanism as one of the best implementations of constructivist learning, and as the only known learning system to model constructivism as described by Piaget.
He gives a very precise description of it in section 5.1. He summarizes it as follows:

To summarize, the Schema Mechanism starts with a set of primitive items and primitive actions. It then explores the environment to create a set of sensorimotor schemas. These schemas form the basis of new synthetic items. They also are used in the creation of goal-directed composite actions. Using these techniques, an agent can build a hierarchy of items to describe its environment, and a hierarchy of sensorimotor schemas that are combined into a plan for achieving some goal.

However, Harold Chaput deplores that the Schema Mechanism faces insurmountable scale-up issues. These scale-up issues make it impossible to use for a robot in a real world situation.
So, Chaput presents the CLA as an alternative for implementing the Schema Mechanism. Basically, the schema construction that was made in a deterministic manner by the Schema Mechanism, is made in a probabilistic manner by the CLA. This probabilistic schema construction uses a clustering mechanism called SOM (Self-Organizing Maps) (Kohonen 1997).

SOM is an unsupervised learning system that maps input data into a feature coordinate system, or feature map. In the CLA's implementation, patterns of behavior constitute the SOM's input data. The SOM is used to cluster them into a set of schemas. Along training, the SOM will organize itself as a network of nodes, where each node represents a prototype vector of the input. Harold Chaput has chosen the SOM because of its neural plausibility. He notices that other methods for vector clustering could be used as well. As an alternative, he cites the Independent Component Analysis (ICA; Hyvärinen, Karhunen & Oja 2001).
CLA uses several layers of SOM. Each layer takes the level below it as an input. This hierarchical architecture allows the robot to learn increasingly abstract schemas.

Chaput shows how the CLA implements the same functionalities as the Schema Mechanism, but without facing the scale-up limitation.
He illustrates it in a very precise way in section 6.2. This section describes an experiment made with a simulator of a robot, involved in a foraging task. Chaput demonstrates how different levels of schemas are learned, from low-level sensorimotors schemas, to more abstract description of strategies. He demonstrates fallback mechanisms to lower-level schemas, when higher-level schemas are failing. He also demonstrates recovery mechanisms, when the robot is damaged.

Finally, he describes a mechanism of reinforcement learning with delayed feedback, that was implemented to provide the robot with goals. A high reinforcement value was attributed to the final schema that completes the foraging task. A mechanism of spreading reinforcement value along prerequisite schemas was implemented. This caused the robot to organize its behavior according to the task.

In conclusion, I can only regret that this work was only validated in a robot simulator, but not in a real robot in a real environment. Chaput is listing future works aiming at validating it in a real world environment. I imagine that could raise new difficulties, for example the problem that schemas might need to be more "continuous", in opposition to the "discrete symbolic schemas", on which this work was focusing. Then would come the problem of linking continuous schemas to discrete schemas. Anyway, I think this work raises facinating research opportunities.

Reference

Chaput, H. H. (2004). The Constructivist Learning Architecture: A Model of Cognitive Development for Robust Autonomous Robots. Unpublished doctoral dissertation, The University of Texas, Austin