Olivier Georgeon's research blog—also known as the story of little Ernest, the developmental agent.

Keywords: situated cognition, constructivist learning, intrinsic motivation, bottom-up self-programming, individuation, theory of enaction, developmental learning, artificial sense-making, biologically inspired cognitive architectures, agnostic agents (without ontological assumptions about the environment).

Friday, June 12, 2009

Ernest 6.2

Ernest 6.2 only enacts schemas when they attain a certain level of reinforcement (set to 7 in this example).

At the beginning, only schemas A1 and B1 are enacted.

Very soon, the sequence B1F-B1S is aggregated into S5, and the sequence A1F-A1S is aggregated into S9, but Ernest has not yet found the stable solution based on these schemas.

Almost at the middle of the trace, Ernest finds the solution by stabilizing the sequence A1F-A1S-B1F-B1S.

From then, S5 and S9 get faster reinforcement until S5 reaches the reinforcement of 7. Then S5 starts being enacted and Ernest searches a new equilibrium until he finds the sequence A1F-A1S-S5S.

Then, S9 reaches a reinforcement of 7 and starts being enacted too. So, Ernest finally finds the sequence S5S-S9S to solve the aA..bB task.

Ernest 6.2 borrows the two mechanisms of Ernest 6.0 and 6.1. Like Ernest 6.0, he starts searching a solution based on lower-level schemas until they get frozen. Then, like Ernest 6.1, he starts enacting higher-level schemas. With this mechanism, Ernest uses what he knows about lower-level schemas to find the correct sequence of higher-level schemas before effectively starting to enact them.

Once the higher-level schemas are stabilized, we can consider them as base-level schemas and continue learning higher-level schemas on top of them.