Olivier Georgeon's research blog—also known as the story of little Ernest, the developmental agent.

Keywords: situated cognition, constructivist learning, intrinsic motivation, bottom-up self-programming, individuation, theory of enaction, developmental learning, artificial sense-making, biologically inspired cognitive architectures, agnostic agents (without ontological assumptions about the environment).

Sunday, November 23, 2008

Ernest4.1's cycle

Construct context: consists of structuring the current context to prepare the schema construction and the schema selection. The context is made up of the three previously enacted schemas, stored in short-term memory. These schemas can be of any level, and can refer to subschemas. This phase indexes these different levels.

Construct new schemas: This phase consists of creating new schemas that match the current context. If they not yet exist, these new schemas are added to long-term memory. They constitute hypotheses about how to deal with a new context, but they still need to be tested.

Select a schema / First step: consists of selecting a schema to be executed in this context. High-level schemas add weight to their subschemas. Weights are positive if they lead to Y and negative if they lead to X. Schemas of any level compete, and the one with the highest weight is selected. If there are several ex aequo, one of them is randomly picked. This phase initialize the selected schema at its first step.

Execute schema step: sends the selected action defined in the current schema step to the environment: A or B.

Environment: computes the response from the environment and send it to Ernest: Y or X. The environment has his own memory and cycle.

Assess environment's response: checks if the schema has succeeded or failed. If the current subschema has succeeded but it is not the last step of the selected schema, then the "ongoing" loop is selected.

Next step / subschema: selects the next step in the subschema hierarchy of the selected schema.

Memorize / reinforce schema: when the schema ends up, if it has succeeded, then it is referred as the last enacted schema in short term memory. The previous two are shifted, and the previous third is drop out of short-term memory.
If the schema has failed at some point, then the actually enacted schema is stored in short-term memory and reinforced in long-term memory. For exemple, if a primary schema expecting Y has been selected, but if the environment actually returned X, then the same schema but with an expectation of X is actually memorized and reinforced.

Trace: is just used to generate the trace of this cycle and to clear the temporary variables.

No comments: