Ernest 4.1

Like Ernest 4.0, Ernest 4.1 learns and exploits second-order schemas to succeed in the "aA..bB..aA..aA" environment.

The way schemas are constructed and enacted has been however slightly modified.

At the beginning of each round, Ernest 4.1 evaluates the context, and if the context is new, he constructs potentially interesting schemas that match this context. For exemple, the intial context is made up of "nils" because Ernest has no initial experience, but this context is structured under the form of a previous act A2 and a previous schema S5 (first line). An act is a couple (Ernest's action, Environment's response). From this context, four initial primary schemas are constructed: S6, S7, S8, S9. Finally, one schema is enacted: in this example S9. Like, Ernest 4.0, if the schema gets X, Ernest 4.1 sais O-o, and if it gets Y, he sais Yee!

The context is the content of Ernest's short-term memory, it corresponds to the three previously-enacted schemas. For example, in this trace, the third context contains S5, S9 and S12. These schemas can be of any level of abstraction. For convenience, the third schema is expanded: S12=(A3,A4). A3 is S12's context and A4 is S12's act. A4 is expanded: A4=(B,Y), meaning that the last thing Ernest did was B and he got Y from the environment.

Like Ernest 4.0, Ernest 4.1 constructs a secondary schema each time a primary schema succeeds. This secondary schema is made up of the three previously-enacted primary schemas. That also corresponds to the context stored in short-term memory.

Later, when a new context matches a secondary schema, this secondary schema is enacted, if he has the highest weight amongst all the schemas matching this context, at any level.

Contrary to Ernest 4.0, Ernest 4.1 does not trace the details of the secondary schema enaction. A secondary schema enaction is displayed as a single line in the trace, despite it actually takes two rounds. Moreover, only the secondary schema is reinforced and stored in short-term memory, but not the primary schemas that are part of it. Thus, when a secondary schema is enacted, short-term memory is not filled with lower-level details. This mechanism allows Ernest to construct tertiary schemas.

At the end of this trace, we can see the construction of tertiary schemas, made up of the three previously-enacted primary or secondary schemas. In this environment, however, these tertiary shemas are not used because there is no reason for them to receive more reinforcement than the secondary schemas, and anyway, Ernest 4.1 is not yet able to recursively manage nested schemas.

It is interesting to notice that, as this trace only displays the highest-level schema enaction, it can again be understood as a description of Ernest's viewpoint on his activity. It is as if Ernest was always focusing on the highest level of control, and automatically performing lower-level behavior without having to pay attention to it.