First steps in space

Ernest 4.2 is the same as Ernest 4.1 but he is now connected to a 2D environment.

This environment is the Vacuum cleaner environment initially developed by Cohen (2005). It is a simple grid environment that allows an agent, represented as a vacuum cleaner, to move in search for dust to clean up. I have implemented three new functionalities: (i) Wall representation (dark gray squares) inside the grid. They flash red when the vacuum cleaner bumps into them. (ii) Speaking aloud the agent's rational in real time. (iii) "Bump" sense, that, after each move, returns true if the agent has bumped into a wall and false if he has not.

Ernest is controlling the vacuum cleaner. His output A is mapped to a move command to the left, and output B to the right. His input Y is mapped to a non-bump feedback, and input X to a bump feedback. So far, Ernest only uses one dimension of this 2D environment: the horizontal. There is no dust in this example, and Ernest cannot sense any other information than bump or non-bump.

We can hear the possible construction of new schemas at the beginning of each cycle. Then we see the vacuum cleaner possibly moving or bumping into a wall. Then we hear which schema has been enacted. If he did not bump, he says "yahoo", if he bumped, he says "Hoho".

We see that he begins randomly exploring his environment and constructing primary schemas, then secondary schemas. Eventually, he finds a stable activity based on a secondary schema (S28) that gives him a "yahoo" each time. From observing his activity, we can infer, if we want, that he does not "like" bumping into walls, and that he learns a procedure to avoid that.

Reference

Cohen, M. A. (2005). Teaching agent programming using custom environments and Jess. AISB Quarterly, 120 (Spring), 4.