Olivier Georgeon's research blog—also known as the story of little Ernest, the developmental agent.

Keywords: situated cognition, constructivist learning, intrinsic motivation, bottom-up self-programming, individuation, theory of enaction, developmental learning, artificial sense-making, biologically inspired cognitive architectures, agnostic agents (without ontological assumptions about the environment).

Wednesday, January 5, 2011

Ernest 8.2 can find his food

Unable to display content. Adobe Flash is required.
Ernest 8.2 is a horseshoe crab. Horseshoe crabs are archaic arthropods whose visual system has been extensively studied. From these studies, we pulled several principles that guided the development of Ernest's distal sensory system:

- Small matrix resolution: the horseshoe crab's most elaborated eyes (two compound eyes among the 10 eyes that horseshoe crabs possess) have a resolution of roughly 40*25 pixels.
- Fixed eyes: eyes are fixed to the animal's body. The animal has to rotate its full body to move its visual field.
- Sensibility to movement: the signal sent to the brain does not reflect static shape recognition but rather reflects changes in the visual field.
- Visio-spatial behavioral proclivity: male horseshoe crabs move toward females when they see them with their compound eyes whereas females move away from other females.

As noted earlier, Ernest's "eyes" have only one pixel — pixel sensible to the distance to the blue square in a 90° visual field (assuming there is only one blue square).

Each eye produces a signal that represents the change in the corresponding visual field during the last interaction cycle:
- Appear: a blue square appeared in the visual field.
- Closer: more blue in the visual field, meaning the blue square is approaching.
- Arrived: the blue square occupies the entire visual field, meaning Ernest is stepping on the blue square and can eat it.
- Disappeared: the blue square disappeared from the visual field.


As opposed to previous versions, Ernest 8.2 has no antenna and has only three possible primitive behaviors:
- [move forward, succeed, 0] Ernest is indifferent of moving forward.
- [move forward, fail, -8] Ernest hates bumping walls.
- [turn left or right, succeed, 0] Ernest is indifferent of turning toward an adjacent empty square.
- [turn left or right, fail, -5] Ernest dislikes turning toward an adjacent wall.


To generate a visio-spatial behavioral proclivity, Ernest's sequential learning mechanism receives an additional inborn intrinsic satisfaction when an eye returns a signal:
- [Appear, 15] Ernest loves blue squares appearing in an eye's visual field.
- [Closer, 10] Ernest enjoys blue squares getting closer.
- [Arrived, 30] Ernest is crazy about stepping on a blue square (and eating it in the process).
- [Disappear, -15] Ernest hates blue squares disappearing from an eye's visual field.


At the beginning, the video shows Ernest learning to coordinate his actions with his sensory input. As before, he needs to learn to generate expectations associated with actions (e.g., turning schemas may shift the blue square from one eye to the other, etc.). He also needs to learn sequences of behavior (or "strategies") to reach the blues square. In this example run, he learned a strategy consisting of following a diagonal, and subsequently a straight line. Other strategies are possible that we will report next.

No comments: