Ernest 7.4 has a basic iconic visual system implemented. He can now perceive his surrounding environment under the form of an icon made of three parts: [left, forward, right]. These parts correspond to the three squares around Ernest. This icon is displayed in the upper-right corner of the video.
This environment offers 5 different possible icons. Ernest must learn them and simultaneously learn what behavior is the most satisfying in the context of each of them.
This video shows that this learning is pretty fast. With this iconic mechanism, Ernest succeeds much better in this environment than with the sequential learning mechanism only. Indeed, this task is more tailored to visual learning than to sequential learning, because all the information needed to make a choice is easily accessible in the visual field.
In this experiment, the "visual" perception is not implemented as sensory-motor schemas. The visual perception rather follows a second parallel process of learning and recollection. New learned icons are stored in Ernest's iconic memory; and new or recollected icons are activated in Ernest's short-term memory. As an element of Ernest's short-term memory, the currently-percieved icon is part of the context of the new learned schemas.
Ernest 7.4 has three primitive schemas with the following settings:
- [move forward, succeed, 10] Ernest enjoys moving forward.
- [move forward, fail, -10] Ernest dislikes bumping walls.
- [turn left or right, succeed, 0] Ernest is indifferent of turning toward an empty square.
- [turn left or right, fail, -5] Ernest dislikes turning toward a wall.
Friday, October 16, 2009
Two Visual Systems and Two Theories of Perception
Norman, J. (2002). Two Visual Systems and Two Theories of Perception: An Attempt to Reconcile the Constructivist and Ecological Approaches. Behavioral and Brain Sciences, 25 (1), 73–144.
I am reviewing this paper because it gives some directions for implementing Ernest's visual system. In this paper, Norman analyzes the so-called dual-process approach to vision. He describes these two processes as follows:
Purpose:
- Process 1: picks up visual information to generate adapted behavior.
- Process 2: recognizes and identifies objects and events in the environment.
Human neuro-anatomy:
- Process 1: dorsal (occipito-dorsal and parietal, toward bimodal areas vision/motor).
- Process 2: ventral (occipito-ventral, toward bimodal areas vision/audition).
Information theory:
- Process 1: related to Gibson's (1979) ecological cognition theory. Invariants are picked-up in the visual scene to constitute affordances.
- Process 2: related to Helmholtz's constructivist theory of vision. Pictoral clues are perceived to construct/recollect a representation of objects.
Consciousness:
- Process 1: triggers behavior while the subject has no explicit consciousness of why this behavior is triggered.
- Process 2: the subject is conscious of seeing the objects that this process identifies.
How to use it for Ernest?
So far, only process 1 has been implemented in Ernest. Norman's paper suggests that process 2 should be implemented in parallel. Process 2 requires an iconic memory associated with skills for learning new icons and recognizing icons. Process 1 and process 2 should be interconnected. The process of recognizing icons (process 1) may require to trigger information-pick-up behavior (process 2). Moreover, recognized icons should participate to the selection of forthcoming behavior.
References
Gibson, J. J. (1979). The ecological approach to visual perception. Boston: Houghton Mifflin.
I am reviewing this paper because it gives some directions for implementing Ernest's visual system. In this paper, Norman analyzes the so-called dual-process approach to vision. He describes these two processes as follows:
Purpose:
- Process 1: picks up visual information to generate adapted behavior.
- Process 2: recognizes and identifies objects and events in the environment.
Human neuro-anatomy:
- Process 1: dorsal (occipito-dorsal and parietal, toward bimodal areas vision/motor).
- Process 2: ventral (occipito-ventral, toward bimodal areas vision/audition).
Information theory:
- Process 1: related to Gibson's (1979) ecological cognition theory. Invariants are picked-up in the visual scene to constitute affordances.
- Process 2: related to Helmholtz's constructivist theory of vision. Pictoral clues are perceived to construct/recollect a representation of objects.
Consciousness:
- Process 1: triggers behavior while the subject has no explicit consciousness of why this behavior is triggered.
- Process 2: the subject is conscious of seeing the objects that this process identifies.
How to use it for Ernest?
So far, only process 1 has been implemented in Ernest. Norman's paper suggests that process 2 should be implemented in parallel. Process 2 requires an iconic memory associated with skills for learning new icons and recognizing icons. Process 1 and process 2 should be interconnected. The process of recognizing icons (process 1) may require to trigger information-pick-up behavior (process 2). Moreover, recognized icons should participate to the selection of forthcoming behavior.
References
Gibson, J. J. (1979). The ecological approach to visual perception. Boston: Houghton Mifflin.
Friday, October 9, 2009
Ernest72 Soar
The Soar 9.1 code of Ernest 7.2 is available here. This code can be edited in Eclipse with the Soar plugin (remove the .txt extension).
Thursday, October 8, 2009
Perception/action coordination
In this experiment, Ernest can touch to his right and to his left in addition to to his front. As before, squares flash yellow when Ernest touches them, and red when Ernest bumps into them (see this post for a detailed legend of the video).
Furthermore, the turn schemas now have feedback status. These status are:
- S, if Ernest turns toward an empty square.
- F, if Ernest turns toward a wall. The wall flashes pink to indicate that Ernest has rubbed it while turning.
In both cases, the turn action is performed.
As before, when Ernest starts, he does not know the linkages between the different schemas.
This experiment shows that Ernest is able to learn to consider certain schemas as perceptions and other as actions, and to learn the linkage between these perceptions and these actions.
At the beginning of the video, Ernest moves frantically. He keeps bumping and scratching walls. He is unable to predict the consequences of his acts and he keeps expressing his "surprise" by saying "Ho!".
After a while, he learns to touch forward before moving forward, to touch to the right before turning to the right, and to touch to the left before turning to the left.
Then, he learns a habit consisting of touching to the left when he is facing a wall. If the square to the left is empty then he turns left. If the square to the left is a wall then he touches to the right. If the square to the right is empty then he turns right. With this strategy, he manages to go all the way around his environment.
The settings for generating this behavior were not easy to find. They are as follows:
- [touch empty (forward, left, or right), 0] Ernest is indifferent of touching empty squares.
- [sense wall (forward, left, or right), -1] Ernest slightly dislikes sensing walls.
- [move forwar, 10] Ernest enjoys very much moving forward.
- [bump wall, -10] Ernest dislikes very much bumping into walls.
- [turn toward empty square (left or right), 0 ] Ernest is indifferent of turning toward an empty square.
- [turn toward wall (left or right), -5] Ernest dislikes turning toward walls.
Two insights from this experiment:
- The learning is faster if schemas that are associated with high emotions are never forgotten, either positive or negative emotions.
- Without the feedback from the turn schemas, Ernest would have a lot of trouble establishing the linkage between sensing to the sides and turning. He would usually end up finding less elegant regularities.
Furthermore, the turn schemas now have feedback status. These status are:
- S, if Ernest turns toward an empty square.
- F, if Ernest turns toward a wall. The wall flashes pink to indicate that Ernest has rubbed it while turning.
In both cases, the turn action is performed.
As before, when Ernest starts, he does not know the linkages between the different schemas.
This experiment shows that Ernest is able to learn to consider certain schemas as perceptions and other as actions, and to learn the linkage between these perceptions and these actions.
At the beginning of the video, Ernest moves frantically. He keeps bumping and scratching walls. He is unable to predict the consequences of his acts and he keeps expressing his "surprise" by saying "Ho!".
After a while, he learns to touch forward before moving forward, to touch to the right before turning to the right, and to touch to the left before turning to the left.
Then, he learns a habit consisting of touching to the left when he is facing a wall. If the square to the left is empty then he turns left. If the square to the left is a wall then he touches to the right. If the square to the right is empty then he turns right. With this strategy, he manages to go all the way around his environment.
The settings for generating this behavior were not easy to find. They are as follows:
- [touch empty (forward, left, or right), 0] Ernest is indifferent of touching empty squares.
- [sense wall (forward, left, or right), -1] Ernest slightly dislikes sensing walls.
- [move forwar, 10] Ernest enjoys very much moving forward.
- [bump wall, -10] Ernest dislikes very much bumping into walls.
- [turn toward empty square (left or right), 0 ] Ernest is indifferent of turning toward an empty square.
- [turn toward wall (left or right), -5] Ernest dislikes turning toward walls.
Two insights from this experiment:
- The learning is faster if schemas that are associated with high emotions are never forgotten, either positive or negative emotions.
- Without the feedback from the turn schemas, Ernest would have a lot of trouble establishing the linkage between sensing to the sides and turning. He would usually end up finding less elegant regularities.
Monday, October 5, 2009
More complex environment
What happens when Ernest is put in a more complex environment? He still starts learning low-level regularities, then he tries finding higher-level regularities that would continue increase his satisfaction.
In this video, Ernest 7.1 drives a little tank. Ernest 7.1 has anticipations associated with his actions. When the result of his action does not meet his anticipation, he says "Oh!". We can see that he first learns the strategy consisting of sensing before moving forward (although the yellow flashing is sometimes too fast to bee seen in this video).
Once this strategy is learned, Ernest stops bumping into walls and becomes more confident in exploring his environment. We can see him going up the left border by being surprised of sensing empty squares and running into them, because he enjoys it.
In this example, Ernest's second-level strategy consists of turning to the right after facing a wall. At the end of the video, this habit gets him trapped in a little loop on two squares. After a few cycles in this loop, he stops being surprised of the result of his acts and stops saying "Ho!".
Two insights from this experiment:
- Ernest 7.1 now has a forgetting mechanism. At the end of each cycle, the weight of all the schemas that have not been enacted during this cycle is decremented of 0.2. When a schema reaches a weight of 0 it is deleted from memory. Forgetting useless schemas helps prevent the continual accumulation of schemas and rather keeps their number stable. This forgetting mechanism actually accelerates the learning because it helps get rid of useless schemas.
- Ernest is tailored for sequential regularity learning but not for spatial regularity learning. Therefore, he has troubles exploiting the possibilities offered by a freedom of movement in a two-dimensional environment. Future developments should provide him with mechanisms to capture spatial regularities, through constructing an internal representation of his surrounding environment.
In this video, Ernest 7.1 drives a little tank. Ernest 7.1 has anticipations associated with his actions. When the result of his action does not meet his anticipation, he says "Oh!". We can see that he first learns the strategy consisting of sensing before moving forward (although the yellow flashing is sometimes too fast to bee seen in this video).
Once this strategy is learned, Ernest stops bumping into walls and becomes more confident in exploring his environment. We can see him going up the left border by being surprised of sensing empty squares and running into them, because he enjoys it.
In this example, Ernest's second-level strategy consists of turning to the right after facing a wall. At the end of the video, this habit gets him trapped in a little loop on two squares. After a few cycles in this loop, he stops being surprised of the result of his acts and stops saying "Ho!".
Two insights from this experiment:
- Ernest 7.1 now has a forgetting mechanism. At the end of each cycle, the weight of all the schemas that have not been enacted during this cycle is decremented of 0.2. When a schema reaches a weight of 0 it is deleted from memory. Forgetting useless schemas helps prevent the continual accumulation of schemas and rather keeps their number stable. This forgetting mechanism actually accelerates the learning because it helps get rid of useless schemas.
- Ernest is tailored for sequential regularity learning but not for spatial regularity learning. Therefore, he has troubles exploiting the possibilities offered by a freedom of movement in a two-dimensional environment. Future developments should provide him with mechanisms to capture spatial regularities, through constructing an internal representation of his surrounding environment.
Subscribe to:
Posts (Atom)
RSS feed
Posts
All Comments