tag:blogger.com,1999:blog-5864532633570942842024-03-14T11:23:47.736+01:00Artifical Developmental Learning<p><a href="http://www.oliviergeorgeon.com/">Olivier Georgeon</a>'s research blog—also known as the story of little Ernest, the developmental agent.</p><p><i>Keywords: situated cognition, constructivist learning, intrinsic motivation, bottom-up self-programming, individuation, theory of enaction, developmental learning, artificial sense-making, biologically inspired cognitive architectures, agnostic agents (without ontological assumptions about the environment).</i></p> Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.comBlogger101125tag:blogger.com,1999:blog-586453263357094284.post-78116348058896971172015-07-01T13:14:00.001+02:002015-07-06T16:19:14.866+02:00AImergence<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; float: left; margin-right: 10px; text-align: center;"><iframe allowfullscreen="" allowtransparency="true" height="405" mozallowfullscreen="" scrolling="no" src="http://e.gsrca.de/131699?s=560&mode=embed" style="border: none;" webkitallowfullscreen="" width="560"></iframe></div><b>AImergence</b> (Artificial Intelligence emergence) is a game in which you play the role of Ernest learning to interact with its environment. Please share and write comments from <a href="http://oliviergeorgeon.com/aimergence">this page</a>.<br />
</div>Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-25336100905012940592014-09-20T19:19:00.000+02:002015-01-02T13:00:56.104+01:00Inverting the interaction cycle<div dir="ltr" style="text-align: left;" trbidi="on">
<a href="http://www.oliviergeorgeon.com/paper/Georgeon-BICA2014.pdf" target="_blank">Inverting the Interaction Cycle to Model Embodied Agents</a> (2014). <a href="http://www.oliviergeorgeon.com/" target="_blank">Olivier L. Georgeon</a> and <a href="http://liris.cnrs.fr/membres?idn=acordier" target="_blank">Amélie Cordier</a>. Procedia Computer Science, 41, pp 243-248. <a href="http://bicasociety.org/meetings/2014/" target="_blank">5th Annual International Conference on Biologically Inspired Cognitive Architectures</a>. <a class="S_C_ddDoi" href="http://dx.doi.org/10.1016/j.procs.2014.11.109" id="ddDoi" target="doilink">doi:10.1016/j.procs.2014.11.109</a>.<br />
<br />
This paper relates very much to the first lesson of the <a href="http://liris.cnrs.fr/ideal/mooc/" target="_blank">IDEAL MOOC</a>. </div>
Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-14210521354507454002014-07-22T14:29:00.000+02:002014-07-23T10:50:31.086+02:00IDEAL MOOC<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; float: left; margin-right: 10px; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='560' height='315' src='https://www.youtube.com/embed/kQPz9InhHjk?feature=player_embedded' frameborder='0'></iframe></div>
We are pleased to announce that our <a href="http://liris.cnrs.fr/ideal/mooc">Developmental Artificial Intelligence MOOC</a> is now open for registration.</div>Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-33354288165882656692014-04-14T09:44:00.002+02:002014-04-23T13:23:27.725+02:00Create your Ernest<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; float: left; margin-right: 10px; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='560' height='315' src='https://www.youtube.com/embed/fhbex8dzNtE?feature=player_embedded' frameborder='0'></iframe></div>
Learn to create your own self-programming agent and robot in my <a href="http://www.oliviergeorgeon.com/radical-interactionism/developmental-learning.htm">course on developmental learning</a>. French original version available <a href="http://www.youtube.com/watch?v=aQWnx1PX7-w&feature=share&list=PLlSPp5EpW5vH3PAPA4FTema1wi-9cfC6m" target="_blank">here</a>.</div>
Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-11158553444016929502014-02-24T14:47:00.000+01:002014-02-25T13:41:04.010+01:00Lyon Startup Weekend<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; float: left; margin-right: 10px;">
<a href="http://1.bp.blogspot.com/-3qdn7aKu1Ys/UwtGXrVVF_I/AAAAAAAAGi0/bzMlRj95sPE/s1600/sw-team.jpeg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://1.bp.blogspot.com/-3qdn7aKu1Ys/UwtGXrVVF_I/AAAAAAAAGi0/bzMlRj95sPE/s1600/sw-team.jpeg" height="240" width="320" /></a></div>
<div style="text-align: left;">
We won the jury's favorite prize ("<i>coup de coeur du jury</i>", 4<sup>th</sup> prize) at the <a href="http://lyon.startupweekend.org/">Lyon Startup Weekend</a> ! The IDEAL team from left to right: <a href="http://www.viadeo.com/fr/profile/renaud.detaille">Renaud Detaille</a>, <a href="https://plus.google.com/+Rapha%C3%ABlCAZORLA/">Raphaël Cazorla</a>, <a href="http://www.oliviergeorgeon.com/">Olivier Georgeon</a>, <a href="http://severin-bruhat.com/">Séverin Bruhat</a>, and <a href="http://www.jmillet.com/">Julien Millet</a>. Great thanks to the <a href="http://lyon.startupweekend.org/#organizers">organizing team</a> for this crazy unforgettable weekend. We love you too !</div>
</div>
<br />
Watch the <a href="http://youtu.be/BDTR1Zd2v9U">finale video</a> (our presentation from time <a href="http://youtu.be/BDTR1Zd2v9U?t=2h47m37s">2:47:37</a> to 2:56:20, low audio quality French).
</div>
Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-75058549231303636032014-01-24T11:26:00.002+01:002014-02-20T20:30:22.168+01:00Inferring actions and observations from interactions<div dir="ltr" style="text-align: left;" trbidi="on">
<a href="https://liris.cnrs.fr/publis/publi_aut/?id=4873" rel="nofollow">Garnier J.</a>,
<a href="http://www.oliviergeorgeon.com/">Georgeon O.</a>, and
<a href="http://liris.cnrs.fr/amelie.cordier/wiki/doku.php">Cordier A.</a>, 2013. <a href="http://www.oliviergeorgeon.com/radical-interactionism/GarnerJ-ACS2013.pdf">Inferring Actions and Observations from Interactions</a>. In the <a href="http://mcox.org/g-reason/">Second Workshop on Goal Reasoning</a> at Advances in Cognitive Systems (<a href="http://www.cogsys.org/conference/2013">ACS2013</a>), Baltimore, ML. 26-35.<br />
<br />
Following the <a href="http://www.oliviergeorgeon.com/radical-interactionism/" target="_blank">Radical Interactionism paradigm</a> introduced <a href="http://e-ernest.blogspot.fr/2013/12/radical-interactionism.html" target="">previously</a>, this paper investigates the construction of <i>intentional actions</i> and <i>meaningful observations</i> from regularities observed in sequences of sensorimotor interactions.<br />
<br />
In doing so, we take the opposite stance of most machine learning approaches that seek to learn a mapping between a predefined space of actions and a predefined space of observations. Instead, our agent begins with a predefined space of interactions and learns actions and observations as secondary constructs. This paper also reports the <a href="http://e-ernest.blogspot.fr/2013/09/ernest-12.html">Ernest 12 experiment</a>.</div>
Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-47260598098232988072013-12-06T12:58:00.002+01:002014-02-20T20:27:02.678+01:00Radical Interactionism<div dir="ltr" style="text-align: left;" trbidi="on">
<a href="http://www.oliviergeorgeon.com/">Olivier L. Georgeon</a> and <a href="http://home.earthlink.net/~dwaha/">David W. Aha</a> 2013. <a href="http://www.oliviergeorgeon.com/radical-interactionism/jagi.htm">The Radical Interactionism Conceptual Commitment</a>. <a href="http://www.degruyter.com/view/j/jagi">Journal of Artificial General Intelligence</a> 4(2): 31-36.<br />
<br />
This is where I turn radical and eliminate <i>perceptions</i> and <i>actions</i> as primitive notions of cognitive models. Now I wonder why it took so many years to come up with such an obvious and elegant formalism.</div>
Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-38385475975201310212013-11-15T14:11:00.001+01:002014-02-19T22:47:48.518+01:00Single Agents Can Be Constructivist Too<div dir="ltr" style="text-align: left;" trbidi="on">
<a href="http://www.oliviergeorgeon.com/">Olivier L. Georgeon</a> and <a href="https://liris.cnrs.fr/membres?idn=shassas">Salima Hassas</a> 2013. <a href="http://www.univie.ac.at/constructivism/journal/9/1/040.georgeon">Single Agents Can Be Constructivist too</a>. <a href="http://www.constructivistfoundations.info/">Constructivist Foundations</a> 9(1): 40-42.<br />
<br />
In this open peer commentary to an <a href="http://www.univie.ac.at/constructivism/journal/9/1/026.roesch">article by Roesch et al.</a>, we wished to argue that multi-agent systems are not the only alternative to cognitivism. We present Ernest in Roesch et al.'s environment to show that Ernest is constructivist too. </div>
Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-18910252818021507702013-09-12T18:23:00.004+02:002013-12-17T15:08:11.442+01:00Ernest 12<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; float: left; margin-right: 10px; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='560' height='315' src='https://www.youtube.com/embed/2F4uIRghlkU?feature=player_embedded' frameborder='0'></iframe></div>
Ernest 12 categorizes the entities in its environment based on the possibilities of interaction that they afford, and adjusts its behavior to categories.<br />
<br />
Top-left: Ernest in its environment. The "eye" (half-circle) takes the color of the entity that get Ernest's attention at any given time.<br />
<br />
Top-right: Ernest's spatial memory. Interactions are localized in space, and Ernest updates their position as he moves. Entities are constructed where interactions overlap. Rectangles and trapezoids represent interactions, circles represent entities.<br />
<br />
Bottom: activity trace. Bottom: interactions (rectangles and trapezoids) enacted on the left, in front, or on the right of Ernest. Middle: the motivational value of the enacted interaction represented as a bargraph (green when positive, red when negative). Top: the actions (half-circles (turn), triangles (try to step forward)) and the entities (blue and green circles) learned over time.<br />
<br />
In this run, Ernest, learns the "<a href="http://e-ernest.blogspot.com/2011/01/ernest-82-can-find-his-food.html">bishop behavior</a>" during the first 50 steps. On steps 78, we introduce two targets in a raw. The spatial memory shows that Ernest interacts with these two targets at the same time. Ernest's spatial memory (associated with its rudimentary attentional system) allows Ernest to focus on one target at a time.<br />
<br />
On step 110, we introduce a "wall brick", and Ernest learns that this kind of entity affords the interaction "bumping". Subsequently, when we introduce a target, Ernest will preferably go towards the target than towards the wall brick because it has learned that the targets are edible.<br />
<br />
Ernest 12 implements <a href="http://e-ernest.blogspot.fr/2013/05/enactive-cognitive-architecture.html">ECA, the Enactive Cognitive Architecture</a>.<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">(Demo implemented with Ernest <a href="http://code.google.com/p/e-ernest/source/detail?r=439">r439</a> and Vacuum <a href="http://code.google.com/p/vacuum-sg-continuum/source/detail?r=392">r392</a>)</span><br />
<br /></div>
Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com2tag:blogger.com,1999:blog-586453263357094284.post-20925405563052745842013-06-18T09:56:00.000+02:002014-03-01T18:07:31.001+01:00Enactive Robot Learning<div dir="ltr" style="text-align: left;" trbidi="on">
<a href="http://www.oliviergeorgeon.com/">Olivier L. Georgeon</a>, <a href="http://liris.cnrs.fr/christian.wolf/">Christian Wolf</a>, and <a href="http://liris.cnrs.fr/simon.gay/">Simon Gay</a> 2013. <a href="http://liris.cnrs.fr/ideal/doc/GeorgeonO-Epirob2013-6.pdf">An Enactive Approach to Autonomous Agent and Robot Learning</a>. IEEE Third Joint International Conference on Development and Learning and on Epigenetic Robotics (<a href="http://www.icdl-epirob.org/">EPIROB2013</a>). Osaka, Japan. August 18-22 2013.<br />
<br />
This paper constitutes a short introductory version of our <a href="http://e-ernest.blogspot.fr/2013/05/enactive-cognitive-architecture.html">ECA paper</a>. It also presents the experiment of <a href="http://e-ernest.blogspot.fr/2012/04/ernest-7-in-e-puck.html">Ernest7 in an e-puck robot</a>. </div>
Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-87024262557043462862013-05-14T11:12:00.000+02:002013-12-09T15:42:07.365+01:00Enactive Cognitive Architecture<div dir="ltr" style="text-align: left;" trbidi="on">
<a href="http://liris.cnrs.fr/ideal/doc/BICA2013-Extended-13.pdf"> </a><a href="http://www.oliviergeorgeon.com/">Olivier L. Georgeon</a>, <a href="http://science.slc.edu/%7Ejmarshall/">James B. Marshall</a>, and <a href="http://www.consciousness.it/">Riccardo Manzotti</a> 2013. <a href="http://liris.cnrs.fr/ideal/doc/BICA2013-Extended-13.pdf">ECA: An enactivist cognitive architecture based on sensorimotor modeling</a>. <a href="http://www.journals.elsevier.com/biologically-inspired-cognitive-architectures">Biologically Inspired Cognitive Architectures</a>, Volume 6, pp. 46-57, <a href="http://dx.doi.org/10.1016/j.bica.2013.05.006">doi: 10.1016/j.bica.2013.05.006</a>. Also presented at <a href="http://bicasociety.org/meetings/2013/">BICA2013</a>.<br />
<br />
This paper introduces a new way of modeling an agent interacting with an environment called an <i>Enactive Markov Decision Process</i>, inspired by the <a href="http://en.wikipedia.org/wiki/Enaction">Theory of Enaction</a>. It also describes Ernest's motivational principle in relation with the <i>autotelic principle</i> (<a href="http://glotta.ntua.gr/StateoftheArt/Motor-Action/MirrorNeurons/steels-04b.pdf">Steels, 2004</a>) and the <i>optimal experience principle</i> (<a href="http://en.wikipedia.org/wiki/Mihaly_Csikszentmihalyi">Csikszentmihalyi</a>, 1990). It introduces ECA, the Enactive Cognitive Architecture that drives Ernest 11, and it reports the <a href="http://e-ernest.blogspot.fr/2012/04/ernest-112.html">Ernest 11.2 experiment</a>. </div>
Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-53945247032168814692013-02-11T16:21:00.000+01:002013-12-06T13:25:55.753+01:00Sensemaking emergence demonstration<div dir="ltr" style="text-align: left;" trbidi="on">
<a href="http://www.oliviergeorgeon.com/">Olivier L. Georgeon</a> and <a href="http://science.slc.edu/%7Ejmarshall/">James B. Marshall</a> 2013. <a href="http://liris.cnrs.fr/ideal/doc/georgeon2013-IJMC7.pdf">Demonstrating sensemaking emergence in artificial agents: A method and an example</a>. <a href="http://www.worldscientific.com/worldscinet/ijmc">International Journal of Machine Consciousness</a>, 5(2), pp 131-144, doi: <a href="http://dx.doi.org/10.1142/S1793843013500029">10.1142/S1793843013500029</a>.<br />
<br />
This paper addresses the <i>sensemaking demonstration problem</i> : the problem of demonstrating that an agent <i>gives meaning to</i> or <i>understands </i>its experience. We present a methodology to produce empirical evidence to support or contradict the claim that an agent is capable of a rudimentary form of sensemaking, based on an analysis of the agent's behavior.<br />
<br />
As an example, we report an analysis of Ernest's behavior in the <a href="http://e-ernest.blogspot.fr/2012/03/small-loop-challenge.html">Small Loop Problem</a> and we conclude that Ernest is capable of a rudimentary form of sensemaking. This paper is an extended version of our <a href="http://e-ernest.blogspot.fr/2012/05/challenge-emergent-cognition.html">previous paper</a> presented at <a href="http://bicasociety.org/meetings/2012/">BICA2012</a>. </div>
Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-2293284650598949482012-11-20T16:05:00.000+01:002014-11-16T12:45:56.577+01:00Training Ernest 7<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="float: left; margin-right: 10px;" trbidi="on">
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='560' height='315' src='https://www.youtube.com/embed/q8WkYbt2BxM?feature=player_embedded' frameborder='0'></iframe></div>
<br /></div>
This video demonstrates that Ernest develops different behaviors depending on the experience he has during his youth. Here, we have two instances of Ernest: Ernest 1 (brown) is initially kept in the small loop and released on step 290. Ernest 2 (bleu) is confronted to the complex environment right from his birth.<br />
<br />
Ernest 1 develops more sophisticated behaviors than Ernest 2 because he is trained to touch both of its sides when he faces a wall. Consequently, after being released, he has a more exploratory behavior than Ernest 2.<br />
<br />
Ernest 2 learns to preferably turn to the right when he faces a wall. Consequently, he tends to keep spinning in limited areas of the environment. Ernest 2's learning is limited by the fact that the environment is initially too complex for him to notice sophisticated sequences that involve touching to both sides.<br />
<br />
The importance of training is an interesting property of Ernest because it accounts for theories of developmental learning.<br />
<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">(Demo implemented with Ernest <a href="http://code.google.com/p/e-ernest/source/detail?r=296">r296</a> and Vacuum <a href="http://code.google.com/p/vacuum-sg-continuum/source/detail?r=203">r203</a>)</span><br />
<br /></div>
Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-1989247330324234522012-10-29T14:19:00.000+01:002012-11-05T13:17:58.441+01:00Ernest's source code<div dir="ltr" style="text-align: left;" trbidi="on">
Ernest's source code is available <a href="http://code.google.com/p/e-ernest/">here</a> with the instructions to use it. The cleaned-up and tested recommended revision is <a href="http://code.google.com/p/e-ernest/source/detail?r=296">r296</a>. This revision demonstrates the exact behavior of Ernest 7 reported in the <a href="http://e-ernest.blogspot.fr/2012/03/small-loop-challenge.html">Small Loop Experiment</a>. </div>
Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-64846334146802112212012-10-15T10:18:00.000+02:002013-12-06T14:01:14.882+01:00Interactional Motivation<div dir="ltr" style="text-align: left;" trbidi="on">
<a href="http://www.oliviergeorgeon.com/">Olivier L. Georgeon</a>, <a href="http://science.slc.edu/%7Ejmarshall/">James B. Marshall</a>, and <a href="http://liris.cnrs.fr/simon.gay/">Simon L. Gay</a> 2012. <a href="http://liris.cnrs.fr/ideal/doc/GeorgeonO2012-Epirob.pdf">Interactional Motivation in Artificial Systems: Between Extrinsic and Intrinsic Motivation</a>. In proceedings of the 2nd Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics (<a href="http://icdl2012.ucsd.edu/">EPIROB 2012</a>), San Diego, pp. 1-2.<br />
<br />
This paper presents the notion of <i>interactional motivation</i> that drives Ernest, and compares it to reinforcement learning as it is traditionally implemented in Partially Observable Markov Decision Processes (POMDPs).
</div>
Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-42023364055266645752012-08-28T16:03:00.000+02:002012-09-11T11:15:04.755+02:00Ernest 11.5 constructs goals<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="float: left; margin-right: 10px; text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: center;">
<object class="BLOGGER-youtube-video" classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0" data-thumbnail-src="http://i.ytimg.com/vi/_5HUb6AvSLg/0.jpg" height="315" width="560"><param name="movie" value="http://www.youtube.com/v/_5HUb6AvSLg?version=3&f=user_uploads&c=google-webdrive-0&app=youtube_gdata" /><param name="bgcolor" value="#FFFFFF" /><param name="allowFullScreen" value="true" /><embed width="560" height="315" src="http://www.youtube.com/v/_5HUb6AvSLg?version=3&f=user_uploads&c=google-webdrive-0&app=youtube_gdata" type="application/x-shockwave-flash" allowfullscreen="true"></embed></object></div>
<br /></div>
Like <a href="http://e-ernest.blogspot.fr/2012/07/ernest-114.html">Ernest 11.4</a>, Ernest 11.5 can recognize objects by the possibilities of interaction that they afford. Additionally, Ernest 11.5 has a specific inborn taste for stepping on flowers.<br />
<br />
Ernest 11.5 simulates different possible sequences of interactions in spatial memory before selecting the best sequence to enact. These simulations are represented in the bottom-right area of the video. Simulations that produce predictable results (due to information available in spatial memory) are represented with orange outlines. Simulations that produce unpredictable results (due to the lack of information in spatial memory) are represented in blue. The video shows that Ernest learns to simulate increasingly elaborated sequences of interactions as time goes on (see blue squares and triangles spreading in all directions around Ernest from step 253 on).<br />
<br />
The high value associated with stepping on flowers favors simulations that lead to even more stepping on flowers. As a result, Ernest learns to make a u-turn to return to a flower when he passes one (see Ernest keeping stepping on the flower from step 260 on).<br />
<br />
We find this experiment interesting because it illustrates how an inborn drive can give raise to an explicit goal. Ernest's inborn tendency to step on flowers makes Ernest identify flowers as an interesting goal to reach. Once this goal is recognized, Ernest performs a rudimentary problem-solving computation to reach it. Perhaps the skill to choose a desirable point in space and find a sequence of operations to reach this point underlays higher-level problem-solving skills.</div>
Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-15046718823727984112012-07-03T15:52:00.001+02:002014-11-16T12:42:55.920+01:00Ernest 11.4 recognizes objects<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="float: left; margin-right: 10px;">
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='560' height='315' src='https://www.youtube.com/embed/DzCkcpFvG8c?feature=player_embedded' frameborder='0'></iframe></div>
</div>
This video shows Ernest 11.4 learning to interact with different objects in this new version of the Small Loop.<br />
<br />
At the beginning, Ernest learns to interact with empty places and with dark-green walls. From step 76 on, he learns to interact with cyan walls. On step 220, we introduce alga, and he starts to learn to interact with them.<br />
<br />
Note the funny hesitation on step 234 when Ernest touches an alga for the first time, turns back, and then returns to the alga. Once this new kind of objects is learned, Ernest moves through them without hesitation.<br />
<br />
Ernest's previous management of bundles (<a href="http://e-ernest.blogspot.fr/2012/04/ernest-112.html">Ernest 11.2</a>) no longer works in this environment because objects can no longer be identified by disjoint bundles of interactions. Some interactions (e.g., bump) are afforded by different objects (dark-green walls and cyan walls). Ernest 11.4, however, does not actually need to fully recognize objects. He adapts to this environment by only learning "<i>compresences</i>" of pairs of interactions. We borrowed the term <a href="http://en.wiktionary.org/wiki/compresence">compresence</a> from the <a href="http://en.wikipedia.org/wiki/Bundle_theory">bundle theory of objects</a> to designate the tie between two interactions that are afforded by the same location in space. In this video, compresences are represented by gray circles that contain interactions (in sequential and spatial memory, top and bottom right areas of the video).<br />
<br />
The question of identifying objects by bundles of interactions that are consistently compresent remains an open and difficult question. The notion of compresence seems to be still controversial in <a href="http://en.wikipedia.org/wiki/Object_%28philosophy%29">philosophy of objects</a>. Identifying objects raises the question of making analogies between objects, and learning categories of objects based on similarities in the interactions that they afford.<br />
<br />
<u>In this experiment: </u><br />
<br />
Touching a cyan wall ahead generates a specific feeling
(cyan squares). Touching a cyan wall on the side generates the same
feeling as touching a dark-green wall on the side (dark-green squares).
Bumping into a cyan wall feels the same as bumping into a dark-green
wall (red triangles). Once learned, touching walls ahead "evokes"
bumping ahead (light-red triangles in spatial memory, bottom right area of the video). As <a href="http://e-ernest.blogspot.fr/2012/04/ernest-112.html">previously</a>, the evocation of bumping refrains Ernest from trying to move forward towards walls.<br />
<br />
Touching
an alga ahead generates a specific feeling (light-green squares).
Touching an alga on the side generates the same feeling as touching an
empty square on the side (white squares). Moving to an alga feels the
same as moving to an empty square (white triangles).<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">(Demo implemented with Ernest <a href="http://code.google.com/p/e-ernest/source/detail?r=261">r261</a> and Vacuum <a href="http://code.google.com/p/vacuum-sg-continuum/source/detail?r=186">r186</a>)</span></div>
Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-72860540973406270162012-05-30T13:05:00.001+02:002012-06-18T13:35:11.587+02:00Ernest 11.3 in e-puck<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="float: left; margin-right: 10px;">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: center;">
<object class="BLOGGER-youtube-video" classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0" data-thumbnail-src="http://i.ytimg.com/vi/LR_IwgqU4fM/0.jpg" height="315" width="560"><param name="movie" value="http://www.youtube.com/v/LR_IwgqU4fM?version=3&f=user_uploads&c=google-webdrive-0&app=youtube_gdata" />
<param name="bgcolor" value="#FFFFFF" />
<embed width="560" height="315" src="http://www.youtube.com/v/LR_IwgqU4fM?version=3&f=user_uploads&c=google-webdrive-0&app=youtube_gdata" type="application/x-shockwave-flash"></embed></object></div>
<br /></div>
</div>
</div>
Ernest 11.3 is an adaptation of <a href="http://e-ernest.blogspot.fr/2012/04/ernest-112.html">Ernest 11.2</a> for the e-puck robot.<br />
<br />
This video shows the e-puck robot in the "Box Environment" (left). The possibilities of interaction and the LED signals are the same as with <a href="http://e-ernest.blogspot.fr/2012/04/ernest-7-in-e-puck.html">Ernest 7 in e-puck</a>.<br />
<br />
The top-right part of the video shows the sequential trace with the same symbols as <a href="http://e-ernest.blogspot.fr/2012/04/ernest-112.html">previously</a>.<br />
<br />
The bottom-center shows the new spatial memory (in an egocentric referential with the robot's front oriented towards the right). When Ernest enacts an interaction, the area that is concerned by this interaction is marked by a halo in spatial memory. Interactions that concern empty places are in white, interactions that concern walls are in green. The superimposition of different interactions in the same spatial location reveals occurrences of <i>empty place phenomena</i> (white halos) and <i>wall phenomena</i> (green halos).<br />
<br />
Note that wrong associations can occur due to false detections. For example: false detection of a wall on the left on steps 221 and 222 (time 2:23). (We turned on additional light to provoke more false detections from step 100, time 0:59.)<br />
<br />
The bottom-right part of the video represents coefficients of spatial overlapping between interactions (red segments). The more consistent the overlapping, the shorter the segment. Over time, interactions that concern the same kind of phenomena become grouped together because they consistently overlap. Two bundles of interactions emerge: white interactions form a bundle that represents empty places, green interactions form a bundle that represents walls.<br />
<br />
On step 229, the false detection made on step 221 and 222 yields to a wrong association between the two bundles (mixed white and green halo in the center of spatial memory on time 2:25, and long red segment between the two bundles). This wrong association, however, does not impact Ernest's behavior too much because it remains weak.<br />
<br />
This experiment demonstrates that Ernest 11.3 handles the imprecision in the robot’s displacements and in the sensors by keeping track of probabilities of presence of phenomena in Ernest's surrounding space. Simultaneously, Ernest gradually learns the notions of <i>empty space phenomenon</i> and <i>wall phenomenon</i> by associating the interactions that they afford. In turn, the recognition of phenomena helps Ernest organize its behavior by prompting interactions adapted to the phenomena that surround him.</div>Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-44702435386847086682012-05-24T15:28:00.001+02:002013-02-11T15:58:04.616+01:00A Challenge for Emergent Cognition<div dir="ltr" style="text-align: left;" trbidi="on">
<a href="http://liris.cnrs.fr/ideal/doc/Georgeon0-BICA2012.pdf">The Small Loop Problem: A challenge for artificial emergent cognition</a>. <a href="http://www.oliviergeorgeon.com/">Olivier L. Georgeon</a>, <a href="http://science.slc.edu/%7Ejmarshall/">James B. Marshall</a>. In proceedings of <a href="http://bicasociety.org/meetings/2012/">BICA2012</a>, Annual Conference on Biologically Inspired Cognitive Architectures<a href="http://chilab.dinfo.unipa.it/bica2012/"></a>. Palermo, Italy, pp. 137-144. (October 31, 2012).<br />
<br />
This paper presents the <a href="http://e-ernest.blogspot.fr/2012/03/small-loop-challenge.html">Small Loop Problem</a> and how <a href="http://e-ernest.blogspot.fr/2012/04/ernest-112.html">Ernest 11.2</a> handles it.</div>
Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-81718616493376758632012-04-25T11:09:00.000+02:002012-06-18T13:37:32.805+02:00Ernest 7 in e-puck<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="separator" style="clear: both; float: left; margin-right: 10px; text-align: center;">
<object class="BLOGGER-youtube-video" classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0" data-thumbnail-src="http://i.ytimg.com/vi/kbQtaBhK8_Y/0.jpg" height="315" width="560"><param name="movie" value="http://www.youtube.com/v/kbQtaBhK8_Y?version=3&f=user_uploads&c=google-webdrive-0&app=youtube_gdata" />
<param name="bgcolor" value="#FFFFFF" />
<embed width="560" height="315" src="http://www.youtube.com/v/kbQtaBhK8_Y?version=3&f=user_uploads&c=google-webdrive-0&app=youtube_gdata" type="application/x-shockwave-flash"></embed></object></div>
Here is <a href="http://e-ernest.blogspot.fr/2010/12/java-ernest-72-in-vacuum.html">Ernest 7</a> in an <a href="http://www.e-puck.org/">e-puck</a> robot.</div>
<br />
We implemented "touch" with the infrared sensors available on the front, left, and right side of the robot. The range of these sensors was set to approximately 5cm. When Ernest "touches", the corresponding led
flashes. When the touching detects a wall, the two additional leds on
the rear flash. When it bumps into a wall, all the leds flash.The symbols in the trace are the same as <a href="http://e-ernest.blogspot.fr/2012/04/ernest-112.html">previously</a>.<br />
<br />
This video shows that Ernest learns to touch ahead before moving forward to avoid bumping, and learns to turn when it reaches a wall.</div>Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-27739739064698931582012-04-24T19:00:00.000+02:002012-04-25T16:35:02.047+02:00Ernest 11.2 in the Small Loop Problem<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="float: left; margin-right: 10px; text-align: left;" trbidi="on">
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='560' height='315' src='https://www.youtube.com/embed/IU-tQ1j6LqU?feature=player_embedded' frameborder='0'></iframe></div>
<br /></div>
This video shows Ernest 11.2's temporal memory (up-left) and spatial memory (right) in the <a href="http://e-ernest.blogspot.fr/2012/03/small-loop-challenge.html">Small Loop Problem</a> (bottom-left).<br />
<br />
Like <a href="http://e-ernest.blogspot.fr/2012/03/ernest-111-lsm.html">Ernest 11.1</a>, Ernest 11.2 keeps track of interactions in his spatial memory. Touch interactions are represented by squares, <i>move</i> by white triangles, <i>bump</i> by red triangles, <i>turn</i> by half-circles.<br />
<br />
Interactions that are enacted at the same spatial place are bundled together to represent the phenomenon that afforded them. The Small Loop Problem offers two kinds of phenomena: <i>empty squares</i> and <i>walls</i>. In temporal memory, the construction of bundles is represented by gray rounded rectangles (upper part). In spatial memory, bundles are represented by gray circles that contain interactions. These circles are fading to represent spatial memory decay.<br />
<br />
For example, on step 13, Ernest bundles together the interactions <i>touch front wall</i> and <i>bump</i>. On step 19, the fact that Ernest touches a wall activates the still-partial <i>wall bundle</i> in front of Ernest (right side of spatial memory). This activation reminded Ernest that moving forward would cause bumping, which made him refrain from trying to move forward.<br />
<br />
Ernest 11.2's spatial mechanism helps him perform better than <a href="http://e-ernest.blogspot.fr/2012/03/small-loop-challenge.html">Ernest 7 in the Small Loop Problem</a>. Ernest 11.2 does not, however, constitute a full solution to the Small Loop Problem because he has pre-implemented knowledge. The position of interactions relatively to Ernest's body were hard-wired in spatial memory. Additionally, the consequences that certain interactions have on spatial memory were pre-implemented: <i>move</i> translates spatial memory and <i>turn</i> rotates spatial memory. A full solution to the Small Loop Problem must eliminate such pre-implemented assumptions.</div>Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-63030426140492737852012-04-10T13:00:00.000+02:002012-08-20T14:19:54.262+02:00Designing Environment-Agnostic Agents<div dir="ltr" style="text-align: left;" trbidi="on">
<a href="http://liris.cnrs.fr/ideal/doc/GeorgeonO-ALA2012.pdf">Designing Environment-Agnostic Agents</a>. <a href="http://www.oliviergeorgeon.com/">Olivier L. Georgeon</a>, <a href="http://users.uom.gr/%7Eiliass/">Ilias Sakellariou</a>. In proceedings of <a href="http://como.vub.ac.be/ALA2012/">ALA2012</a>, Adaptive Learning Agents workshop, at <a href="http://como.vub.ac.be/ALA2012/"></a><a href="http://aamas2012.webs.upv.es/">AAMAS 2012</a>, 11th International Conference on Autonomous Agents and Multiagent Systems. 25-32. (June 4-5th 2012). <br />
<br />
In this paper, we define the notion of environment-agnostic agents. We also presents <a href="http://e-ernest.blogspot.com/2011/11/ernest-in-netlogo.html">Ernest in NetLogo</a>.</div>
Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-59031924786282482872012-03-27T16:27:00.000+02:002012-03-28T12:46:51.230+02:00The Small Loop Problem<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="float: left; margin-right: 10px;">
<iframe allowfullscreen="" frameborder="0" height="315" src="http://www.youtube.com/embed/Mstl8BRCbSI" width="560"></iframe></div>
We submit the <i>Small Loop Problem</i> to the community of artificial developmental cognition.<br />
<br />
The Small Loop Problem consists of implementing an artificial agent that would
"smartly" organize its behavior through autonomous interaction with the <i>Small Loop Environment</i> shown in this video. <br />
<br />
See our <a href="http://liris.cnrs.fr/ideal/demo/small-loop/">Small Loop Platform</a> in NetLogo for a comprehensive explanation and illustration of this problem.<br />
<br />
Our work thus far has only partially solved this problem. This video shows that Ernest indeed manages to learn to perceive its environment and to organize its behavior as we have demonstrated previously in this blog. Ernest, however, still does not manage to capture spatial regularities that would be required to "smartly" handle the passage in the upper-right corner of the loop.<br />
<br />
While this problem may seem simplistic, we believe that it captures fundamental questions about self-motivation and early-stage
developmental cognition. We suspect that the solution requires making the agent capable of some form of rudimentary reflexivity.</div>Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com2tag:blogger.com,1999:blog-586453263357094284.post-10048508344222725532012-03-16T12:17:00.001+01:002012-04-26T16:59:28.516+02:00Ernest at Innorobo<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="float: left; margin-right: 10px;" trbidi="on">
<div class="separator" style="clear: both; text-align: center;">
</div>
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='560' height='315' src='https://www.youtube.com/embed/jWtiM95kGiA?feature=player_embedded' frameborder='0'></iframe><br />
<br /></div>
This presentation of Ernest was shown at the <a href="http://innorobo.com/">Innorobo</a> symposium in Lyon. Here is a <a href="http://youtu.be/oDktIp98qLY">shorter video</a> without the audio presentation.</div>Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0tag:blogger.com,1999:blog-586453263357094284.post-41439934750891122062012-03-08T16:47:00.001+01:002014-11-16T12:56:42.390+01:00Object persistence<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="float: left; margin-right: 10px;">
<object data="http://content.screencast.com/users/oliviergeorgeon/folders/Default/media/05a47db7-8273-415a-8033-3ce2ee1690b4/bootstrap.swf" height="450" id="scPlayer" type="application/x-shockwave-flash" width="600"> <param name="movie" value="http://content.screencast.com/users/oliviergeorgeon/folders/Default/media/05a47db7-8273-415a-8033-3ce2ee1690b4/bootstrap.swf" />
<param name="quality" value="high" />
<param name="bgcolor" value="#FFFFFF" />
<param name="flashVars" value="thumb=http://content.screencast.com/users/oliviergeorgeon/folders/Default/media/05a47db7-8273-415a-8033-3ce2ee1690b4/FirstFrame.jpg&containerwidth=600&containerheight=450&content=http://content.screencast.com/users/oliviergeorgeon/folders/Default/media/05a47db7-8273-415a-8033-3ce2ee1690b4/Ernest11_1_object2.swf&blurover=false" />
<param name="allowFullScreen" value="true" />
<param name="scale" value="showall" />
<param name="allowScriptAccess" value="always" />
<param name="base" value="http://content.screencast.com/users/oliviergeorgeon/folders/Default/media/05a47db7-8273-415a-8033-3ce2ee1690b4/" />
Unable to display content. Adobe Flash is required.</object></div>
<br /></div>
This video shows how we expect Ernest to learn about objects in the world. To do so, Ernest needs a <i>Spatial Memory</i> that keeps track of the places where Ernest enacted interactions. We implemented a mechanism that emulates a <i>vestibular system</i> to allow Ernest to keep track of his displacements in space.<br />
<br />
In Ernest's Spatial Memory, blue half-circles represent places where Ernest turned; blue triangles represent places where he moved forward; red circles represent places where he bumped; green circles represent places where he situates the origin of visual perception that "attracts" him. These symbols are shrinking over time to represent memory decay.<br />
<br />
This video shows that Ernest constructs a rough outline of the object in Spatial Memory, and localizes <i>possibilities of interactions</i> relatively to this object.<br />
<br />
The big question now is how to make Ernest exploit this knowledge and how to entangle this knowledge construction mechanism with Ernest's "intrinsic motivation".
<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">(Demo implemented with Ernest <a href="http://code.google.com/p/e-ernest/source/detail?r=228">r228</a> and Vacuum <a href="http://code.google.com/p/vacuum-sg-continuum/source/detail?r=166">r166</a>)</span></div>
Olivierhttp://www.blogger.com/profile/01065028060343045890noreply@blogger.com0