© Stefano Nolfi, 2021 | How to cite this book | Send your feedback | Collaborate
Index First Chapter
This book describes how to create robots capable to develop the behavioral and cognitive skills required to perform a task autonomously, while they interact with their environment, through evolutionary and/or learning processes. It focuses on model-free approaches with minimal human-designed intervention in which the behavior used by the robot solve its task and the way in which such behavior is produced is discovered by the adaptive process automatically, i.e. it is not specified by the experimenter.
The first objective of the book is to introduce autonomous robots and adaptive methods: evolutionary robotics, reinforcement learning, and learning by demonstration. The book cannot and does not aim to be exhaustive in that respect. It focuses on the most effective methods at the current state of the art and on the relation between methods that are closely related but that are normally studied independently within separated research communities.
A second objective is to use the analysis of the behavioral and cognitive solutions discovered by adaptive robots in concrete experiments to illustrate the fundamental aspects of embodied intelligence: the relation between the body and the “brain” of the robot, the role of sensory-motor coordination, the consequences of under-actuation, the implications of the dynamical and multi-levels nature of behavior, the importance of robustness, the role of emergence and self-organization, the impact of the learning experiences on the adaptive process, the role of anticipation and world models, the role of cooperation and competition among robots, the factors that can promote continuous and open-ended learning.
Finally, the third objective is that to enable the reader to acquire hand-on knowledge by experimenting with adaptive robots. This final objective is realized by introducing the reader to easy-to-use and powerful software tools that permit to create adaptive robots, replicate representative state-of-the-art experiments, and acquire the practical skills required to carry on high-quality research on this area. Moreover, it is realized by providing directions on how to set the hyperparameters and how to implement the reward function. This last objective is covered in the last chapter of the book and on the “learn how” sections, which introduce the exercises suggested after each Chapter. For examples of experiments which can be performed, see Video 1.
The electronic format of the book allows the usage of videos to illustrate the experiments reviewed, an aspect that is particularly important for robotics. Moreover, it enables the author to update the content of the book more easily. This is crucial to keep the book updated with respect to the theoretical and technical progresses of the field.
The author thanks Paolo Fazzini for detailed feedbacks on the manuscript.
Copyright: © 2021 Stefano Nolfi. This is an open access book distributed under the terms of the Creative Commons Attribution License 4.0 (CC BY) which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
The figures shown in the left and right banners are modified versions of the original figures included in (from left to right and from top to bottom): Nolfi (2009); Sagita & Tani (2008) https://www.youtube.com/watch?v=n9NYcG8xlYs; Simione & Nolfi (2021); Andrychowicz et al. (2017), https://www.youtube.com/watch?v=Dz_HuzgMxzo; Lipson & Pollack (2000), www.demo.cs.brandeis.edu/golem/; Carvalho & Nolfi (2016), https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0160679; Sperati, Trianni & Nolfi (2011); Achiam (2018); Nolfi (2005); Andrychowicz et al. (2018), https://www.youtube.com/watch?%3F&v=jwSbzNHGflM; Floreano et al. (2007), https://www.youtube.com/watch?v=K3s5uh6-sXo; Balsal et al. (2018), https://www.youtube.com/watch?v=OBcjhp4KSgQ; Cheney, Clune & Lipson (2014), https://www.youtube.com/watch?v=z9ptOeByLA4; Coumans & Yunfei, 2016-2019, https://pybullet.org/; Floreano, Nolfi & Mondada (1998); Nolfi (2021); Tuci et al. (2011); Baldassarre et. al. (2007).