This is the repository of the POETICON++ EU project.

Video by Istituto Italiano di Tecnologia, IIT, iCub Facility (credits Vadim Tikhanoff):

More videos are available on the YouTube channel.

Mandatory dependencies of the core build:

• YARP
• icub-contrib-common
• OpenCV, version 2.4

Dependencies of probabilistic planning:

• libPRADA, mirrored in the 3rdparty directory

Dependencies of affordance perception:

• MATLAB
• YARP Java bindings (in the YARP build directory, call CMake and enable YARP_COMPILE_BINDINGS, CREATE_JAVA, PREPARE_CLASS_FILES)
• YARP for MATLAB (add the directory with the Java *.class files to the classpath, and the directory with libjyarp.so to librarypath.txt)
• pmtk3 (installation: launch MATLAB as superuser, run initPmtk3.m, click on setPath, select the pmtk3 directories then Save to permanently add them to MATLABPATH)

The following external projects are needed to execute the full POETICON++ demo:

• Hierarchical Image Representation
• Interactive Objects Learning
• karma
• segmentation
• stereo-vision

Dependencies of the optional speech recognition component:

• YARP Lua bindings
• rFSM (just clone it, no need to compile)

First install libPRADA, required by the probabilistic planner:

cd poeticon
tar xzvf 3rdparty/libPRADA.tgz && cd libPRADA
patch src/MT/util.h < ../extern/libPRADA/prada_unistd.patch
patch test/relational_plan/main.cpp < ../extern/libPRADA/prada_readgoalfromfile.patch
make
cp test/relational_plan/x.exe ../app/conf/planner.exe

Then install the core POETICON++ build:

git clone https://github.com/robotology/poeticon
cd poeticon && mkdir build && cd build && cmake .. && make

Note: the modules belonging to the probabilistic planner part (planningCycle, affordanceCommunication, geometricGrounding, goalCompiler) must have access to the same "contexts/poeticon" directory. One way to accomplish this is to run them on the same machine.

Affordance perception sensorimotor data and documentation are here.

Instructions on how to run the simulated symbolic reasoner (to test probabilistic planning under noisy conditions and with different heuristics) are here.

Further documentation is available inside each module source, and via the help commands available via RPC interfaces.

• Alexandre Antunes, Lorenzo Jamone, Giovanni Saponaro, Alexandre Bernardino, Rodrigo Ventura. From Human Instructions to Robot Actions: Formulation of Goals, Affordances and Probabilistic Planning. IEEE International Conference on Robotics and Automation (ICRA 2016).
• Tanis Mar, Vadim Tikhanoff, Giorgio Metta, Lorenzo Natale. Self-supervised learning of grasp dependent tool affordances on the iCub Humanoid robot. IEEE International Conference on Robotics and Automation (ICRA 2015).
• Tanis Mar, Vadim Tikhanoff, Giorgio Metta, Lorenzo Natale. Multi-model approach based on 3D functional features for tool affordance learning in robotics. IEEE-RAS International Conference on Humanoid Robots (Humanoids 2015).
• Afonso Gonçalves, João Abrantes, Giovanni Saponaro, Lorenzo Jamone, Alexandre Bernardino. Learning Intermediate Object Affordances: Towards the Development of a Tool Concept. IEEE International Conference on Development and Learning and on Epigenetic Robotics (ICDL-EpiRob 2014).
• Afonso Gonçalves, Giovanni Saponaro, Lorenzo Jamone, Alexandre Bernardino. Learning Visual Affordances of Objects and Tools through Autonomous Robot Exploration. IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC 2014).
• Vadim Tikhanoff, Ugo Pattacini, Lorenzo Natale, Giorgio Metta. Exploring affordances and tool use on the iCub. IEEE-RAS International Conference on Humanoid Robots (Humanoids 2013).

Released under the terms of the GPL v2.0 or later. See the file LICENSE for details.