WP3: Cognitive architecture and deep representations supporting longlife assistive robotics
The main goal of WP3 has a clear scientific bias involving research in cognitive robotics and human intelligence theories and can be stated as the design and development of CORTEX, a novel robotics software architecture able to support the activity of a socially assistive robot working in a smart home during long periods of time.
This goal can be divided in the following sub goals and associated tasks:
T3.1 Inner-modeling of the robot, human interactors, and its environment
CORTEX will contain a Deep hybrid State Representation (DSR) of the current state, including people, the robot itself and the environment. The DSR will include geometric relations between these elements. It will set and modify kinematics chains as required, to match observed cues. It will also integrate symbolic concepts and relationships, being these symbolic links able to store action-related (affordances) concepts (e.g. ‘is’, ‘can be picked’, ‘is in touch with’…). Geometric data within the model will allow the robot, for instance, to avoid self-collisions and dangerous movements (e.g. concerning to the person’s presence) or to safely manipulate objects. Symbolic data will be used to conduct natural interactions through multiple channels, to share attention, to influence the interactive responses of the robot according to the context, or to define the robot’s personality, and its intentions (course of action coming from high-level modules). The use of an unified representation will allow agents to interface more efficiently the data represented in the DSR, and will allow to easily implement features such as semantic annotations for geometric objects or kinematics relations influenced by symbolic states. The agents themselves maintain this representation synchronized with the real world, by updating continuously its contents with new sensor readings.
T3.2 Integration of the DSR and the agents in the CORTEX architecture
The DSR will be interfaced by a set of task-related networks of software components (agents), which will provide broad functionalities such as navigation, dialog or multimodal person monitoring. These agents will manage data coming from the robot or other external sensors and they will be fully developed within other work-packages on this project (WP4, WP5, WP6 and WP7). Specifically, within this task, we must guarantee that the agents are able to feed the DSR with new geometric models or symbolic concepts, and that the data stored in the DSR will be kept synchronized with the real world by updating actions performed by different agents. Also, the DSR graph will be kept synchronized among the agents by using the publishing mechanism of RoboComp. Finally, these data will be learnt online, changing in time to achieve lifelong adaptation to the user.
T3.3 Emulation and mid-level planning
Two additional agents will be implemented in this WP. These agents will be in charge of performing short-term and mid-term cognitive emulations, where foreseen courses of action can be inferred and tested. They will be able to include and update virtual elements in the representation. These virtual data will be managed as real data by the remaining agents. The simile could be as if CORTEX simultaneously moves a real robot and a virtual copy of this robot, this second one considering the virtual data included in the DSR. The first included agent is a short-term emulator, which allows the robot to be locally predictive. The second agent performs mid-term emulations, which allow the robot to determine the consequences of its actions or to show a pro-active behavior. The long-term ability for planning, finally, is provided by the high-level decision maker itself (WP4).