A landscape of current research
This page presents different projects that use explicit knowledge encoded in an ontology to support some aspects of robot autonomy. Robots may reach different levels of autonomy depending on what restrictions their architecture imposes, and what type of cognitive capabilities it supports. It is hard, however, to pinpoint a definition of autonomy. It is more common to list features that a cognitive system needs to address, such as:
This page has been created to summarize the findings of the literature review paper entitled "A Review and Comparison of Ontology-based Approaches to Robot Autonomy" that was published at the Knowledge Engineering Review journal in 2019.
This is the abstract of the paper:
Please use the following bibtex entry for citing us:
In total, six frameworks/projects have been subject of study in our review. For each of them, their underlying principles and foundations are discussed, as well as what application domain the system was designed for. We also describe how the frameworks evolved over time, and what impact they have had so far. The selection of the presented projects has been done based on the selection criteria presented within the article. To the best of our knowledge, we have included all projects that satisfy the criteria.
Department of Informatics, Bioengineering, Robotics, and Systems Engineering, University of Genoa
CARESSES is an international, multidisciplinary project whose goal is to design the first robots that can assist older people and adapt to the culture of the individual they are taking care of. The robots will help the users in many ways including reminding them to take their medication, encouraging them to keep active, helping them keep in touch with family and friends. Each action will be performed with attention to the older personâs customs, cultural practices and individual preferences.
Institute of Industrial and Control Engineering (IOC), Polytechnic University of Catalonia (UPC)
Perception and Manipulation knowledge framework (PMK) presents a standardized ontology framework for autonomous robot perception and manipulation, which follows the IEEE standards 1872 of representing knowledge for the robotic domain. Moreover, an inference mechanism for reasoning over the knowledge is included, which enhances the planning of manipulation tasks. A perception module can be integrated with the framework to capture a rich semantic description of the scene, knowledge about the physical behavior of the objects, and reasoning about the potential manipulation actions. The reasoning scope of PMK is divided into four parts: reasoning for perception, the reasoning for object features, the reasoning for a situation, and reasoning for planning.
LAAS-CNRS, France
ORO is a project focused on the implementation of a common representation frameworkfor autonomous robots with special emphasizes on human-robot interaction. The proposed framework was meant to enhance robotâs interactionwith complex and human-inhabited environments, were robots are expected to exhibit advancedcognitive skills, such as: object recognition, natural language interaction, task planning withpossible dynamic re-planning, ability to cooperate with other robots or humans, etc.
Institute for Artificial Intelligence, University of Bremen
KnowRob is a knowledge processing system that combines knowledge representation and reasoning methods with techniques for acquiring knowledge and for grounding the knowledge in a physical system and can serve as a common semantic framework for integrating information from different sources. KnowRob combines static encyclopedic knowledge, common-sense knowledge, task descriptions, environment models, object information and information about observed actions that has been acquired from various sources (manually axiomatized, derived from observations, or imported from the web). It supports different deterministic and probabilistic reasoning mechanisms, clustering, classification and segmentation methods, and includes query interfaces as well as visualization tools.
Computer Science, Faculty of Engineering of Lund University
ROSETTA stands for RObot control for Skilled ExecuTion of Tasks in natural interaction with humans; based on Autonomy, cumulative knowledge and learning. It consists of a set of ontologies of robot skills implemented with the goal to create an intelligent support system for reconfiguration and adaptation of robot-based manufacturing cells. Along the years, the ontology has been used to enhance cognitive abilities of robots that are required to plan and execute assembly tasks, robot programming, and skill reusability in industrial scenarios.
Polytechnic Institute of Castelo Branco, Portugal
An Ontology for Robotic Orthopedic Surgery (OROSU) was developed and then applied for hip resurfacing surgery (e.g., for trimming the femoral head). In this scope, the main goal of the research, related to ontologies, was to build a knowledge-based framework for this surgical scenario, along with a formal definition of components and actions to be performed during the surgery. The developed ontology was partially based on the 1872-2015 - IEEE Standard Ontologies for Robotics and Automation. The work was developed under the HIPROB and ECHORD projects, funded by the Portuguese Science Foundation and the EU-FP7, respectively. The framework is among the first to integrate robotic ontologies in the domain of surgical robotics.
The scope of reasoning is our second classification criterion for the comparison between ontology-based approaches in autonomous robotics. It consists of a categorization of nine ontology-based reasoning tasks that are in particular relevant for autonomous robotics, and that have been considered in previous works.
For the purpose of establishing a contact between its environment and its knowledge, a robot must be able to recognize events or situations (static and dynamic) and categorize them as named instances of already known patterns. For instance, let’s consider a collaborative industrial robotic arm which picks pieces from a conveyor and places them on a table where a human operator can access to. The robot must recognize and cateogorize the conveyor and the different pieces to manipulate (static), as well as a the human collaborator’s movements and actions (dynamic). Both, recognition and categorization, operate on the output generated by perception systems and often are seen as a unique capability. Nevertheless, they are addressed separately because both can operate on abstract mental structures. In order to support recognition and categorization, a cognitive architecture shall provide a form to represent patterns and situations in memory.
An autonomous robot requires the ability to choose among several alternatives, which usually is considered together with the recognition and categorization problem in a recognize-act cycle. Nonetheless, we consider the capability of decision making independently. It is important not to mistake this capability with planning, whose focus is on the achievement of a goal and it will be explained later. For example, a collaborative robot would apply decision making to choose between moving or not when the operator is close, while planning would be used to find the sequence or sequences of actions which are likely to lead to a successful collaborative task. A cognitive architecture should be able to represent the different choices in a format the robot understands. Indeed, that representation would also be used to improve the decision making through learning.
The environment where the robot exists, must be sensed, perceived and interpreted. First, the robot senses its surroundings through possibly muti-modal sensors. Then, using the gathered information and relying on recognition and categorization, discussed earlier, and on inferential mechanisms, which will be covered shortly, the robot is able to perceive the environmental entities (e.g. objects and events). Finally, the situation assessment takes place when the perceived objects and events are interpreted. Following with the example used before, a collaborative industrial robot would look at the conveyor, the pieces and at the operator's movements to sense the environment. The pieces, their poses and other information would be recognized and categorized in order to assess the environmental situation so that the robot could, for example, interpret that an approaching piece should be picked. Just as occurred with previous cognitive capabilities, the inherent knowledge of the whole process must be represented in a manner the robot understands. Note that the representation requires memory, a resource which is often limited. Hence, the notion of attention emerges, meaning that the robot not only has to perceive but also it could be asked to decide to focus only on a specific region of the environment.
Prediction is a cognitive capability which requires the representation in memory of a model (e.g. ontology-based) of the environment, the actions that can take place and their effects. Therefore, the robot could predict future events and situations which did not occur yet by means of a proper mechanism which utilizes the representation. Applied to the collaborative robotics' example, it would be possible for the robot to predict the operator's actions, so that the robot could adapt better to what it is expected. Note that prediction enables robots to also monitor processes. When the perceived situation differs from the expected one, it means that either our knowledge is not complete or something did not go as it was supposed to. In the former case, it would be possible to store the facts in memory for posterior learning, in the later case, an alarm or error could be triggered. In the example, the robot could detect a malfunctioning in the conveyor if the pieces stopped arriving close to the robot's workspace.
In novel situations where robots are meant to achieve their goals, it is necessary for them to be able to plan and solve problems. For the purpose of generating a plan, the robot needs a model of the environment utilized to predict the effects of its actions. Furthermore, the cognitive architecture must be able to represent a plan as an (at least partially) ordered set of actions, their expected effects, and the manner in which these effects enable subsequent actions. Sometimes, a robot could have a memory with previous plans which could be re-used with and without further modifications. Note that it is also considered the case of having conditional actions and different branches which depend on the outcome of previous events. Despite often being viewed intimately related, planning is somewhat less general than problem solving. In particular, the former usually refers to cognitive activities within the robot's head, whereas the later can also occur in the world. Concretely, when a problem to be solved is complex and the available memory is limited, a robot may search for solutions by executing actions in the environment, rather than constructing a complete internal plan. As an illustration, a collaborative robot could solve a problem by mixing the execution of actions such as 'asking for operator's help' (external behavior) and the generation of actions' sequences (internal planning).
Reasoning is a cognitive activity which allows a robot to expand its knowledge state, drawing conclusions from other beliefs or assumptions the robot already maintains. Thus, it is required the existence of a representation of beliefs and the relationships among them. A common formalism used to encode such knowledge is first-order logic (FOL). Ontologies are often written in languages based on less expressive formalisms than FOL (e.g. OWL-description logic) in order to reduce the computational cost of inference. These formalisms, allow the use of different sorts of reasoning such as: deductive or inductive. For the robot of the previous example, it would be possible to infer how operators will react to unexpected interactions by knowing general information about human-robot interaction (deductive). Or the opposite, from specific operator's behaviors, inferring the norms to follow during human-robot interaction (inductive). Note that reasoning is not only relevant to infer new beliefs but also to decide whether to hold existing ones (belief maintenance). Such belief maintenance is especially important for dynamic environments in which situations may change in unexpected ways, with implications for the robot's behavior.
Cognition takes place to support and drive activity in the environment. To this end, a cognitive architecture must be able to represent and store motor skills that enable such activity. In the example, a collaborative robotic arm should have skills or policies for manipulating its surroundings and for collaborating with other agents (e.g. humans). A robot should also be able to execute those skills and actions in the environment, what can happen in a reactive form. Nevertheless, a cognitive architecture should enable a robot to maintain a continuum loop of execution. Hence, the robot would be able to interpret how the execution of actions is affecting the state of the environment and could adapt its behavior. A proper representation of the ongoing actions occurring in the environment, is essential for aspects related to robot action execution: robot adaptation, new skills learning, action-execution-related knowledge, etc.
Sometimes, the most effective way for a robot to obtain knowledge is from another agent (e.g. humans, robots, etc.), making communication another important ability that an architecture should support. Going back to the example used before, a collaborative robot could request for further information about how to perform a task or which are the preferences of the specific plant operator about where to place the picked pieces. Regardless of the modality or mean of communication, there should be a way to represent the transferred knowledge so that it is accessible to and understandable for the robot. Indeed, this should be bi-directional, meaning the robot must be able to transform stored knowledge into the concrete medium through which it will be communicated.
There are some capabilities which cut across those described before, whose use could enhance the performance of autonomous robots while not being strictly necessary for robot autonomy: remembering, reflection and learning. Remembering is the ability to encode and store the results (facts) of cognitive tasks so that they can be retrieved later. Once again, based on the previous example, a collaborative robot could store the results of an entire day of work (e.g. successful experiences, human-robot interactions, etc.). On the other hand, reflection stands for the serious thought or consideration about something which usually is represented and stored in memory and can be retrieved. A collaborative robot could take into account a stored memory in order to explain which was the rationale behind its actions. Finally, learning, which usually involves generalization beyond specific beliefs and events. In the example, the collaborative robot would use the stored memories about successful and failed actions to generalize and learn from them. The knowledge used to learn might come from distinct sources, the observation of another agent, the result of previous experiences, or through kinesthetic teaching. No matter the source of experience, all of them require the existence of a memory in which the experiences are represented.
Should you have any doubt/comment about our work, please, do not hesitate to contact us.
Alberto Olivares-Alarcos - aolivares@iri.upc.edu
Daniel Beßler - danielb@uni-bremen.de