HARI -- It’s why we like talking to robots Posted on : Dec 17 - 2018

We feature speakers at 3rd Annual Global Artificial Intelligence Conference - 2019 Jan 23 - 25 – Santa Clara to catch up and find out what he or she is working on now and what's coming next. This week we're posting blog written by  Robert Zhang, President & CEO, Cloudminds Technology Inc

HARI -- It’s why we like talking to robots

By Robert Zhang

Introduction

The discussion of machines that can think and make decisions similar to a human existed even before John McCarthy neologized the term artificial intelligence (AI) in 1956. Today, the term AI is known by many and used loosely. Experts in the field have a multitude of definitions for AI. However, the nucleus of AI is the ability of a machine to make decisions to achieve a specific goal by learning and reasoning from the observed data. Or simply, the ability to simulate human ways of reasoning and decision making.

Similar to how intelligent thinking machines led to the term artificial intelligence, the term robotics first appeared in the 1940s in science fiction by Isaac Asimov. In Asimov's fiction, the term robotics represented an artificial lifeform and its specific field of study. By definition, a robot, generally, is a machine that is capable of autonomous movements. A robot can function autonomously without AI. However, with the advancement in technology, it has become imperative that robots need AI for more human-like functioning.

While AI can exist as software without a physical body, robots exist physically. A physical robot interacts with the objects in the setting in which it resides. Amongst the many entities of the environment, humans are the trickiest entity with whom robots need to interact. A robot needs to learn the social ways of human-human interaction (HHI) to emulate the free-flowing nature of HHI in human-robot interaction (HRI). Social participation by social robots requires computer vision, speech and language processing, as well as  emotion and gesture recognition. Cloudminds Technology offers a human-augmented robotic intelligence (HARI) platform that combines these capabilities to provide a robust and engaging HRI.

So, what is HARI? 

HARI is a form of collaborative artificial intelligence in which the robot along with the human-in-the-loop collaboratively carry out decision-making processes and social interaction with a system user. This human aided robotic intelligence is HARI.

What is the HARI workflow for HRI?

A system user interacts with a social robot using linguistic and non-linguistic methods. The social robot is connected to cloud AI through a robot control unit. The cloud AI consists of the HRI system, HARI, and the AI learning system. The interaction design of HRI consists of receiving and processing the audio-video signals, semantic understanding of the processed signals, and, subsequently, generating a response to the user. HARI is embedded in all stages of the HRI system to augment the AI capabilities of HRI system. The HARI design consists of a HARI switch, a human-in-the-loop(HIL) interface, and a human operator. The function of the HARI switch is to switch between the AI of the HRI system and the human operator (human intelligence) in the HARI interface. The role of a human operator is to make decisions for the HRI system when the HARI switch is triggered. The AI learning system collects the interaction data from both HRI and HARI systems. The learning system trains and learns from the human operator decisions to update the HRI system AI in the cloud.

High-level system diagram of HARI architecture

High-level system diagram of HARI architecture

Why HIL?

With HIL, the human operator helps the robot when it encounters unseen or incomplete data and is unable to make decisions with  high confidence level from the trained model. When the HARI switch is triggered, the human operator corrects the erroneous results generated by the specific stage of the HRI system. During the process of correction and updating, the system logs the following: (a) input data from the preceding stage, (b) the incorrect result from the current stage, and (c) the updated result by the  human operator. At every step of the interaction, the system generates and stores the log in the database of the AI learning system of the HARI set-up. The AI learner of the AI learning system uses the log data as a training example, learning from demonstration to map the knowledge to results. Therefore, the AI learner helps to update the trained model of the HRI system and makes the HARI for HRI a constant process of learning and updating AI. This  improves overall AI system performance.

What is confidence scoring and its role in HARI for HRI?

In machine learning, the functions responsible for confidence scoring are called activation functions. Relying on the confidence scoring mechanism using activation functions may still produce results with a higher confidence score that are incorrect. Therefore, in HARI for HRI system, the confidence scoring mechanism embeds further steps to analyze the error forms, including false accept and false reject, in the result. Similar to the work in understanding the uncertainty in a model using deep models and Gaussian process by Yarin Gal from University of Cambridge, the confidence scoring function in HARI for HRI system uses a Gaussian process to train the results to analyze errors and break down the confidence scoring approach [currently, research in this area is ongoing]. Incidentally, the confidence score acts as a switch to call HIL to action.

Transitioning from HIL to no HIL?

While HIL helps the robot perform better in carrying out interactions with users, the HRI system should steer towards less usage of HIL. The central reason behind having HIL for HRI is to aid the robot with decision making and that the robot learns from the decisions made by the human operator. Therefore, HIL is an integral part of the learning process in the HRI system which helps the robot in learning from demonstration. Assuming that the AI is never complete and an ever learning system, HIL will continue to exist in some capacity. However, the involvement of HIL must decrease over time or include HIL in those areas where the robot needs assistance to achieve the task. It is a subjective decision as to when the HIL must stop aiding the learning process.