A System to Detect and Reduce Understanding Bias in Intelligent Virtual Assistants

Many companies and researchers are surprised and frustrated when promising methods of machine learning perform so well on standard benchmarks or in literature but fail miserably once deployed into the wild. This occurs when the latent biases in the training data and benchmarks do not match those present in real-world data. While publicly available datasets are a valuable resource, rarely do they directly translate to features within private datasets. In this talk, we will cover the common pitfalls of using publicly available data to train or evaluate models for use on private data. We then introduce Human-in-the-Loop methods, and Active Learning in particular, as a means to overcome these pitfalls. This allows companies and researchers to leverage existing datasets to create initial models and quickly adapt them to real-world data to minimize annotation costs.

Who is this presentation for?
Enterprises looking to improve their AI soultions, executives, and IT leaders.
Prerequisite knowledge:
Basic understanding of AI, automation, machine learning, and Big Data.
What you'll learn?
We'll learn Active Learning and how to no only use it to leverage public data but also get the results needed on the actual task at hand.

Ian Beaver, PhD is the lead research engineer at Verint Intelligent Self Service, a provider of conversational AI systems for enterprise businesses. Ian has been publishing discoveries in the field of AI since 2005 on topics surrounding human-computer interactions such as gesture recognition, user preference learning, and communication with multi-modal automated assistants. Ian has presented his work at various academic and industry conferences and authored over 20 patents within the field of human language technology. His extensive experience and access to large volumes of real-world, human-machine conversation data for his research have made him a leading voice in conversational analysis of dialog systems. Ian is currently focused on the means to detect and resolve misunderstandings between humans and machines. He is also leading a team in finding ways to optimize human productivity by way of automation and augmentation, using symbiotic relationships with machines.