Edge Intelligence: Orchestrating Low-Latency Machine Learning Services at the Edge

Currently, there is a proliferation of Artificial Intelligence (AI) enabled applications, particularly those based on deep learning in the areas of computer vision and natural language processing, heavily using cloud resources aiming to deliver more significant experiences to their customers. In general, these solutions consume a massive volume of data, relevant communication capacity, and expensive and specialized processing capabilities. On the other hand, there is an increasing interest in the industry to reduce latency, save bandwidth, improve availability, and protect data privacy. These two trends combined, created the need for a different modality of execution platform: Edge Intelligence. However, existing distributed computing solutions are designed to work in the cloud, and some crucial principles are not directly applicable to the edge. Computing power limitations and the mismatch between machine learning algorithms and the edge platform bring significant challenges to the software development community. This work describes an Edge Intelligence platform enabling real-time machine learning services leveraging low-latency microservices architecture, hardware-agnostic scheduling of resources, GPU sharing, and serverless execution pipelines. The work also illustrates the utilization of the Edge Stack in real-time object detection and visual inspection of parts, as well as real-time topic summarization and sentiment analysis.
Who is this presentation for?
software engineers and solutions architects
Prerequisite knowledge:
The talk is open to all levels
What you'll learn?
This session describes our practical experience in implementing real-time deep learning pipelines at the edge. The goal is to share with the audience different options to deploy AI-enabled services at the edge, orchestration of low-latency deep learning pipelines using serverless functions at the edge, and a technical overview of the developed edge intelligence architecture.

Franco Vieira is a senior technologist working for HP Personal Systems Software in Brazil. He leads the development of edge computing solutions, in particular, AI-enabled applications. Franco is part of the PS Software Data Science leadership team, founder member of machine learning center of expertise in HP Brazil, and an active contributor of data science community at HP. He holds a bachelor’s degree in computer science, and significant experience in distributed systems, cloud computing, and machine learning.