Abstract:

Currently, there is a proliferation of Artificial Intelligence (AI) enabled applications, particularly those based on deep learning in the areas of computer vision and natural language processing, heavily using cloud resources aiming to deliver more significant experiences to their customers. In general, these solutions consume a massive volume of data, relevant communication capacity, and expensive and specialized processing capabilities. On the other hand, there is an increasing interest in the industry to reduce latency, save bandwidth, improve availability, and protect data privacy. These two trends combined, created the need for a different modality of execution platform: Edge Intelligence.

However, existing distributed computing solutions are designed to work in the cloud, and some crucial principles are not directly applicable to the edge. Computing power limitations and the mismatch between machine learning algorithms and the edge platform bring significant challenges to the software development community.

This work describes an Edge Intelligence platform enabling real-time machine learning services leveraging low-latency microservices architecture, hardware-agnostic scheduling of resources, GPU sharing, and serverless execution pipelines. The work also illustrates the utilization of the Edge Stack in real-time object detection and visual inspection of parts, as well as real-time topic summarization and sentiment analysis.

 

Who is this presentation for? This presentation is targeted to software engineers and solution architects.

 

Prerequisite knowledge: The talk is open to all levels.

 

What you'll learn? This session describes our practical experience in implementing real-time deep learning pipelines at the edge. The goal is to share with the audience different options to deploy AI-enabled services at the edge, orchestration of low-latency deep learning pipelines using serverless functions at the edge, and a technical overview of the developed edge intelligence architecture.