Federated deep learning offers new approach to model training Posted on : Apr 19 - 2019

Training deep learning models puts a massive strain on enterprise infrastructure, but federated learning, which trains models on endpoint devices, could lessen some of the demand.

Training machine learning and deep learning models requires massive compute resources, but a new approach called federated learning is emerging as a way to train models for AI over distributed clients, thereby reducing the drag on enterprise infrastructure.

In one example, engineers at Google working on the company's Android mobile platform used federated deep learning to improve the performance of speech recognition and predictive text applications for phones in a way that reduces privacy concerns, increases model performance and reduces communication overhead.

"Federated learning is a new frontier in AI where you leverage the massive compute available in the form of distributed devices, thereby allowing for learning to be local, private and yet relevant," said Ramya Ravichandar, vice president of product management at FogHorn Systems, an IoT platform provider.

Traditional AI approaches require powerful compute resources where training data from all sources is aggregated and the models are trained. In federated learning, developers use the compute power of individual devices to distribute the learning process. And, because data never leaves the device that created it, federated learning can help support user privacy.

Federated learning can be applied to deep learning problems or more traditional machine learning problems, said Mike Lee Williams, research engineer at Cloudera Fast Forward Labs.

"Deep learning is cool and can be more accurate than traditional machine learning, but you need a good reason to use it because it introduces engineering complexity and may require specialized hardware."

A new paradigm

Most existing AI tools train on data in the cloud and then push better algorithms to devices. With federated deep learning, there's no need for training data to be sent in all its completeness to a central data store, Ravichandar said.

Approaches can vary in terms of which subset of edges to include in the training updates, how to capture those updates, when to trigger retraining and when to push out the new model to all users. All these approaches will vary based on the use cases and applications involved. View More