Adaptive Reinforcement Learning For Connected Devices

We live in a connected world where, thanks to our smartphones, laptops, tablets and other devices, we are never offline. The billions of devices, sensors and actuators, connected to the internet, are generating exponentially more data than before. Not only will we be able to forecast when these devices might need maintenance, we may also be able to predict when we need support. A use-case might be related to earlier detection of diseases/disorders based on insights from our wearable device. However, just like everything else, this isn’t an easy task. Sensors break, actuators get swapped and the ecosystem changes in all sorts of ways. Thus, timely upgrade recommendations for these connected devices would enable customers maintain their seamless interaction with these IoT devices. A traditional supervised learning approach doesn’t work well for such IoT data. Moreover, standard classification and regression models are not that capable of handling the relationship between sensor changes and actuator commands. One solution is to go for an adaptive feedback-driven state-action based reinforcement learning techniques, which might help in finding reward or penalty and the cost incurred to the customer. This can ultimately lead to an intrinsically smart and connected ecosystem.

Pranjal Daga is a Data Scientist focused on strategizing and developing Deep Learning Proof of Concepts at Cisco Services Machine Learning R&D. He graduated from Purdue University with a Masters in Computer Science, specializing in Deep Learning and NLP. In the past, Pranjal has worked with researchers at Adobe Research, University of Alberta, Northwestern University, IBM Research and MIT. He has spent the past few years exploring new technologies, hacking on the quirky side projects and bringing together his research and engineering experiences. To understand the practical aspects of identifying business ideas and moving them forward, Pranjal recently joined Stanford Graduate School of Business' Ignite program.
On a side for fun, Pranjal loves building new things from scratch, which is why he regularly goes for hackathons.