Big Data and Deep Learning: A Tale of Two Systems

From determining the most convenient rider pickup points to predicting the fastest routes, Uber uses data-driven machine learning to create seamless trip experiences. Within engineering, big data powers machine learning to inform decision-making processes across the board. As Uber expands to new markets, the ability to accurately and quickly use data to make predictions becomes even more important. Zhenxiao Luo explains how Uber tackles data caching in large-scale deep learning. Zhenxiao begins by detailing Uber’s machine learning architecture and discussing how Uber uses big data to power deep learning jobs. He then explores the efficiency and scalability challenges Uber is facing as a result of its growing business, data, and job complexity and how Uber is designing and implementing data caching solutions to speed up Uber’s machine learning platform. Zhenxiao concludes by sharing artificial intelligence use cases powered by Uber’s machine learning platform.

Zhenxiao Luo is Sr. Staff Engineer, leading Interactive Query Engines team at Twitter, where he focuses on Druid, Presto, Spark, and Hive. Before joining Twitter, Zhenxiao was running Interactive Analytics team at Uber. He has big data experience at Netflix, Facebook, Cloudera, and Vertica. Zhenxiao is PrestoDB committer. He holds a master’s degree from the University of Wisconsin-Madison and a bachelor’s degree from Fudan University.