Back

Speaker "Monal Daxini" Details Back

 

Topic

Netflix Keystone - The Cloud Event Processing Pipeline

Abstract

Keystone - Processing over Half a Trillion events per day with 8 million events & 17 GB per second peaks, and at-least once processing semantics. We will explore in detail how we employ Kafka, Samza, and Docker at scale to implement a multi-tenant pipeline. We will also look at the evolution to its current state and where the pipeline is headed next in offering a self-service stream processing infrastructure atop the Kafka based pipeline and support Spark Streaming.

Profile

Monal Daxini is a Senior Software Engineer at Netflix working on building a scalable and multi-tenant event processing pipeline. He has also worked on Cassandra, Dynomite (Redis), and was instrumental in developing the encoding compute infrastructure for all Netflix content. He has 14 years of experience building scalable distributed systems at organizations like Netflix, Cisco, and NFL.com.