Back

Speaker "Peter Bakas" Details Back

 

Topic

Netflix Keystone - How we built a 700B/day stream processing cloud platform in a year

Abstract

Keystone processes over 700 billion events per day with at-least once processing semantics in the cloud. We will explore in detail how we have modified and leverage Kafka, Samza, Docker, and Linux at scale to implement a multi-tenant pipeline in the Amazon AWS cloud within a year. * Pipeline Evolution & Architecture * Why we chose Kafka, Samza, Docker * How to effectively use these technologies together in the Cloud * Alterations to Kafka & Samza * Scaling and Managing Kafka, Samza & Docker * Deployment & Monitoring details * Fault tolerance and failover strategies * Performance numbers

Profile

Peter leads the Real Time Data Infrastructure team at Netflix. His team is responsible for building common infrastructure to collect, transport, aggregate, process and visualize over 700 billion events a day. Prior to Netflix, Peter has built and led the teams responsible for developing, deploying, scaling and managing some the world's largest platforms at Ooyala, Yahoo! and PayPal.